OpenThoughts: Data Recipes for Reasoning Models
Authors: Etash Kumar Guha
Ryan Marten
Sedrick Keh
Negin Raoof
Georgios Smyrnis
Hritik Bansal
Marianna Nezhurina
Jean Mercat
Trung Vu
Zayne Rea Sprague
Ashima Suvarna
Benjamin Feuer
Leon Liangyu Chen
Zaid Khan
Eric Frankel
Sachin Grover
Caroline Choi
Niklas Muennighoff
Shiye Su
Wanjia Zhao
John Yang
Shreyas Pimpalgaonkar
Kartik sharma
Charlie Cheng-Jie Ji
Yichuan Deng
Sarah M Pratt
Vivek Ramanujan
Jon Saad-Falcon
Stutee Acharya
Jeffrey Li
Achal Dave
Alon Albalak
Kushal Arora
Blake Wulfe
Chinmay Hegde
Greg Durrett
Sewoong Oh
Mohit Bansal
Saadia Gabriel
Aditya Grover
Kai-Wei Chang
Vaishaal Shankar
Aaron Gokaslan
Mike A Merrill
Tatsunori Hashimoto
Yejin Choi
Jenia Jitsev
Reinhard Heckel
Maheswaran Sathiamoorthy
Alex Dimakis
Ludwig Schmidt
Venue: ICLR 2026 Oral (2026)
Keywords: Reasoning
Data
LLM
Primary area: foundation or frontier models, including LLMs
TL;DR: Data pipeline analysis for training reasoning models

Abstract: Reasoning models have made rapid progress on many benchmarks involving math,
code, and science. Yet, there are still many open questions about the best train-
ing recipes for reasoning since state-of-the-art models often rely on proprietary
datasets with little to no public information available. To address this, the goal of
the OpenThoughts project is to create open-source datasets for training reasoning
models. Our OpenThoughts2-1M dataset led to OpenThinker2-32B, the first model
trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard
reasoning benchmarks such as AIME and LiveCodeBench. We then improve
our dataset further by systematically investigating each step of our data genera-
tion pipeline with 1,000+ controlled experiments, which led to OpenThoughts3.
Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields
our OpenThinker3-7B model, which achieves state-of-the-art results: 53% on
AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Dia-
mond – improvements of 15.3, 17.2, and 20.5 percentage points compared to the
DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on
openthoughts.ai.

Published as a conference paper at ICLR 2026
Open Thoughts DATA RECIPES FOR REASONING MODELS
Etash Guha 1.2, Ryan Marten Sedrick Keh*1, Negin Raoof* Georgios Smyrnis Hritik Bansals7, Marianna Se Jean Mercat Trung Vu Zayne Sprague Ashima Suvarna", Benjamin Feuer10, Liangyu Chen', Zaid Khan 1 Eric Frankel", Sachin Grover'?, Caroline Choi', Niklas Mucnnighoff Shiye Su², Wanjia Zhao', John Yang', Shreyas Pimpalgaonkar", Kartik Sharma?, Charlie Cheng-Jie Jiº, Yichuan Deng". Sarah Pratt?, Vivek Ramanujan", Jonn Saad-Falcon' .Jeffre Li, Achal Dave, Alon Albalak 13
Kushal Arora', Blake Wulfe', Chinmay Hegde Greg Durrett Sewoong Oh?, Mohit Bansal 11 Saadia Gabriel', Aditya Grover", Kai-Wei Chang', Vaishaal Shankar, Aaron Gokaslan 14 Mike A. Merrill', Tatsunori Hashimoto', Yejin Choi', Jenia Jitsey 8,916 Reinhard Heckel's Maheswaran Sathiamoorthy", Alexandros G. Dimakis f3.5 LLddiig Schmidt
Stanford University, Universityy of Washington, BespokeLLabs.at *Toyota Research Institute, UC Berkeley, SUT Austin, TUCLA, *JSC, "LAION, "NYY, "UNC Chapel Hill, 12 ASU, Liia Sciences, 1CConell Tech 15TUM 16 O'pen-V (Open-Sci) Collective
ABSTRACT
Reasoning models have made rapid progress on many henchmarks involving math, code, and science. Yet, there are still many open questions about the best train- ing recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to public information available. To address this, the goal of the OpenThought project is to create open-source datasets for training reasoning models. Our Open Thoughts- 1M dataset led to OpenThhnkeer23322 the first model trained on public reasoning data to match DeepSeek-R -eigsSee--R--istill---3--- --Distill-322B on standard reasoning benchmarks such AIME and Live We then improve our dataset further by systematically investigating each step of our data genera- tion pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields our Opennhiiner''s AIME 2025, 51 % on LiveCodeBench 06/24-01/25, and 54% on GPQA Dia- mond - improvements 15.3, 17.2, and 20.5 percentage points compared to the All of our datasets and models are available on
model, which achieves state-of-the-an results: 53% on
DeepSeek-RI R-Di-- openthoughts.ai.
AIME 2025
LiveCodeBench
GPQA Diamond
60-
60
60
Accuracy (%6)
40
40
40
30
30
30
20
10
10
10
"I
"LOK" 100K
11M
1K
"OKK
100K
I'M
1K
'LOK"
100K
1M
Dataset Size
Dataset Size
Dataset Size
OpenThoughts3
AM
LIMO
Nemotron Nano
Dwwen225-77--Inste
Figure 1: OpenThouuhtss outperforms existing SFT reasoning datasets across data scales. All models are finetuned from Owwen-2.5-BB-Instruu We compare to large SFT datasets (AM, Nemotron Nano) and small curated datasets (s1.1. LIMO) on AIME 2025 (left), LiveCodeBench 06/24-01/25 (middle), and GPQA Diamond (right). Scaling curves for all evaluation benchmarks are in Figure 6.
+ denote equal contribution. Ç denotes additional care contributors. The order determined randomly.

---

Published as a conference paper at ICLR 2026
OpenR1 -Distill- 7B
Nemotron- Nano-8B 8B
Qwen2.5 1-.-5-7B-. 3-7B--Bnstruut Instructt
OpenThinker3-ng
DS-RI- DS-RI-Qwen-Th Hell---en-7B 7B
NemoNano- IMM
AceReason 7B
AM-1.4M
-FF Rt
Skywork-7h
Bas Model
Benchmark
Base Model
M
M
O
N/A
Train Size
1.2M
800K
IM
1.4M
350K SFT
3.9M
57K
119K
N/A
Method
SFT
SFT
SFT
SFT
SFT/RL
RL
RL
N/A
Trained by us
Yes
No
Yes
Yes
No
No
No
No
N/A
Open Data
N/A
Average
55.3
42.9
47.3
42,1
47.2
53.2
52.9
51.6
24.0
Math
AIME24
69.0
51.3
55.0
28.3
57.7
62.0
71.0
68.3
15.0
AMC23
93.5
92.0
87.0
82.2
87.0
94.0
93.8
91.0
53.0
MATH500
90.0
88.0
86.8
87.4
88.0
89.4
89.8
90.2
70.8
Code
CodeElo
31.0
19.9
28.6
21.0
30.1
30.9
32.9
37.0
5.5
LCB 05/23-05/24
64.5
48.7
58.0
54.5
37.9
68.0
60.5
60.4
36.2
CodeForces
32.2
28.3
24.8
29.3
32.9
30,9
32.5
10.2
Sei
GPQA-D
53.7
33.2
52.9
48.3
58.9
52.9
52.9
50.2
24.6
JEEBench
72.4
50.4
61.0
61.1
68.7
70.7
64.3
55.3
33.9
Held Out
HMMT 02/25
42.7
25.0
24.7
19.0
25.7
26.7
33.3
32.7
2.0
HLE MCQ
10.2
12.4
2.1
9.5
12.4
12.0
10.9
10.7
12.7
AIME25
53.3
38.0
41.3
28.7
39.7
48.0
50.7
47.3
8.0
LCB 06/24-01/25
51.7
34.5
42.2
40.3
30.7
50.9
44.3
43.8
16.3
Table 1: OpenThinker3-7B outperforms all open-data 7B and 8B reasoning models across domains. Our model also performs well on held out benchmarks which are not measured during our main experimentation, such HMMT and AIME25. ble, Vp denotes model trained from D.ven22.57BBIInstruec, & for 2ien-225-Mathh-ass 00 for the starting checkpoint of the training strategy. "Method" denotes the model's optimization algorithm. In each row, we bold values within two standard errors of the highest-scoring model.
our ta-
& DeepSeek-RR DeepsSek-RI-Dissee RI-Distilll-Qenn--2 "Base Model" denotes
INTRODUCTION
Recent models, such as DeepSeek-R1 (Guo et al., 2025) and o3 (OpenAI, 2024), have demonstrated strong performance in reasoning-baad domains, including math, coding, and science. These models often start from strong base model, then introduce reasoning capabilities through series of post-training techniques like supervised finetuning (SFT) or reinforcement learning (RL). This post- training process equips these models with the ability to output long chains of thought, or "thinking tokens," during inference time, which can guide the model toward the correct answer. Yet, the complete recipes for frontier reasoning models are not public, making research for building reasoning models difficult.
Innovating SFT data curation powerful method for building reasoning models (Abdin et al., 2024; Lin et al., 2024). For instance, the RI-Distill models show that it is possible to get statc-of-the- small- to mid-scale reasoning models, with performance of 51% on AIME and 33% on GPQA, without any RL steps, based only on supervised fine-tuning of question-thinkinn tokens-ansen answer triplets. where the thinking tokens and answers are generated using reasoning teacher model or are taken from the real data that contains reasoning traces. Existing works, such as SkyTI (NovaSky-Team, 2025b) and S1 (Muennighoff et al., 2025), adopt nearly identical model architectures and training setups as typical instruction tuning, still achieve performance improvements by focusing on improving the training datasets. These examples highlight the importance of curating high-quality SFT data key lever for reasoning performance.
large, carefully curated dataset
2

---

Published as a conference paper at ICLR 2026
Most of these projects, however, explore only a limited fraction of possible design choices, such as relying on human- written questions or using DeepSeek-RI as the sole teacher. Recreating reasoning models requires exploring large design space for various strategies of generating question-an -uestion-answer pairs for reasoning (Face, 2025). This exploration is prohibitively expensive for many researchers due the high costs of teacher inference and model training. In the absence of these expensive experiments, many papers rely on existing heuristics and intuitions to inform their data design choices.
To demystify the SFT data curation process and gain a deeper understanding of what contributes to strong reasoning SFT dataset, we conduct an empirical investigation into data curation techniques for improving reasoning capabilities. Through more than 1,000 ablation experiments across three data domains (math, code. and science). we develop a simple, scalable, and highly performant pipeline, as shown in Figure 2. We scale up this pipeline to produce Open OpenThoughts3-1.25 fine-tune Qwen2.5-7BB 7-- on it to yield Open Thinker3-7B As seen in Table 1, the result- ing OpenThinkerr377B is a state-of-the-an art open-data model at the 7B scale on several reasoning benchmarks, outperforming the RI-Distill-7B model by 12.4 points on average across 12 tasks, and outperforming the next-best open-data model WeemorronNaano-ie
and
by 2.1 points. We release our models and data artifacts to the community and share several key findings from our study. Some of our insights include: (1) Sampling multiple answers per question from a teacher model is an effective technique increase the size of data source by at least 16x. The increased dataset scale drives significant performance gains. (2) Models with better performance are not necessarily better teachers. QwQ-32B is a stronger teacher than DeepSeek-RI. although scores lower on target reasoning benchmarks. (3) We experimented with numerous verification and answer filtering methods, and none gave significant performance improvements. (4) Selecting questions from a small number (top I or 2) of high- sources leads to better downstream performance compared to optimizing for diversity (i.e., top or 16 sources). (5) Filtering questions by LLM labeled difficulty or LLM response length yields better results than filters typical to pre-training data curation using embeddings or fastText.
2 RELATED WORK
The release of models such as Gemini (Gemini-Team et al., 2023), QwQ (Qwen-Team, 2025), and DeepSeek-RI (Guo et al., 2025), which made long reasoning traces visible to users, opened the possibility of training small models via the distillation of traces from larger ones. DeepSeck released strong distilled models together with DeepSeek-RI (c.g., DeepSeek- RI- RL--Dstill-QQenn-7B) showing how promising this strategy can be. Following this, many open-data efforts have attempted to replicate these models by building SFT reasoning datasets through distillation from teacher models such as QwQ-32B (NovaSky-Team) 2025b) DeepSeek-R1 (Bespoke-Latee 2025).
Many datasets target math. code, and science to develop reasoning capabilities. Datasets such as OpenR (Face, 2025). Oaessath (Moshkov et., 2025), and OpenCodeReasoninng (Ahmad et al., 2025b) collect questions from public forums and competition sites like CodeForces, AoPS, and StackOverflow while others like Natural Reasoning (Yuan et al., 2025) use large pre-training corpora as seed data for generating reasoning traces. Efforts like S1 (Muennighoff et al., 2025) and LIMO (Ye al., 2025) emphasize manual curation of small datasets of challenging, high-quality prompts. In practice, many reasoning projects (e.g., Deep 103K (He et al., 2025c). OpenR1 (Face, 2025), and Nvidia Nemotron (Adler et al., 2024)) introduce innovations across multiple stages, such as data sourcing, filtering, and scaling. Beyond SFT, works such as AceReason (Chen et al., 2025) and Skywork-OR1 (He et al., 2025a) build reasoning datasets for reinforcement learning.
3 OPENTHOUGHTS3 DATA PIPELINE
3.1 EXPERIMENTAL SETUP
Training Our goal is to create the best dataset of question-response pairs for SFT reasoning. The best dataset is the one that produces the highest-performing model. To approach this systematically, we ablate each step of our pipeline individually, isolating the effect of given strategy while keeping the rest of the pipeline constant. For each experiment, we utilize the full pipeline to generate 31 31,600 data points for each data strategy, and we finetune Siven2.5-7B-Instruee (Yang et al., 2024b) on each dataset. Our experiments are conducted at a dataset scale that is small enough to be cost-effective yet
3

---

Published as a conference paper at ICLR 2026
Source Questions
Mix Questions
Filter Questions
Generate Answers
Filter Answers
Select Teacher Model
Final Recipe
The Thoughts experiment pipeline aims to build the strongest reasoning dataset recipe. We investigate (1) sourcing questions from existing and newly generated datasets, (2) mixing questions from the top-perforning sources, (3) filtering for high-quality questions using fastText or LLMs, (4) deduplicating questions and sampling multiple answers per question, (5) filtering out low-quality answers using LLM verification or majority consensus, and (6) selecting the best teacher model.
Figure 2:
large enough to provide a meaningful signal. We choose 31,600 as a log-scale midpoint between 10K and 100K, as v10 = 3.16. These experiments inform the design choices for the final Open Thoughts3 pipeline. Appendix A contains details on hyperparameters and training setup.
Evaluation Setup We evaluate models set of reasoning benchmarks containing math, code, and science questions. Per domain, these benchmarks are: AIME24 (MAA, 2024), AMC23 (MAA, 2023) and MATH500 (Hendrycks al., 2021b) for math; CodeElo (Quan et al., 2025), CodeForces (Penedo et al., 2025a), and LiveCodeBench 05/23-05/24 (Jain et al., 2024) for code; GPQA Diamond (Rein al., 2024) and JEFBench (Arora al., 2023) for science. We score each model based on average performance on these eight tasks. Evalchemy (Raoof et al., 2025) is our primary evaluation tool, and we use the default setup provided for each benchmark. Further details on evaluation setup are in Appendix B. We also decontaminate our datasets against our benchmarks by removing samples with high similarity. Details for this process are in Appendix C. To measure generalization, our pipeline experiments exclude held out set of benchmarks, which are only measured once pipeline experiments are over. This held out set consists of AIME 2025 (MAA, 2025), HMMT 02/25 (Balunović et al., 2025), Humanity's Last Exam (multiple choice questions subset) (Phan et al., 2025), and LiveCodeBench 06/24-01/25 (Jain al., 2024).
Pipeline At each pipeline step, we select the pop-performing approach based on the average benchmark score across all domains, and then proceed to the next step in the pipeline experimentation with this selection. DeepSeek-RI is the default teacher model unless specified otherwise.
3.2 QUESTION SOURCING
The first step in our data generation pipeline is finding questions for each data domain. We can broadly categorize our question sourcing techniques into three types: (1) Fully synthetic an existing LLM generates questions with little-to-no seed material. Examples include CodeAlpaca (Chaudhary, 2023) and CamelChemistry (Li et al., 2023a). These often involve prompting an LLM with a template to generate multiple questions. (2) Semi-
LLM uses existing data sources such as CommonCrawl or Web (Penedo et al., 2024a) as seeds to form questions. Examples include TigtrLabMath (Yue et al., 2024) and Teth (Zhang et al., 2024b). (3) Non-synthetic humans write the questions. Examples include StackExchange and ShareGPTCode. These questions
often arise from online forums, contests, chatbot interactions, and other sources.
Our experiments cover 27 different question sources for code questions, 21 sources for math, and 14 sources for science. The details of these sources are in Appendix O.I. The first step of ablation is to generate 31,600 questions using each source. For sources that produce fewer datapoints, we repeat the questions until we reach the desired amount. We use GPT-4o-mini for all sources that we generate which require an LLM. Finally, we use DeepSeek-RI to generate responses for each question, even if a pre-existing answer exists.
The experimental results are in Table 2. For code, CodeGolf questions from StackExchange and com- petitive coding questions from OpenCodeReasoning (Ahmad et al., 2025b) perform well, achieving scores of 25.3 and 27.5 on average code benchmarks. For math, both L.-M-generated questions in openmath-2-nn m-th (Toshniwal et al., 2024a) and numan-written questions in NuminaMath (LI et al.,
4

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Code Question Source
Average
Code Avg
Math Avg
Science Avg
StackExchange CooeeGf
38.8p.
25.3p.6
50.9
40.70.s
OpenCodeReusoning
38.4
27.5
47.90.7
40.70.6
KodCode-VI
37.7
23.90.4
49.80.7
40.4g.3
bugdaryan/ssl-oreate....
21.60.6
7.0
34.1
24.70.9
Math Question Source
Average
Code Avg
Math Avg
Science Avg
OpenMath- 2--Mthh MutinaMath-1.5 MathPile
38.1p.3
12.4
58.8
45.60.2
37.40.5
11.40.5
58.51.0
45.01.2
36.2p.s
11.5p.7
55.1p.9
44.61.1
Laplofficia//Mtth
24.4p.3
7.30.3
38.61.0
28.50.3
Science Question Source
Average
Code Avg
Math Avg
Science Avg
StackExchange-Phyee OrganicChemistry-PDDF
34.3p.4
11.9
50.9
43.2g.7
34.0p.3
8.40.3
52. 12.1
45.3
CQADupStack-Phssies
33.3n.4
7.40.3
51.9
44.10.9
AdaperrOceanbiioogy__aaass
21.90.1
3.10.3
41.31.1
21.1o.8
Table Evaluating question sources and generation strategies. We show only the top 3 scoring sources for each domain; descriptions of each source are in Appendix O.1 and full results are in Tables 31 to 33. Each row represents a unique source of questions. Question quality greatly affects performance, yielding 17.2 gap between the strongest and the weakest code datasets. The symbol denotes a new dataset we created with a programmatic generation strategy. Gray subscripts represent standard errors, and bold values within two standard errors of the highest-scoring data strategy.
SFT Datasets
Benchmarks
Code Question Mixing Strategy
Average
Code Avg
Math Avg
Science Avg
Top I Code Sources
39.9
23.1 1.0
54.5
43.1 1.2
Top 2 Code Sources
41.30.4
27.3
54.70.9
42.1 1.0
Top 4 Code Sources
38.60.4
24.20.6
52.2
39,80.9
Top 8 Code Sources
37.0p.4
21.8p.3
51.91.2
37.7a.6
Top 16 Code Sources
36.40.4
20.80.4
50.1p.9
39.11.0
Table 3: Mixing different code question sources. Our experiments show that choosing only the two best question sources outperforms mixing more question sources. Similar results hold for science and math data domains. Full results including the math and science datasets are in Tables 34 to 36.
2024) score the highest, achieving 58.8 and 58.5 on average across math benchmarks. Lastly, for science, the highest-scoring question generation strategies physics questions from StackExchange and LLM-extracted questions from organic chemistry textbooks, which achieve an average score of 43.2 and 45.3, respectively, on science benchmarks. No clear pattern emerges across question generation strategies simple synthetic methods perform comparably to, and occasionally better than, more complex manually curated pipelines. These top-performing question sources provide the foundation for subsequent stages of the pipeline.
5

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Math Question Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
Response Length Selection (GPT-4.1-mini)
41.90.3
13.4
66.0
48.60.4
Response Length Selection (GPT-4.I-nand
39.4u.3
11.0g.4
64.5
44.3u.7
AskLLM Selection
36.30.4
9.5n.5
58.1
43.8n.6
FastText (P: Numina; N: Lapl lffloffiiill
35.60.4
11.0g.2
54.91.1
43.5
Code Question Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
Difficulty- Selection
43.0p.5
27.7a.4
56.0 1.3
46.40.7
Response Length Selection (GPT-4.1-nano)
42.20.1
26.60.5
55.41.3
46.0p.2
AskLLM Selection
41.60.5
28.8
52. 1.2
45.2
Response Length Selection (GPT-40-mini)
40.80.5
25.60.5
53.1
45.2
Table 4: Filtering questions provides an effective tool for extracting high-quality questions. Using LLMM-based methods to find the best questions from the question sources outperformed classical filtering methods such as fast and embedding- filters. This table shows the top-performing strategies. Full results including science datasets are reported in Tables 37 to 39.
3.3
MIXING QUESTIONS
After obtaining high-quality questions from various question sources, the challenge becomes how to combine them effectively should we rely on a single best tersfoerrooming strategy, or blend multiple strong ones maximize downstream performance? This mixing strategy key design choice in many generation pipelines (Yue et al., 2024; Moshkov et al., 2025; Lambert et al., 2024). Intuitively, adding more question sources into the mix introduces the risk of incorporating lower-quality strategies in exchange for greater diversity. Our experiments aim assess whether the additional question diversity justifies this tradeoff in terms of question quality.
For simplicity, we use the rankings of the previous step in our pipeline (Section 3.2) heuristic for candidate dataset selection. Our mixing strategy selects the top-ranked datasets, randomly samples 31,600 questions from each source, and concatenates them to form dataset of size 31,600. We sweep values of N E 21, 4, 8, 16). The results for the code domain are in Table 3 while the other results are in Appendix P.2. Our experiments show that mixing many question sources degrades performance: mixing at most two sources yields the best results across all data domains. Using two high-quality code question sources instead of 16 strategies results in a 5% accuracy increase on average across all benchmarks. This result indicates that downstream performance benefits from increased quality of source data rather than diversity induced by mixing multiple question sources.
N
Takeaway: We use OpenMath-2-MM 2-atthh-hthhhh as our sole math question source, CodeGolf and Open- CodeReasoning as our code question sources, and StackExchaneehhysies and OrganicChemistry- PDFs our science question sources.
3.4 QUESTION FILTERING
Since each data source can contain millions of potential questions, answering and training on every possible question is infeasibly expensive. Therefore, the next step is to select a high-quality subset of questions from each source. Across the literature, wide range of filtering strategies have consistently improved overall dataset quality (Soldaini et al., 2024; Su et al., 2024; Li et al., 2024; Wettig et al., 2025; Penedo al., 2024a; Gao et al., 2020; Shum et al., 2025). Using the best question sources from Section 3.3 as a starting point, we extensively explore various filtering methods, including fastText classifiers, difficulty scores, and embedding distance, to select higher-quality questions. A detailed description of these filtering methods is in Appendix O.2. The results of these experiments for math and code are reported in Table 4 while the science results are in Appendix P.3 (Table 39).

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Science Answer Generation Strategy
Average
Code Avg
Math Avg
Science Avg
Exact Dedup w/ 16x sampling
36.20.5
9.0g.4
54.5
49.71.2
Fuzzy Dedup w/ 16x sampling
36.10.4
10.9
52.9
48.8p.5
Exact Dedup w/ 4x sampling
35.8
10.6
51.81.0
49.61.2
No Dedup w/ 4 X sampling
35.8
10.0g.4
55.2
45.40.9
No Dedup w/ 16x X sampling
35.70.4
7.60.5
53.8
50.90.5
No Dedup w/ 1x sampling
35.5
9.30.3
54.2
46.90.2
Exact Dedup w/ 1x sampling
35.00.4
7.60.4
54.0
47.5p.5
Fuzzy Dedup w/ 4x sampling
34.90.4
7.40.5
55.01.0
46.0p.7
Fuzzy Dedup w/ 1x sampling
34.20.3
5.8u.4
52.50.7
49.50.4
Table 5: Deduplication and repeated teacher sampling provide an axis of scale. Using fewer questions and annotating more times performs similarly even outperforms annotating more questions fewer times. There does not seem to be clear trend in types of deduplication that improve performance. Full results including math and code datasets are in Tables 40 42.
The two highest performing question filtering methods difficulty-basee filtering and response length filtering. Difficulty-baser filtering asks an LLM (GPT-40-mini) to assess the difficulty cach question, then retains the most difficult questions. Difficulty-basee filtering is the winning strategy for code. Meanwhile, response length filtering asks an I.LM to respond to each question directly, then selects the questions with the longest LLM-ggenrrted responses. Response length filtering performs the best for math and science. For math and code domains, using the best question filtering strategy resulted in an average improvement of 4% and 6% over the random filtering baseline, respectively. We test different LLMs such as GPTT-4-miin --iii and GPT-4.1 --nmo for response length filtering and find that using stronger models for response length filtering typically outperforms using weaker models. For all domains, using LLM-based filtering methods outperformed classical filtering methods such as embedding- based and fast Text filters.
Takeaway: We difficulty-based filtering with GPT-44-mini for code questions, and response length filtering with GPT-4.1-mine for math and science questions.
3.5 DEDUPLICATION AND SAMPLING MULTIPLE ANSWERS PER QUESTION
Deduplication is powerful strategy for improving dataset quality (Li et al., 2024; Lee al., 2022; Penedo et al., 2024b; Fang et al., 2025; Liu et al., 2024). Our ablations investigate the effeets downstream reasoning performance. We explore three degrees of
of question deduplication deduplication strictness: no deduplication, exact match deduplication, and fuzzy deduplication using threshold-basee string similarity. Further details are in Appendix O.3. While deduplication enhances question diversity by reducing repetition, a natural counterpart for enhancing answer diversity is to query the teacher multiple times elicit distinct responses. This strategy trades off higher answer diversity for lower question diversity and provides another axis of data scale. We explore three levels of sampling multiple answers per question, at 1x, 4x, and 16x.
To address the interplay between naturally occurring duplicate questions in the source datasets and the need to query the teacher multiple times per question, we sweep all combinations of deduplication levels (none, fuzzy, exact) and sampling multiple answers (1x, 4x, 16x). The results for the science domain are presented in Table 5, while the other domains' results are in Appendix P.4 (Table 40 and Table 41). For code and science data, various combinations of deduplication and multiple answer generation yield similar results. For example, the baseline of no deduplication with x answer per question performs 0.7 points worse average than exact deduplication with 16x answers per question for the code domain. Meanwhile, for math, exact deduplication with 4x answers per question performs the best, and 16x answers per question is the second-best option. We adopt the second-best option moving forward, as it provides better scalability. Similar to Section 3.3, the results here indicate that the benefits of question diversity may be limited for the reasoning datasets measure performance on, at least when answer diversity increases. Thus, for math and science, we
7

---

Published as a conference paper at ICLR 2026
Math Answer Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
No Filtering (not compute-controlled))
41.9
15.20.5
65.60.9
46.4p.7
Random Filtering
41.60.4
14.90.4
64.8
46.7
Shortest Answers Selection
41.10.4
14.80.4
63.7
46.7
Removing Non-English Answers
41.10.5
14.20.5
63.1
48.6
GPT Verification
40.0
13.10.3
61.41.1
48.3
Removing Long Paragraphs
38.0p.4
5.70.2
64.50.9
46.81.0
Table 6: Filtering answers did not improve over not filtering at all. Using verification techniques such majority consensus filtering or response-length-basee filtering did not improve upon training on all samples. Full results including code and science datasets are in Tables 43 to 45.
SFT Datasets
Benchmarks
Teacher for Code
Average
Code Avg
Math Avg
Science Avg
Qwen/QwQ-32B deepseek- 1
44.2
29.5
58.7
44.61.0
42.30.5
27.20.5
54.7
46.5
microosof/PPii---reaaonnnng--o
29.0p.4
0.5
52.1
37.2o.6
Table 7: Using a weaker teacher outperformed using a stronger teacher. Across all domains, QwQ-32B was the strongest teacher model, despite being a weaker model than DeepSeck-RA. Further results can seen Table 48.
select the optimal strategy, which is exact deduplication with 16x answers per question. For code, we employ the second-bes best strategy, which involves no deduplication with 16x answers per question.
Takeaway: Our final pipeline uses 16x answers per question for all domains. It uses exact deduplication for math and science and no deduplication for code.
3.6 ANSWER FILTERING
Verification removing low-quality annotations is common step in many reasoning data pipelines. Intuitively, removing data that may be incorrect should improve downstream performance. Our experiments explore various answer filtering techniques. To ensure that we can still obtain datasets of size 11,000 after filtering, we first generate 63,200 answers, apply each answer- filtering strategy, and then sample 31,600 question-answer pairs from the filtered dataset. Our ablations also include baseline with no filtering, which is not contrlled,, as it contains 63,200 questions.
Table shows the result of each filtering method for math datasets, and the results for code and science datasets are shown in Appendix P.5. For math datasets, the random filtering baseline outperformed all other filtering methods. fast Text: (Joulin et al., 2017) classifier was the best answer-filtering method for code question-answer pairs. The positives for the FastText classifier came from CodeForces (Penedo et al., 2025a) answered with DeepSeck-RL. and the negatives came from CodeForces answered with GPT-4o-mini. For science, keeping the top 8 longest answers was the strongest filtering strategy. However, across all domains, the no- fillttering strategy (training on all samples without controlling compute) led to performance similar that of all other methods of filtering. This result suggests that the benefits of answer filtering are not significant enough to justify reducing the number of samples in the dataset. As such, we opt to skip this part in the pipeline.
Takeaway: We do not perform answer filtering because filtering strategy outperformed the baseline, which uses all the answers.
8

---

Published as a conference paper at ICLR 2026
Filter Questions
Deduplicate Questions
Elaia
Science 60k
Randomly Sample Questions
49%
Generate Multiple Answers
Sclence
Source Datasels
TTM
60%
Sclence
6k
X16
Code
Code
75k
Final
Math
1.2M
Math 80k
Math
T8OK
Figure 3: The OpenIhhoughts--1.MM Full Data Pipeline.
3.7 TEACHER MODEL
The previous experiments have relied on using DeepSeek-R2 as a teacher model. However, there are many possible candidates for teacher reasoning models, including DeepSeek R1, Phi-4-Reasoning- Plus-14B (Abdin et al., 2025), and QwQ-32B. Our experiments measure the downstream effects of selecting different teacher models for each strategy, descrihed in Section 3.6. The sampling hyperparameters are kept constant across all teacher models we studied. The results of this experiment in Table Across all domains, using QwQ-32B
teacher model outperforms all other teacher models, yielding an average accuracy improvement of 1.9% and 2.6% over using DeepSeek-R RR as teacher for code and math, respectively. This despite the fact that QwQ-32B scores lower on average when compared to DeepSeek-R1 R1. For example, DeepSeek-RI outperforms QwQ-32B by 9%, 8%, and 23% on CodeElo, GPQA Diamond, and JEEBench, respectively. A comparison of empirical strengths of each teacher is in Table 28.
Takeaway: We use QwQ-32B as the teacher model.
4 SCALING OUR PIPELINE TO OPENTHOUGHTS3-1.222
Dataset scaling plays key role in achieving strong performance. We investigate how well our pipeline scales by identifying the winning strategy in each successive pipeline step and plotting its performance from 3 316 to 31.6k examples. Figure 5 demonstrates that the scaling behavior improves as we successively stack the best choices from each stage in the pipeline. Additionally, Figure also shows strong positive correlation between scale and performance. This suggests that further scaling the dataset size could yield even greater gains. We thus mix and scale up the data pipelines 1.2 million-sized dataset. Open'T hhoughts3-1112M contains 850,000 math, 250,000 code, and 100,000 science datapoints. We chose this ratio following the OpenThoughts2- 1M mixture used train Thinkein, which exhibited strong and balanced performance on with the DeepSeck- - -Distill models. To arrive at the target number of samples each domain, we work backwards estimate how many questions we need at the beginning of the pipeline. Then, we apply the highest performing strategy at each stage in the pipeline, opting for the more scalable choices if performance is equal. This construction process of OpenTToouhhts-11 1.2M illustrated in Figure 3.
from Section 3 build Open[Thoughts311222 3.....
As seen in Table 1, OpenThinker3-777 the best open-data reasoning model at the scale, regardless of optimization algorithm choice (SFT, RL, both). OpenThinker3-77 also generalizes well to evaluations held-out throughout the pipeline process, exhibiting the best scores on HMMT, AIME25, and LCB 06/24-01/25. Further results scaling can be seen in Appendix D.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
Through iterative experimentation, our pipeline surfaced several key insights into effective SFT reasoning data curation. These findings collectively shape our final pipeline, allowing us build OpenTToughtt3-I.2NN a state-of-the-ae open-data SFT reasoning dataset, composed of science, math, and code data. Our final model, OpenThinker3-775 trained on this data, is the SOTA open-data reasoning model at its model scale.
This work has several limitations. We did not explore datasets for reinforcement learning, a standard training regime for building reasoning models. Within the SFT realm, we did not explore the use of staged SFT or curriculum learning to further improve performance. We nonetheless believe this work serves valuable foundation for the community's s continued progress on open reasoning models.
6 REPRODUCIBILITY STATEMENT
We have provided all adequate details to fully reproduce our datasets, including techniques used, code, and prompts. We have also discussed hyperparameters for the training and evaluation setup in the appendix. We believe this is sufficient information for reproducing our experiments.
ACKNOWLEDGEMENT
We would like to thank Alex Fang. Matthew Wallingford, Jiaqi Zeng, Oleksii Kuchaiev, Asad Aali, Benjamin Burchfiel, and Russ Tedrake for helpful discussions and support at various stages of the project. We would also like to thank Adam R. Klivans, Mike Garrison, and Satya Kotari for support with compute and infrastructure. Further thanks go for support provided by supercomputing facilities and their teams, especially Damian Alvarez and Mathis Bode from Juelich Supercomputer Center (ISC, Germany) and to Laura Morselli from CINECA (Italy).
This research is supported by Toyota Research Institute (TRI), the Vista GPU Cluster through the Center for Generative AI (CGAI) and the Texas Advanced Computing Center (TACC) at UT Austin, the nigh-performance computer the NHR Center of TU Dresden, and the compute resources of the AI Systems provided by the Leibniz Supercomputing Centre (LRZ) through the Munich Center for Machine Learning.
This research is also supported by NSF Grants AF 1901292, CNS 2148141, IFML CCF 2019844, and research gifts by UT Austin Machine Learning Lab (MLL) and the Onassis Foundation -Scholarhiip ID: F ZS 056- 176-1/2022, 1/20122222222222.2 AGD was with UT Austin for a part of this work. HB is supported in part by AFOSR MURI grant FA9550-22- 10380. MN and acknowledge funding by the Federal Ministry of Education and Research of Germany (BMBF) under grant no. 01 011S24085C (OPENHAFM), under the grant 16HPC117K (MINERVA) and partial funding under grant no. 011S22094B (WestAl AI Service Center West), well as co-funding by EU from EuroHPC Joint Undertaking program under grant no. 101182737 (MINERVA) and from Digital Europe Programme under grant no. 101193 (openEuroL.LM))
This work would not be possible without the support of the Gauss Centre for Supercomputing e. V. through the John von Neumann Institute for Computing (NIC) on the supercomputer JUWELS Booster at Juelich Supercomputing Centre (JSC). We further acknowledge EuroHPC Joint Undertaking for providing this project access to the EuroHPC supercomputer LEONARDO, hosted by CINECA (Italy) and the LEONARDO consortium through an EuroHPC Extreme Access grant EHPC-EXT-2023E00- 068, storage resources on JUST granted and operated by JSC and supported by Helmholtz Data Federation (HDF), compute resources provided via WestAl compute grant "Measuring and enhancing advanced reasoning capabilities of foundation models via local model deployment" (westai0007) at JSC and compute resources provided via JSC at JURECA. This work was supported in part by RS-20224-00558882, National AI Research Lab Project.
REFERENCES
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, al. Phi-3 technical report: A highly
10

---

Published as a conference paper at ICLR 2026
capable language model locally your phone. arXiv preprint arXiv:2404.142199 2024.
Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, et al. Phi-4-reasoning technical report. arXiv preprint arXiv:2504.213// 2025.
AdapterOcean,
s biology_dataset dataset batiaaset_standaddiiieee unniiid unified. https: zeleeeeddddieeeeeeeeeeeeeeeeeeee unified, 2024. Dataset on
htgg://wwwntttttth/gg/ggggggeee http///gggggggggg.gege/ ct co/datase
Hugging Face.
Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, al. Nemotron-4 340b technical report. arXiv preprint arXiv:2406. 1704, 2024.
Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jo- celyn Huang, Vahid Noroozi, and Boris Ginsburg. Opencodereasoning:: Advancing data distillation for competitive coding, arXiv preprint arXiv:2504.012 01943,, 2025a.
Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jocelyn Huang, Vahid Noroozi, and Boris Ginsburg. Opencodereasoning:: Advanc- ing data distillation for competitive coding. arXiv preprint arXiv:2504.01943, 2025b. URL 01943.
https: http:////araiv..r///////////
Ajibawa. Maths-college. https: / huggg ng sagffce.... co htt..//aaaatttt/aaaaaaaaaaa 20222/22aaa 2003/Mathh s e-Colleges 2023. Dataset Hugging Face.
ajibawa 2023. Code 2ode-290k--hareGPP https: astps::/hugginggface ctt htt.://g.aa..cc/aattttt///aaaaaa
023/Code- 2alee--2220--SaeeePePee 2023. Hugging Face Dataset.
Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and Nick Haber. Big-math: A large- scale, hugl-quality math dataset for reinforcement learning in language models, 2025. URL
https://aaee /atpiii/ariivv///a httpps//aarxx.ccrraa////2..... oxg//abs/2502..13337
Allen Institute for AI. ai2addpp.dee/ogeemaaab--mmee https: : htttps://hugiingee htt.hgggggggacceeecccooc ctaetaacceco///aeeeeee datts ets/ai2 aalaai--aapptta.. www capps///pemaaathhee hath, 2025. Accessed: September 19, 2025.
Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Konzel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation- -ased formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers). pp. 2357-2367, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi:
1245. URL https : aclantholooooyyyyooooooooooooooa org/N19 9-119110 1245.
Daman Arora, Himanshu Singh, and Mausam. Have Ilms advanced enough? challenging problem solving benchmark for large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, pp. 7527-7543, Singapore. December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2022 1/wrr3.2222222 wwwweemwwnlmm.mmmeeeeemm URL https: httptt:ss///aaaaahhhooggeeo wwwpttthwwwooooo.o//2oeee//2 emnl p-main 48.
777p hww.hhhrokkk--aat----lllaaa Hugging
s /aap777a/heeddyykk--aatttt 11m0a,,am,ammmmmm 2025. Accessed: 2025-09-19.
mc2. sql-create-context dataset, 2023. URL
c2/sql-create--ontee
Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. Matharena: Evaluating Ilms on uncontaminated math competitions, February 2025. URL https: httas://mmtteee / arena. ai/
11

---

Published as a conference paper at ICLR 2026
Anshul Bansal. Basies of Organic Chemistry: A Textbook for Undergraduate Students. Bentham Science Publishers, 2024. ISBN 978-981--523-23--1 doi: 10227449979911552222222200000 URL https:/ /w///i.oo// orgg /tt.://.......111111112222222220 22-4.
Online ISBN: 978-981-5223-
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El- Ea-Yaniy, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, al. Llama-nemotron: Efficient reasoning models. arXiv preprint SxXIv:25055009443 2025.
Bespoke-Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning distilla- http://wwii...pse.eeooeeeeeeeeee distillation, 2025.
tion.
Accessed: 2025-01-22.
BigCode. seffcoss-nsttuccssseexeelllll Hugging Face, 2024. htthts:
gface. http://aaattts//sess httpraateeteigiieeeeeeeeeeeeee oos insttnnntttttteeee -acrruccttccccccce fee2-exec-fiie filter-lltrrrrre Accessed: 2025-09-19.
F.A. Carey and R.J. Sundberg. Advanced Organic Chemistry: Part Structure and Mechanisms. Advanced Organic Chemistry. 2007. ISBN 9780387448978. URL https:::///oottt google
www/bookokkeeee..oo id=g5dkyyMBBhoc..
Francis A. Carey. Organic Chemistry. McGraw-Hill, edition, 1996. ISBN 978-0072905014.
Francis A. Carey. Organic Chemistry. TMH, New Delhi, 7th ed. edition, 2010.
W. Carruthers and lain Coldham. Modern Methods of Orgamic Synthesis, Cambridge University Press, 3rd ed. edition, 1978. ISBN 0-521-217194 URL https: /bookks www.lesssoooooo.....ooooo www.le.oooooooooooooooooooo..oo com/books /about /Modern Methods_tssssseeee of Beethods_oo_Organii_______ httpai/_ssstttsiii..emeeeeeeeee.
Linzheng Chai, Shukai Liu, Jian Yang. Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, Zekun Wang, Boyang Wang, Xianjie Wu, Bing Wang, Tongliang Li, Liqun Yang, Sufeng Duan, and Zhoujun Li. Mceval: Massively multilingual code evaluation, 2024. URL pattps://arxie org/ a 07436.
Sahil Chaudhary, Code alpaca: An instruction-following llama model for code generation. https: //gittubb//////// com sahi 1ww.11200111111 cow.aa/aaaceaapaaaaaacaaa
2023. 2023.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like llms. arXiv preprint 4rX 2024.
Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. neeretoon-nemotron: Advancing math and code reasoning through reinforcement learning. arXiv preprint, 2025.
2025.
J. Clayden, N. Greeves, and S. Warren. Organic Chemistry. OUP Oxford, 2012a. ISBN 9780199270293. URL https::: /bwoksooss////////e/ google. www./owwo/eeeeeoooooooo Fid--wwUe-----eee 2e- 4MC.
Jonathan Clayden, Nick Greeves, and Stuart Warren. Organic Chemistry. Oxford University Press, Oxford: New York, 2nd ed. edition, 2012b.
Colin B. Clement, Matthew Bierbaum, Kevin P. O' O'efffe, and Alexander A. Alemi. On the use of arxiv as a dataset, 2019.
Rosetta Code. Rosetta code rosetta code,, URL wtt.tttss//rostttccc..rrrr/////e dex www.:pppttitle-tsstttt_cccodeeee
[Online; accessed 8-December-2022].
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai- Chang. Openvithinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503. 17352, 2025.
Paul M. Dewick. Essentials of Organic Chemistry. Wiley, New Delhi, 2011.
12

---

Published as a conference paper at ICLR 2026
Ricardo Dominguez-Olmeddo Vedant Nanda, Rediet Abebe, Stefan Bechtold, Christoph Engel, Jens Frankenreiter, Krishna Gummadi, Moritz Hardt, and Michael Livermore. Lawma: The power of specialization for legal tasks. In International Conference Learning Representations (ICLR), 2025.
Jonathan Drechsel, Anja Reusch, and Steffen Herbold. MAMUT: A novel framework for modifying mathematical formulas for the generation of specialized datasets for language model training. Transactions on Machine Learning Research, 2025, ISSN 2835-8856. URL buttps::/openn. view. nww. forum? d=kh
Hugging Face. Open rl: A fully open reproduction deepseek-rl, January 2025. URL https: //github. www.co////ooooo hoom/hugginggaee openn-l..
Christian Fahlgren. react-code-innstrucionn https: http://////ggggaaaaaaoaaaaaaaaaa purenn/reacte code oon 2025.
Alex Fang, Hadi Pouransari, Matt Jordan, Alexander Toshev, Vaishaal Shankar, Ludwig Schmidt, and Tom Gunter. Datasets, documents, and repetitions: The practicalities of unequal data quality, 2025. URL /
Joseph G. Flowers. Par-four ott.fuufttttttuuaattddddddoooeod
ht Four Coour-Fineweo-
tps:
2024. Dataset
Hugging Face.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy, The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint 100iv:2101. 100000027 2020.
L. Gattermann, W.B. Schober, and V.S. Babasinian. The Practical Methods of Organic Chemistry. Macmillan, 1917. URL https: /books gwogle. wwt.cooose/ooeeeeeoo books ? 2id-iiiii LQQAAAAYYAUAAAA
Gemini-Team, Rohan Anil, Sebastian Borgeaud, Jean- can-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.1 //805, 1805, 2023.
Glaive. saaivaaillaive-code--usstttttt A dataset for code assistance. https://mm htt.///uggiggg ace , httd/aaaatssgaaaaaaaaeaaeeeeeeee 2025-09-19.
code http://aaaaeeoocaaiisstttatttaaa set.ee-sssssttttttrr.aa 2023. Accessed:
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
H. Hart, C.M. Hadad, L.E. Craine, and D.J. Hart. Organic Chemistry: Short Course. Available Titles OWL Series. Cengage Learning, 2011. ISBN 9781111425562. URL //books.
google. www.boowos.eeeo.ooooooooooo.ooww 21d==
Eric Hartford, Lucas Atkins, Fernando Fernandes, and Cognitive Computations. Dolphin-coder. htt.://ceeoeceooooooeeeeeeetttae www.://////ooppiiiooooooeoeoeeoo 2024.
https://hugginggeee
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng. Xiaoyu Zhang, Fuxiang Zhang. Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. Skywork open reasoner 1 technical report. arXiv preprint 2:X2v:2505..2312. 2025a.
Zhiwei He, Tian Liang. Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang. Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath- 103k: A large-scale, challenging, decontaminated. and verifiable mathematical dataset for advancing reasoning. 2025b. URL http:://
org/ abs 2504. 11456.
13

---

Published as a conference paper at ICLR 2026
Zhiwei He, Tian Liang. Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang. Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang. Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath- 103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. arXiv preprint arXiv:2504. /456, 2025c. URL https: /apps://axxiii htt.v/ixivvv. org/ alos/ 2.00..444 11456.
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. Measuring coding challenge competence with apps. NeurlPS, 2021a.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurlPS, 2021b.
Robert V. Hoffman. Organic Chemistry: An Intermediate Text. John Wiley & Sons, 2004. ISBN 978-0471450243.
Doris Hoogeveen, Karin M. Verspoor, and Timothy Baldwin. Cqadupstack: A benchmark data set for community question-answering research. In Proceedings of the 20th Australasian Document Computing Symposium (ADCS), ADCS 15, pp. 3:1-3:8, New York, NY, USA, 2015. ACM. ISBN URL htttp://dofi. acm org / 10. 114
978-1-4503-4404-33 doi: 10. 10.1144//838933128888388 5/2838931.28388
Siming Huang, Tianhao Cheng, Jason Klein Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J. Yang, H. Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Zhaoxiang Zhang, Jie Fu, Qian Liu, Ge Zhang, Zili Wang, Yuan Qi, Yinghui Xu, and Wei Chu. Opencoder: The open cookbook for top-tier code large language models. 2024. URL htwps: UUU.xppps//aaxx...orr///////1112 04905
04905.
ibivibiv. math instruct dataset. https: http:///ttt/t//ggggggggaaaa h_instruct, 2023.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint vrXiv:2403.07974 2024.
Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 427-431, Association for Computational Linguistics, April 2017.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin. Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124, 2024.
Laplofficial. Math. https: : / /huggi i www..cc.cccooommmm wtt.ce.co/atttttt//aaaaaa.a
2024. Dataset on Hugging Face.
Katherine Lee, Daphne Ippolito. Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 8424 8445, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 11118665/112222... -long.577 577. URL https: /ates///aaaaaataa /aclanthologyy org/ 2022 . acl- wcw.llng..........l 1 577/
Guohao Li, Hasan Abed AI Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large scale language model society, 2023a.
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large scale language model society, 2023b.
14

---

Published as a conference paper at ICLR 2026
Jeffrey Li, Alex Fang, Georgios Smymiss Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Kch, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng- Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song. Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby. Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G. Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, and Vaishaal Shankar. Datacomp-Im: In search of the next generation of training sets for language models. A. Globerson, L.. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 14200-14282. Curran Associates, Inc., 2024. URL https:: /ttps::/prroeeddngss.. 2 Saaaapar-Datasets_aa__aeeeeae
ile http://e/eeeeeeeeeeeeddddeee33ea Trasss__rrcckccpc.
Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin. Bin Dong. Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https: https////hgggngngggeeeee //uuminaMathhh111 1. ttttps://gitt s- s-prize/blon http:///a///eeettt///aaaaaaaaaaa http:://mmmmaaaaaatteeeeeeee pdf). 2024.
CO/AI-M wamoo-progres
O/
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago. Thomas Hubert, Peter Choy, Cyprien de Masson d' Autumee Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel Mankowitz, Esme Sutherland Robson, Pushmeet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-lever code generation with alphacode. arXiv preprint arXiv:2203.077 0784444 2022.
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang. Jian Jiao, Nan Duan, et al. Rho-1: Not all tokens are what you need. arXiv preprint arXiv:2404.07900 079655 2024.
Zhengzhong Liu, Bowen Tan, Hongyi Wang, Willie Neiswanger, Tianhua Tao, Haonan Li, Fajri Koto, Yuqi Wang, Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Liqun Ma, Liping Tang, Nikhil Ranjan, Yonghao Zhuang, Guowei He, Renxi Wang. Mingkai Deng, Robin Algayres, Yuanzhi Li, Zhiqiang Shen, Preslav Nakov, and Eric Xing. Llm360 k2-65b: Scaling up fully transparent open-source llms. 2024.
Dakuan Lu, Xiaoyu Tan, Rui Xu, Tianchu Yao, Chao Qu, Wei Chu, Yinghui Xu, and Yuan Qi. 1cp-116k: A high-quality problem-solution solution dataset and a generalized pipeline for automated extraction in the higher education science domain, 2025. URL https: wwwaaaivvvv.... orgoorg//aoooo abs// 2501. 15587.
C.G. Lyons, S. McLintock, and N.H. Lumb. A Concise Text-Book of Organic Chemistry: The Commonwealth and International Library: Chemistry Division. Pergamon, 2016. ISBN 9781483141862. URL https:/ /books......// htt.o//ccooggeecooooooooooa/dda.
MAA. Ame 2023 1tr.:22__MMM1122__rooeeeeeeeeee, Accessed: 2025-05-11
2023. URL http:///..//////rrrooooooooaaoo/ ndex. phppppppp22222 20122__MM____0
MAA. Aime 2024
2024. URL utt.pppp//aatttoooooooooooooooo. ndex. Dtbxx_php/2224AAAMEEE
ems. Accessed: 2025-055111
MAA. Aime 2025 problems, URL http:///nntttt/////oroooooaaaaa.
ndex, p2p/ AIME B_tMMM____Paaeea....ee Accessed: 2025-005-1
thub. http:///gggthtbc.coooooaoaaa co.b/bggie// 2025. GitHub repository.
Lign/Magpie--uee Staa://aappieenne2222-----------
Mantas Mazeika. Long Phan, Xuwang Yin, Andy Zou, Zifan Wang. Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint 240.:2200..044499 2024.
15

---

Published as a conference paper at ICLR 2026
John McMunyy Fundamentals of Organic Chemistry. Brooks/Cole, Cengage Learning, 7 edition, 2010. ISBN 978-1439049730.
John McMurry. Organic Chemistry, Cengage Learning, 10 edition, 2023. ISBN 978-0357853290.
millawell. wikipedia_ field of science. https: httgs////gggggg/ceegeeee co datasattsss//// htt.ssetssmmlaeeellm
sccf_ssceeneee. 2024. Dataset on Hugging Face.
J.R. Mohrig, C.N. Hammond, and P.F. Schatz. Techniques in Organic Chemistry. W. H. Freeman, 2010. ISBN 9781429219563, URL - 929wC.
R. Monson. Advanced Organic Synthesis: Methods and Techniques. Academic Press, 2012. ISBN 9780323159975. URL https: /booksbos.////oo/ooo google wtt.ccooooooooooo 3i03:si1ld-wwwwww0333300
Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal. Christof Henkel, Benedikt Schif- ferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building sfate-of-the-ar mathematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504. 16891, 2025.
Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. Octopack: Instruction tuning code large language models. arXiv preprint XxXiv23088.07124 2023.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL hwtph:tt//// http://aawwww../a/a///////eeee/e
Marianna Nezhurina, Lucia Cipolina- Kun, Mehdi Cherti, and Jenia Jitsev. Alice in wonderland: Simple tasks showing complete reasoning breakdown in state-of-the-art large language models. arXiv preprint arXiv:2406.0200 02061, 2024.
James Flack Norris. The Principles of Organic Chemistry. International Chemical Series. McGraw- Hill Book Company, Inc., New York, second edition, 1922.
NovaSky- -eamm Think less, achieve more: Cut reasoning costs by 50 https://novasky- 2025a. Accessed: 2025-01-23.
ai. ggghhbbbooooss/rrcueeeovttttaaai
NovaSky-Team. Sky-tl: Train your own ol preview model within 450. huts://novasky-
ai. n.github.io/postt/kkyyt. 2025b. Accessed: 2025-01-09.
OpenAl. Openai o3 and 04-mini system card, 2024. /2221c875-02dc- d. ddf.f. Accessed: 2025-05-11.
OpenStax. Chemistry 2e, Rice University, 2019. URL http:::tttpssssseeeeeees wwwprsstta..oo//etallllaee ooks/ htt/cchene /cheeistry-2ee Licensed under Creative Commons Attribution 4.0 International (CC
BY 4.0).
Rameswar Panda, Ruchir Puri, Gaoyuan Zhang, Mayank Mishra, and Matt Stallone. tiny-codes: A dataset for learning reasoning with language and code, 2024. URL attps://huggings co/ co/dataseda aseta /aasstts//amddt. nampdn ai/tiny codes.
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024a. URL https: SCkn2QaG.
/ http:///ppeeeeeeeeeeeeeeeee net formmmiduummm
Guilherme Penedo, Hynek Kydlfček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024b. URL https: opwnrevveenn net forum 1d=n6 SCkn2QaG.
16

---

Published as a conference paper at ICLR 2026
Guilherme Penedo, Anton Lozhkov, Hynek Kydliček. Loubna Ben Allal, Edward Beeching, Agustín Piqueres Lajarin, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Codeforces. https: htLDD://uugiingfaoo.oo/aaaaassss 2025a.
Guilherme Penedo, Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Edward Beeching, Agustín Piqueres Lajarin, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Codeforces. https: http://hgggggnacc.eeee co/ http://ggaacccctttttteeeee//e//e /ttp/aaassssppeen-eeeaeaa r1/codeforeees 2025b.
https:///uggiiggfee
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, and Sean Shi et. al. Humanity's 's last exam, 2025, URL https: wttpav..rrg/ab//251....23
Prime Intellect. Prinehmneeeetsssseeccceeeeeeeeee Hugging Face, 2025. URL
http:///w......otttatt////eeeena stion-answering.
Primelntellect. Synthetic-1. https: www.unggngaaaan
htt/attaastt//Prnnnnneeeelleeeee /SYNTHETIC-1, 2025. Dataset available on Hugging Face.
PrimeIntellectt, http:///wltttt.aaaooooodw---oooo http://///gttbbo.o.//////eeettnt
-xorld-swe-prroblemm 2025. GitHub repository.
prithivML.mods. Coder-Stat dataset. htttps://huge haaeeet.cc.aaeacaaaeeeeeeeeeeeee htt.eeee///dattttsssssrtthi ods/Coder-Stat. 2024. Accessed: September 19, 2025.
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023. URL httpst /ww.vaa.vvvvvv////va//eee/eaaa o abs 2.3 0. 03693.
Shanghaoran Quan, Jiaxi Yang, Bowen Yu, Bo Zheng, Dayiheng Liu, An Yang, Xuancheng Ren, Bofei Gao, Yibo Miao, Yunlong Feng, et al. Codeclo: Benchmarking competition- level code generation of Ilms with human-comparable elo ratings. arXiv preprint arXiv:2501 01 257, 2025.
Qwen-Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL btttss://wwennns htwpw//nm...ihhb.b....... ht/blog/////oooooooooooooooooooo httbblog//ww//2..////////////// g2g-32b/
Negin Raoof, Etash Kumar Guha, Ryan Marten, Jean Mercat, Eric Frankel, Sedrick Keh, Hritik Bansal, Georgios Smyrnis, Marianna Nezhurina, Trung Vu, Zayne Rea Sprague, Mike A Merrill, Liangyu Chen, Caroline Choi, Zaid Khan, Sachin Grover, Benjamin Feuer, Ashima Suvarna, Shiye Su, Wanjia Zhao, Kartik Sharma, Charlic Cheng-Jie Ji, Kushal Arora, Jeffrey Li, Aaron Gokaslan, Sarah M Pratt, Niklas Muennighoff. Jon Saad-Falcon, John Yang, Asad Aali, Shreyas Pimpalgaonkar, Alon Albalak, Achal Dave, Hadi Pouransari, Greg Durrett, Sewoong Oh, Tatsunori Hashimoto, Vaishaal Shankar, Yejin Choi, Mohit Bansal, Chinmay Hegde, Reinhard Heckel, Jenia Jitsev, Maheswaran Sathiamoorthy, Alex Dimakis, and Ludwig Schmidt. Evalchemy: Automatic evals for Ilms, 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. URL ht http::////eeeeeeeeeeeeeeee net for um um?id=Ti (id--16669998
Houxing Ren, Mingjie Zhan, Zhongyuan Wu, Aojun Zhou, Junting Pan, and Hongsheng Li. Reflec- tioncoder: Learning from reflection sequence for enhanced one-off code generation, 2024.
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263. 2023.
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities neural models. In International Conference on Learning Representations.
17

---

Published as a conference paper at ICLR 2026
Kashun Shum, Yuzhen Huang, Hongjian Zou, Ding Qi, Yixuan Liao, Xiaoxin Chen, Qian Liu, and Junxian He. Predictive data selection: The data that predicts is the data that teaches. arXiv preprins arXiv:2503.00808, 2025.
M.B. Smith and J. March. March's Advanced Organic Chemistry: Reactions, Mechanisms, and Structure. Wiley, 2007. ISBN 9780470084946. URL
google com/boo
ks7id=by05kNKmm_. xYC.
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani. Oyvind Tafjord, Evan Walsh, Luke Zettlemoyer, Noah Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. Dolma: an open corpus of three trillion tokens for language model pretraining research. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15725-15788, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18663/1122022.allllooo URL https: / /aa//caaantt aawlanthlllooyyo org 2024 aall-long 840/.
Stack Exchange. Stack exchange data dump, 2024. URL https: aarciiiee httpcchwieeerreeeeeeooooooooe /etg///eaal///////// stackexchange. Dataset all Stack Exchange sites, released under CC BY-SA.
Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Nemotron-ce: Transforming common crawl into refined long-horizon pretraining dataset, 2024. URL https::///// aarriiiiiivv///aaa/aaaaa/aaiavve org/ abs absp/412.. 2595.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
Haodong Sun, Yi Xu, Ling Dong, and Yunpeng Liu. Reflectioncoder: Learning from reflection sequence for enhanced one-off code generation. arXiv preprint arXi::2405.170577 2024.
TIGER-Lab. MATH-plus, 2025. URL https: htttts://huggineen /tt.g/hggaaccceeeee//// co//atasetstsssseeeaaaoa / TIGER-IGG ab/ ab/MATH-pl us. TIGER-LabbMATHH-pluu: A benchmark for evaluating large language models on math reasoning problems.
Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruutt-2: Accelerating ai for math with massive open-source instruction data. arXiv preprint 24X 2024a.
Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-:: Accelerating ai for math with massive open-source instruction data. arXiv preprini arXiv:2410.01560. 2024b.
Zengzhi Wang, Xuefeng Li, Rui Xia, and Pengfei Liu. Mathpile: A billion-token-scalee pretraining corpus for math. The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL httpt:tts///oot //tp:rev///ppeevveew .nt. Eorumooumiioooo fttt//orrm?idsss ?id=Rs vhU69sbG.
Zijun Wang, Haoqin Tu, Yuhan Wang, Juncheng Wu, Jieru Mei, Brian R Bartoldson, Bhavya Kailkhura, and Cihang Xie. Star-1: Safer alignment of reasoning lims with Ik data. arXiv preprint arXiv:2504.01903. 2025.
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang, Magicoder: Source code is all you need, 2023.
Alexander Wettig, Kyle Lo, Sewon Min, Hannaneh Hajishirzi, Danqi Chen, and Luca Soldaini. Organize the web: Constructing domains enhances pre-training data curation. arXiv preprint arXiv:2502.10341 2025.
18

---

Published as a conference paper at ICLR 2026
Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran. Kodcode: A diverse, org/
challenging, and verifiable synthetic dataset for coding. 2025. URL abs/2503. 02951.
An Yang. Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang. Peng Wang. Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint 2iv: 240.124000..611 2024a.
An Yang, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412. 15115, 2024b.
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025. URL https: / arxiv. org/ abs 2502. 03387.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12288 2023.
Weizhe Yuan, Jane Yu, Song Jiang, Karthik Padthe, Yang Li, Dong Wang, Ilia Kulikov, Kyunghyun Cho, Yuandong Tian, Jason Weston, and Xian Li. Naturalreasoning: Reasoning the wild with 2.8m challenging questions, 2025. URL ht https: http//axvvvvo/////////./ org/ abs 22002.50... 13124.
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprins (.Xiv:3009006633, 2023.
Xiang Yue, Tuney Zheng, Ge Zhang, and Wenhu Chen. Mammoth2: Scaling instructions from the web. Advances in Neural Information Processing Systems, 2024.
Be-Wen Zhang, Yan Yan, Lin Li, and Guang Liu. Infinitymath: A scalable instruction tuning dataset in programmatic mathematical reasoning. 2024a. URL https: :ttptpps://aii....// 7089.
Yifan Zhang, Yifan Luo, Yang Yuan, and Andrew Chi-Chih Yao. Autonomous data selection with language models for mathematical texts. arXiv preprint arXiv:2402.07625, 2024b.
Yifan Zhang, Yifan Luo, Yang Yuan, and Andrew Chi-Chih Yao. Autonomous data selection with zero-shot generative classifiers for mathematical texts. The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025 Findings), 2025.
Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencoddeinteppreeer: Integrating code generation with execution and refinement. http::/aarii.orgkaa22400..11111 2024.
19

---

Published as a conference paper at ICLR 2026
A Training Details
22
A.1 Training Framework
22
A.2 Hyperparameters
22
A.3 Packing
22
A.4 Chat Template
22
A.5 System Prompt
B Evaluation Details
23
C Decontamination
25
Additional Scaling Experiments
27
D.I Base Model
29
Additional Data Recipe Experiments
30
E.1
Verification
30
................................
E.1.1
Verification Impact on the Original Open Thoughs
Fea.raogh.......................
................................
30
E.1.2
Removing proof-based questions
30
E.1.3
Extraction- Math Verification
30
E.1.4
LLM-generated Unit Test Verification
31
Teacher Model
E.3
Compressing Reasoning Traces
33
E,4
Poorly Performing SFT Datasets from Weak questions
33
E.5
Stacking Gains from Out of Domain Transfer
34
Model Reasoning Performance Analysis
36
F.1
Performance Math Difficulty
36
F2 Performance Scaling by Code Difficulty
................................
F3 Sampling by Longest, Shortest, Majority
36
G Surpassing the Teacher with Distillation for Legal Reasoning
38
H All Teachers Ablations
39
I Safety Analysis of OpenThinker Models
39
J Existing Frontier Model Evaluations
40
Testing reasoning robustness: Alice in Wonderland evaluation
40
Compute Requirements
44
M Sourcing reasoning traces from the Web
44
20

---

Published as a conference paper at ICLR 2026
N Licenses of Existing Assets
45
Pipeline Details
46
0.1
Question Generation Strategies
46
O.1.1
Code Question Generation Strategies
46
O.1.2
Math Question Generation Strategies
48
O.1.3
Science Question Generation Strategies
50
0.2
Information question filtering strategies
54
0.2.1
FastText Details
54
O.2.2
Code Filtering Strategies
................................
54
0.2.3
Math Question Filtering
55
0.2.4
Science Question Filtering
57
0.3
Deduplication and Teacher Sampling
0.4
Question Answer Filtering
59
0.4.1
Question Answer Filtering for Math
59
0.4.2 Code Question Answer Filtering
59
0.4.3
Science Question Answer Filtering
62
P
Pipeline Experiments Expanded Results
65
P.I
Question Sourcing
65
P.2
Mixing Question Generation Strategies
67
P33
Filtering Questions
68
P.4
Deduplication and Multiple Sampling
70
P.5
Question Answer Filtering
P.6
Teacher Model Experiments
72
21

---

Published as a conference paper at ICLR 2026
A TRAINING DETAILS
A. TRAINING FRAMEWORK
Our training is done using LlamaFactory. This repository is publicly available at https://gtt ub. com/ http://yyouo///// Laaaaa/aLMMMaaarrry
A.2 HYPERPARAMETERS
For the different scales in Figure 1, we use different hyperparameters for each scale. In general, we want to larger batch sizes and larger learning rates, but doing this datasets that are smaller would lead us to take too few steps. We performed hyperparameter sweeps to find an appropriate set of hyperparameters in each model scale. We ultimately ended up with sets of hyperparameters micro (for 0.3K scale), small (for IK, 3K scale), medium (for 10K, 30K scale), and large (for 100K scale and above). These hyperparameter sets are identical, except for number of epochs, batch size, learning rate, and packing.
For all hyperparameter sets, we train on DeepSpeed v3 with a cosine learning rate, warmup of 0.I, weight decay of 0, and with the AdamW optimizer with betas 0,9 and 0.999. The specific differences between the hyperparameter sets can be found in Table below.
Hyperparameter Set
Dataset Size
LR
Batch Size
Epochs
Packing
Micro
< IK
le-5
32
13
No
Small
IK 3.16K
2e-5
96
7
No
Medium
3.16K 31.6K
4e-5
128
5
No
Large
31.6K
8e-5
512
5
Yes
Table 8: Settings for different hyperparameter sets with corresponding dataset sizes
As shown in Table 8 above, the micro, small, and medium sets are trained without example packing, while the large set is trained with packing. For the large hyperparameter set, we use packing in order to save compute time. On the other hand, for the micro, small, and medium sets, we do not use packing because we want have a larger number of training steps. We show in Appendix A.3 below that the presence/absence of packing does not significantly affect our performance.
Aside from packing. we adjust certain hyperparameters to optimize for training speed. We use DeepSpeed v3 without memory offloading, In addition, we use persistent dataloader workers, with mum_workers=4. We conducted small tests with these settings to ensure that they did not negatively affect performance.
A.3 PACKING
We investigate the effect of sequence packing on model performance across diverse reasoning and coding benchmarks. Sequence packing concatenates multiple training examples into single sequences to improve computational efficiency, but may affect learning dynamics. Table 9 compares models trained with and without packing on dataset with shorter sequences. Overall, we see that adding packing does not negatively affect performance. This observation is in contrast with some of the findings reported by Face (2025), where they found that packing negatively affected performance on their setup. We believe this drop in performance can likely be attributed to truncating long sequences into multiple parts, whereas Llama Factory's 's packing implementation, packing is implemented in greedy way, and only shorter sequences are packed together.
A.4 CHAA TEMPLATE
One other axis we explore is the chat template. More specifically, this refers to how reasoning models can be prompted to produce thinking tokens. This often comes in the form of special- ized templates. For instance, the RI template encloses the thinking tokens in <think> and </think>. In Appendix A.4, we compare this RI template with the SkyTI template which
22

---

Published as a conference paper at ICLR 2026
Configuration AIME24
AMC23
MATH500
JEE
GPQA-D
LCB
CodeFlo
AM100K
18.7
62.0
79.6
45.6
46.5
34.7
6.3
w/o packing
22.0
62.0
77.6
46.4
34.7
31.7
5.3
Table 9: Performance comparison between models trained with and without sequence packing. Packing shows mixed effects.
uses < beg n_ n__o__houghtttttttt >> and < the initial ThinTrinke on OpenThoughts- 114k. From Appendix A.4, we see that the results for the two different chat templates are roughly equivalent, suggesting that the chat template may not be as important as long as they exist (see Appendix A.5 below for ablations where we remove this completely). For simplicity, we stick with the R1 chat template of <think> and </think> in all of our pipeline experiments.
> and was originally used to train
Model
AIME24
AIME25
AMC23
MATH500
GPQA-D
LCB 05/23-05/24
OpenThinker-77B w/ RI template
31.3
28.0
72.0
84.4
42.9
41.8
32.7
22.0
72.5
83.8
43.9
33.6
Table 10: Performance comparison of OpenThinker-77B model with and without R1 tem- plate across different benchmarks. The table shows that using simpler <think> and </think> tokens instead of the more complex SkyTI tokens (< (bbeeinnooo gerin_off_thughtt ttthhhhhh >> and < <lend of tooughht >>) yields comparable or slightly improved performance on most bench-
marks.
A.S SYSTEM PROMPT
In this subsection, investigate the effect of removing the chat template completely and simply prompting the model directly. We report our results in Appendix A.5. Our evaluation reveals several key findings. First, enabling explicit reasoning consistently improves performance across mathematical and scientific reasoning tasks, with particularly dramatic improvements on AIME benchmarks (45.3% vs. 2.0% on AIME25). Second, even when reasoning explicitly disabled, many responses still begin with <think> tokens (1681/3127) indicating the model has learned to engage reasoning mechanisms by default. Third, the choice between system prompts shows task-dependent effects: while "reasoning on" helps mathematical tasks, removing system prompts entirely can sometimes yield better results (70.0% vs. 61.3% on AIME24), suggesting that explicit instruction may occasionally constrain the model's 's natural reasoning patterns.
Model/Configurationn AIME24
AIME25
AMC23
MATH500
GPQA-D
LCB 05/23-05/24
[lama-3. Tee---mmmtrronNaoo--e (Ours)
Reasoning On
61.3
45.3
94.0
89.0
55.9
68.4
Reasoning Off
4.0
2.0
38.0
48.8
34.7
67.1
No System Prompt
70.0
42.7
94.0
88.8
23.2
67.2
Llama-3. Nemooronn-aano-a (Official)
Reasoning On
47.1
95.4
54.1
_
Reasoning Off
0.0
36.6
39.4
Table 11: Performance comparison across reasoning benchmarks with reasoning enabled and not. Turning reasoning for existing models greatly improves performance. However, using system prompt at all also performs similarly well to Reasoning On.
B EVALUATION DETAILS
All model evaluations across benchmarks were performed using Evalchemy (Raoof et al., 2025), unified, multi-GPU evaluation framework. Evalchemy partitions each task into independent
23

---

Published as a conference paper at ICLR 2026
shards, runs them in parallel (via data parallel sharding), and streams per-shard metries back to central coordinator for real-time aggregation. This architecture ensured consistent generation settings, reproducible logging of model checkpoints and sampling parameters, and enabled to compute average model accuracy and standard error over repeated runs. We used Evalchemy to cache full model completions for long chain-of-thouge tasks (e.g., AIME, LiveCodeBench, GPQA Diamond), which reduced redundant inference costs and enabled easy analysis of failure cases.
For each benchmark, we report:
AIME24, AIME25, AMC23, and HMMT: mean accuracy and SEM over 10 iterations
LiveCodeBench: mean accuracy and SEM over 6 iterations
CodeForces, CodeBlo, GPQA Diamond, JEEBench, and HLE: mean accuracy and SEM over iterations
MATH500: single pass evaluation on the full 500 sample set
All runs used unified generation configurations: temperature 0.7, top_p= 1.0, and sax__ne_tokeen 32,768.
Reasoning models with long CoT often generate multi-step explanations before providing a final answer. The response typically begins with a <think> token, includes intermediate reasoning steps, and ends with </think> token to indicate the end of the reasoning process. The final answer follows this block. For code generation questions, the final answer is usually marked within a fenced code block with a language tag.
We provide brief descriptions for each of our benchmarks.
1. AIME24: mathematics competition for high-school students held in 2024. It involves 30 questions of different levels of difficulty. Answers are a single integer from 0 to 999.
2. AIME25: mathematics competition for high-school students held in 2025. It involves 30 questions of different levels of difficulty. Answers are single integer from 0 to 999.
3. AMC23: mathematics competition for high-school students held in 2023. It consists of 40 questions with different difficulty levels. The answers are numerical.
4. MATH500: consists of 500 diverse problems in probability, algebra, trigonometry, and geometry.
5. CodeForces:
consists of 453 real-world programming problems sourced from the Code- Forces platform. The benchmark measures unit test-based execution accuracy with human-comparable Elo rating.
6. CodeElo:
consists 391 real-world programming problems curated from a variety of contests. The benchmark measures unit test-based execution accuracy with difficulty- calibrated Elo rating.
7. LiveCodeBench: a benchmark real-world programming tasks that evaluate model's ability to generate, execute, verify, and iteratively repair solutions using unit-test feedback. LiveCodeBench 05/23-05/24 subset has 511 problems released between May 2023 and May 2024, whereas the 06/24-01/25 subset has 369 problems released between May 2024 and Jan. 2025.
8. GPQA Diamond: a set of 198 challenging questions from the Graduate-Less Google-Proof Q&A Benchmark (GPQA). Questions are in multiple-choice format.
9. JEEBench: contains 515 questions spanning Physics, Chemistry and Mathematics subjects collected from the Joint Entrance Examination (JEE): Advanced held from 2016 2023. Questions are in multiple-choice and numerical formats.
10. HMMT: 30 questions from the HMMT high school mathematics competition held in
February 2025. Questions in Combinatorics, Number Theory, Algebra, and Geometry.
11. HLE: subset of 512 multiple-choice, text-only questions from the Humanity's 's Last Exam (HLE) benchmark.
24

---

Published as a conference paper at ICLR 2026
Benchmark
Domain / Description
Number of Questions
Code Generation
CodeElo (Quan al., 2025)
Code generation with human comparable Elo ratings.
391
CodeForces (Penedo et al., 2025aa
Benchmurking competition-leww code generation.
453
LiveCodeBench 05/23-05/24 (Jain al., 2024)
Holistic code benchmark with iterative repair.
511
LiveCodeBench 16/24001/25 (lain et al., 2024)
Holistic code benchmark with iterative repair.
369
Mathematical Problem Solving
AIME (MAA, 2024)
2024 AIME math-reusoning dataset. 2025 AIME math reasoning dataset. AMC muth-reusoning dataset. High school mathematics competition. 500-problen split from "l.et's Verify Step by Step."
30
AIME: 25 (MAA, 2025)
30
AMC 23 (MAA, 2023)
HMMT (Balunović et al., 2025)
30
MATH500 (Hendrycks et al., 2021h)
500
Science
GPQA Diamond (Rein et ol., 2024)
Graduate-Ievel, Google-preof Q&A benchmark.
198
JEEBench (Arora et al., 2023)
Pre engineering IIT JEF Advanced exam questions.
515
General Tasks
HLE (Phan et al.. 2025)
Subject-matter questions.
512
Table 12: We evaluate on 12 tasks across multiple data domains. We validate experiments on 8 of these tasks, and keep the remaining (AIME 2025, LiveCodeBench 06/24-01/25, HMMT, HIEE) as held-out sets.
C DECONTAMINATION
Contamination with the evaluation datasets is an important issue, since poses the danger of misleading results over the actual usefulness of the training set. It is expected that training data that contains evaluation questions in some form will lead to improved performance on those same questions. Such an effect could potentially affect the conclusions of our experiments. To avoid this issue, we perform decontamination against our evaluation sets, via two separate criteria.
The first method used is Indel similarity of each training sample and each evaluation sample. This similarity refers to the number of characters that need be inserted or deleted from one sample to match the other, and is calculated relative to the sample length. More precisely, consider the Normalized Indel similarity score between a pair of strings, as computed via the Longest Common Subsequence (LCS) metric: indelsim 100 x max [81], (82))
LCS jongth (51, 82)
We consider a similarity of 75% between our two strings to indicate contamination with respect to this metric.
Our second method is an N-gram-based similarity metric. In this setting, we first tokenize both the training and the evaluation sample using the same tokenizer as the Swen2-7B-Instrrue model. We then examine the sets of N-grams in each of the samples, for N = 13. If we find that the two samples share N- N-gram with each other, then we consider the training sample be contaminated.
For our pipeline, we consider a training sample contaminated if it is marked as contaminated by either of our methods, and we discard it. The thresholds for our methods are chosen empirically, in order to minimize both false negatives (samples that are contaminated but are not detected) and false positives (samples that are marked as contaminated but are in fact unrelated to the evaluation samples).
We systematically tune our decontamination schema through rigorous experimentation. Our testbed manually contaminated dataset; an ideal decontamination scheme can accurately filter out contaminated questions from the normal questions.
Contaminated Dataset Construction noncontaminated questions. We construct this dataset from several sources.
Our experiments require a dataset contaminated and
1. We take test sets (MATH500, GPQA Diamond, LiveCodeBench) and sample exact questions from each test set.
2. We sample questions from test sets and apply three types of alteration. Our first alteration is embedding the question in longer context, such as "Please help me solve this problem: ".
25

---

Published as a conference paper at ICLR 2026
Decontamination Results
3000
True Uncontaminated
2500
2958
42
2000
1500
Contaminated
1000
3080
500
Uncontaminated
Contaminated
Predicted
Figure 4: Our decontamination algorithm accurately identifies contaminated samples. Our decontamination algorithm has 99.6% negative accuracy rate. The algorithm also throws minimal amounts of noncontaminated samples.
The second alteration is replacing several words with synonyms, numerical expressions with equivalent expressions, and variable names. Our final alteration changing the formatting the question by altering paragraph breaks, sentence order, and punctuation.
3. We add uncontaminated questions by creating completely original questions manually.
Overall, our dataset has 3092 contaminated samples and 3000 uncontaminated samples. We tuned our decontamination algorithm to produce nearly 0 false negatives (marking contaminated questions decontaminated) while having many false positives. The results of final decontamination schema are in Figure 4. Our final decontamination algorithm only misses 12 questions out of 3092 manually contaminated items, representing 99.6% true negative rate. Decreasing the threshold for fuzzy string matching or the in n- g-gaam count significantly raises the false positive rates, which could potentially affect downstream performance. The decontamination schema only throws out 1.4% of noncontaminated samples.
26

---

Published as a conference paper at ICLR 2026
D ADDITIONAL SCALING EXPERIMENTS
Math Average
Code Average
Science Average
60
70
Accuracy (%)
60
40
50
30
40
1K
10K
1K
10K Dataset Size
1K
10K Dataset Size
Dataset Size
Question Source
Question Mix
Question Filter
Multiple Answers
Teacher Model
Qwen-225-77-Instrucc
Figure 5: Scaling the top strategies from each pipeline step. Across dataset scales, the datasets created by subsequent stages in the pipeline shift the scaling curve upwards. The largest gains come from the selection of question sources, question filtering strategies, and teacher model selection. The average performance the final "Teacher Model" curve not being the top science performer is consequence of our design choice, where we select winning strategies based on average performance across domains.
math and code has clearer scaling trend than the performance science
27

---

Published as a conference paper at ICLR 2026
AIME24
AMC23
MATH500
100
100
100
90
90
90
80
80
80
Accuracy (%)
70
50
50
40
40
40
30
30
30
20
20
20
10
10
1K
10K
100K
1M
1K
10K
100K
1M
1K
10K
100K
1M
HMMT 02/25
AIME25
LCB 05/23-05/24
100
100
100
90
90
90
80
80
80
Accuracy (%)
70
70
40
40
40
30
30
30
20
20
20
10
10
1K
100K
10K
100K
1M
1K
10K
100K
1M
1K
10K
100K
1M
LCB 06/24-01/25
CodeÉlo
CodeForces
100
100
100
90
90
90
80
80
80
Accuracy (%)
70-
70
70
40
40
30
30
30
20
20
20
10
10
10
1K
10K
100K
1M
1K
10K
100K
1M
1K
10K
100K
1M
GPQAD
JEEBench
HLE MCQ
100
100
90
90
90
80
80
80
Accuracy (%)
70
70
70
60
60
30
30
20
20
20
10
10
10
1K
10K
100K
1M
1K
10K
100K
1M
1K
10K
100K
1M
Dataset Size
Dataset Size
Dataset Size
OpenThoughts3
AM
LIMO
Nematron Nano
s1.1
Qwen-2.5- 2wvn-2.5-78-Instrus
Figure 6: Downstream model performance after finetuning Qwen-2.5-7B-Instruud on increas- ingly larger subsets from OpennThoughts--12M Across wide variety of math (AIME, AMC, MATH, HMMT), code (LiveCodeBench, CodeElo, CodeForces), and science (JEEBench, HLE) benchmarks, OpenThoughts3- 1.2M outperforms existing reasoning datasets. HMMT, AIME 2025, LiveCodeBench 06/24-01/25, and HLE are "held out", which means that did not use them to evaluate any intermediate models during our experiments to inform our data recipe.
Studying scaling trends allows us to see if data recipe is consistent across scales and helps determine whether further scaling promising. Figure 6 shows that the OpenThoughts3 recipe dominates other reasoning dataset strategies across scales and benchmarks.
Performance on many of the studied benchmarks continues to improve up to the 1M scale. However, some benchmarks are saturating (AMC23, MATH500) at the largest scale, and others do not respond
28

---

Published as a conference paper at ICLR 2026
Math Average
Code Average
Science Average
80
40
Accuracy (%)
60-
30
50-
40
20
40
10
30
20
1K
10K
100K
1M
1K
10K
100K
1M
1K
10K
100K
IM
Dataset Size
Dataset Size
Dataset Size
3
No Pipeline
Nemotron Nano
Twen-2.5-7B-Instruue
Math subset
Code subset
Science subset
The OpenThoughts3 data recipe within each domain shows strong scaling baselines. Math performance averaged over AIME24, AMC32, and MATH500. Code performance averaged over LCB 05/23-05/24, CodeElo, and CodeForces. Science is averaged over GPQA Diamond and JEEBench. The largest scale for the OpenThoughts3 math and science subsets are 250K and 100K, respectively.
Figure 7:
to scale (HLE). Scaling the reasoning datasets to larger sizes beyond 1M is an exciting future direction.
The scaling curves do not always exhibit smooth increases in performance, exhibiting dips and jumps. There is variance our experimental procedure in training and evaluation, so even with a fixed dataset, the downstream performance will fluctuate. However, found in our experimentation that re-training and re-evaluating models on a fixed dataset did not fully explain these dips and jumps.
To further study the scaling trends, first isolate data from each domain and measure the average performance of the in-domain evaluation benchmarks. This matches the same setting as our pipeline experiments in Section 3, in which data recipes are sweeped for each domain (math, code, and science).
Figure shows the individual domain recipes continue scaling nicely beyond the largest dataset size used in the pipeline experiments, 31.6K, for another order of magnitude. We chose these sizes to study scaling half order of magnitude resolution.
We include "No Pipeline" as naive baseline to demonstrate the full gains from the data recipe determined by our extensive experimentation Section 3. "No Pipeline" constructed by taking the union of 31.6K samples from all candidate question sources from the first stage in the pipeline 3.2. Therefore, "No Pipeline" does not include the selection of questions only from high-quality sources, does not employ the filtering questions, uses DeepSeek-RI instead of QwQ-32B does not include multiple answer samples per question, and does not have any filtering based on answers. Figure 5 further breaks down the gains due to these choices individually step by step.
teacher model,
D.1 BASE MODEL
We train Thoughts using the Llama-3. 1-0--3..-8B--nntttee models. This experiment shows that the scaling gains that we observe are not just limited Qwen models, and our dataset is indeed scalable and generalizable. The results of this experiment are shown in Appendix D.I. Overall, we see that for some datasets, the Llama models see more significant performance gain as compared to the Qwen models. This is most prominent in some math datasets such as AMC23 (from 15.8 to 75.2) and MATH500 (from 43.2 to 83.8), though the Qwen models, which start from a stronger starting point, still perform better overall. In the future, we would like to expand this to further consider stronger models such as the recent Qwen series models.
29

---

Published as a conference paper at ICLR 2026
Base Model
AIME24
AIME25
AMC23
MATH500
GPQA-D
LCB 05/23-05/24
Qwenn225-7B-lnasruuct
54.3
41.0
86.8
89.0
51.0
43.7
(+39.3)
(+33.0)
(+33.3)
(+18.4)
(+27.3)
(+10.7)
(lama-3.--BB-nma
37.0
30.3
75.2
83.8
45.1
44.4
(+32.3)
(430.0)
(+59.4)
(+-40.6)
(+19.3)
(+31.3)
Table 13: Performance comparison between base models when fine-tuning on 100k samples from Open Thonghough The table shows the absolute performance scores achieved by fine-tuned models, with improvements over the respective hase models shown in parentheses. Both fine-tuned models demonstrate substantial improvements
all benchmarks when trained OpenThoughtss3 data. While Llama-3. 1--aa-3.--8B-Insae I-8B-Instruct experiences larger lifts on AMC23, MATH500, and LCB 05/23-05/24, using 2.0---2-------teree results in the overall best performance.
Models
Average
Code Avg
Math Avg
Science Avg
OpenThinker-322B
64.50.3
45,8
83.7 1.0
63.7p.2
OpenThinker- ------nverB-ie--e
62.1o.3
43.80.4
81.40.9
60.5g.3
OpenThhinker-7B-Unneee --Unverified
45.0g.4
24.20.3
64.60.9
46.90.6
OpenThiiner-7B
41.9 0.6
21.80.5
62.0
41.9p.6
Table 14: Impact of Verification on original OpenTTheniee Thinker models. We see verification hurts at the 7B model scale but helps the 32B model scale.
E ADDITIONAL DATA RECIPE EXPERIMENTS
Due to space constraints, we could not present all of our data curation ablations the main text. This section discusses several interesting experiments that provide further insights into tools for improving reasoning models.
E.1
VERIFICATION
Verification played a large role in OpenThoughts-11444 114K. examine how verification impacted 114K
OpenThoughts- 14K in the following sections.
E.
VERIFICATION IMPACT ON THE ORIGINAL OPENTHOUGHTS
Verification played an important part in OpenThoughts-11 14K and OpenThoughts2. However, OpenThoughts3 does not rely on any form of verification. A natural question is how important empirically was verification for the original Open'Thought experiments. Table 14 demonstrates the findings of this study. We trained a 7B and 32B model on the unverified version of OpenThoughts- 114K and evaluated the difference between the unverified models and the original models. Our results show that verification may hurt performance at the 7B level but help at the 32B level.
1.1. REMOVING PROOF-BASED QUESTIONS
We push further on the impact of verification on OpenThoughts- 14K. Some math questions in OpenThoughts-11 14K are proof-based. This characteristic can make our numerical verification less accurate. simple question is whether removing proof-based questions improves downstream performance due to more accurate verification. Table 15 contains the results of this ablation. Re- moving proofs degrades performance on relevant benchmarks despite being unverifiable with our methodology.
1.3 EXTRACTION-RASED MATH VERIFICATION
Another verification strategy we explore is filtering question-answer pairs based answer correct- ness. For math examples with known ground truth answer, we compare the model-generated answer
30

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Datasets
Average
Code Avg
Math Avg
Science Avg
Open'T OpenThinker-777 7B
41.9 0.6
21.8
62.0
41.90.6
OpenThinker-777 7B w/o proofs
39.41.3
15.22.9
60.4
44.2
Table 15: Comparison of Thenghosshss with and without proof-based questions. Throwing out proof-based questions harms performance overall by 5.6 points on average.
Extraction Method
Dataset Size
Extraction Method
AIME25
MATH500
(Training)
(Evaluation)
LLM judge
114K
Hendrycks-Math (default)
31.3
84.4
LLM judge
114K
HF Math-Verify
44.0
89.0
Math- Verify
83K
Hendrycks (default)
23.0
55.0
Math- Math-Verify Verify
83K
HF Math-Verify Verify
22.7
82.2
Table 16: Comparison math answer verification strategies. We filter the OpenThoughts-111 114K dataset using either LLM-based Math-Verify-bbseer answer correctness. The resulting models evaluated using both Hendrycks and Math-Verin answer extraction tools.
SFT Datasets
Benchmarks
Datasets
LiveCodeBench
CodeElo
CodeForces
Verified via Unit Tests
36.0
9.4
10.4
Unfiltered (Random Sample)
38.5
10.7
13.54
Table 17: Effect of using I.MM-Geerraedd unit tests for code data verification. Downstream performance of models trained on 16,000 code examples: one set filtered include only samples verified by LLM-generated unit tests, and the other unfiltered. No improvement observed from cerification-based filtering.
to the reference and discard samples with incorrect responses. However, extracting and evaluating the model's final answer, which is often embedded in complex mathematical expressions, non-trivial.
To address this, we experiment with two answer extraction methods: (i) using the Math- Marh-Verify toolkit from Hugging Face, and (ii) using an LLM-based extractor (OpenThinkerr7BB)
We apply both methods to filter the OpenThoughts- 14K dataset and train downstream models. For evaluation, we again compare Math-Verify against the default answer extractor from Hendrycks et al. (2021b). Table 16 summarizes the results across two benchmarks AAMME25 and MATH500- -under different combinations of data generation and evaluation verifiers.
E.1.4 LLM-GENERATED UNIT TEST VERIFICATION
We investigate the effect of filtering code examples by LLM-generated unit tests. From initial pool of 45,000 question-answer pairs, we use GPT4o-mini to (1) detect which answers contain Python code and (2) generate standalone, executable unit test for each Python instance. We then apply our "verification" filter only to those Python examples, while non-Python examples remain untouched. Next, we fine-tune Qwen2.5-7B -nnstuuttt on two separate subsets of 16,000 samples each: one filtered to include only examples whose generated tests pass, and one drawn at random without filtering. The results shown Table 17 suggest that an --geeereeteedd unit test verification does not improve downstream code-generation accuracy.
E.2 TEACHER MODEL.
In this section, we study Claude 3.7 (with thinking mode) as an annotator. First, we show that Claude-thinking traces contribute to better performance in code, maths, and general question answer-
31

---

Published as a conference paper at ICLR 2026
ing. Longer thinking traces lead better results in all three categories of benchmarks. Then demonstrate that using Claude 3.7 to re- re-annotate the SIK Muennighoff et al. (2025) dataset (math) as well as our science or code data actually leads to worse performance than using RI.
with Thinking
With Thanking
77.5
With Thinkin
50
Without Thrnking
Weteet Thinking
75.0
Withou: Thinking
Accuracy (%)
68-
40-
72.5
70.0
30-
64
67.5-
65.0
60
62.5
58
60.0
1,024
3,012
7,368
15,360
31.764
62,976
2,000
4.008
0,102
12.388
28,672
4,008
12.208
28,472
59
Number of Thinking lökens
Number of Thinking Tokens
Number of Thinking Tökens
(a) AIME 24
(b) LiveCodeBench
(c) GPQA Diamond
Figure 8: Claude 3.7 accuracy improves consistently with larger thinking-token budgets across three benchmarks. Each panel plots mean accuracy (markers) and +1 standard error (error bars) over multiple independent runs (5 for AIME 24, 3 for LCB and GPQA Diamond). The horizontal axes are logarithmic in the number of thinking tokens; the answer budget is 024 tokens for AIME 24 and 4 199 tokens for LCB and GPQA Diamond and is not counted in the thinking tokens budget. AIME 24; accuracy rises from a no-thinking baseline of 18.0% (red diamond) to 51.3% when the model is allowed 62 976 thinking tokens. LCB: performance climbs steadily from 60.5% to 70.4% at 28 672 thinking tokens. GPQA Diamond: accuracy increases from 60.1% without thinking to 76% at 12 288 tokens, after which the curve plateaus, illustrating diminishing returns beyond this budget.
Claude 3.7 with thinking. The API interface to Claude 3.7 allows the user to set a budget for the number of thinking tokens. This permits us to study the evolution of the benchmark performance as test-time compute is increased. Figure shows that for all types of tasks tested (mathematical reasoning, coding, and question answering), increasing test time is beneficial. This is especially true for mathematical reasoning and coding; GPQA Diamond performance saturates earlier.
Claude vs R1 as an annotator for code. To assess Claude 3.7 as an annotator for code, we consider the Open Tpenghougt 14K dataset and re-annotated 10K random coding problems from Open' Thought-- 1 114K with Claude and verified them, which yields 5.8K verified examples. We mixed this with the OpenThoughts math and science parts, so that the proportions of the code, math, and science parts of the resulting dataset are the same as for OpenThoughts 1. This swaps the code annotator from RI to Claude. Table 18 shows that Claude performs slightly worse code annotator in this experiment.
Code Annotators
AIME24
GPQA
MATH500
LiveCodeBench
R1
0.233
0.399
0.816
0.341
Claude
0.266
0.404
0.806
0.323
Table 18: Scores for switching the code annotator from R1 to Claude.
Claude vs RI as an annotator for science To assess Claude 3.7 as an annotator for science, we con- sider the OpenThoughtt I with Claude, analogously as above for code, we swapped the science annotator from RI to Claude. The results in Table 19 show that Claude performs slightly worse a science annotator in this context.
dataset and re-annotated and verified the science part from OpenThoughts
Claude vs R1 vs Gemini as annotators for S1 To further assess annotators for math, we consider the SIK dataset (Muennighoff et al., 2025) and re-annotated its answers and reasoning traces with Claude and RI. Table 20 shows much better performance with R1 annotations. However, Claude annotations did not show as strong improvements.
32

---

Published as a conference paper at ICLR 2026
Science Annotators
AIME24
AIME25
AMC23
MATH500
GPQA
LiveCodeBench
Claude 3.7
0.3733
0.2733
0.740
0.840
0.2138
0.4207
R1
0.3867
0.2933
0.765
0.872
0.2121
0.4586
Table 19: Scores for switching the science annotator from R1 to Claude.
Models
AIME24
AIME25
MATH500
GPQA
Gemini simplescaling/sIK
56.7
26.7
93.0
59.6
R1 simplescaling/s K-1.1
56.7
60.0
95.4
63.6
Claude-3-7 simplesaling'sSK-luddde 3-n-snnctt
40.0
87.0
51.5
Table 20: Scores for switching the annotator from Gemini to R1 or Claude.
E.3 COMPRESSING REASONING TRACES
So far, we performed supervised fine-tuning on long reasoning traces (up to 16K tokens) before predicting the final answer. Recent work (Chen et al., 2024; NovaSky-Tean 2025a) highlights the significant inference cost associated with long reasoning traces and the tendency of reasoning models to overthink. In this section, study how reducing reasoning traces during training affects downstream performance. We employ two methods: (a) removing self-reflection components from the reasoning traces, and (b) filtering out instances where the reasoning trace length is above specified threshold.
To examine the first approach, begin with random subset of 12K instances from the OpenThoughts3 dataset, ensuring that each instance contains a complete thought (i.e., <\th i nk> is present in the reasoning trace). Typically, reasoning traces long due the model's self-reflective behavior, where it (re- (re-)analyzes prior solutions and proposes alternative approaches to the problem. To investigate the role of self-reflection, we remove keywords such as "wait", "but wait", and "but the question" from the reasoning traces, following the approach of Deng et al. (2025). This reduces the average reasoning trace length from 11.6K 0.3K tokens. We present the results Table 21. Notably, we observe that removing self-reflection leads to average relative performance drop of 49.1% % across diverse downstream benchmarks. These findings suggest that self-reflection and long-form reasoning structures essential for enhancing the reasoning capabilities of OpenThought models.
Our second approach to reducing the reasoning trace is to filter instances whose lengths exceed given threshold (e.g., 2048, 4096, or 8192 tokens). This method reduces both the length of the reasoning traces and the overall dataset size, while preserving the self-reflection capabilities within the retained traces. We present the results in Table 21. We observe that the downstream performance of models trained on the filtered datasets drops significantly compared to the default dataset. Specifically, the filtered-2044 setting results in relative performance degradation of 33.3% across downstream benchmarks. Furthermore, higher filtering thresholds show improved model performance. This indicates that the presence of long reasoning structures in the dataset beneficial, and that self- reflection alone is not sufficient for achieving strong reasoning performance. Nevertheless, the ability reduce overthinking post-hoc remains an active and highly relevant area of research (Sui al., 2025).
E.4 POORLY PERFORMING SFT DATASETS FROM WEAK QUESTIONS
When benchmarking existing SFT reasoning datasets, we finetune models on the full sample sets from multiple available sources. Evaluating downstream performance our fixed benchmark suite, we observe a wide range of dataset quality.
To investigate further, finetune on small subset of 31,600 randomly sampled question-answer pairs from each original dataset and re-annotate those questions with DeepSeek-R1, using the same procedure described in the sourcing stage of our pipeline (Section 3.2). Once again, we observe significant variance in downstream performance, which appears to stem primarily from differences in the quality and nature of the question sources.
33

---

Published as a conference paper at ICLR 2026
Setup
Avg. length
AIME24
MATH150
GPQA
LCB 05/23-05/24
Average
Baseline
11593
34.0
84.0
45.6
40.7
51.4
Self-Reflection
328
61.8
31.5
19.2
26.3 (-49.1%)
Filter 2048
1343
16.7
70.2
32.3
26.9
34.2 (-33.3%)
Filter 4096
2305
18.3
74.6
42.6
31.7
39.9 (-22.3%%
Filter 8192
4775
22.0
79.6
44.4
30.3
42.3 (-17.6%)
Table 21: Evaluating the role of compressing reasoning traces on downstream performance. The first row shows the performance of the model trained the default OpenThoughts3 dataset (12K instances), where the average reasoning trace length is 11.5K tokens. The second row reports performance when self-reflection capabilities are removed, reducing the average trace length to 0.3K tokens. The subsequent rows present results for filtering strategy that removes instances with reasoning trace lengths exceeding a certain threshold (2048, 4096, or 8192 tokens). Overall, results underscore the importance of both self-reflection and long reasoning structures for achieving strong performance across diverse evaluation benchmarks.
SYNTHETIC-I-SFT.DE I-SFT--a--- Data
KodCode VI-SFT-RI -I-SS-I-V
code stack exchange
code understanding
Full Controlted
code codegolf
code kodcode
real, world swe
Benchmark
Train Size
894K
268K SFT
31.6K
31.6K
31.6K
31.6K
31.6K
Method
SFT
SFT
SFT
SFT
SFT
SFT
Average
42.6
26.9
37.3
36.2
27.3
26.9
29.8
AIME24
40.7
15.0
17.7
20.3
14.0
15.7
12.3
Math
AMC223
78.5
50.5
58.0
56.3
52.2
50.7
52.2
MATH500
87.6
71.8
77.0
72.8
74.0
70.0
72.4
MMLUPro
31.0
28.0
30.6
28.2
25.2
25.6
24.4
Code
CodeElo
17.2
7.4
14.4
12.5
3.8
2.6
7.8
LCB 05/23-05/24
48.3
39.1
44.4
43.9
9.3
0.6
15.9
CodeForces
21.8
10.5
17.2
15.3
4.4
3.2
8.4
Sci
GPQA-D JEEBench
45.1
31.6
42.6
41.6
33.7
42.4
38.6
57.2
31.0
38.8
39.3
29.5
31.9
36.3
Table 22: Performance comparison: Full-scale datasets vs. controlled ablation study. SENTHETIC-I-SFT-DEE (894K) achieves the highest performance with average of 42.6. significantly outperforming all controlled datasets. Among the size-controlled 31.6K datasets, oade_codegolf serves the best baseline (average score of 37.3), with code kodcode competitive performance (average score of 36.2). The vertical line separates full-scale mixed datasets from sample-size controlled ablation experiments.
code_kodcode achieving
E.5 STACKING GAINS FROM OUT OF DOMAIN TRANSFER
During our experimentation on the dataset pipeline in Section 3, it was clear that reasoning ability transferred across domains. For example, scores science evaluations would increase when model was finetuned on only math reasoning data. We often observed such significant out-of- domain performance gains between candidate pipeline choices that the model with the highest in-domain average performance would not be the same as the model with the highest overall average performance.
34

---

Published as a conference paper at ICLR 2026
We studied whether these gains due cross-domain transfer persisted when all the domains mixed together. To do this, we selected datasets from the answer filtering 3.6 portion of the pipeline experiments the strongest performing science reasoning dataset in-domain science evaluations (Iongest answer filtering), the strongest performing code reasoning dataset on out-of-domain science evaluations (longest answer filtering), and the weakest performing code reasoning dataset out-of- domain science evaluations (shortest answer filtering).
Then, compared the downstream science performance between the mixes created by combining the science dataset with the two different code datasets. The large difference in the GPQA Diamond scores of the two code datasets did not show up after mixing with the strong science dataset. In other words, the gains from the out-of- transfer seen code datasets disappear when the in-domain science data is mixed in.
Finetuning Dataset
JEE
GPQA-D
LCB 05/23-05/24
CodeElo
CodeForces
Science Code (high GPQA)
48.7
48.8
21.8
6.3
7.9
46.6
47.3
44.6
15.9
18.9
Code (low GPQA)
44.3
36.7
45.8
15.1
19.7
Science + Code (high GPQA)
50.4
52.7
20.2
15.3
17.6
Science + Code (low GPQA)
51.1
52.7
20.7
14.6
19.6
Table 23: Out-of-domain (code science transfer) gains do not persist when mixed with the in-domain (science) dataset. The individual datasets (above the midline) contain only 31K samples from that domain and the mixes (below the midline) contain 62K samples of both code and science. When combining code dataset with strong performance science evaluations with a strong science dataset, there is no difference in the downstream model GPQA-D scores over mixing with code dataset with weak performance science evaluations.
35

---

Published as a conference paper at ICLR 2026
GPT-4o-mini Annonated Difficulty on Math in Sespoke-Stratos
Accuracy (%) with DeepSeek R1
75%
70%
65%
60%
GPT-4o-mini Annonated Difficulty
Figure 9: GPT-4o-mini can reliably determine the relative difficulty of questions. DeepSeek R. performs substantially worse on the hardest questions (difficulty ten).
F MODEL REASONING PERFORMANCE ANALYSIS
F.1 PERFORMANCE ON MATH DIFFICULTY
While developing OhenThoughts, we explored using language models to filter math data questions by difficulty. We applied the difficulty labeling prompt from Sky-T1 NovaSky-Team (2025b) the math questions from Bespoke- Despoke-Stratos with GPT-4o-mini as the annotator. This prompt uses example problems judge questions on a 1-10 scale, with one corresponding to beginners questions and corresponding to the hardest IMO problems.
We found that GPT-4o-mini could reliably predict which questions DeepSeek-RI would correctly answer. At the lowest level of difficulty (one) R1 scored over 75%, while at the highest (ten) R1 scored less than 57% (Figure 9). This built confidence that LLM-annotated difficulty labeling could be used to filter the hardest (and therefore potentially most useful for training) questions.
F.2 PERFORMANCE SCALING CODE DIFFICULTY
While testing the scaling of Open Thoughts3 (shown in figure of the main paper), we noticed slight drop in performance on all code benchmarks 100K scale. To study this phenomenon in more detail, we studied LiveCodeBench 05/23-05/24 and represented the contributions of each difficulty level to the average accuracy. We expected to see a saturation of the easy category and very low performance levels in the hard category. Surprisingly, in figure 10, we observe that the models' accuracies are actually increasing nicely with scale for both the medium and hard tasks, and the slight drop is entirely happening in the easy category.
F.3 SAMPLING BY LONGEST, SHORTEST, MAJORITY
When sampling multiple responses, we have multiple aggregation strategies to predict the final answer number MCQ choice: (1) "shortest": using the answer of the shortest response the final answer: (2) "Iongest": using the answer of the longest response as the final answer; (3) "majority": using the majority prediction as the final answer.
Our experiments reveal that the shortest response strategy consistently outperforms the longest response strategy across models and datasets. While majority voting often achieves the best overall performance, the shortest strategy provides an effective single-response selection method that typically outperforms both vanilla sampling and longest response selection.
On the AIME24 mathematical reasoning dataset (Table 24), majority voting generally achieves the best performance, but the shortest response strategy still outperforms the longest strategy for most models. As shown in Table 1, while majority voting achieves the highest scores (e.g., 76.67 % for
36

---

Published as a conference paper at ICLR 2026
OpenThoughts3 Performance by Dataset Size
0.6
0.554
hard (24.1%)
medium (40.3%)
Accuracy (weighted contribution)
0.480
0.5
easy (35.6%)
0.461
0.435
Global Average
0.400
0.4
0.326
0.3
0.224
0.197
0.2
0.1
0.0
0.3k
1k
10k
30k
100k
300k
1000k
Dataset Size (thousands, log scale)
Figure 10: OpenTToughts, performance scaling LiveCodeBench 05/23-05/24 the dataset size increased. The contribution of the performance on each category of problem represented.
DeepSeek-RIi -i---i-iilll--ee----- the shortest strategy (66.67%) significantly outperforms the longest strategy (36.67%).
Model
Shortest
Longest
Majority
Vanilla
DeepSeek- -i-eeekk----D -Dist-itill--eee Distill-Qwen-75
66.67
36.67
76.67
55.00
OpenThinker-78
43.33
13.33
43.33
30.33
simplescaling-s1. NovaSky-Sky-T-332B
40.00
30.00
46.67
35.00
30.00
26.67
43.33
30.67
Table 24: Performance comparison of sampling strategies on A1ME24 dataset. "Vanilla" refers to the average pass rate across all sampling.
The trend is even more pronounced on the GPQA Diamond scientific reasoning dataset. Table 25 presents results from our most comprehensive experiments with higher run counts. The shortest strategy consistently outperforms the longest strategy across most models, with improvements ranging from 7-11 percentage points.
Model
Runs
Shortest
Longest
Majority
Vanilla
DeepSeek-RR -Die--
43
51.01
43.94
29.80
48.81
OpenThinker-7B
44.44
39.39
24.75
42.28
1aaplescaling-s13332 NovaSky-Sky-TI.
44
50.00
48.99
30.30
52.79
40.91
42.42
27.78
49.99
Table 25: Performance comparison on GPQA Diamond dataset (high-run experiments)
Response Length Analysis An interesting pattern emerges when examining the relationship between response length and correctness. Table 26 shows the average token lengths for correct versus incorrect responses in vanilla sampling. Across most models, incorrect responses tend to be significantly longer than correct ones, suggesting that verbose reasoning may actually indicate uncertainty eπror-
error-prone prone reasoning paths.
37

---

Published as a conference paper at ICLR 2026
Dataset
Model
Correct
Incorrect
Difference
DeepSeck- R R--Distil--Qwen
7,817
18,198
+10,381
OpenTThinkrr-77
8,966
19,517
+10,551
AIME24
simplescaling-s1-322
5,048
7,481
+2,433
NovaSky-Sky-T1-32B
1,802
3,327
+1,525
SeesSeek-RI-iisilll-O-e
5,034
6,622
+1,588
OpenThinker:7B simplescaling-s1-222 NovaSky-Sky-T1-32B
7,471
8,772
+1,301
GPQA Diamond
3,617
3,790
+173
911
896
-15
Table 26: Average response length comparison: Correct vs. Incorrect responses
The superiority of the shortest response strategy suggests that concise reasoning often captures the most direct and correct solution path. Longer responses may indicate the model is uncertain, exploring multiple approaches, or getting lost unnecessary complexity. This finding has important implications for practical deployment, selecting the shortest response is computationally efficient you stop generating all samples and often yields better results than more complex aggregation methods.
However, it's important to note that the shortest strategy is not universally optimal. For some models, like SryaSk--Sky-T1-33- optimal sampling strategy may be model- -eneedddeeendnt. Additionally, majority voting can sometimes achieve the highest performance when computational resources allow for multiple response generation and aggregation.
certain datasets, other strategies may perform better, indicating that the
G SURPASSING THE TEACHER WITH DISTILLATION FOR LEGAL REASONING
In this work, we focus primarily on reasoning for math, science, and coding. However, reasoning can also be beneficial for other domains, for example, for legal reasoning. For such domains, reasoning models out of the box can be significantly improved through finetuning.
We consider classification task from the Lawma benchmark (Dominguez-Olmedo al., 2025), specifically, we consider the task of classifying the ideological direction of an opinion from the Supreme Court as conservative, liberal, or unspecificable. This is a challenging task, since general models perform relatively poorly on this task, for example, GPT4 only performs slightly above 50%. Nominguez-Olmedd
[Dominguez-Olmeet as well as 1.52K labeled test examples.
al., 2025) provide 5,44K labeled training examples
We consider three data generation strategies: i/ We finetune Qwen2.5-77 on dataset obtained by taking 2K examples, each annotated 5x independently by R1, and verified with a majority vote. Only the traces where outcome agrees with the majority are kept (8.3K many the 10K), the others are filtered out. This strategy does not use the provided expert labels. ii/ We finetune Qwen2.5-7B on a dataset obtained by taking 2K examples, each annotated 5x independently by R1, and verified with dataset
the expert labels, resulting in 7.36K verified examples. iii/ We finetune Qwen2.5-7B obtained by taking all 5.4K examples each R -annotaaee once and verified with the expert-provinee label, resulting in 4.02K verified examples.
Table 27 contains the accuracies in predicting the ideological direction of an opinion from the Supreme Court as conservative, liberal, or unspecificable for the different data generation strategies. It can be seen that all finetuned Qwen2.5-7B models outperform the much larger annotator (i.e., R1)
by significant margin.
Those results demonstrate that finetuning with a strong teacher model, like R1, followed by verification can yield strong models for specialized tasks, such as this legal reasoning task. Interestingly, finetuning without the expert labels based on consensus/majority verification performs almost as good as using the expert labels for verification.
38

---

Published as a conference paper at ICLR 2026
Model Setup
Accuracy
Owen2.5-7B (no finetuning)
0.271
Qwen2.5-7B finetuned on 2K examples (5x R1-annotated, majority verified)
0.819
Qwen2.5-7B finetuned on 5.4K examples (annotated, verified)
0.820
Qwen2.5-7B finetuned on 2K examples (5x RI-annotated, verified)
0.828
RI (annotator) accuracy
0.739
Table 27: Comparison of model performance under different finetuning setups.
microsofi/Phi-4 reasoning- ppuus
Open Openthinker3-7B --BB
deepseek- deepseek-reasoner rensoner
Qwen/Q=#Q32B -2B
Benchmark
Average
55.3
65.3
64.2
45.2
Math
AIME24
69.0
76.0
78.3
76.0
AMC23
93.5
98.8
98.8
96.2
MATH500
90.0
89.6
90.6
84.0
CodeElo
31.0
53.7
44.3
2.4
Code
LCBv2
64.5
75.2
88.6
0.8
CodeForces
32.2
47.4
46.7
3.5
Sci
GPQA-D
53.7
73.7
65.0
66.8
JEEBench
72.4
92.3
69.9
83.5
Held Out
HMMT
42.7
43.0
47.7
53.0
HLE
10.2
14.3
10.3
AIME25
53.3
60.0
62.7
68.0
LCBv5
51.7
60.2
67.2
0.5
Table 28: Comparison of OpenThhinker377B teacher models. We see that DeepSeek-RI is empirically the best model overall and our actual teacher model, QwQ-32B, is empirically worse. Mhii--Reaaoning-PP performs empirically poorly on code evaluations since it outputs code without code tags which Evalchemy marks as incorrect.
H ALL. TEACHERS ABLATIONS
We report the benchmarks of all of our teacher models in Table 28. Our results indicate that DeepSeek- RI is the most performant model despite QwQ-32B being the stronger teacher. mhii--Reassoing-PPP Plus also strong on certain benchmarks, but performs poorly on code. This is because it often fails to produce code tags such as """ython" which Evalchemy uses for code extraction. One notable number that OpenThinkerr, Thinker3-778 outperforms QwQ-32B on JEEBench, demonstrating single example of weak-to-strong generalization.
I SAFETY ANALYSIS OF OPENTHINKER MODELS
As enhance the reasoning capabilities of open-source models, we aim to ensure that our models refuse to respond unsafe requests while complying with benign requests. To this end, we evaluate the safety capabilities of Openthinker models on the following benchmarks:
39

---

Published as a conference paper at ICLR 2026
Model
Harmbench (L)
XSTEST (L)
Owen2.5-7BBInne Instrucct
14.5
4.4
30.5
2.4
Op
36.8
4.4
OpenThinker2-7B
42.8
2.4
OpenThinker3-7B
55.5
5.6
Table 29: Performance of Open'T ThenkThiner models on safety and over-refusal benchmarks. Here, we report the harmfulness rate and the over-refusal rate.
XSTEST Röttger al. (2023) consists of 250 safe prompts that syntactically resemble unsafe prompts. We report the over-refusal rate based on whether GPT-4o classifies the response refusal compliance.
Harmbench Mazeika et al. (2024) consists of 400 prompts based on harmful behaviors such as cybererime, unauthorized intrusion, handling of copyrighted material and misinfor- mation/disinformation.. We use GPT-4o the evaluator and report the proportion of cases that got the maximum score of 5 as the harmfulness rate.
In Table 29, observe that supervised fine-tuning on reasoning inadvertently degrades the pre- existing safety alignment of Qwen2.5 models, consistent with prior findings Qi et al. (2023). Among the OpenThinker models, OpenThinker-7B achieves relatively low harmfulness rate (36.8) alongside moderate over-refusal rate (4.4). In the subsequent generation, Open' OpennThinker2788 7B slightly improves the over-refusal rate (2.4), but this comes at the cost increased harmfulness (42.8). OpenThinker3- 7B continues this trend, reaching the highest harmfulness score (55.5) and modest rise in over-refusal rate (5.6). Notably, OpenThinker3-77 was trained without any explicit safety-tuning or alignment- focused data, which likely contributes to its degraded safety performance.
Interestingly, when comparing Tables 29 and 1, we find a clear trade-off between reasoning ca- pabilities and safety. These findings underscore the challenge of balancing safety and utility in reasoning models Wang et al. (2025); Bercovich et al. (2025). Future work on the OpenThoughts would benefit from incorporating safety-sp specifcc datasets to mitigate these risks while preserving their strong reasoning capabilities.
J EXISTING FRONTIER MODEL EVALUATIONS
Table 30 shows the benchmark results of the models available through APIs. Gemini-2.5-pe displays the strongest performance despite some of its answers being empty (and thus incorrect) due running out of token budget when its thinking process too long (especially visible in JEEBench). Claude 3.7 was given 32K token budget. 03 was used with its default medium reasoning effort. Our Open Tpenkhinner337B model outperforms, on average, the models to the right of the separator.
K TESTING REASONING ROBUSTNESS: ALICE IN WONDERLAND EVALUATION
Here, we build on the work on Alice in Wonderland problems (Nezhurina et al., 2024), which use variations in simple problem templates that do change both problem and solution structure. Given an instance of problem P and its corresponding solution S, reasoning (i.e. abstract solution) R and the final answer A, the reasoning-invann invariann perturbation the problem statement P will have solution ST and answer AJ. indicates an arbitrary ID of perturbation, depending template, can have infinitely many perturbations, e.g. if varying variables that hold arbitrary natural numbers. Importantly, P will have the same abstract solution (or reasoning) R the original template P. Models capable strong generalization should show similar performance solving the problem across all its structure- presuurvng variations
problem
We measure model sensitivity to reasoning- invariant problem perturbations to test models' ability to generalize. We follow Nezhurina et al. (2024) in our evaluation setup: set sampling temperature to 0.1 and sample 100 times, we set a maximum number of output tokens to 30720. Assuming beta-binomial distribution for models' answers for each variation, find the mean (average
40

---

Published as a conference paper at ICLR 2026
gemini-2.5 pro 5--rr--previie preview-05500
claude-3 1oade-3-7-ssnn 7-sonnet-thinking thinking
04-mini- 202--0225444 b4-minii-025-044 -0-1--1--66
deepseek reasoner
1o1-4.1-022-04-14
63-2025-04-16
gpi-4. I I-mini
gpt-4, I-nano
xai/grok-3
API Provider
Average
69.6
67.2
67.0
63.4
56.3
53.0
50.7
47.4
35.3
Math
AIME24
92.3
81.0
80.7
76.0
47.0
48.3
60.0
50.7
31.0
AMC23
100.0
99.0
97.5
99.0
73.2
87.5
89.8
83.2
71.7
MATH500
93.8
90.8
86.0
91.6
78.2
88.0
85.0
83.6
79.2
Code
CodeElo
59.5
$2.9
35.2
32.1
58.4
29.5
29.8
31.1
8.4
LCB 05/23-05/24
73.7
64.8
79.2
76.6
72.0
72.1
32.7
65.6
39.5
CodeForces
57.5
55.4
37.5
47.4
56.3
37.2
35.8
35.4
14.5
Sci
GPQA-D
82.5
77.9
80.0
73.7
80.8
64.3
66.5
34.5
46.0
JEEBench
44.6
80.8
86.2
88.5
71.4
73.4
88.5
78.3
55.7
HMMT
76.7
58.0
62.3
43.0
28.0
29.7
33.3
18.7
10.0
Unseen
HI.E
15.8
16.3
22.7
14.3
14.4
9.5
8.6
8.4
12.6
AIME25
76.3
72.3
70.3
60.0
39.7
41.3
50.0
33.0
23,3
I.B 06/24-01/25
62.3
57.5
59.0
55.7
55.2
28.3
46.6
31.5
Table 30: Performance of current API-based frontier models. A significant gap remains between current open-source reasoning models and frontier reasoning models. The vertical line denotes division between models that underperform and overperform Open Thenneenne3377 according to average benchmark performance. The largest gaps arise from benchmarks such as CodeElo, CodeForces, and GPQA Diamond. Gemini-2.5-pru is the most performant model.
41

---

Published as a conference paper at ICLR 2026
10
0.-
57
Na 02
Hi VS 532
0.6
VS
02
0.0 plpeiiow DeesSeak ML 97:8 UMG 328
325
LyNG VI-OWwn0-30 32b 013 Deen-19 K3 Marier Lonono
Owin 328
Qwion77
Figure 11: Distilled reasoning models show deficits in generalization. All distilled reasoning models exhibit strong performance fluctuations on AIW Friends variations 1-6 Nezhurina et al. (2024), despite the variations not changing problem structure at all. This points generalization deficits. The fluctuations affect to same extent SFT only (eg $1.1 32B, LIMO-32B) and SFT+RL. (eg Light-RI-32B, QwQ 32B) reasoning models. Smaller scale models, eg OpenThinner3377B perform worse than larger scale ones, eg SpenThiiner-32B showing overall lower correct response rates. Larger scale 32B models, while having higher overall correct response rate, show strong fluctuations, e.g. OpenThinker-322 going from close to I on variations 2 and 3 down to close to 0 on variation 1. For reference, ol op-evvwwww and 03-mini are shown, which have much smaller fluctuations and higher overall correct response rates. Distilled models still do not possess robust zero-shot generalization on simple problems. Numbers the legend are prompt IDs, see Nezhurina et al. (2024) and AIW repo
correct response rate) and the variance o², We visualize the correct response rate for each model and problem variant as a bar with corresponding error bars to indicate variance (Fig. 11). Despite improved performance on average (compared to non-reasoning models, Fig. 12), distilled reasoning models still show substantial fluctuations simple task variations, in accord with what was observed in Nezhurina et al. (2024), Fig. 11.
As evident from Fig. 12. reasoning models clearly and strongly outperform their conventional LLM counterparts, showing higher average correct response rates. Despite still persisting clear generalization deficits, reasoning models exhibit a strong boost across AIW problems compared to previous SOTA LLMs, with mid-scale reasoning models (32B) strongly outperforming LLMs trained at the largest scales (e.g. Llama 3.1 405B or DeepSeek v3 671B).
Takeaway: Distilled reasoning models, while strongly improving over standard LLMs, still suffer from generalization deficits, as evident from strong performance fluctuations across natural problem and solution structure preserving variations problem templates.
42

---

Published as a conference paper at ICLR 2026
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1.0
o1-preview
(amn-2025-022
s1.1-32b
ciQ 32b
doopsecio distil-llams-70b
3penthinneer-77--qeee 7b-qwen-32b
lightr1-320
ol-mini
deepseek-r1 r1 caaiillllmmaeeeee 8b
deepseek-rl distill distill--een-300 3wen-32b
owen2 3-72b-instruct
(iits-llamm-331
deepseek-v3
openthinker3-777 (attnnerrr----wooor 7b-qwen
meta-liama33 3. 1-8b-instruct
deepseek-ri diietil--wee-7b
owen2..5 in7sttttttttt
meta-llama-3. 1at-mmm-...----nn5-----
owen2.5
Reasoning models outperform conventional LLMs. Albeit suffering from strong fluctuations, larger-scale distilled reasoning models themselves apart from the conventional language models from which they were distilled. Shown are correct response rates averaged across all variations of AIW Friends, AIW Plus, and AIW Circles Colleagues problems Nezhurina al. (2024). Larger reasoning models (the only exception being RI-QWen-32B). Conventional LLMs, including the largest scale Llama 3.1 405B and DeepSeek v3 67IB, stay confined to the low correct response rate region below 0.2. As a reference, we show closed reasoning models o3-mini and ol ol-preview that show only weak fluctuations and settle in the upper performance region above 0.7%
Figure 12:
32B scale are populating the upper correct response rate range
43

---

Published as a conference paper at ICLR 2026
L COMPUTE REQUIREMENTS
The three main compute requirements for this effort are for annotation, training, and evaluation. Annotating OpenThoughts3-1.2M with QwQ-32B required 22,000 H100 GPU hours on 16 IxNvidia GH200 nodes. One single run of training Open Thenkhiine33-71 required 25,000 A100 GPU hours on 128 nodes, each equipped with 4x Nvidia A100 GPUs (512 GPUs in total). One evaluation run for Open'Tinne Thinker3-3 required 32 GPU-hours on 16 IxNvidia GH200 nodes. Throughout the pipeline experiments, annotating each 31, 31,600-size dataset cost roughly $300 in API costs through the DeepSeek API.
M SOURCING REASONING TRACES FROM THE WEB
In this work, we generate reasoning traces with annotator models. As an alternative, we also experimented with finding reasoning traces from the web, then processing them with language models in order to bring them in suitable form for SFT, following Yue et al. (2024), which generated catructtio-finetuinng data that manner.
We searched for long reasoning traces in Dcbe (Li et al., 2024), which is a large text corpus sourced from CommonCrawl text data heuristically filtered and deduplicated. We trained a FastText classifier by taking positive data the reasoning traces from OpenThoughts- fifitts-144K, 14K, and as negative data an equal amount of text from DCI.MRRefineWWet We then examined some of the sequences that most similar to the reasoning traces according to the FastText classifier (see Figure 13 for an example), and found that those are not similar to the long reasoning traces from OpenThoughts-1 14K. This indicates that CommonCrawl does not contain large amounts of long reasoning traces similar to those used by current reasoning models. While CommonCrawl can contains useful questions and answers that can serve base for effective instruction answer pairs for instruction finetuning and as question and answers as a bases for reasoning traces, it does not seem to contain many long reasoning traces.
Algebra 1
Published by Prentice Hall ISBN 10: 0133500403 ISBN 13: 978-0-13350-040-1
Chapter 4 - An Introduction to Functions
Chapter Review 4-2 Patterns and Linear Functions Page 282:8
Domain: 0, 1, 2, 3 Range: 18, 21, 24, 27
Work Step by Step: The relationship is between the number of snacks purchased and the total cost.
If snack is purchased, then the cost is 18. If snack is purchased, then the cost is 21. If 2 snacks are purchased, the cost is 24.
If 3 snacks are purchased, then the cost is 27. can see a pattern in the range:
each cost term is separated by 3.
So, we can assume that each snack costs 3.00.
Update this answer! Update this answer
Figure 13: Example of reasoning trace found in DCLM-Baseline. This is example reasoning trace found in OCLM-Baseline. This text does not resemble reasoning traces from modern reasoning models.
44

---

Published as a conference paper at ICLR 2026
N
LICENSES OF EXISTING ASSETS
Dwen-2.5-7B-Inntruun model is distributed under Apache 2.0 license as indicated in Qwen- 2.5-7B-Instruct. (Yang al., 2024a).
Open2Math dataset is distributed under Creative Commons Attribution 4.0 license as indicated in openmath-2-math. (Toshniwal et al., 2024b).
StackExchange CodeGolf dataset is distributed under cc-by-sa 4.0 license as indicated in StackExchange Data Dump. (Stack Exchange, 2024).
OpenCodeeRessoning dataset is distributed under cc-by-4.0 license as indicated in Open- CodeReasoning. (Ahmad et al., 2025a).
StackExchange Physies dataset is distributed under cc-by-sa 4.0 license indicated in StackExchange Data Dump. (Stack Exchange, 2024).
OpenAl Models are distributed under OpenAI Terms Use.
QwQ-32B model distributed under Apache license 2.0 indicated in QwQ-32B. (Yang et al., 2024b).
SCP-116K dataset distributed under ce-by-nc-sa-4.0 license as indicated in SCP- 1CK. (Lu al., 2025).
Organic Chemistry by Jonathan Clayden, Nick Greeves, and Stuart Warren has all rights reserved. (Clayden et al., 2012b).
Organic Chemistry/Fourth Edition by Francis Carey has rights reserved. (Carey, 1996).
Organic Chemistry by John McMurry is distributed under the Creative Commons Attribu- tion Non-Commereial ShareAlike 4.0 International License. (McMurry, 2023).
Principles of Organic Chemistry by James Flack Norris has standard copyright (Norris, 1922).
Organic Chemistry by Robert V. Hoffman with all rights reserved (Hoffman, 2004).
March's Advanced Organic Chemistry by Michael Smith and Jerry March has all rights reserved. (Smith & March, 2007).
Essentials of Organic Chemistry by Paul M. Dewick has all rights reserved. (Dewick, 2011).
Fundamentals of Organic Chemistry by John McMurry has all rights reserved. (MeMurry, 2010).
Advanced Organic Chemistry by Francis Carrey and Richard Sundberg have all rights reserved. Carey & Sundberg (2007).
Introduction to Organic Chemistry has no license.
Principles and Techniques by Unknown Author is distributed under NCERT Educational Use Permitted.
Organic Chemistry by Francis A. Carey has all rights reserved. (Carey, 2010).
A Concise Textbook of Organic Chemistry by C. G. Lyons, S. McClintock, and Nora Lumb with all rights reserved. (Lyons al., 2016).
The Practical Methods of Organic Chemistry by Ludwig Gatterman with all rights reserved and Community Resource Educational Use. (Gattermann et al., 1917).
Modern Methods of Organic Synthesis by W. Carruthers and Iain Coldham has all rights
reserved. (Carruthers & Coldham, 1978).
Organic Chemistry by J. Clayden, Greeves, Warren, and Wothers has all rights reserved. (Clayden al., 2012a).
Techniques in Organic Chemistry by Jerry R. Mohrig, Christina Hammond, and Paul Schatz has all rights reserved. (Mohrig et al., 2010).
Organic Chemistry by David J. Hart, Christopher Hadad, Leslie Craine, and Harold Hart has all rights reserved. (Hart et al., 2011).
45

---

Published as a conference paper at ICLR 2026
Basics of Organic Chemistry: A Textbook for Undergraduate Students by Anshul Bansal has all rights reserved. (Bansal, 2024).
Advanced Organic Synthesis by Richard Monson has rights reserved. (Monson, 2012).
Chemistry by Rice University subject Creative Commons Attribution 4.0 International License. (OpenStax, 2019).
O PIPELINE DETAILS
In this Section we go into more details for each step of our pipeline, which we briefly described in Section 3.
O.1
QUESTION GENERATION STRATEGIES
We will now go over the different ways we generated questions.
O.1.1 CODE QUESTION GENERATION STRATEGIES
We begin by detailing all the code question generation strategies:
StackExc CodeGolf (Number of Questions: 85.9K): A StackExchange forum of
coding puzzles, specifically aimed at solutions with the least number of characters possible. (Stack Exchange, 2024).
Opennodeeeeaooning (Number of Questions: 459K) A large reasoning-basee synthetic dataset date for coding, comprises 735,255 samples in Python across 28,319 unique competitive programming questions (Ahmad et al., 2025a).
oogntiivcompttaiossddhhhiioooooe (Number of Questions: 101K): Synthetic questions
evolved from LeetCode questions. (Hartford et al., 2024).
Fewoodeeeeeeae--FFeee cntteredd-Insructioon (Number of Questions: 150K): Mixture of synthetic and real coding questions filtered by an LLM. (Zheng al., 2024). KodCode/KodCodee V1 (Number of Questions: 384K): Fully synthetic and diverse coding dataset with questions ranging from algorithmic to package specific knowledge. (Xu et al., 2025).
(Number of Questions: 35.8K): Mul- tilingual code dataset on oode-understanding, completion, and generation. (Chai et al., 2024).
christopherrrroscta-cod (Number of Questions: 75.4K); Multilingual code dataset basic coding exercises. (Code, 2022).
(Number of Questions: 946K): Code problems and solu- tions generated using Glaive's ' synthetic data generation platform. (Glaive, 2023).
glaivaiighive-cod---sitttt
StackExchangg CodeReview (Number of Questions: 183K): Code review questions from wwwetttsttacexccanee.coom (Stack Exchange, 2024).
CrtthivMLmods/Codeer (Number of Questions: 41.9K): Coding dataset for the analysis
coding patterns, error types, and performance metrics. We transform code and an associated error into question for the L.M to solve using GPT-4o-mini with the prompt in Figure 14. (prithivML.mods 2024).
A mixture of synthetic python questions generated from python documentation, educational material and more. We use both OpenCoder- LLM/oop-sft--ttage2 and
OpenCCoder-LLNV/ocssstttttt
Specifically, we use the pack- age_ jee_inttuctt subset of DpenCCadr-LLL//opesf--tttte and the filtered_ srtescale_divereee _inttruct, and realuser instrrct subsets of OpenCoder-LLLMoppe-fff stagel. (Huang al., 2024).
instruct,
-e- JNSS-Inntruet-75K (Number of Questions: 73.4K): Coding instruction- tuning set generated with gpt-3.5-turbo- 1106. (Wei et al., 2023).
46

---

Published as a conference paper at ICLR 2026
You are to generate a question or task for a Language model based the following error and code pairs.
Error Type: (foriginal_status)) Code: [origgnnllsscc)]
Include only the question and task. include anything like Here is the instruction". Include the code in your question and make the task sound like what a human would ask a language model.
Figure 14: Coder Stat Prompt
codeparrot/apps (Number of Questions: 3.7K): Python dataset for generating code from natural language specifications. (Hendrycks al., 2021a).
aiibavaa-2222CCdd-290--hhaae
(Number of Questions: 283K): Human-aske ques-
tions to ChatGPT regarding code. (ajibawa 2023, 2023).
nampdn-ai/tiny-codes (Number of Questions: >1M); Dataset of coding questions from textbooks transformed into questions using an LLM. (Panda et al., 2024).
bigcode//ommittaccktt (Number Questions: >MM): Commits on GitHub turned into coding questions using LLM. We specifically look the Python, C++, Java, C, C#, CSS, JavaScript, Shell, and Ruby commits. We ask GPT-4o-mini to turn a pair of commit message and code into question using GPT-4o-mini and the prompt in Figure 15 (Muennighoff et al., 2023).
deepmind/code conteesss (Number of Questions: 8.8K): Competitive programming ques- tions. (Li et al., 2022).
(Number of Questions: 9.7K): Python dataset with ques- tions formed using compiler feedback with an LLM. Ren al. (2024). (Sun et al., 2024).
SenseLLMM/Rffeetionneee
(Number of Questions: 538K): Set of programming coding questions from CodeForces website. (Penedo et al., 2025b).
(Number of Questions: 47.6K): Questions generated by using an LLM to turn code snippets from GitHub into difficult questions. (BigCode, 2024).
(Number of Questions: 299K): Coding questions generated by letting Qwen2.5 Coder 32B Instruct generate coding ques- tions. AMagpie-Align, 2025).
Primelntellect/rell-ooort seerollssss (Number of Questions: 69.6K); Real SWE prob- lems generated by PrimeIntellect. (PrimeIntellect, 2025).
StackExchange StackOverflow: Coding questions from the StackOverflow online forum. (Stack Exchange, 2024).
Beablgeen/rracct--od----nttrur (Number of Questions: 70.4K); LLM generated ques-
tions regarding the React framework. (Fahlgren, 2025).
(Number of Questions: 309K): Cu-
rated questions from StackExchange StackOverflow. (Prime Intellect, 2025).
(Number of Questions: 59.9K): Coding questions to teach an LLM to predict the output coding snippet. (Primelntellect, 2025).
(Number of Questions: 78.6K): Coding ques-
tions about SQL from the WikiSQL and Spider forums. (b mc2, 2023).
47

---

Published as a conference paper at ICLR 2026
You are to generate a question or task for a Language model based the following instruction and code pairs.
Instruction: {(essage))
Code: [[old_contents]]
Include only the new question and task. include anything like Here is the instruction" Include
the code in your question and make the task sound like what human would ask a language model.
Figure 15: CommitPack Prompt
0.1.2 MATI QUESTION GENERATION STRATEGIES
We also detail all the math question generation strategies:
al2=adapttdeeoopeemaah--m- (Number of Questions: >IM); The MATH subset of OpenMattInstrrut22 (Allen Institute for AI, 2025).
AL-MO/NuinnaMathh (Number of Questions: 853K): Scanned math problems from
competition math problem sources. (LI et al., 2024).
GAIR/MathPile (Number of Questions: 99.5K): Math text shards that become seed infor- mation for generating math questions with GPT-4o-mini. Specifically, MathPile ontains unstructured text about topics in math. GPT-4o-mini uses the prompt in Figure to turn each text into question. (Wang al., 2024).
MeMEMath-AIME (Number of Questions: >IM): The MetaMath pipeline applied to the AIME and AOPS sections of NuminaMATH. We reproduce this pipeline using GPT-40-mini since the original MetaMath dataset was based GSM8K and MATH train sets (Yu et al., 2023).
math-al/AAtoMMathTett (Number of Questions: >IM): Math text shards that become seed information for generating math questions with GPT-4o-mini. Specifically, math- ai/AutoMathText contains unstructured text about topics in math. GPT-4o-mini uses the prompt in Figure 16 to turn each text into a question. (Zhang et al., 2025).
(Number of Questions: >1M): The OpenMathInstrutt pipeline applied the AIME and AOPS sections NuminaMath. We only do the question augmen- tation part of the OpesMuttlnstruet pipeline and use GPT--4-mini for our augmentation. (Toshniwal al., 2024b).
OpennMathinstruuteAAANM
zwhe?9/DeepMath 103K (Number of Questions: 95.9K): Curated math questions from several different sources filtered for difficulty. (He et al., 2025b).
T (Number of Questions: 256K): Mixture of existing math
datasets with questions generated with LLMs and Common Crawl. (Yue al., 2023).
wvida//OpenMahhusstuueee (Number of Questions: > >1M): Synthetic questions sourced from MATH and GSM8K train sets. (Toshniwal et al., 2024b).
ddrg/named math formulas (Number of Questions: > >IM): Math text shards that become seed information for generating math questions with GPT-4o-mini. Specifically, we take each formula and put in the prompt for Figure 17 and ask GPT-4o-mini to form a question. (Drechsel et al., 2025).
facebook/natural_ccm ceterriilrrsainngggg (Number of Questions: >1M): High-quality challenging reasoning questions backtranslated from DCLM and FineMath pre- pre-training corpora. (Yuan et al., 2025).
(Number of Questions: 45.6K): Heavily filtered verifiable math questions. (Albalak al., 2025).
SynthL Biissl//ig-MathRRL---.
48

---

Published as a conference paper at ICLR 2026
You are to reform the following math text snippet into question with quantitative verifiable answer such that would be included the USAMO or the Putnam Exam.
Text: [[text))
Include only the question task. De include anything like Here is the instruction". You can either extract question from the text or form a new one based on the text. Make the question sound like what a human would ask language model.
Figure 16: AutoMathText Prompt
You to reform the following math text snippet into question with quantitative verifiable answer such that would be included the USAMO or the Putnam Exam.
Text: Ilformula)
Include only the new question or task. De not include anything like Here is the instruction". You can either extract a question from the text or form a new one based on the text. Make the question sound like what human would ask language model.
Figure 17: Formulas
Napp7722hendyydkkmmmttmm---laa (Number of Questions: 79.9K): No details provided. (Asap7772, 2025). 2025).
TIGER-Lab/MATH (Number of Questions: 847K): Mixture of MetaMath, MATH- orca and some additional MATH-augmented dataset with GPT-4. (TIGER-Lab, 2025).
instruct (Number of Questions: >1M): No information provided. (ibivibiv,
2023).
BAAI//ninntyMAATH (Number of Questions: 99.9K): A scalable instruction tuning dataset
for programmatic mathematical reasoning. (Zhang et al., 2024a).
(Number of Questions: 937K): Questions spanning a diverse domains of college level mathematics. (Ajibawa, 2023).
Babbawa-2223Maths--aaaa
MetaMath (Number of Questions: >1M): Our reproduction of MetaMath. The original
MetaMath built with GPT-3,5-turbo. We replace this with GPT-4o-mini in our pipeline (Yu et al., 2023).
allenai/math _q (Number of Questions: 29.7K): Math word problems sourced from AQuA- RAT. (Amini et al., 2019).
deepmind/math dataset (Number of Questions: 1M): Math questions at roughly school level. We specifically use questions from the algebra_ 2 2d composed, probabil- ity_ sty_ssr_p____ s___ss_____e_eel__l_____________ level set, Sollyoomils__eeallleel composed, polynomials. _iaaollils_siiiiiffyyooo_o simplify_ppot _owerr calculus tifffreentiteecoomppss and probability_ swr sequncee et al.).
sequence subsets. (Saxton
Laplofficial/Math (Number of Questions: >IM): No information provided. (Laplofficial, 2024).
49

---

Published as a conference paper at ICLR 2026
0.1.3 SCIENCE QUESTION GENERATION STRATEGIES
We also detail all the science question generation strategies:
StackExchange Physics (Number of Questions: 547K): Questions from https: /ph ysics. stackexchange. com, the Physics StackExchange Forum. (Stack Exchange, 2024).
Organic Chemistry PDF Pipeline (Number of Questions: 46.2K); LLM extracted organic chemistry questions from SCP PDFs and more Organic Chemistry Textbooks. We start the PDFs from SCP-116K alongside organic chemistry textbooks, solution manuals, and more. We use gemini/gemini-2 flash-litc-previeww
with the prompt in Figure 22 to extract the text from the PDFs. GPT-4o-mini then uses the prompt in Figure 21 to extract the question and answers from each page of the extracted text. GPT-4o-mini then refines the questions into cleaner questions using the prompt in Figure 20. GPTT-4o-mini then filters out questions not related to math, science, and code using prompt in Figure 19. We use structured decoding to get classification as a boolean and a reasoning as string. GPT-4o-mini then further filters questions not related to organic chemistry with the prompt in Figure 18. We use the same structured decoding technique for the organic chemistry filtering as we did for math, science, and code questions.
mtth/cqaduusstackplyee physics (Number of Questions: 38.3K): A dataset for community question-ansserrigg research focussed on physics. We use the prompt in Figure 23 to turn each unstructured text into question with GPT-4o-mini (Hoogeveen et al., 2015).
Camel-AA/Physie (Number Questions: >IM): Our reproduction the Physics questions from the Camel pipeline. We reproduce this pipeline from scratch using GPT-4o-mini. (Li
et al., 2023b).
Josephgflowers/Par- Sates/ParrrrFFFrFeeeeeeee
Math-Reason (Number of Questions: 988K): LLM generated questions from science text on FineWeb We use the prompt in Figure 23 to turn each unstructured text into a question with GPT-4o-mini (Flowers, 2024).
millawell/wikkppedia field_of ociennc (Number of Questions: 304K): LLM generated
questions from Wikipedia science articles. We use the prompt in Figure 23 to turn each unstructured text into a question with GPT-4o-mini. (millawell, 2024).
zeroshot/axxiv-biolggy (Number of Questions: 1.2K): LLM generated questions using Arxiv Biology papers. We take the abstracts from the original source use the prompt in Figure 24 to turn the abstract into question using GPT-4o-mini. (Clement et al., 2019).
Camel-Al/Chemistry (Number of Questions: >IM); Our reproduction of the chemistry questions from the Camel pipeline. We reproduce this pipeline from scratch using GPT-4o- mini. (Li et al., 2023b).
Stack Biology (Number of Questions: 60.3K): Questions from https: /bi ol ology. stackexchange.e.........o. com, the Biology StackExchange Forum, (Stack Exchange, 2024).
Camel-AIBBiology (Number of Questions: >IM): Our reproduction of the biology questions from the Camel pipeline. We reproduce this pipeline from scratch using GPT-4o-mini. (Li et al., 2023b).
hawpolooyyyydssset.eeeeee iiitasttsttnnardizeee unified: (Number of questions: 22K)
information provided. (AdapterOcean, 2024).
50

---

Published as a conference paper at ICLR 2026
Yes or No, is this question an organic chemi stry question? Question: ((problem)
Figure 18: Organic Chemistry Filtering
Yes or No, is the quest in an answerable difficult science, math, or coding question? If the questi on refers to content (figures, equations, additional text) that you cannot see, then it unanswerable.
Provide your reascning.
Question: [LLmporee_gueetiooossdttlool]]
Figure 19: Code, Math, and Science Question Filtering Prompt
creating ing quest I will provide you with given question and the text fron which it xas extracted from.
You will enauze that the question is ansverable, meaning tast there is enough context to ansver the ton,
this, you will questions instantistion it chere is, you will provide the new extra text before restating the quastion but ze suze you always add the question itself at the end. instruster creating questions, will include solution in Because question.
extracted
eneure
nothing is missing
the
Here is example your task: Extracted Questions dalculate the chenical 5.000 a comple Extracted Text: cample at sitterent nineral is anslysed by the same nethodo. This mineral also
(in mool) of nitric acid that with the containe only Ph2, cD3, Oi,, and oz tons. 5.000 cample (BNG5), 0.5214
nitric acid 03 2.000 renains. When subjected to chermal decoroosition, 5.000 of this mineral lesas 0.5926 a. (g) Calculste the chenical amount (in mel or mol)) nitric acid that seasts with the 5.000
minersi treszed reimared,
this nineral.
You would tell no:
sample diffevent ninersl
nethods. ninerol contains only Pb2, co2
When 5,000 e aample of this nineral treased wich 25.03 au of 2.000 nol & nitric acid (BB05), 0.5214 of carbon dioxide is zelcared, and 0.01051 mol of the acid zenains.
themsl decorposition, 5.000 this mineral 0.5926 Calculate chenical reacta with
anount (in amol:
5.000 a sample of this ninerai.
Here is the qunations [esstracterLgmestioooo Boze is the extracted text: [iousput_oxtractiionn
Do not include filler Like "hore is the improved question". Include only the relevant information and Itselt. Inslude solution you it. This is exan, so you Lhe quest ion.
cholces applicable.
answez
Figure 20: Question Refinement Prompt
51

---

Published as a conference paper at ICLR 2026
Extract the questions, ansver choices, and solitions fren the extracted text Crom the pác below.
response
QUESTION;
*the quastion from text and selavant context excluding anavez choices" the question is "Watt waS E.vis Presicy's 5 (a) ravorita sanowich? (b) most hazed song? (e) birthdzy 7" You should have guestion "QUESTION: that wad Elvis Presley's tavorite eancwich.... and taen Presicy's
*QUESTION:
hated sene?..."
AMSKER CHOICES; "ansvez choice
answer cheiccs are available just say "f:ue sesponse", if anaver cheices are available please welle then out 1.8, "A: 0.57 III B: John Hancozk I1 etc.)
"ansver choice ...
SOLUTION;
choice" "fres DEFECIED"
determine
solution
say
is important that each QUESIION: is selt cortaines, meaning, if you mere rozd QQESTION: ..." by itself, should sandwich? (b) nost hated sond? (e) birthday?", you should say "QUBSTION:: ravorica sandxicas" Lhis nakes to sense. Ins.ead you should say *CUESTION; What was Elvis Preslay's a favorite sandvich.
diver. the exarple "that Bivis Fresley's ta) favorite
2a
subquestions should restated individusl questions thanseivas relevant infermat ion requiced Lo answer clen.
All
Coly break up the aubquestiens. I.B. only break up questions that have (a) ib) and (e) or (1) (2) and (3)
do not break up questions that do not have narkers indicating they have parts or subquesticns
glease.
creating
questions that actonatically extracted from sdfa, so is Ixporative you
get this right.
Extracted Text:
[botpal_.nttreccion)
Figure 21: Question Extraction Prompt
Extract all texe tron this Por, including all text tzon images. Sxample extractions could look like this: htmlo conlyy cta wew.eaaabeeeeeteeeee oott2/t abd 14/tibeeed seccsc/eppttt-eeeseeeee ottpr//oooooooaotooooooooo////ee
6c/tabaeib (acabbab--1144tte 14c/t/b//---- raced 12tp///
Notics that the rinal sun zero, tolling us that 2 is zero. aut the nize thing about syathetic quotient
divicion, that civisicn, polynonial,
contrictents the ccofcicionts
The Remsiader Theoren states that when divicing polynonial by a factor et the torn 9ix-ar6 there is quotient polynonial. and constant renainder, The remainder Pla) since (x)(x-a)) Ledot 0(x1+r) Rightarsow (8))
Miscell lareous Facts;
any polynoniel simatami()s y-interceps and equal.y as (1)5 is the sun of all the confficiants.
Problems:
1. Giver. the cublc polynental 92 ix] xx x'-72x--x-28: Two of the zeros are additive inverses, Find tho zecos.
(inathen
polynonial
zational contricicats and
Slegrté least?
(31) 4. When Madiacn's a dog chesed up has nathenat los assiqueent, ane parcieular equation was ripped apast. I found picco of the beçincing of the equation. and piece at the end, but the middie was missing. beginning piece 9x" (3)- (9)+9 and ending zoots would intecers. How nany times (-118 root? [Farman
degres of #imathrrskipp) (Neneursmath (taathh ix
9+11-09 Fortunately
teacher
pronised that
22721
5. The tollowing is polynenial. Find the cun of the equares or its contticiants. Sigget 191-1x (8) FIRISH
-2xx" (61+84x" (5) -42x" (8)x((())))))))) 72x**2+*
IE cabic polynonial Pheperatornan: l'mathbe (015 has zoots as -1, 2, 3, @imachfrak (0)=15 then the semaindes when Makthffak (p) (temaurenath Naathbl (x)1) is divided by S (00
-14 19:
Ir colution 9xc (3) x+10-95
equals:
cunber distinct real solut iona of the squation 84x" -x3 (2)
9. What is the sun of the squares of the roots of 5x" (4)-5x" (2)-55x(2))5-00 (2)+5-05
10. For how nany integers 3_inathrn (N16 6 is $a0 (01401-61 (3) -80 121
12. Madison's doz choved graph sC (3) (2)-2x-45
Lines
Statthbi axis?
could finish The feeement saved from henework before
canine' nouth swoeal only the two teems of highest degrew of the polymonial Flmathirakip)) IN iisurematiimattae (4)1)8
Now please give nw your exizant of all Lex., including Lext images.
Figure 22: Gemini OCR Prompt
52

---

Published as a conference paper at ICLR 2026
You to reform the following math text snippet into question with quantitative or verifiable answer such as one that would be included in the Biology, Physics or Chemistry Olympiad. Text: [[text))
Include only the question task. De include anything like Here is the instruction". You can either extract question from the text or form a new one based on the text. Make the question sound like what a human would ask language model.
Figure 23: Prompt for Generating Science Questions
You to reform the following math text snippet into question with quantitat verifiable answer such one that would be included in the Biclegy, Physics or Chemistry 0lympiad. Text: (abstract)}
Include only the question task. Do not include anything like Here is the instruction", You can either extract question from the text or form a new one based on the text. Make the question sound like what a human would ask language model.
Figure 24: Prompt for Generating Science Questions
53

---

Published as a conference paper at ICLR 2026
O.2
INFORMATION ON QUESTION FILTERING STRATEGIES
We will now provide more information on our question filtering strategies.
O.2.1 FASTTEXT DETAILS
We provide some more details on our FastText filters.
Our hidden dimension of the FastText Filter was 256.
We train the classifier for 3 epochs.
We use learning rate of 0.1.
We use bigrams = 2.
We use a minimum n-gram count of 3.
O.2.2 CODE FILTERING STRATEGIES
We provide more details about our code filtering strategies.
Diffieulty-based Selection: Ask GPT-4o-mini to rate the question scale of 1 to 10 using a rubric for ICPC problems and take the hardest rated problems. When asking GPT- 4o-mini help with question filtering, only temperature to 1.0. We do not set other sampling hyperparameters. The prompt for difficulty filtering is in Figure 25. We use structured decoding to extract numerical response from GPT-40-mini. For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
Length-based Selection (GPT-4.1-nano): Annotate questions with GPT-4.1NNam and keep the questions with longest responses.
AskLLM Selection: Ask GPT-4o-mini to rate on a scale of to 100 how similar a question is a of good questions and different from set of bad questions. When asking GPT- 4o-mini to help with question filtering, we only use temperature set to 1.0. We do not set other sampling hyperparameters. The prompt for AskLLM filtering in Figure 26. We use structured decoding to extract numerical response from GPT-4o-mini. For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
Length-based Selection (GPT-4o-mini): Annotate questions with GPT-4o-mini and keep the questions with longest responses.
FastText (P: Codeforces; N: CodeReview): Classify questions with FastText classifier trained with positives that are CodeForces and negatives that are questions from StackEx- change Code Review. More info in Appendix 0.2.1.
Length-based Selection (GPT-4.1-minij Ask GPT-4.1-mi to rate scale of to 100 how similar question is to a set of good questions and different from a set of bad questions.
Random Selection: Randomly select questions.
Fast (P: LeetCode; N: SQL): Classify questions with FastText classifier trained with positives that are from LeetCode and negatives that are questions from bugdaryan/sql-crear context-instruccion. More info is in Appendix O.2.1.
FastText (P: Code Golf; N: SQL): Classify questions with a FastText classifier trained with positives that are from StackExchange Code Golf and negatives that are questions from http:///-creatt-cotte---iie
More info is in Appendix 0.2.1.
FastText (P: All; N: SQL): Classify questions with a FastTTxx classifier trained with positives that are from CodeForces, LeetCode, Code Golf, and IOI and negatives that are questions from ouggaaayaasssl--rettt--otttt-tte instruccion.......o...ooo More info is in Appendix 1.2.1.
FastText (P: I0I; N: SQL): Classify questions with FastText classifier trained with positives that are from IOI and negatives that are questions from Iugdaryan/sql-creatte context-instruction.. More info is in Appendix O.2.1.
FastText (P: Codeforces; N: All): Classify questions with FastText classifier trained with positives that are CodeForces and negatives that are questions from StackExchange Code Review and geeedraannkllleeeee--mmmeet--tte
More info is in Appendix O.2.1.
54

---

Published as a conference paper at ICLR 2026
FastText Selection (P: Codeforces; N: SQL): Classify questions with FastText clas- sifier trained with positives that are CodeForces and negatives that are questions from hatpapaaa///slrraattt-ooottttt--
More info is in Appendix O.2.1.
Libeddding-bssee Selection: Embed a set of positives, which are CodeForces, and a set of negatives, which are model, coxt-embeddiin-3-Beee similarity to positives and mean cosine similarity to negatives, and generate the final score by taking the difference.
using OpenAI's embedding First, embed a given question and measure its mean cosine
O.2.3 MATH QUESTION FILTERING
Length-based Selection (GPT-4.1-nano): Annotate questions with GPT-4.I-Nano and keep the questions with longest responses.
AskLLM Selection: Ask GPT-4o-mini to rate scale of 1 to 100 how similar question is to a set of good questions and different from a set of bad questions. When asking GPT- 4o-mini help with question filtering, we only use temperature
1.0. We do not set other sampling hyperparameters. The prompt for AskLLM filtering is in Figure 26. We use structured decoding to extract numerical response from GPT-4o-mini, For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
Random Selection: Randomly select questions.
Embedding-based Selection: Embed a set of positives, which are the "amc mime and "olympiads" subsets of ALMMONuminaMath-Coo1 and of negatives, which is Laplofficial/Math using OpenAl's embedding model, cext-embeddiig-33-1 bed given question and measure its mean cosine similarity positives and mean cosine similarity to negatives, and generate the score by taking the difference.
First, em-
FastText (P: S1.1; N: Laplofficial): Classify questions with a FastText classifier trained with positives that are questions from SI and negatives that questions from Laplofficial/MMattsssaa Battaatthhssma______ _acpll__________ More info is in Appendix O.2.1.
FastText (P: Olympiad; N: Laplofficial): Classify questions with a FastText classifier trained with positives that are questions from brando/olmmpiad-bench sareeeiioommmab-boeee 025-v2-21-08-22 2024 and negatives that are questions from Lap Laplofficial/Math small corpus. More info is in Appendix O.2.1.
Difficulty-bssed Selection: Ask GPT-4o-mini to rate the question on a scale of 1 to 10 using& a rubric for AOPS problems and take the hardest rated problems. When asking GPT-4o-mini to help with question filtering, we only use temperature set to 1.0. We do not set other sampling hyperparameters. The prompt for Difficulty filtering is in Figure 27. We use structured decoding to extract numerical response from GPT-4o-mini. For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
FastText (P: OpenRI; N: Laplofficial): Classify questions with a FastText classifier trained with positives that are questions open-rl/OpenRl 1-M00pnnn--Math-200-- and negatives that are questions from Lfffffhicial/Mathh small _coppu. More info is in Appendix O.2.1.
Fast (P: All; N: Laplofficial): Classify questions with a FastText classifier trained with positives which are the "amc_aime" and "olympiads" subset of Al-MO/NuninaMath CoT, saandooddyppaa---eee----------- hat---mmat--boedd22------- 220K, and S1 and negatives that is questions from Laplofficia//Math_snn harraaaath__mallo_opuus More info is in Appendix 0.2.1.
open-rl --ath- Mathh-
FastText (P: Numina; N: All): Classify questions with a FastText classifier trained with positives, which are the "ame aime_imme" and "olympiads" subsets of NI-MO/NuminaMath-CCTT and negatives, that are questions from offiiiaaiia//M fiaithhsmal__eee small_corpus and facebook/natu- ral nal_reasoning. More info is in Appendix 1.
FastText (P: Numina; N: Natural Reasoning): Classify questions with a FastText clas- sifier trained with positives, which are the "amc aime" and "olympiads" subsets of AI- MO/NuminaMath-Col and negatives, that are questions from facebook/natural reasoning. More info is in Appendix O.2.1.
55

---

Published as a conference paper at ICLR 2026
You will be given code problem. Your job is to grade the difficulty level from 1-10 according to the ICPC standard. Here is the stancard:
A 10-point scale for ICPC problems could be structured as follows, where level represents the easlest problems and level 10 represents the most challenging:
Level 1: Basic implemenaation problems requiring simple input/cutput
handling and straightforward calculations. Typically solvable with single coop or basic conditional statements. Examples include summing numbers or finding the maximum in an array.
Level 2: Froblems involving basic data structures like arrays and strings, requiring simple algorithma like linear search or basic sorting. May include simple mathematical concepts like prime numbers or basic geometry.
Level 3: Problems requiring knowledge of standard algorithms like
binary search, complete sorting algorithms, or basic graph traversal (DFS/BES). May include simple dynamic programming problems with clear state transitions.
Level 4: Froblems combining multiple basic concepts, requiring careful implementation and moderate aptimization. Includes medium- difficulty dynamic programming problems and basic graph algorithms like shortest paths.
Level 5: Problems requiring solid understanding of data structures
like segment trees, binary indexed trees, disjoint set unions. May include more corplex graph algorithms like minimum spanning trees or network flow basics.
Level 6: Advancea dynamic programming problems with non-obvious state representations. Problems requiring combination of muitiple algorithms
data structures. May include basic game theory basic number theery concepts.
Level 7: Problems requiring advanced algerithmic knowledge like heavy- light decomposition, suffix arrays, or advanced geometric algorithms. Includes complex optimization problems and harder network flow applications.
Level Froblems requiring deep mathematical insights combined with complex algorithmic implementations. May include advanced number theory, complex geometric algorithms, problems requiring multiple non-ebvicus observations.
Level Problems requiring extensive knowledge advanced algorithms and mathematicns concepts, often needing multiple key insights to solve. May include advanced string algorithms like suffix automata, or complex mathematical optimizations.
Level 10: The most challenging problems, eften requiring novel approaches insights not covered standard competitive programming material. These problems might combine multiple advanced concepts in non-obvious ways, require complex proofs for correctness, or need highly optimized implementations to meet strict time limits.
This scale correspends roughly to the difficulty progression you might see from early regional contests (levels 1-4) through regional finals (levels 4-7) to world finals problems (levels 7-10). Problem to be labeled: [Iquestion))."
Figure 25: Prompt for Code Difficulty Filtering. This text is the prompt for Code Difficulty Filtering.
Length-based Selection (GPTT44-mini):: Annotate questions with GPT-4o-mini and keep the questions with longest responses.
56

---

Published as a conference paper at ICLR 2026
I sont you to junge whether the tolloving question is like the pocicive examples provided or 1ike the iie examples. Here are fex positive exerples of ionst
Fositive questions
plann cortains 5405 lines, no 925 of which are perallel, Suppose that there are $38 points where wxeet ly $38 lines intersect, $43 puints where exectly $48 Lines Intersect, 558 points where exeetly than
555 lines intersect, 969 points where exactly 569 lines intersect, lines Vdomsrrorw of its spin
points
points 2. spin-tait particis is in Linsar superpociciont. h and spin-down states, It Nuparsoviraagge and lddomerrowrrangl are the eigerstates or \signe_ then what the
fon value signa_ [X] Bere, symbols have their useal meanings.
decimal place, of the
10vsigns (8)48)
3. An establithes group or sclentiets are vorcing on tinding solution to RP hare problens, They clain Sobset Sun as an Ni hp-hard problen. The probl is detern ne whether there exiats subset of engineer
ciain at non-negstive
this
problen given integers, find whether there exlats subset whose sun is K, Toput First 1ine input containa T, the nunser of test cases. tast cases tollow, Each test case contains 2 lines, First 1ine containe two integers
rumbers
nor-negative. Given
non-negative output Cutput lines, for each test case. Every Line should ze asther 0 or deperding on whether euch a susset existe or not, Example Input 1 2 S 10 3 6.6 3.2 655
Next line containe
(each
lcss than 100900) g
Dutput: 1
independenzlly and $0 ir the order they divides 28.28 25 and sa divides $5 is 8lcfraeimi where And and 9n8 are relativcly prine positive intagezs. Find $n.5 What
positive replacement
divicors 920-9.5 Three
at randon
labeied $a_1,8_2,
chosen. The probabi lity that both
concentration calcium iane solution csntaining 0,02 stcichionetrie Ca-EDTA complex (va assune that pä ideal, 25C), KCa-EDIA 1x.
Negative Questions:
Bolve
foz
Simplity
(1210) sqet (1210) sqgt (20) /eget --611 ((sgrt (1800) sart (1800) sqgt (1000) egre -11=))+=1++3++222
3. civer. 1ist of abjects that have an "La_orçanized" mothod that returns a boclean value, weita those objects which "isoorannized"
takes list and retures
retuzna True.
6. Can you provide a Pyzhon code snippet that doncastrates how to uss a dezorater to lo7 the exccution time of runction?
sodiun
acid
Here is your question:
[qqestton]]]
Beturn apore belveen and 100, whare mwans exactly like the positive questions whereas exactly like the negativa cuestions.
Figure 26: Prompt for AskLLM Filtering. This text is the prompt for AskLLM Filtering
Length-based Selection (GPT-4.1-mini): Annotate questions with GPT-4.1-min and keep the questions with longest responses
FastText Selection (P: Numina; N: Laplofficial): Classify questions with a Fast- Text classifier trained with positives, which are the "ame_aime" and "olympiads" and negatives, that is, questions from
subsets of Al-MO/NuminaMath-Con Laplofficial/Matth small corpus. More info is in Appendix O.2.1.
O.2.4 SCIENCE QUESTION FILTERING
FastText (P: ExpertQA; N: Arxiv): Classify questions with FastText classifier trained with positives that are questions from katielink/experrqq and negatives that are questions from AdapterOcean/biolgyy carooooyyyddtsse...emoe stanteet_sttadddeee__aee
More info is in Appendix 0.2.1.
Length-based Selection (GPT-4o-mini): Annotate questions with GPT-4o-mini and keep the questions with longest responses.
Length-based Selection (GPT-4.1-nano): Annotate questions with GPT-4.1Nano and keep the questions with longest responses.
Length-based Selection (GPT-4.1-mini): Annotate questions with GPT-4.1-min and keep the questions with longest responses.
AskLLM Selection: Ask GPT-4o-mini to rate on a scale of to 100 how similar a question is to of good questions and different from set of bad questions. When asking GPT- 4o-mini to help with question filtering. we only use temperature set to 1.0. We do not set other sampling hyperparameters. The prompt for L filtering is in Figure 26. We
57

---

Published as a conference paper at ICLR 2026
sifffiulty
acooeding spstew
deeast Aottor Kist Higs fetool Cha crgiing figl Setuall Diyppial leselry oiser cenceat our be Interpelcted oprieet (NO
Holl thee
they
Test-n skery cor see the oveer chotees Hiety cornect
quastiare than Racteporce AIE airliss cifficsity.
Sisaplade
quastises, naabscd
For chese the first half of
1-1)
1: Troblies
school wisstte acheol Tewes NEM6, HareCoonms Coertery set 1-70, Me co 1-10, 1-y, inceodaced up sie sidtis actood Levasl, seat cexiational ecdzse/tign ackeod uned pectlens, with extrensly comtin pooblem selviro.
and
21-21,
Detecmellateii
suce ADE
oupplaad-atpee prdkiana tearly aato quscioss, sssinat CRADD L/41-33
1/41.
2t's
ars, eatcemety
30
medtar 55M00 and D50 215).
4. Kistiinon Sisquaariieed
testi 1000 2/h, cansenr SEMR and DE 12o
(sach 1h0y that is uoceriinnyy inturnyy kong and aitiieest
very for reodects i capable sutring bastal.
Geappies
reternatry ane
Judy
facks.
for sany Lataged axtiary did 31pa22 (2322 spring kad 200, recolen
nidpeirs of meeet trotrnational ErrNcs
M018 are 400, bulyy to
the colkosses iocaerala cons
leguadiceeoaad 1199, sasdegt
cast aee 2013 euch collowing spaces (e (at- ta: 2 ATTa te Foed aia,
is Loe stockangaa daccy ADSS wetteer
Promngs and les aas to the Lececnnctics ias of the acciteces of Picecargle ABC.S sipposs tte
poloce and set with divistble
end 183-6.5 aree Pitelinale
cem.
Skeathoull555,
Leat snovece may
any cheee puists saa, 18t, ace in the Eo skece peact, ger
Sinathealldls such
Decsendre all Lecepece Salged 50 for which theee andana balanced destce free conatactng of ted potnee. X
de for tosts diffacare deconiaations and (oo ficice odd
2nd,
Ist proce that Le possible to apiis this collection loss 52600 or deves
Fllrasied Broms Las on
by
nost
3: Lee dio posicion murr/aa
irom that shace in as neat ose any tip to eccacion and redlactions to
12: kcoce thas shace ectate postsive cocatans auds follossen anaceners seas: cocatdar imcegee to 1s zee Dinatborl 00271-22310 for miny two prltcca Bootthed erensea Birtid 3190/11
snpacating dlassacea do aacs that the dissanoe tean any poins of Slascioed at co plalis Pecolan lsceled: (rquestion))*
Figure 27: Prompt for Math Difficulty Filtering. This text is the prompt for Math Difficulty Filtering.
use structured decoding to extract numerical response from GPT-4o-mini. For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
FastText (P: SciQ; N: Wikipedia w/ Arxiv): Classify questions with a FastText classifier trained with positives that are questions from allenai/scig and negatives that is questions from adapperOcean/biology_ddaeee daataaet_standeeeee srnifffdzzedduiifieedd unified and questions generated from millawell/wikipedia. _aeeleeeddddidddd_ddee- of science in Appendix O.1.3. More info is in Appendix 0.2.1.
Embedding based Selection: Embed a set of positives, which are questions from allenai/s- ciq. and a set of negatives, which is AdapterOccan/biology Satollogyyydatt_t_a_aeee using OpenAl's embedding model, text-embedding-3-kee and measure its mean cosine similarity to positives and mean cosine similarity to negatives, and generate a score by taking the difference.
Lunfffiie First, embed a given question
FastText (P: SCP, ExpertQA, SciQ; N: Arxiv): Classify questions with a FastTexx classifier trained with positives that are questions from allenai/sciq. 1/iiL//SCP-116K, and katielink/expertqa and negatives that are questions from AdapeerOeenn/iology__aa hatpiioooooyyyyoooooaeeeeeee sttes_stadddddd_niiiiiiii _niified and questions generated from millawellwwikppedia. _teedd of science in Appendix O.1.3. More info is in Appendix 0.2.1.
Random Selection: Randomly select questions.
Difficulty-based Selection: Ask GPT-4o-mini to rate the question on a scale of 1 to 10 using rubric for science Olympiad problems and select the hardest-rated problems. When asking GPT-4o-mini to help with question filtering, we only use temperature set to 1.0. We do not set other sampling hyperparameters. The prompt for Difficulty filtering is in Figure 28. We
58

---

Published as a conference paper at ICLR 2026
structured decoding to extract numerical response from GPT-4o-mini. For AskLLM Filtering, we request a numerical response as an integer and a string response for reasoning.
O.3 DEDUPLICATION AND TEACHER SAMPLING
There are several methods for performing deduplication. We choose use Indel Similarity to measure the similarity between two questions. This similarity refers to the number of characters that need to be inserted or deleted from one sample match the other, and calculated relative to the sample length. More precisely, we consider the Normalized Indel similarity score between pair of strings, as computed via the Longest Common Subsequence (LCS) metric:
LeSienghh (51, 82)
indel 100
max (1 (82))
0.4
QUESTION ANSWER FILTERING
We will now discuss the details of each question-answer filtering method.
O.4.1 QUESTION ANSWER FILTERING FOR MATH
Comprehensive Large Dataset: Take all 63,200 responses for one dataset without filtering.
This not compute-controlled.
Random Filtering: Filtering questions-answer pairs randomly.
Shortest Answers Selection: For any given question, take the 8 shortest responses out of the 16 responses and create 8 question-answer pairs for the dataset.
Majority Consensus Selection: For any given question, provide all responses to GPT-4o- mini Ask GPT-4o-mini to return a list of indices of responses that agree with the majority. Only use the last 1, 000 characters of each response to get the final answer. Use temperature Figure 29.
1.0. The prompt
Classify question-answer pairs with FastText filter. Form the query strings by the following format: "Question: [question) AnAnswer: {answer_ooe (answer_column] We do this for both training and using the FastText classifier. The classifier's positives are S1.1, which contains responses from DeepSeek R1. The classifier's negatives are from ceniddatioonsdv/strttteee verified mix annotated with GPT-4o-mini. More info is in Appendix 0.2.1.
Test Selection:
colummn )
Longest Answers Selection: For any given question, take the longest responses out the 16 responses and create 8 question-answe pairs for the dataset.
GPT Verification: Ask GPT-4o-mini whether provided answer is the correct answer for the provided question. The sampling hyperparameters are temperature of 0.0, top_p of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is a boolean value, and a reasoning, which is string. The full prompt in Figure 30.
Removing Non-English Answers: Ask GPT-4o-mini whether provided answer contains only English. The sampling hyperparameters are temperature of 0.0, top_p of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and a reasoning, which is a string. The full prompt in Figure 32.
Removing Long Paragraphs: Ask GPT-40-mini whether provided answer is free of long paragraphs. The sampling hyperparameters are temperature of 0.0, top p of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and reasoning, which is string. The full prompt is in Figure 33.
0.4.2 CODE QUESTION ANSWER FILTERING
Classify question-answer pairs with FastText filter. Form the query strings by the following format: "Question: (question) InAnswer: monswer_columee We do this for both training and using the FastText classifier. The classifier's positives CodeForces which contains responses from DeepSeek RI. The classifier's negatives are CodeForces annotated with GPT-4o-mini. More info is in Appendix 0.2.1.
FastText Selection:
59

---

Published as a conference paper at ICLR 2026
You will be given a science preblen, Your 3ob 15 to grade the ditticuity level fron 1-10 according to the international science olyspiad standard, \
Here is the standards \
A. 10-poing scale tor international ecience olyrelad proniems could be strustures as tollows, where level "epresents is the esstest problens and level 10 represents the most chell lenginng
Lavel 1s Saxic Knovladge Applicatioon Straight forvard recall of fundamenta:: scient facts and principles, Simple calculations requiring solent consegt.
basic tormu.as. Direct applicstion ot single typically solvable
typical'y covered in standard high schocl curriculun. Examples includo ldentitying simple chemical compounds, basic circuit celculatioon, or classitying organisne.
stepe,
Tevel 2: Nclzi-Step Basic ipplications
Prob ens requi 2-5 dist steps to solve, Appl lestion of multiple basic concepts within singie tield. aasic data interpretation from grapho tables. Simple laberetory techaiques and measurements. Content typice. ly tound in advarces high school courses. Pxamples include staichionetry eillaaaionn basic i nenaties problens, or analyning simple biologicai processer,
Level
Advanced Application et Standard Concepts Integration mitiple scientitic concepte. Noderate quant ive ressoning with milti-atep cal stlnne Interpretation of experinental date requizing anclyticsl tainking, Problens requiring deeper unsorstanding beyond menorization. Typiczi of chal lenging
school science competition guestions, Exergles include prebl combining tharnadynenics and xinetics, malti-stop mechorics problens, cociogical relationship analysic.
Level 6: Serly National Olymplad Level Frobiens requiring specinlized knoviedge in specific sciertitic doneins, Application or advanced concepts not typicsily covered in regular curriaulom. Moderate laboratery techniques and experimentei design understanding, Analytical thicking with nen-sbwious solution paths, Typicsi at cariy rounds in natioral science o-ympiads. Exampler include chemicel equilibrion problens with miltiple werisbles, cireuit snalysis with non-ideal components, or molccular hiology mecharisns.
Level
multiple ecientitic consins. Craative application of szandard principles in aon-standard contexts. Anelysia complex experincatal setups and data, Noltiple ccaceptual hurdles requising insight. typical of naticaci olympiad timal rounds, Examples incluse complex organic syntherie pathways, nor-iceal themosynamic pystans, advanced geneties problens.
National olyrpiad Standard Probiens integrating csncepts
Level
Advances Natisnal/Early International Level Prebione requitin deep concoptual understarding beyond standard curticalun, Integration or tseeretical knoviedge with practical laboratory techniquas. Creative prosicm-soiving with muitiple possible approaches, Application of nataenaticsl nodels complex scientitic phenonena. Typical international olympiad proparation camps. Examples include quantinn nechanical nodels, complex bischanical pathways, or atariscical anclysia of bialegical systems,
Level 7: International Olympiad Standard Froblens at the level of Icho, IPhO, or IBC theoreticsl waninaations. Requires specialized knexledoa conbined
creative insicht. Complex quantitative modeling of scicatific pheranena. Integration of concepts accoss scientific disciplincs. Miltiple concestual layers requiring systematic enalysis. Exarples include advanced spectrosespy interoretation, complex physical systens with maltiple forces, or detailed biochenical nechanisn analysis.
Level Advanced International Olyngiad Froblens sequiting beih boeadth and depth of aiient.fic knovladge, Novel apelications of scientific priaciples not typically taucht. sophiscicated experinental design and analytis. Miltiple solution pataways requiring evaleation and selection. Typical of challsnging insecnational olyspiad problens. Examples include challenging quantan chanistry problons, advanced lsberatory protocols with multisle variables, or complex evolationary or ocalegical modeis.
Level 9: alite International Olynpiad - Problens seguiring exceptional seiantific insight and iviativity. Integratien of outting-adre sciencific knowledge. Muitiple conceptual broakthrauchs neaded for solut iun, Problens that shallenge eoen the nost talented students. Beserved for she moxt diffioult ions in international
congeticiona 3xamples include novel application of plysical principles, complex nulti-stop syatheris with sterecthanical consideratins, or systems biology analysls.
Level 10: Fiatorieslly Challenging Problens Prublens of legerdary diffimulyy in science compet ians.. Requizes innovative approaches bayond standard nethodologies. May integrate advanced university- level concepts. Problans that vesy sew comperitors worldwids can solve compiseely often renembered part icularly challanging in olympiad hiatory. Exampley include problens that sequired ion of asproachea that stured alnost all participants
year.
This scals coscesponds roughly to the difficulty progeassion you night sue from suhool science competitions (levels 1-3) through national selection rounda (Levels 4-5) to internacional olymplad problens flavels 6-10).
jubject-Specitet Notes:
Physics (IPh0): lavelx 1-3 cover standard high school physies content Imecaaiiss electricity, thernalynaniual: hevela include advanced Lapics like opties, basic quanaun physics, and non-ideal aystena; Levels 7-10 incosporate university leval content including quantun nachanios, statistioal physics, and relat relativity.
Cheniatry (TCHOI: levels 1-3 cover basic inorganis, organio, onelycical chenistry concepti; include complex reaction mechanisns, advarced analytical nethoda, physicel ster levela 7-10 incosporate sophiat icated labosatory melleds, quantun chenistty, and est edge chenlcal concegts.
Biology (IB0)s Levels 1-3 cover basic callular, nolesulas, and organianal biologys Levela inolude advanced de cullr processes, genetics, evol mnllutionar biology, and ecology; Tevels 3-10 incorporete curplex esperimental desigr, advanced biochamisiny, systems blology, and bioinforne iex. Problem to be labeled: (iquestion1s."
Figure 28: Prompt for Science Difficulty Filtering. This text is the prompt for Science Difficulty Filtering.
60

---

Published as a conference paper at ICLR 2026
I will provide you the last words of 16 math problem solutions. They candicate solutions problem.
They will typically contain the solution to a math problem. want you to return the indices of the responses with the most common final numerical answer.
Only the final numerical
matters.
Question: What is 5?
Solution answer is 15.
Solution 1:
15.0 is the solution this problem.
Solution The answer is 14.
You would return: [0, 11 since they both saying 15 is the answer only one response is saying the answer.
Here is your question: ((question))
Here are your canoidate solutions: I1list of all solutions]]
Now tell me the solutions. Please remember to zero index these solutions. Do not include the nunber 16 as index.
Figure 29: Prompt for Math Question-Answer Majority Consensus Verification. This text the prompt for Math Question-Answer Majority Consensus Verification
1s the provided answer correst solution to the following ng probl em?
Question: [questicn)| Response:
{ [[answer]]
Figure 30: Prompt for Math Question-Answer GPT Verification. This text is the prompt for Math Question-Answe GPT Verification
Comprehensive Large Dataset: Take all 63,200 responses for one dataset without filtering.
This not compute-controlled.
Shortest Answers Selection: For any given question, take the shortest responses out of the 16 responses and create 8 question-answer pairs for the dataset.
Python Tag Based Selection: Filter out any questions that don't have python tags:
"""pthhon.
For any given question, provide all responses GPT-40- mini. Ask GPT-4o-mini to return a list of indices of responses that agree with the majority. Use temperature 1.0. The prompt is in Figure 35.
Majority Consensus Selection:
Longest Answers Selection: For any given question, take the 8 longest responses out of the 16 responses and create 8 question-ansse annwerr pairs for the dataset.
61

---

Published as a conference paper at ICLR 2026
GPT Verification: Ask GPT-4o-min -ini whether provided answer is the correct answer for the provided question. The sampling hyperparameters are temperature of 0.0, top p of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and a reasoning. which is a string. The full prompt in Figure 31.
Removing Non-English Answers: Ask GPT-4o-mini whether a provided answer contains only English. The sampling hyperparameters
temperature of 0.0, top_p P of 1.0, and presence penalty of 1.0. We use structured decoding to get a response, which is a boolean value, and reasoning. which is string. The full prompt is in Figure 32.
Random Filtering: Filtering questions-answer pairs randomly.
Removing Long Paragraphs: Ask GPT-4o-mini whether provided answer is free of
long paragraphs. The sampling hyperparameters are temperature of 0.0, top_ of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and reasoning, which is string. The full prompt in Figure 33.
0.4.3 SCIENCE QUESTION ANSWER FILTERING
FastText Selection: Classify question-answer pairs with a FastText filter. Form the query strings by the following format: "Question: [question] \nAnswer: ponswer_columnn We do this for both training and using the FastText classifier. The classifier's positives are S1.1, which contains responses from DeepSeek R1. The classifier's negatives from anffoundations-de//staaasos verified_ mix annotated with GPT-4o-mini. More info is in Appendix 0.2.1.
Comprehensive Large Dataset: Take all 63,200 responses for one dataset without filtering. This not compute-controlled
Longest Answers Selection: For any given question, take the longest responses out the 16 responses and create 8 question-answer pairs for the dataset.
Removing Non-English Answers: Ask GPT-4o-mini whether a provided answer contains only English. The sampling hyperparameters are temperature of 0.0, top_ of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and reasoning, which is string. The full prompt in Figure 32.
Random Filtering: Filtering questions-answer pairs randomly.
Shortest Answers Selection: For any given question, take the 8 shortest responses out of the 16 responses and create 8 question-anss -nnooer pairs for the dataset.
Removing Long Paragraphs: Ask GPT-4o-mini whether provided answer free of long paragraphs. The sampling hyperparameters are temperature of 0.0, top_p of 1.0, and presence penalty of 1.0. We use structured decoding to get a response, which is boolean value, and reasoning. which is string. The full prompt is in Figure 33.
Majority Consensus Selection:
For any given question, provide all responses to GPT-4o- mini. Ask GPT-4o-mini to return a list of indices of responses that agree with the majority. Only use the last 1, 000 characters of each response to get the final answer. Use temperature 1.0. The prompt in Figure 34.
GPT Verification: Ask GPT-4o-mini whether provided answer is the correct answer for the provided question. The sampling hyperparameters are temperature of 0.0, top_p of 1.0, and presence penalty of 1.0. We use structured decoding to get response, which is boolean value, and reasoning, which is string. The full prompt is in Figure 30.
62

---

Published as a conference paper at ICLR 2026
Is the provided code snippet correct sclution following problem?
Question: ((question))
Response: [(nswer)]
Figure 31: Prompt for Code Question-Answer GPT Verification. This the prompt for Code Question-Answe GPT Verification
Does provided answer only contain one language?
Question: [[questinn]] Response: [(answer))
Figure 32: Prompt for English Verification. This text is the prompt for English Verification.
Is the provided answer free of any long paragraphs? A paragraph is any block of text separated by a blank line. paragraph is *to0 long* if it has **>750 words**
Question: [|questien) Response: [[answer]]
Figure 33: Prompt for Long Paragraphs Verification. This text is the prompt for Long Paragraphs Verification.
will provide you the 1ast words of 16 science problen em sol polutiens They are candićate soluti ons to problem.
They will typically contain the solution to science problem. want you to return the indices of the zesponses with the most common answer.
Hore is your question: ((question))
Here your candidate solutions: (1list of all solutions]
Now tell the solutions. Please remember to zero index these solutions. Do not include the number 16 as index.
Figure 34: Prompt for Science Question-Answer Majority Consensus Verification. This text is the prompt for Science Majority Consensus Verification
63

---

Published as a conference paper at ICLR 2026
will provide you sanp_san.les.
They are candidate solutions to a coding probien.
list indices cozresponding comman solutions that funccionally equivalent. thare of solationt iora Lhat asw of the same sime, pick use of the sels at ramion, sent you to elau provide your reasening
the samples that constitute code samples functionally
you conpase
for indicts
beine fuactionally equivaleat. Bare examiie:
Question: Solve Fizsbuzz.
Sulatiion fiazbussl for i in range (1,, 11: output -
output 1- rPize! $ 0:
autput "Boze!
print coutput 11
Bolution.
def Fixeburee
fox i in range (l, 11:
("Pinsnuzz" #1 1 9 psint ("Buxx")
print ("Fine")
elaes print (1)
Solution
def Fixrbarx3(n11
for i in range 11:
Maltiple logical errors:
brackets for sinple if
print ('Fize' elif i 0:
print ("Buxx")
Krong Logic operato:
grint ('Finzhurss)
cise: (1)
You vould retern: [6, 1) since they are functionslly equivalent bet the third respense is different.
Here questiion: [questton]]
Beze are your candidate soluticas: Hilst solutions:
Figure 35: Prompt for Code Question-Answer Majority Consensus Verification. This text the prompt for Code Question-Answer Majority Consensus Verification
64

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Question Generation Strategy
Average
Code Avg
Math Avg
Science Avg
StackExchange CodeGolf
38.8
25.3u.6
50.9 1.1
40.70.5
ovidia/OpemnCoddeResssmm
38.4
27.5u.4
47.9u.7
40.70.6
KodCode/KodCond V1
37.7p.3
23.9n.4
49.8n.7
40.40.3
coguitivccompptttooo/dddppphiiie
37.1 0.5
20.8n.4
49.2
43.30.7
Pratpp/CooeFcccacck.....
37.00.4
18.90.3
51.8 1.1
41.80.6
Muttinguui-MMllimndd--APAMNau
35.40.3
16.20.4
09.5
42.90.6
OpenCColerLLL//ipcsffttttaaz/
35.40.4
16.70.2
51.0
39.8p.5
gjibawa-2023/CCodd229----
35.1u.4
18.50.4
49.4
38.7
christoppher/rosettta-code
35.0u.4
14.70.3
49.60.7
43.31.0
glaivalglaave--od--ssisstt----
35.0p.3
16.60.3
50.2
40.0o.3
prihiv-il..odd/Cooee--ttta
34.20.4
14.10.7
49.70.9
41.20.7
ise-
se--uiuc/Magicodder-reeerro OSS
33.90.5
17.70.4
47.41.0
37.91.2
codeparrot/apps
33.5
15.60.6
49.80.9
35.80.8
StackExchange Codereview
33.40.4
13.50.5
48.40.9
40.7
32.8p.4
12.00.5
48.00.9
41.1g.5
bigcode/commitpacktt
32.1
12.2
46.90.9
39.61.6
deepmind/code__omme contests
31.8
18.50.5
45.1
31.80.5
31.5p.4
14.80.4
45.60.9
35.2
MarrixSuudioCCooefffree---tiiiie
31.4u.5
19.30.4
39.31.
37.71.1
MMepii-Alig/MMggii---vve...
31.20.5
12.30.4
47.81.3
34.61.1
bigoddeseffooss-issruccs.......
31.2
13.70.4
16.
35.00.6
30.5g.4
10.70.2
45.71.3
37.40.7
StackExchange
30.30.4
8.50.4
47.5
37.30.3
28.50.4
7.80.3
45.5
34.00.5
Primehntelleesstaceeechhanggg
27.60.3
5.90.2
46.81.0
31.60.5
Primelntellecd/yyntheiic
27.20.3
2.2
45.50.7
37.2
bugdaryanfsgl-creatte...
21.60.6
7.0p.7
34.11.4
24.70.9
Table 31: Full Ablation for Code Question Sources
P PIPELINE EXPERIMENTS EXPANDED RESULTS
Due to spatial constraints, we do not show every table and every data strategy in the main text. We elaborate and show the full results here.
P.1 QUESTION SOURCING
Our results for the code question sourcing ablation are in Table 31. Our results for the math question sourcing ablation are in Table 32. Our results for the science question soureing ablation are in Table 33. The gap between the top-performing datasets and the lowest-performing datasets is large. In the code data domain, this difference is 17.2 points on average
65

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Question Generation Strategy
Average
Code Avg
Math Avg
Science Avg
n12=adapt-eee//ppemmtth22 2-mtm
38.10.3
12.4
58.8
45.60.2
A-MOONuminaMath-155 openmathinstrict aime"
37.40.5
11.40.5
58.51.0
45.0
37.20.5
12.5
57.10.9
44.31.4
GAIR/MathPile
36.20.5
11.50.7
55.10.9
44.61.1
MetaMath from Numina"
35.80.0
10.90.8
56.31.4
42.5
math-ai/AutoMathh
35.7
8.7
55.50.s
46.60.8
mvddaa/OpenMMathhnnnnnee22 Aime
35.40.4
10.70.4
55.81.2
41.80.4
zwhee9/DeepMath-1003
34.8
7.20.4
55.61.9
44.8
ddrg/namedd_math_foomm
33.90.4
10.30.5
53.9
39.4n.4
TTGFR--LabbMahhnssruutt
33.8
12.4
50.60.8
40.8
avidia//OpeuMaathhnntnuut-2
33.50.5
7.90.5
55.2
39.11.0
facebook/nnatural reasoning
33,4
7.6
52.11.0
43.9
SynthlabssA//iig-Matth
33.0
9.40.2
53.21.0
38.1p.4
TIGER-Lab/MATHH.p
32.7a.4
8.80.3
51.60.9
40.3
Asap7772henddrckk-matt--
32.30.4
8.90.5
52,41.1
37.4
httphi/mmat_jiistruete
30.60.4
8.2
48.21.0
37.60.7
ajbawa-2023/Maths-SC College
30.1o.4
2,40.2
51.61.1
39.5
BAAL//nfinitMAAHH
29.70.3
7.40.3
47.61.0
36.50.2
MetaMath
29.30.4
6.10.5
48.41.2
35.3
allenai/math qa
27.80.4
4.90.3
46.4a.8
34.11.1
deepmind/math dataset
25.9
5.10.2
43.21.2
31.1p.8
Lap Laplofficial/Math
24.40.3
7.30.3
38.61.0
28.5
Table 32: Full Ablation for Math Question Sources
SFT Datasets
Benchmarks
Question Generation Strategy
Average
Code Avg
Math Avg
Science Avg
StackExchange Physics
34.3
11.9 0.5
50.9
43.2p.7
Organic Chemistry PDF Pipeline
34.0p.3
8.4g.3
52.19.7
45.3
mtcb/cqaduusstuck-lyye physscss camel-ai/physics
33.3
7.4
51.9 1.1
44.10.9
30.9
8.60.2
48.0
38.91.2
Josephgffoweer/Par--Weee -aaeeees/Paaarr-r-iiiieeeeeeee Four-Fineweb.....
30.9
8.2
48.4
38.80.6
mattany//iikippeiaa-blllogg
29.30.4
5.20.2
47.3
38.40.7
millawell/wikipediia wttpedddddiiiilooo_eeeeeeeee feld_oo_scieen
29.10.4
4.8
47.7
37.50.9
zeroshot/arxiv-biokke
28.7 28.7u.4
5.5
46.7u.9
36.41.2
camel-ai/chen aichemis//
27.8
3.6p.3
46.4
36.10.7
Sangeetha/ Kaggle.
27.5
6.0
43.8
35.2
marcov/pubmed. qa......................
25.6n.5
5.5
41.8
31.4
StackExchange Biology
25.1
4.0
39.6
34.8
camel-ai/biology AdupterOOcca//iology dataset...
25.0p.4
2.70.2
44.1
29.71.0
21.9
3.10.3
41.3
21.10.8
Table 33: Full Ablation for Science Question Sources
66

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Mixing Strategy
Average
Code Avg
Math Avg
Science Avg
Top 2 Code Sources
41.30.4
27.3u.3
54.70.9
42.1
Top 1 Code Sources
39.9u.6
23.11.0
54.5u.8
43.1
Top 4 Code Sources
38.60.4
24.20.6
52.2n.8
39.8p.9
Top 8 Code Sources
37.00.4
21.80.3
51.91.2
37.70.6
Top 16 Code Sources
36.40.4
20.80.4
50.10.9
39.11.0
Table 34: Full Ablation for Code Question Source Mixing
SFT Datasets
Benchmarks
Mixing Strategy
Average
Code Avg
Math Avg
Science Avg
Top 1 Math Sources
37.60.5
12.10.5
60.1
41.90.7
Top 8 Math Sources
35.80.5
12.60.4
54.80.9
42.3
Top 4 Math Sources
34.7a.3
9.0g.2
56.0u.9
41.40.5
Top 2 Math Sources
34.30.3
6.90.3
55.9u.9
43.0p.5
Top 16 Math Sources
33.8p.9
11.20.4
50.3g.8
43.1
Table 35: Full Ablation for Math Question Source Mixing
SFT Datasets
Benchmarks
Mixing Strategy
Average
Code Avg
Math Avg
Science Avg
Top Science Sources
33.7p.4
9.5g.3
50.3
44.8
Top Science Sources
33.6p.5
12.0p.3
49.60.9
42.01.3
Top Science Sources
32.2p.4
8. 1o.2
49.3
42.50.6
Top 8 Science Sources
31.10.4
6.40.4
49.0
41.40.9
Top 16 Science Sources
30.8p.3
6.70.2
48.41.0
40.3u.6
Table 36: Full Ablation for Science Question Source Mixing
P.2 MIXING QUESTION GENERATION STRATEGIES
Our results for the code question mixing ablation are Table 34. Our results for the math question mixing ablation are in Table 35. Our results for the science question mixing ablation are in Table 36. The trend of less mixing being better holds across all domains.
67

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
Difficulty-baid based Selection
43.0
27.70.4
56.0,.a
46.4u.r
Length- baath- Selection (GPT-4.I-nano)
42.2
26.60.5
55.4
46.0u.2
AskLLM Selection
41 41.60.5
28.80.5
52.11.2
45.2
Length-based Selection (GPT-400mini)
40.8
25.60.5
53.1
45.2 1.1
FastText (P: Codeforces; N; CodeReview)
40.50.3
26.30.1
53,90.9
41.80.5
Length-bar based Selection (GPT-4.I-ming)
40.5
26.80.3
51.30.8
44.90.8
Random Selection
39.7p.s
28.60.5
50.21.3
40.5u.7
FastText (P: LeetCode; N: SQL)
39.6p.4
25.20.5
52.81.0
41.70.5
FastText (P: Code Golf; N: SQL)
39.0p.s
23.90.5
52.41.3
41.5u.7
FastText (P: All; N: SQL)
38.90.5
26.30.4
50.5
40.7 1.2
FastText (P: IOI; N: SQL)
38.50.4
24.80.4
50.2
41.40.7
FastText (P: Codeforces; N: All)
38.50.4
25.1o.1
50.91.1
39.90.8
FastText (P: Codeforees; N: SQL)
38.40.4
25.00.2
50.0
41.10.6
Embedding-basee Selection
36.90.6
16.20.4
53.4
43.1
Table 37: Full Ablation for Code Question Filtering
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
Length-based Selection (GPT-4.1-ming
41.9
13.4
66.0p.s
48.60.4
Length-based Selection I-nano)
39.40.3
11.0p.4
p4.
44.3u.7
AskLLM Selection
36.30.4
9.50.5
58.1
43.8n.6
FastText (P: Numina; N: Lapl
35.60.4
11.0p.2
54.9
43.5n.8
Difficulty- Selection
35.5
8.20.3
59.31.4
40.81.1
Embedding Selection
35.40.4
6.30.3
57.01.0
46.50.9
Random Selection
35.2
8.
56.61.3
43.8
FastText (P: $1.1; N: Laplofficial
34.90.4
10.8
55.5
40.5
FastTexx (P: Olympiad: N: Laplofficial
34.9
8.7g.3
56.21.0
42.2
FastText (P: OpenRI; N: Lapl official)
34.40.4
8.70.3
55.1
41.7
FastText (P: All; N: offfciiia
34.40.4
9.80.5
54.3
41.40.4
FastText (P: Numina; N: All)
32.80.4
7.8p.2
53.91.2
38.50.4
FastText (P: Numina; N: Natural Reasoning)
32.6
6.80.4
53.91.2
39.41.5
Length-based Selection (GPT-4o-mini)
6.80.3
2.30.4
10.2
8.4
Table 38: Full Ablation for Math Question Filtering
P.3 FILTERING QUESTIONS
Our results for the code question filtering ablation are in Table 37. Our results for the math question filtering ablation are in Table 38. Our results for the science question filtering ablation are Table 39, For each data domain, we each question filtering strategy from Appendix O.2. The ablations contain different combinations FastTTxx positives and negatives. They also include various models and GPT-4. I-nano. Across both
for the length-based filtering, such as GPT-4o-mini, GPT-4.1 science and math, length-based filtering with GPT-4.1 models works well. Around half of the filtering strategies improve over random filtering for each data domain. On all data domains, AskLLM filtering and difficulty- based filtering work relatively well.
68

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
Length-based Selection (GPT-4.1-mini
35.9
7.30.5
54.4
50.9u.9
Length-based Selection (-PTo4.I-nano)
35.6p.3
7.70.3
53.7p.7
50.2
AskLLM Selection
35.3
11.1g.1
51.5
47.2
FastText (P: SciQ; N: Wikipedia w/ Arxiv)
35.10.5
9.3p.3
51.8
48.7
Embedding-based Selection
34.90.4
9.70.4
51.21.0
48.41.0
Length-based Selection (GPT-4o-mini)
34.90.3
10.80.3
52.9
44.30.6
FastText (P: SCP, SciQ, ExpertQA: N: Arxiv)
34.40.6
9.50.4
51.3
46.41.7
Random Selection
33.80.4
8.10.4
52.11.1
44.60.6
FastText (P: SciQ; N: Wikipedia w/ Arxiv)
33.5p.3
8.20.3
51.31.0
44.60.4
Difficulty-basee Selection
33.5o.4
7.70.4
50.7
46.4
FastText (P: SciQ; Wikipedia w/ Arxiv)
33.40.4
10.2
50.7
42.20.5
FastText (P: ExpertQA: N: Arxiv)
31.50.4
11.4
45.9
40.20.8
Table 39: Full Ablation for Science Question Filtering
69

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Annotation Strategy
Average
Code Avg
Math Avg
Science Avg
Exact Dedup w/ 4x sampling
p1.3
28.6u.4
53.5,
42.1
No Dedup w/ 16x sampling
41.3u.4
25.9u.3
53.8 1.2
45.8
No Dedup w/ 4x sampling
41.2n.4
27.3n.3
54.3
42.20.7
Fuzzy Dedup w/ 4x sampling
41.2p.4
28.1
51.8
45.0p.6
Fuzzy Dedup w/ 16x sampling
41.20.4
24.80.4
54.3
46.10.4
Exact Dedup w/ 16x sampling
40.60.5
25.50.4
53.5 1.2
44.1
No Dedup w/ 1x sampling
39.8p.6
26.50.4
53.0
39.91.5
Table 40: Full Ablation for Code Deduplication and Multiple Sampling
SFT Datasets
Benchmarks
Annotation Strategy
Average
Code Avg
Math Avg
Science Avg
Exact Dedup w/ 1x sampling
41.7u.x
14.0
66.0
46.60.2
Exact Dedup w/ 16x sampling
40.1o.4
11.0p.a
63.61.3
48.70.5
Exact Dedup w/ 4x sampling
39.2n.4
12.0
62.6
44.70.5
Fuzzy Dedup w/ 4x sampling
38.81.2
12.4p.5
60.63.9
45.70.9
No Dedup w/ 1 x sampling
38.3
10.8
59.83.7
47.4
No Dedup w/ 4x sampling
37.70.4
3.3p.3
64.61.2
49.00.6
Fuzzy Dedup w/ 1x sampling
37.4
9.3
60.6
44.82.0
No Dedup w/ 16x sampling
36.51.2
13.90.4
55.43.9
42.1
Fuzzy Dedup w/ 16x sampling
36.0p.4
5.5p.2
61.11.1
43.9
Table 41: Full Ablation for Math Deduplication and Multiple Sampling
SFT Datasets
Benchmarks
Annotation Strategy
Average
Code Avg
Math Avg
Science Avg
Exact Dedup w/ 16x sampling
36.2n.5
9.0p.4
54.5
49.7
Fuzzy Dedup w/ 16x sampling
36.1 n.4
10.9
52.9 1.3
48.8p.5
Exact Dedup w/ 4x sampling
35.8
10.6
51.81.0
49.61.2
No Dedup w/ 4 x sampling
35.8
10.0g.4
55.2
45.40.9
No Dedup w/ 16x sampling
35.70.4
7.6p.5
53.8
50.9
No Dedup w/ 1 X sampling
35.5
9.3p.3
54.2
46.90.2
Exact Dedup w/ x sampling
35.0p.4
7.6p.4
54.0
47.50.5
Fuzzy Dedup w/ 4x sampling
34.9p.4
7.40.5
55.0
46.0p.7
Fuzzy Dedup w/ 1 sampling
34.2p.a
5.8p.4
52.5n.7
49.5o.4
Table 42: Full Ablation for Science Deduplication and Multiple Sampling
P.4 DEDUPLICATION AND MULTIPLE SAMPLING
Our results for the code deduplication and multiple sampling ablation are in Table 40. Our results for the math deduplication and multiple sampling ablation are Table 41. Our results for the science deduplication and multiple sampling ablation are in Table 42. Across all data domains, doing exact deduplication or no deduplication was better than doing fuzzy deduplication. Moreover, doing multiple sampling performed main exception here is in Table 41, where doing annotation per question is better. However, the second-best strategy empirically is doing exact deduplication with 16x sampling. The 16x provides OpenThoughts:: hhoughts3-1 1.2M, so we chose this annotation strategy for our
well or equal to annotating each question time. The
an axis for scaling data more for Open'l
final pipeline.
70

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
FastText Selection
42.3
27.20.5
54.7
46.50.3
No Filtering
42.2
27.4
54.5
46.1g.9
Shortest Answers Selection
42.00.5
26.50.5
54.4
46.9 1.1
Python Tag Based Selection
41.70.5
27.80.3
54.61.1
43.4
Majority Consensus Selection
0..30.5
26.60.2
53.61.2
45.01.0
Longest Answers Selection
41.00.4
26.90.4
55.4
40.5
GPT Verification
40.70.4
25.50.4
53.6p.8
44.21.0
Removing Non-Enge Non-English Answers
40.60.4
26.20.5
54.51.1
41.60.6
Random Filtering
39.8p.4
25.10.s
52.21.1
43.0u.5
Removing Long Paragraphs
30.30.5
19.60.2
40.5
31.30.9
Table 43: Full Ablation for Code Question-Answer Filtering
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
No Filtering
41.90.
15.20.5
65.60.9
46.4n.7
Random Filtering
41.6
14.9 0.4
64.8
46.70.s
Shortest Answers Selection
41.10.4
14.8
63.71.1
46.70.7
Removing Non-English Answers
41.1o.5
14.20.5
63.1
48.6
Majority Consensus Selection
41.00.4
14.5p.s
62.3p.s
48.8
FastText Selection
40.70.s
13.50.2
62.8
48.4
Longest Answers Selection
40.5o.5
12.90.4
63.9
46.7
GPT Verification
40.0g.5
13.1
61.4
48.3
Removing Long Paragraphs
38.0p.4
5.70.2
64.5
46.8
Table 44: Full Ablation for Math Question-Answer Filtering
SFT Datasets
Benchmarks
Filtering Strategy
Average
Code Avg
Math Avg
Science Avg
No Filtering
38.3
10.6
56.9
51.9
Longest Answers Selection
37.5
12.0
55.4
48.8 48.80.7
Removing Non-Ehg Answers
37.40.4
13.4
54.5
47.6u.9
FastText Selection
36.8p.4
11.5p.4
54.71.1
47.7u.3
Random Filtering
36.5o.4
11.0p.4
53.9
48.8p.7
Shortest Answers Selection
36.20.5
11.40.7
53.61.2
47.5n.4
Removing Long Paragraphs
35.9 0.5
11.40.6
53.71.0
45.71.2
Majority Consensus Selection
35.70.5
9.10.5
54.6
47.10.9
GPT Verification
35.60.5
11.4g.5
52.61.3
46.41.0
Table 45: Full Ablation for Science Question-Answer Filtering
P.5 QUESTION ANSWER FILTERING
Our results for the code question- question-answer answer filtering ablation are in Table 43. Our results for the math question-answer filtering ablation are in Table 44. Our results for the science question-answer filtering ablation are in Table 45. Across all data domains, not doing filtering at all performs similarly or outperforms the best question-answar filtering strategy.
71

---

Published as a conference paper at ICLR 2026
SFT Datasets
Benchmarks
Teacher Models
Average
Code Avg
Math Avg
Science Avg
Qwen/QwQ-322
44.20.5
29.5
58.71.1
44.61.0 .
38.01.s
19.23.4
54.31.6
41,8
microosti/hhi-4rreaonnnnngggg
29.0p.4
0.5g.
52.11.2
37.20.6
Table 46: Full Ablation for Teacher Model for Code
SFT Datasets
Benchmarks
Teacher Models
Average
Code Avg
Math Avg
Science Avg
Qwen/QwQ-32H
44.20.4
10.90.4
71.6
53.2
Seepseek-ai/DeepSeeee
40.60.3
13.3
62.5
48.5
microsoti/Phii44reasonnnng
30.60.6
7.10.4
49.0
38.20.9
Table 47: Full Ablation for Teacher Model for Math
SFT Datasets
Benchmarks
Teacher Models
Average
Code Avg
Math Avg
Science Avg
Qwen/QwQ-32B
39.10.4
10.10.5
62.1
48.0p.2
ainbseee-ai/DeepSeee-.Re
35.9p.7
7.1
55.90.9
49.0a.4
mrcrosi
21.70.4
4.8g.2
28.81.3
36.40.7
Table 48: Full Ablation for Teacher Model for Science
P.6 TEACHER MODEL EXPERIMENTS
Our results for the teacher model ablations for code are in Table 46. Our results for the teacher model ablations for math are in Table 47. Our results for the teacher model ablations for science are in Table 48. Across all data domains, QwQ-32B is the best teacher by a statistically significant margin.
72


Reviews:
[{"review_id": "tNr9sFn1L0", "reviewer": "Reviewer_wnkN", "summary": "This paper introduces OpenThoughts2-1M and OpenThinker2-32B, the first publicly available open source dataset and model for reasoning tasks that match the performance of DeepSeek-R1-Distill-32B on AIME and LiveCodeBench. They also perform a series of 1000+ experiments to systematically improve their data generation pipeline to develop OpenThoughts3, which, combined with scaling to 1.2M examples and using QwQ-3B, yields OpenThinker3-7B, which vastly outperforms DeepSeek-R1-Distill-Qwen-7B on AIME, LiveCodeBench, and GPQA. Key findings include that sampling multiple answers from a teacher model is an effective strategy to scale the size of the training data, models with better performance are not necessarily better teachers, verification and answer filtering methods don’t lead to significant performance improvements, data quality trumps data diversity, and filtering questions based on LLM labeled difficulty or response length yields better results than using embedding based filters typical to pre-training.", "questions": "Do you explore potential explanations of the QwQ being the best teacher model as an observation of the capacity gap phenomenon [1]?\n\n[1] Zhang, Chen, et al. \"Towards the law of capacity gap in distilling language models.\" arXiv preprint arXiv:2311.07052 (2023).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Well-motivated and carefully designed experiments leading to clear takeaways about best practices for SFT training data design for reasoning models.\n2. OpenThinker3-7B achieves the best average performance across all the evaluation benchmarks.\n3. Efforts to decontaminate training data by removing samples with high similarity to the benchmark instances. \n4. Interesting results: Question sourcing (simple synthetic questions perform comparably or even better than complex or manually curated pipelines), Question filtering (difficulty filtering and response length filtering work well for code and math, compared to fasttext classifiers or embedding-based methods that work well for pre-training), Best teacher model is not necessarily the best performing model.", "weaknesses": "1. Should further investigate the impact of deduplication and sampling multiple answers because the results are very inconclusive, and exhibit too much variance across code, math, and science. It is also weird that the authors choose varying deduplication strategies across math, code, and science, but choose to pick x16 answer sampling per question for all domains, even though it is not the best across the board. This choice seems arbitrary, and the reasoning that it is better for scaling seems weird since training on much more data for essentially the same or worse performance is potentially just a waste of compute.\n2. Again, the conclusions about answer filtering seem weird, since they bring up the point that training on all the data instead of filtering low-quality data makes no difference, so training on all the data is better for scaling. However, an alternative perspective would be that training on these low-quality instances doesn’t add to the performance, so why should one waste compute on training with these instances? In other words, answer filtering strategies could be used to find a smaller, more effective dataset that retains the same level of performance as the full dataset.\n3. The difficulty filters used for answer filtering do not use any verifiers, like code execution with test suites, etc., which, while hard to scale, have shown a lot of promise in rejection fine-tuning-based pipelines (like RAFT) [1, 2, 3]. \n4. There is seemingly a big gap in performance between the best-performing OpenThoughts3-7B model and the teacher models used (QwQ, DeepSeekReasoner). While this is reasonable given the parameter size difference, the 10-point difference is pretty significant.\n5. Some of the more intriguing results warrant further exploration, like why a less capable teacher model is a better teacher or why different question filtering strategies work for specific domains.\n\n[1] Zheng, Kunhao, et al. \"What Makes Large Language Models Reason in (Multi-Turn) Code Generation?.\" arXiv preprint arXiv:2410.08105 (2024).  \n[2] Xiong, Wei, et al. \"A minimalist approach to llm reasoning: from rejection sampling to reinforce.\" arXiv preprint arXiv:2504.11343 (2025).  \n[3] Dong, Hanze, et al. \"Raft: Reward ranked finetuning for generative foundation model alignment.\" arXiv preprint arXiv:2304.06767 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WWYYx4NjW8", "reviewer": "Reviewer_1PHv", "summary": "This paper presents OpenThoughts3, a scalable and systematic data curation pipeline for building reasoning models through supervised fine-tuning. The authors carefully analyze each step of the data generation process—question sourcing, filtering, deduplication, answer sampling, and teacher selection—through more than 1,000 controlled experiments, resulting in a high-quality open dataset of 1.2M examples. The resulting model, OpenThinker3-7B, achieves state-of-the-art results among open-data models, surpassing comparable baselines like DeepSeek-R1-Distill-7B and Nemotron-Nano-8B across math, code, and science benchmarks. The paper emphasizes that careful dataset design can rival or exceed RL-based approaches, and provides a reproducible open-source recipe for training reasoning models.", "questions": "Please refer to Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe work excels in its thorough dissection of the data curation process, covering multiple strategies at every pipeline stage. This level of experimental rigor goes beyond most existing work, making the conclusions highly credible and reproducible.\n2.\tBy scaling the dataset to 1.2M examples and carefully selecting teacher models, the authors deliver competitive results on multiple reasoning benchmarks. The scaling curves and ablation studies clearly demonstrate the effectiveness of their approach.\n3.\tUnlike many proprietary reasoning models, the authors commit to full open-source release of datasets and models. This can significantly accelerate community research and lower the entry barrier for reasoning model development.", "weaknesses": "1.\tI appreciate the design of the SFT data pipeline, but it’s hard not to notice how the entire work leans almost exclusively on this single training paradigm. In recent reasoning models, RL or curriculum strategies often play a big role in pushing performance further. Even if the authors didn’t run those experiments, a more thoughtful discussion or positioning would have made the contribution feel less one-dimensional.\n2.\tI’m left unsure whether the gains translate to broader reasoning capabilities. A few results on less standard or more language-heavy tasks would help a lot here. Right now, the claims around generalization feel more suggestive than demonstrated.\n3.\tThe paper runs a huge number of experiments, but says very little about why certain design choices actually help. For instance, why does a relatively simple teacher mix outperform a seemingly stronger one? What kinds of examples drive the improvements? Without some interpretability or qualitative perspective, the work risks feeling like an “empirical recipe” rather than a deeper contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vd5rhMk09V", "reviewer": "Reviewer_KbX7", "summary": "This paper presents a large-scale empirical study aimed at identifying effective data recipes for improving LLM reasoning performance. The authors systematically analyze the impact of various data curation strategies, including question sourcing, filtering, deduplication, teacher selection, and dataset scaling. Many controlled experiments are conducted to evaluate how different design choices affect model reasoning ability under supervised fine-tuning. The paper also introduces an open dataset and provides detailed ablations to ensure reproducibility and transparency.", "questions": "I do not have specific questions for the authors.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper constructs a large and well-curated dataset specifically designed to enhance LLM reasoning capabilities. This contribution is practically meaningful, as data quality and composition have become increasingly crucial for reasoning-oriented LLM development. The released dataset and accompanying analysis provide a useful foundation for future research in reasoning and instruction tuning.\n\n- The empirical study is extensive and carefully executed. The authors evaluate a wide range of data curation factors, such as filtering, question mixing, deduplication, teacher selection, and dataset scaling. The conclusions are supported with detailed ablation studies. The experiments are systematic and transparent, offering clear evidence for each design choice and contributing valuable insights for the broader community.", "weaknesses": "- This work feels closer to a technical report than a research paper. The study is solid and well-executed, but it lacks significant technical novelty. The proposed pipeline is straightforward and largely follows existing practices in data curation and reasoning dataset construction, with limited conceptual innovation.\n\n- The proposed dataset provides only marginal improvements over existing baselines. As shown in Table 1, the average performance gain over previous datasets is relatively small (around 2.1), which raises questions about the practical significance of the proposed data recipes. While the results are consistent and reproducible, the improvement is quite modest and may not justify the large experimental effort.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vUCYB5tMWM", "reviewer": "Reviewer_1sri", "summary": "The paper addresses the challenge of creating publicly available data recipes for training reasoning models. It introduces a data generation pipeline for creating open-source reasoning data by empirically selecting the most effective approach at each stage. Using this pipeline, the authors construct the OpenThoughts2-1M and OpenThoughts3 datasets. A model trained on this data, named OpenThinker3-7B, achieves state-of-the-art performance.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "S1: The paper conducts thorough experiments and constructs open-source datasets, which paves the way for future research on reasoning models.\n\nS2: The paper is well-structured and easy to follow.\n\nS3: The paper addresses a practical data problem for reasoning model training and research, which is an important contribution to the development of AI.", "weaknesses": "W1: The paper's aim to create open-source datasets is undermined by the proposed pipeline's high dependency on closed-source LLM APIs, such as GPT-4o. This creates a contradiction and harms the reproducibility of the work, as results can vary significantly depending on the specific API version used.\n\nW2: Some experimental conclusions lack rigor. For instance, in Section 3.6, the paper dismisses answer filtering strategies as ineffective because they did not outperform the baseline. However, this conclusion is not well-supported because the comparison is unfair: the baseline was trained on 63,200 samples, while the answer filtering strategies used only 31,600. This confounding variable makes it impossible to rule out that the baseline's superior performance stems from having more training data, rather than the ineffectiveness of filtering.\n\nW3: The paper lacks an in-depth analysis of some interesting and counterintuitive experimental results, failing to provide deeper insights. For example, Section 3.7 shows that while Qwen-32B has a lower average score than DeepSeek-R1, it outperforms all other models when used as a teacher. This phenomenon is counterintuitive, and a more thorough analysis of why a lower-scoring model makes a better teacher could offer valuable insights for future work.\n\nW4: While it is understandable that some experimental results are placed in the appendix due to space limitations, the paper frequently has a significant separation between the presentation of results and their corresponding analysis. This disjointed structure makes the paper difficult to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)