Learning to Solve Complex Problems via Dataset Decomposition
Authors: Wanru Zhao
Lucas Caccia
Zhengyan Shi
Minseon Kim
Weijia Xu
Alessandro Sordoni
Venue: NeurIPS 2025 poster ()
Keywords: language models
decomposition
math reasoning
Primary area: deep_learning

Abstract: Curriculum learning is a class of training strategies that organizes the data being exposed to a model by difficulty, gradually from simpler to more complex examples. 
This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components.
We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system to measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. 
Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared to standard training on original datasets.

Learning to Solve Complex Problems via Dataset Decomposition
Wanru %
Lucas Caccia
Zhengyan Shi
Minseon Kim
Weijia Xu
Alessandro Sordoni
University of Cambridge Miirosooft Research 'Mila Quebee Al Institute
Abstract
Curriculum learning class of training strategies that organizes the data being exposed to model by difficulty, gradually from simpler to more complex examples. This research explores reverse curriculum generation approach that recursively de- composes complex datasets into simpler, more learnable components. We propose a teacher-student framework where the teacher is equipped with the ability to reason stsep, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared standard training original datasets.
Introduction
When teaching language models (LMs) to solve mathematical problems, one common solution is to apply supervised learning on a dataset of problems with worked-out solution steps. This is, to some extent, in stark contrast to how humans educated and generally taught solve problems, e.g. with a curriculum, where a teacher presents simpler concepts first, to build a foundation for later tackling more challenging tasks.
Transposing this strategy to train machine learning models, e.g. via curriculum learning [Elman, 1993, Bengio et al., 2009], has historically yielded mixed results. We speculate that one of the reasons that approaches largely relied on ranking by difficulty examples within a dataset [Wu et al., 2020, Lalor and Yu, 2020]. Not only is accurately estimating difficulty challenging, but datasets may also lack the diversity or granularity needed isolate and teach the fundamental "skills" required for complex problem-solving. Recent studies suggest that curriculum learning is more effective when "decomposed" datasets that isolate atomic skills are available to the learner [Lee al., 2024].
We build upon this intuition and, assuming access to a teacher model that can reason step- by-sepp, we propose way of "decomposing" dataset of mathematical problems into hicrarchy of problems with different difficulty levels. Our approach, DECOMPX, enables smaller LMs to progressively acquire elemental skills before tackling more intricate ones. Although we focus on mathematical datasets such as MATH [Hendrycks et al., 202 1] and AIME [of America, 2024], as well as code generation datasets such as [Chen et al., 2021], we think the approach might applied to a broader range of domains in the future.
39th Conference on Neural Information Processing Systems (NeurTPS 2025).

---

Our dataset decomposition approach starts with dataset of mathematical questions and step-by-step solutions. The main idea is that a teacher model can generate simpler problems by looking at sub-steps in the solution. Every sub-step is by definition simpler than the original problem and thus can be used as the answer for a latent, simpler problem. Crucially, this process can be recursively iterated: we can now ask a teacher model to create a step-by-step solution to the sub-prot sub-problem and recursively generate simpler problems in the same manner until a stopping criterion such as recursion depth or problem simplicity- is met. Given the strong inductive bias of step-by-step reasoning, the teacher is encouraged to work out solutions to progressively simpler problems. This process creates a tree of sub bob-problems for each question in the original dataset.
To facilitate curriculum construction, we assign difficulty scores to sub-problems based on their structural complexity. We tag sub- uub-problems and construct a graph of tags by connecting each sub- problem's 's tag to its parent problem's 's tag. The graph synthesizes tag relationships across the entire dataset. The simplicity of a problem is then computed both using the "depth" of the tags associated with given problem and the number of sub-problems it generates.
We use our dataset decomposition to train small LMs to do mathematical reasoning and code generation via supervised fine-tuning (SFT). Even when decomposed problems are presented i.i.d. to the models, our approach improves the ability of small LMs to learn from small set of examples. We see further gains when sub-problems are presented in the order of increasing complexity.
Apart from the dataset augmentation and curriculum construction aspect, one of the useful byproducts of our method is the potential for creating a "cartography" of given dataset (Swayamdipta et al., 2020]. Our induced graph of skills can be used to gain more insights into coverage of the dataset and how additional samples might increase the coverage, or even how two seemingly different datasets are related to each other.
2 Related Work
Data Augmentation Data Augmentation, synthetically generated from an LLM can be seen as a form of distillation [Hinton et al., 2015, Kim and Rush, 2016]. This approach has been especially successful in distilling reasoning capabilities into smaller models [Mitra et al., 2023, Li et al., 2023, Mitra al., 2024]. This approach has also been applied to generate mathematical reasoning traces. For instance, Self-Taught Reasoner [Zelikman et al., 2022] generates and then learns from its own chain-of- of-thought reasoning steps. MuggleMath [Li et al., 2024] and MetaMath [Yu et al., 2024b] both amplify diversity by evolving problem queries and sampling multiple reasoning traces, and by applying paraphrases and backward reasoning transformation, respectively. MuMath [Yin et al., 2024] further enriches examples through multiple rewrites using several "perspectives" e.g., paraphrase, symbolic reformulation. MathGenie [Lu et al., 2024] back-translates a small seed set through cenerator-verifier loop to create high-fidelity new problems. PersonaMathQA [Luo et al., 2024] adds "persona diversification" plus self-reflection to generate richer problem contexts. Beyond direct data augmentation, Shridhar et al. [2023] generates subquestions and solutions to distill the knowledge from the teacher model to students, Huang et al. [2025] synthesizes QA pairs by extracting key points from problems. To simultaneously address the quality, diversity, and complexity of the dataser, Davidson et al. [2025] proposes Simula, a unified framework for generating and evaluating synthetic data. However, existing data augmentation techniques somewhat neglect prior knowledge and difficulty. To address this gap. we propose hicrarchically decomposing problems with multi-step augmentation so models progressively acquire elemental skills.
Curriculum Learning Bengio et al. [2009], Elman [1993] originally propose to apply curriculum to train LMs. Wu et al. [2020] shows mixed results when applying curriculum learning methods based on example difficulty. In the past, many have studied different curriculum learning strategies to either increase the context length or more efficiently train LMs [Press al., 2020, Nagatsuka et al., 2021], with unconvincing results. More recently, studies started to investigate synthetic data creation associated with curriculum learning, yielding promising results for training small LMS for code [Nair et al., 2024]. Work on Tiny [Eldan and Li, 2023] shows that carefully synthetically curated datasets can be helpful in teaching tiny LMs basic English proficiencies. These works align with ours in the hypothesis that the synthetic data creation of easier examples can be used as a driving factor underpinning a successful curriculum. In the robotics domain, [Florensa et al., 2017] proposes a "reverse" learning strategy that starts RL training from the goal state and gradually guides the policy

---

cot
split
depth +1
Graph of associated problems
StePi
Stapp
Staips
Stepp
question
Kalve
cot1
d1
00.0%
tagi
depth
Step
Step1s
Scoppa
question 12g
Q11
COL1
Figure Left: We recursively decompose a math example (g. cot, a) into set of smaller problems (depth in the figure). We first split the cat into steps, then create question for each step and an associated concept tag. We then ask the teacher model to solve the question step-by-ster step. We verify the final answer by ensuring it is the same the until a stopping criterion. Right: create a graph of tags, where dependency relation given by the hierarchy the decomposition tree. The graph tags used quantify the difficulty generated sub-problem.
obtained without the ground truth step in context. We then recursively apply this procedure
to learn to reach the goal from a set of start states increasingly far from the goal. This approach also builds a curriculum with increasing difficulty levels (in a guaranteed way) by reversing the original task. However, our reasoning domain different because it allows us to leverage the compositionality of language decompose the reasoning tasks at much meaningful abstraction levels.
DECOMPX: Dataset Decomposition
In this section, we propose a recursive algorithm, DECOMPX, that decomposes complex math problems into simpler subproblems based on their underlying reasoning steps. Our algorithm consists in two phases: first, we decompose each example into a hierarchical structure of sub-prot problemob; second, we connect these sub-problems at dataset level using tagging approach and we build graph of tags, useful to infer the difficulty of every sub-problem. Finally, we describe how we use the difficulty score in our curriculum learning procedure.
3.1 How to generate subproblems? Recursive Problem Decomposition
We propose recursive dataset decomposition framework that constructs verified, grounded sub- problems from multi-step solutions. Each sub-problem corresponds to a clear, atomic mathematical reasoning operation, explicitly linked to core mathematical concept and validated for correctness. We assume access to teacher model T (a large LM, we use GPT-4o in our experiments), which we assume is proficient at the task of the dataset of interest.
Step Extraction. Given solution trace cor for math problem q, we first decompose it into at most & reasoning steps. This is achieved by prompting the teacher model to segment the text based on conceptual granularity: cat [s}. S2, S....., ........... sg]. where each step s, introduces new and distinct mathematical operation.
Concept Tagging. For each step s,, we extract atomic concept tag I, by querying the teacher model to identify the most specific mathematical concept that governs the reasoning step.
Subproblem Generation. Given the original problem q. step 5;, and its tag 1, generate new subproblem 4 (i.e. a question) grounded in the original context (f,- Q). Then, we ask the teacher model solve the generated problem q; step-by-step leading to solution cot, and final answer a,
Verification. To assess that the answer the generated sub- problem q; is correct and answerable. we also ask the teacher model to solve without access to the original context (f,, q) resulting in an answer å, Given that a, has been generated with access to the original context, is more likely to be correct. Therefore, Only subproblems satisfying (a,, 2,) = True are accepted. Otherwise, the subproblem is regenerated with retry budget of R attempts.
compare the resulting numerical answer å, a, using symbolic verifier }.
3

---

Salie
Greatest Garmme
Rezirate Po opaliin
jodp pwn 2-4, mutesicttive
Chéfase femiindes
Graad
Comman
Sqare
di iquare, find value ofa.
Facttor
Exponentiation
Root
Modysay Arth
Mumpil cellin
Prine
Feel prre factors of
3<2=7
6/++
Perfect
Adafiion
Square
3"3-5
FALESA
4'3ml
Inforred Graph Concept Tags
Decom position of an example
Figure 2. Left: We show the dccomposition of the math problem on the top obtained with our method, along with the associated concept tags. The problems in the solid white boxes are the generated sub-proue sub-problems. Right: The graph of concept tags, obtained by connecting the tags across the dataset examples.
Recursive Expansion. The sub- nub-problem reasoning cot, contains further multi-step reasoning by construction. Therefore, we can apply the process recursively by decomposing cot, using the same procedure up to maximum depth D. This produces, for every example in the dataset, nested structure:
9: cot, a, (().j. cot 4,))
where each child 9., corresponds to problem grounded in the constituent sub-step of cot,
In summary, the final output dataset composed of sub probleoobl associated concept tags, sub- problem depth, reasoning steps and answers (see Figure 1, left). We also illustrate the whole workflow with concrete problems Figure 2.
Algorithm 1 Recursive Dataset Decomposition
Input :Problem set D pa teacher model T, maximum decomposition depth D, maximum steps per layer k
Output :Decomposed dataset D cure
Initialize empty dataset Denr = ()
Function DecomposeExampleegg cor, d. our?
Split into steps 55) using T
for = 1 to k do
Generate concept tag for step s, using T
for retry = to MaxRetries do
Generate question 4, from (x)
Generate step-by-step solution col, and answer a, from (q, s, using? Verify via auto-solver and consistency check (see paper) if verified then Break end end
12
14
D.n. = D D.nr U
15
then DecomposeExamppe(gg cot, a, +1,11 (m.)) Recursive call end
17
end
19
end
20 foreach (g. col, a) E D,
21
DecomposeFxample (q, cot, a, )
22
De.. = D) D... U (q. cot, a, -.))
23
end

---

3.2 How difficult is a certain (sub)problem? Concept Dependency Graph
We construct a directed acyclic graph (DAG) over concepts derived from the recursive decomposed problem-solvee solving process above. This graph encodes the prerequisite relationships between mathemat- ical operations across all the examples in the dataset, thus enabling curriculum design.
Let G = (V,E) denote the Concept Dependency Graph, where 1 is a set of concept tags (such "GCID" "Square Root"), and & C V x V is of directed edges representing prerequisite relationships between concepts. A directed edge (u, D) E & indicates that concept u depends on concept u.
The construction algorithm proceeds as follows. We initialize an empty directed graph G. We add the ensemble of tags across examples the node set V. A parent tag l has child tag &e if they obtained while decomposing the same example and their depth differ by 1. For every parent tag ' and child tag f, insert edge (pp. E &, unless =
Nodes with zero in-degree in & represent foundational concepts. Formally, the set of root nodes is defined R {o V in-degree(v) 0). To quantify concept difficulty, we define depth function d - such that:
{0 + max d(4)) otherwise.
if E R.
d(v)=
{
This depth serves as a proxy for the reasoning complexity required to apply concept v.
3.2.1 Concept Clustering via Embedding Similarity
Our generated tags include variations such as "GCD" and "Greatest Common Divisor" that refer to same mathematical concept. To unify these concepts and minimize semantic redundancy in our graph, we use unsupervised clustering on tags' embedding representations.
Specifically, each concept tag is mapped to dense vector representation d((t) E Id using pre-trained L.M. We then compute pairwise cosine similarities between all tag embeddings. Denote, the similarity (i..
To cluster tags, we employ a greedy clustering algorithm with a predefined similarity threshold Specifically, sequentially select unassigned tags cluster representatives and group all other tags exceeding the similarity threshold under them. Formally, the mapping from original tags to cluster representatives is concisely defined as:
r(1,) arg max (S(().4) (S(),))) 8). 1,EYm
where Vep S represents the set of selected representative tags.
After clustering, we relabel the nodes in the concept dependency graph G accordingly. Edges are rewired based these updated labels, explicitly removing any self-loops arising from clustering:
E' = [(r(u), r(b) (u, v) E, r(4) # r(v)).
This results in refined concept dependency graph G' (Y', E'), where each node uniquely represents a distinct conceptual skill.
3.2.2 Difficulty Measurement via Structural and Conceptual Features
The difficulty of a data sample stems from both the number of mathematical operations it includes and the complexity of the concepts involved. For example, consider solving the following three problems: 1+1; b) 1+1+...** +1; (sum of thousands of 1); c) sqrt (3) (5/76) **2. e) is harder than P1 due to conceptual complexity (the complexity of operations involved) while b) is harder than a) because of structural complexity, the number of atomic operations involved.
While the Concept Dependency Graph provides a data-driven way to gauge concept's ss complexity by measuring the depth of the concept's tag in the graph, it may fall short in accounting for the structural
5

---

complexity of a specific reasoning step that involves that concept, such as the number of sub- sub-problems it can be decomposed into, or how important the sub-prot uul-problem solve other problems in the dataset. To address this, we introduce a composite difficulty score that combines both conceptual and structural factors for a more comprehensive characterization.
Given reasoning step (or data sample before the first iteration of decomposition) in the recursive decomposition tree (e.g., s in Figure 1), we compute:
structural complexity SC(s): the number of direct children of the reasoning step, reflecting its structural branching factor. In the example in Figure 1, ) decomposed into three lower-level reasoning steps (i.e., step11 step12. and step13).
because step, can be further
conceptual depth Cs(s): the maximum depth of the concept tag in the Concept Dependency Graph & that is associated with this reasoning step. In the example in Figure 1, (D(step)) equals the depth of tag, in G.
Therefore, the overall difficulty score ((s) E RR* is defined as weighted combination of these two terms:
=s) = aj SC(s) #2 CD(s),
where aj a2 E RR are tunable coefficients that balance structural complexity and conceptual depth.
3.3 Curriculum Learning via Difficulty Measurement
To leverage the difficulty scores / during training, we implement staged curriculum learning framework where the model exposed data in increasing order of difficulty. This approach enables the model to first acquire capabilities on simpler sub-problems before attempting harder examples.
Let 1 (q. cot, (4)) be the decomposed dataset where the question in each sample is annotated with our difficulty score. We partition D' into K non-overlapping subsets D' D!.....2 ....,'' based on the quantiles of difficulty:
D'U....UD =D, D' = ((q, cot) E |qbb-11 S (g) < qb,
where (qbo-
are the quantile breakpoints computed from the score distribution. We adopt an easy-to-hard curriculum, where the model is trained sequentially from D' to D'x Each stage is trained with early stopping to avoid overfitting and to allow controlled progression:
Stage i: 0, arg min C(: D()
0
where 0, denotes model parameters after stage i. We split the training budget across the full curriculum, so that experiments presented in this paper are compute matched for given setting.
4
Experiments and Results
In this section, we describe our evaluation procedure used to validate the effectiveness of DECOMPX.
4.1 Experimental Setup
Models
We adopt the Qwen2.5-1.5B [Qwen et al., 2024] and Qwen3-4B-Base [Qwen et al., 2025] models as our student models. Our dataset decomposition driven by GPT-4o and 04-mini for mathematical reasoning and code generation, respectively, which we leverage our teacher models.
Datasets Our setup uses MATH [Hendrycks et al., 2021] and the American Invitational Mathematics Examination (AIME). MATH [Hendrycks et al., 2021] is a benchmark of competition math problems of varying difficulty. We evaluate on the same 500 samples the prior work [Lightman et al., 2023]. AIME contains challenging mathematical competition problems. For training, we use AIME 224 as training set and AIME '25 as test set. Both datasets contain 30 problems that were used the AIME in 2024 and 2025, respectively. More details can be found in Appendix A.

---

Table Comparison of testing accuracy to LLMs the MATH-500 benchmark. data refers to the number of examples used for fine-tuning. *We evaluate based the checkpoint released
using lighteval [Habib et al., 2023].
Model
# data
MATH-500
Open-Weights Models
Owen2.5-1.5B [Qwen et al., 2024]
N.A.
35,0
Qwen2.5-3B [Qwen al., 2024]
N.A.
42.6
Llama-3-70B [Dubey et al., 2024]
N.A.
42.5
Mixtral-8x221 [Jiang al., 2024]
N.A.
41.7
Gemma2-277 [Team et al., 2024]
N.A.
42.7
MiniCPM3-4B [Hu et al., 2024]
N.A.
46.6
Gemma2-90 -nnttrrrut [Team et al., 2024]
N.A.
44.3
Llama3.1-8B --SStIntttt [Dubey et al., 2024]
N.A.
51.9
Qwen2.5-1.5B SFT MATH
Base
N.A.
4.2
SET (full dataset)
7500
2.2
SFT
360
48.4 2.2
SFT-Direct Distillation
360
49.6
PET-MetaMATH-AAE
2638
37.2 6.8
Mag-MuggleMath
147787
50.4
SFT-DecompX (Ours)
4500
50.8 + 2.2
SFT-DecompX + Curriculum (Ours)
4500
51.6 : 2.2
To further demonstrate the generalization capability and versatility of our method, we conduct additional experiments in a non mon-maateeatical domain that demands complex reasoning: coding. Specifically, we use the CodeForces- CoTs dataset [Penedo al., 2025] (competitive programming solutions in C++) from HuggingFace Open-R1 R1 [Hugging Face, 2025] for training. This dataset contains solutions generated by DeepSeek-RI [DeepSeek-Al et al., 2024], where long and complex reasoning traces are common. For evaluation, we employ the HumanEval benchmark [Chen et al., 2021] (Python function completion), which serves as an explicitly out-of-distribution scenario.
Training Details Follow the fine-tuning setup in the previous work [Muennighoff et al., 2025] we train each model for 5 epochs with batch size of 16. We train the models using bfloat16 precision with a learning rate of 10-5, warmed up linearly for 5 5% and then decayed to 0 over the rest of the training, following cosine schedule. We use the A optimizer [Loshchilov and Hutter, 2019]. Unless otherwise specified, we evaluate with a temperature accuracy (equivalent pass @ 1). The results averaged three different training seeds. Our experiments are conducted on NVIDIA A A100 GPUs with 80GB VRAM.
(greedy decoding) and measure
Baselines We compare our dataset decomposition and curriculum learning method with set of baseline systems: (1) closed-weights models such as GPT-4o: (2) open-weights models; (3) Supervised Fine-Tuning (SFT), which uses the training set of the original datasets; (4) direct distillation from the same teacher model and the same original datasets. (4) Data augmentation methods including MetaMath [Yu et al., 2024a] and MuggleMath [Li et al., 2024]: We applied MetaMath's 's augmentation pipeline for mathematical datasets by rephrasing questions well as generating answers four augmentation types (rephrasing, self-verification, seserraugmeettaiion and backward reasoning).
4.2 Main Results
We present our main results, evaluated across different benchmarks, and compared with baseline approaches. We summarize the findings below.
7

---

Table 2.
Comparison of testing accuracy to LLMs on the AIME 2025 benchmark. # data refers the number of examples used for fine-tuning. *We evaluate based the checkpoint released http:///wgiiggeee.ooooo/mieeaaaa without budget forcing.
Model
# data
AIME2025
Open-Weights Models
Owen2.5-722-Inmse [Qwen et al., 2024]
N.A.
15.0
S1-32BB [Muennighoff et al., 2025]
1000
13.3
Close-Source Models
GPT-4o [OpenAl et 2023]
N.A.
7.6
Qwen3-4B-Ban BBse SFT on AIME2024
Base
N.A.
10.0
SFT
30
3.3
SFT-Direct Distillation
30
6.76
SFT-MetaMATHH-Aug
114
4.4
SFT-Decompx (Ours)
385
13.3 + 6.3
SFT-Decomph Curriculum (Ours)
385
16.7 + 6.9
DeepSSek-RR--Disill---eeer SFT MATH
Base SFT-DDcompX (Ours)
N.A.
23.33 + 7.85
4500
30.00 8.51
DECOMPX improves performance across different benchmarks. We start analysis by inves- tigating performance on smaller LLMs. We see that across two different base models and datasets, DECOMPX shows consistent gains over standard baselines; Table 1 presents results using Qwen2.5- 1.5B finetuned and evaluated on MATH. We note that it achieves a 2.4% improvement over SFT and 13.6% relative gain over SFT-MetaMATTH-AUU COMPX over direct distillation. Table 2 reports test accuracy on AIME2025 obtained from fine-tuning Qwen3-4B-Base the AIME2024 data. Again, DRCOMPX performs well, showing improvements of 10% over SFT and 8.9% over SFT-MctaMATTH-AU It even outperforms significantly larger model, using only 385 training samples. Overall, these results highlight the effectiveness of our method and validate the benefit of structured decomposition and curriculum learning in mathematical reasoning tasks. Finally, we note that DECOMPX better than MetaMATH on both of the benchmarks, which is used for generating augmented data for finetuning. MuggleMath also underperforms our approach despite leveraging substantially more training data and richer prior knowledge from the seed datasets (MATH and GSM8K [Cobbe et al., 2021]). These results further support the advantage our structured, decomposition based curriculum. Last but not least, Table demonstrates the advantage of our methods compared to the baselines on the code generation tasks.
also demonstrates the advantages of DE-
Performance reduced during post-training for traditional SFT. Surprisingly, find that SFT on mathematical datasets may reduce the performance compared to the base model in our experiments. This behavior is especially visible AIME (Table 2), suggesting that the model may be prone to overfitting given the small dataset size.
DECOMPX shows better generalisation. S1 [Muennighoff et al., 2025] is a reasoning model obtained via supervised finetuning based on Qwwn2.5532BBInnstrre Instruct using 1,000 samples. Although this sample-efficient baseline achieves strong performance on MATH-500 and AIME2024, it does not perform well on AIME2025. The model is over-parameterized relative to the amount of signal in the data. which means that the dataset contains less information than the model can represent. In contrast, DECOMPX outperforms S1 even with a much smaller base model (4B) on AIME2025. Model trained only on decomposed data generated from ALME2024 performance suggests its effectiveness in generalizing to unseen mathematical tasks. SFT on AIME 2024 fails to generalize to another year of AIME, whereas our data decomposition can help model better to learn the features that stay predictive not only for in-domain generalisation, but also under the distribution shift. With curriculum learning added to DECOMPX. it can reshape the training trajectory that the model first captures
8

---

Table 3. Comparison of testing accuracy LLMS on the HymanEval benchmark. # data refers the number of examples used for fine-tuning.
Model
HumanEval pass@1
Open-Weights Models
Llama3-8B [Dubey et al., 2024]
33.5
Mistral-7B [Jiang et al., 2024]
29.3
Gemma2-99 [Team et al., 2024]
37.8
Qwen2.5-1.5B-Instruue SFT on Codeforces-Coß's
Base
34.15 # 3.71
SFT
35.37 + 3.74
SFT-DecompX (Ours)
42.68 + 3.87
DeepSsek-RI-Disiil--Weee-- SFT on Codeforces-Cofs
Base
36.01 + 1.85
SFT-DecompX (Ours)
57.90 + 0.98
universal mathematical skills, then gradually adapts itself against wording and distribution drift. We evidence that curriculum learning can lead to better generalization and help model competitive performance on AIME 2025 even when the other strong baselines do not.
Curriculum learning beneficial. In regular SFT, the examples are randomly shuffled. However, with the sample difficulty measurement yielded by DECOMPX, we create a learning curriculum, starting with the easiest samples and progressively moving towards harder ones. This explores whether difficulty measurements are useful in forming curriculum without changing the set of training examples. From Table I and Table 2, we find that even when training on the same set of examples, difficulty measurements a random sample order. Indeed, curriculum yields a 0.8% and 3.4% on MATH and AIME datasets for DECOMPX, relative improvements of 1.6% and 25.6% respectively over randomly ordered sampling.
useful for improving performance. compared to training with
4.3 Case Study
Table presents both the original data sample from the MATH training set (Left) and three cor- responding generated data samples (Right), which are decomposed based on the original example. Using the difficulty measurement defined earlier, we compute the difficulty scores and categorize the samples into different levels. In both the MATH and AIME datasets, the difficulty scores range from to 20. We define three levels of difficulty: low (scores around 6), medium (around 6 to 10), and high (from 10 up to 20). As shown in Table 4, we successfully decompose complex problems into simpler subproblems and effectively quantify the difficulty of each subproblem.
5 Discussion and Conclusion
In this work, we propose a novel curriculum learning approach via recursive dataset decomposition, enabling smaller language models to progressively master mathematical reasoning tasks. Our experi- ments on math benchmarks (MATH and AIME) show significant performance improvements over baseline methods, highlighting the effectiveness of our structured decomposition and difficulty strategies. In the future, we plan to improve DECOMPX so could effectively used for generating more general datasets in broader range of domains. However, this will require extensive study on teacher models' capability and reliability in task decomposition in other specific domains.
Implications and future work. Our work provides stronger evidence for self- improvement. While our current work adopts a "large teacher to small student" setup, the method was designed to leverage the previous generation of models not only data consumers but also as data generators. The key idea is to create a high-quality synthetic curriculum that teaches complex concepts by shaping data for learning, rather than simply augmenting the dataset.

---

Table 4. Examples of an original data sample from MATH training set and its generated data samples.
Original Data
Decomposed Data
Problem: Simplify 35-3"
a'43²
Generated Data with Lower Difficulty
Difficulty Score: 4.0
Express your answer as a common fraction.
Problem: What is the value of 3 Solution: To solve for 32, multiply by itself three times: x 3 then 9 x = So, 3 27.
Solution: The common factor of 3 in the numerator and the denominator can be factored
Tag: Exponentiation
Generated Data of Medium Difficulty
Difficulty Score: 10.0
3*+3² 3(3'+3) +'+3
out: 33-3 303-1) 32-1"
Problem: What the greatest common divisor (GCD) of the numbers 30 and 8?
Now compute: numerator is
Solution: To find the greatest common divisor (GCD) of 30 and 8, need to identify the largest number that divides both 30 and without leaving a remainder. 1. List the factors of 30:- 1, 2, 3, 5, 6, 10, 15, 30 2. List the factors of 8:- 1, 2, 4. Identify the common factors of 30 and 8:- The common factors are 1 and 2. 4. The greatest of these common factors is Tag: GCD Calculation
27 3 = 30, denominator is
9
so
30
15
8
Generated Data with Higher Difficulty
Difficulty Score: 18.0
Problem: Simplify the expression 3'+3
3-1
Solution: We first compute the powers3' 27.32 9. Next, substi- tute into the expression: 9-1 Perform addition and subtraction
2743
And simplify the fraction 20 = Final Answer is
15
4
Tag: Fraction Simplification
This design naturally supports scenarios where teacher and student have similar capacity: the benefit comes not from the size gap but from the teacher's ability organize and distill knowledge into seaching-oriente
examples. In the weak-to-strong generalization setting, this can be achieved as long the weaker model already capable of producing clean and well-structured decompositions. We leave this as a promising future direction, especially at scale, where our proposed hierarchical data decomposition and curriculum learning could have even greater impact.
Limitations and broader impacts. This work proposes using stronger LLM teachers to recursively generate simpler data that builds up curriculum for training student models. It assumes access to strong enough teacher models that are capable of understanding a math problem, decomposing the problem into meaningful sub-tasks, and faithfully describing the sub-tasks. As mentioned above, for tasks beyond math, where the reasoning path to solve a task is less divisible in an obvious way, the teacher models may face challenges in generating the curriculum. This may require the design of better scaffolding and/or the use of more advanced teacher models. Moreover, LLMs have been shown hallucinate in various ways, our method is inherently vulnerable because LLM usage is at the core of the system design. For example, a hallucinating or fabricating teacher model may generate inaccurate reasoning chains and decompose them the wrong ways. The student models trained on the generated curriculum in such a way may result in poor performance or learn unexpected behaviors due to the suboptimality of the curriculum. Finally, we acknowledge that our work not yet at stage to be used in many real-world tasks, especially in domains involving high-risk decision making such as law enforcement, legal, finance, or healthcare.
Acknowledgement
The authors are deeply grateful to Eric Yuan and Marc-Alexandre Côté for their invaluable help. We would also like to thank Colin Raffel, Matthew Macfarlane, Ayush Agrawal, Gyung Hyun Je, Vedant Shah and Zhihao Zhan for many stimulating and helpful discussions.
10

---

References
Yoshua Bengio. Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Andrea P. Danyluk, Léon Bottou, and Michael L. Littman, editors, Proceedings of the 26th Anmual International Conference on Machine Learning (ICML 2009), volume 382 of ACM International Conference Proceeding Series, pages 41-48, Montreal, Quebec, Canada, 2009. ACM. doi: 10. (145/1553374.153333 URL https: /ttp::/oooooooo..////oo 1...0.1110 1145/ 01555/55553774.5555
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol. Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin. Suchir Balaji. Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 10. 2021.
Tim R. Davidson, Benoit Seguin, Enrico Bacis, Cesar Ilharco, and Hamza Harkous. Orchestrating synthetic data with reasoning. In Will Synthetic Data Finally Solve the Data Access Problem?, 2025. URL https::/opeeeteee http:::///pp/eeeeeeeeeeee
DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng. Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, A. X. Liu, Bo Liu, Wen Liu, Xiaodong Liu, Xin Liu, Yiyuan Liu, Haoyu Lu, Shanghao Lu, Fuli Luo, Shirong Ma, Xiaotao Nie, Tian Pei, Yishi Piao, Junjie Qiu, Hui Qu, Tongzheng Ren, Zehui Ren, Chong Ruan, Zhangli Sha, Zhihong Shao, Junxiao Song. Xuecheng Su, Jingxiang Sun, Yaofeng Sun, Minghui Tang. Bingxuan Wang, Peiyi Wang. Shiyu Wang. Yaohui Wang. Yongji Wang, Tong Wu, Y. Wu, Xin Xie, Zhenda Xie, Ziwei Xie, Yiliang Xiong, Hanwei Xu, R. X. Xu, Yanhong Xu, Dejian Yang, Yuxiang You, Shuiping Yu, Xingkai Yu, B. Zhang, Haowei Zhang, Lecong Zhang, Liyue Zhang, Mingchuan Zhang, Minghua Zhang, Wentao Zhang, Yichao Zhang, Chenggang Zhao, Yao Zhao, Shangyan Zhou, Shunfeng Zhou, Qihao Zhu, and Yuheng Zou. Deepseek Ilm: Scaling open-source language models with longtermism, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, et al. The llama 3 herd of models, 2024. URL https: http///axvvvv./////e//eeeee.aev/ org/ abs/2407 21783.
Ronen Eldan and Yuanzhi Li. Tinystories: How small can language models be and still speak coherent english?, 2023.
Jeffrey I.. Elman. Learning and development in neural networks: the importance of starting small. Cognition, 48( 48(1) 1):71-99, 1993. doi: (111111600100007779))0000000
Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel. Reverse curriculum generation for reinforcement learning. In Conference on robot learning, pages 482-495. PMLR, 2017.
Nathan Habib, Clémentine Fourrier, Hynek Kydliček, Thomas Wolf, and Lewis Tunstall. Lighteval: lightweight framework for Imm evaluation, 2023. URL hattts::/ggtthaee com com/huggingface/ lighteval.

---

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL https: //ops::/aane /tt////axvvvv..e//ee//eeeeeeeev/ .rgg abs/ / 211./ 03874.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.
URL https: //axiiv////i///////aaeaa/araaaaa org/ abs/ abs//503, 02531.
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He. Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang. Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm: Unveiling the potential of small language models with scalable training strategies, 2024.
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen. Key-point-driven data synthesis with its enhancement mathematical reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24176 24184, 2025.
Hugging Face. Open rl: A fully open reproduction of deepseck-rl, January 2025. URL https: //github. com/huggings ace/ open-r1.
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux. Pierre Stock, Sandcep Subramanian, Sophia Yang. Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts, 2024.
Yoon Kim and Alexander M Rush. Sequence-level knowledge distillation. In Proceedings of the 2016 conference on empirical methods natural language processing, pages 1317-1327. 2016.
John P. Lalor and Hong Yu. Dynamic data selection for curriculum learning via ability estimation. In Findings ACL: EMNLP 2020, pages 545-555, 2020. URL httpp:://aclanthol.oo org/ 2020. findings-emeen emnlp.48 48.
Jin Hwa Lee, Stefano Sarao Mannelli, and Andrew Saxe. Why do animals need shaping? a theory of task composition and curriculum learning, 2024. URL https///t /ttpas////xxvvv.///e oroooogg//ao /ata//242.222 18361.
Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, and Chang Zhou. Mugglemath: Assessing the impact of query and response augmentation math reasoning. Association for Computational Linguistics, 2024.
Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, and Yejin Choi. Symbolic chain-of of-thoughtt distillation: Small models also "think" step-by-step. In Anna Rogers, Jordan Boyd-Graber and Naoaki Okazaki, editors, Proceedings of the 61 Anmual Meeting of the Associa- fion for Computational Linguistics (Volume I: Long Papers). pages 2665 Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.addm https: http:s/:/aaaaaahhhlcoo www.oooggorrr.ooooo 2023. acl-long 150/.
URL
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. 2023. URL pettps:/aartis org/ abs/ 2205..205..0 20050.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. 2019.
Zimu Lu, Aojun Zhou, Houxing Ren, Ke Wang, Weikang Shi, Junting Pan, Mingjie Zhan, and Hongsheng Li. Mathgenie: Generating synthetic data with question back-translation for enhancing mathematical reasoning of 1lms. Association for Computational Linguistics, 2024.
Jing Luo, Run Luo, Longze Chen, Liang Zhu, Chang Ao, Jiaming Li, Yukun Chen, Xin Cheng, Wen Yang, Jiayua Su, et al. Personamath: Enhancing math reasoning through persona-driven data augmentation. arXiv preprint 2xXiv:2410.01504 2024.
12

---

Arindam Mitra, Luciano Del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes Ribeiro, Sahaj Agrawal, Xuxi Chen, Anastasia Razdaibieddma, Erik Jones, Kriti Aggarwal. Hamid Palangi. Guoqing Zheng, Corby Rosset, Hamed Khanpour, and Ahmed Awadallah. Orca-2: Teaching small language models how to reason, arXiv, November 2023.
Arindam Mitra, Luciano Del Corro, Guoqing Zheng, Shweti Mahajan, Dany Rouhana, Andres Codas, Yadong Lu, Wei-ge Chen, Olga Vrousgou, Corby Rosset, Fillipe Silva, Hamed Khan- pour, Yash Lara, and Ahmed Awadallah. Agentinstruct: Toward generative teaching with agentic flows. arXiv, July 2024, URL attps://wwxe microsoftt. conn/en-uus/esearchh towadd-oardddrd seeerdd-geeraaiiiee waanerrativeteachnnne
wwth whteggwwwthhhhttteeeeee seth--agentic-flosss....
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URL btttps://arxive brrg/abs//0 at///50b/55...... 19393.
Koichi Nagatsuka, Clifford Broni-Bediako, and Masayasu Atsumi. Pre-training a BERT with cur- riculum learning by increasing block-size of imput text. In Ruslan Mitkov and Galia Angelova, editors, Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 989-996, 996, Held Online. September 2021. INCOMA Ltd. URL attps://aclanthologg wwg.lloooooooooooooooo.ee wrg/2021...22. 2e222211aa11..... ranlp- 112/.
Marwa Nair, Kamel Yamani, Lynda Lhadj. and Riyadh Baghdadi. Curriculum learning for small code language models. Xiyan Fu and Eve Fleisig, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 390-401 Bangkok, Thailand, August 2024, Association for Computational Linguistics. ISBN 979-8-89176-09742 doi: 10. 12218663/1/22224.cclssa 2024 sww.
URL https://aclanthhol/
org/
Mathematical Association of America. Aime, February 2024.
URL https:// ortofprobbemsooving wwwiiinn...m////i// www.nxxwwppp...eeeeeeeeeee wht.ppppp//aaa Phtp/AMME__robleems and_ mnd_Solution//
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, et al. Gpt-4 technical report, 2023.
Guilherme Penedo, Anton Lozhkov, Hynek Kydlíček, Loubna Ben Allal, Edward Beeching, Agustín Pi- queres Lajarín, Quentin Gallouédec, Nathan Habib, Lewis Tunstall, and Leandro von Werra. Codeforces cots. https://hugginge:
httpppeeonree/ooooooooeeeeeeeeee cots, 2025.
Ofir Press, Noah A. Smith, and Mike Lewis. Shortformer: Better language modeling using shorter inputs, 2020.
Qwen,
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li. Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su. Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2024. URL https: //arxiv////xvv.//// org/ ab//abb//24// 2412. 15115.
Qwen, :, An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu. Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li. Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yugiong Liu. Zekun Wang. Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URI. https: // /arxiv.vvvv org/ abs//ab//// /200..255... 2505.09388.
Kumar Shridhar, Alessandro Stolfo, and Mrinmaya Sachan. Distilling reasoning capabilities into smaller language models. Findings of the Association for Computational Linguistics, 2023.
13

---

Swabha Swayamdipta. Roy Schwartz. Nicholas Lourie. Yizhong Wang, Hannanch Hajishirzi, Noah A. Smith, and Yejin Choi. Dataset cartography; Mapping and diagnosing datasets with training dynamics. arXiv preprini arXiv:2009. 10795, 2020. https://arxiv. wwwggabbs/200/// 10795.
doi: 10.485500/a2xiv20009.000 107999 URL
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupati- raju, Léonard Hussenot, Thomas Mesnard. Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, et al. Gemma 2: Improving open language models at practical size, 2024. URL https http:///aavvvvv.. org/ abs 2408. 00118.
Xiaoxia Wu, Ethan Dyer, and Behnam Neyshabur. When do curricula work? CoRR, abs/2012.03107, 2020. URL https: /ttp//aaxvvv///////a///ee//aaava wttp://..rr//bb///.....m 231200001000 ICLR 2021.
Shuo Yin, Weihao You, Zhilong Ji, Guoqiang Zhong, and Jinfeng Bai. Mumath-code: Combining tool- use large language models with multi-per pettpipteeepeeiveeee data augmentation for mathematical reasoning. arXiv preprint arXiv:2405.0755/ 2024.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models, 2024a. URL https: //axxiv////xvvv///////ea org/ abs/2309. 12284.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models, 2024b. URL https: pnttts://arxii: /ttpxvrivvv//////////////////ava org/ aaaaabb/2000 12284.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning, 2022. URL https: / aarxiv.... org/ abs/aaa2000 14465.
14

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
Our abstract and intro reflect the contributions accurately.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicaalle, the authors should diseuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
15

---

Justification:
We do not have any theoretical results in our paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
We explain our experimental setup in Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-weights models, it may be that access to the model is limited some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: We will release the code when the paper gets accepted. Our implementation is based on open-source training and evaluation scripts and is reproducible.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/
httec/gguiese/coeeeeeeee wwwpabiiissoooooooooooooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results, See the NeurlPS code and data submission guidelines (https: / //nips. ct/pcc/pplcc ccc/public/ppe SeniiddesCooeSuuben wwssssssssiiiiiioi for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: See Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: See Table and Table 2.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma emor bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
See Section 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics fcctss//neuurpp:
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there a special consider- ation due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that there no societal impact of the work performed.
the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained LMs, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper focuses solely on the mathematical dataset and does not pose a high risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited related work appropriately.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We did not conduct any crowdsourcing and/or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We did not conduct any user studies requiring IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: We use GPT-4o the teacher model for distillation. See Section and Section 4.1.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https http://eens../eeeeee/eeee /ttp//rips.cc/ooneeeeeeee c for what should should not be described.
21

---

Technical Appendices and Supplementary Material
Contents
Introduction
2 Related Work
3 DECOMPX: Dataset Decomposition
3.1
How to generate subproblems? Recursive Problem Decomposition
3.2
How difficult a certain (sub)problem? Concept Dependency Graph
3.2.1
Concept Clustering via Embedding Similarity
3.2.2
Difficulty Measurement via Structural and Conceptual Features
3.3
Curriculum Learning via Difficulty Measurement
Experiments and Results
4.1
Experimental Setup
4.2
Main Results
4.3
Case Study
5 Discussion and Conclusion
Details of subset of MATH training data
23
B Examples of Concept Dependency Graph
23
Zero-shot Small Model Performance
23
Tag Clustering Details
24
Examples of Decomposed Data
24
22

---

A Details of subset of MATH training data
In Table 5, we present the original and sampled sizes of the MATH dataset used in our experiments, broken down by subject domain and difficulty level. Sampling was performed uniformly at random within each group ensure representative coverage of the various topics and levels.
Table 5. Statistics of the original and sampled MATH dataset by subject domain (left) and by difficulty level (right). Samples drawn uniformly random within each group ensure representative coverage across topics and difficulty tiers.
By Domain
By Difficulty Level
Domain
#Total
#Sampled
Level
#Total
#Sampled
Algebra
1744
84
Level
566
26
Counting and Probability
771
36
Level
1348
65
Geometry
870
43
Level
1592
77
Intermediate Algebra
1295
63
Level
1690
81
Number Theory
869
41
Level
2304
111
Prealgebra
1205
58
Precalculus
746
35
#Sampled #Total
300 / 7500
B Examples of Concept Dependency Graph
Figure 3 presents the concept dependency graphs constructed during the data decomposition process. observe that an atomic mathematical operation, such as Addition, has many edges linking it to more advanced operations.
Figure 3. Concept dependency graph constructed during the AIME data decomposition process. Nodes represent mathematical concepts, and edges indicate prerequisite relationships between concepts.
C Zero-shot Small Model Performance
We partitioned the decomposed AIME2024 dataset into five equal-sized bins (quintiles) based on our proposed difficulty measurement, shown in Table 6. This measurement is derived from the concept dependency graph, designed reflect the conceptual complexity of each problem. We then evaluated the zero-shot performance of the Boen3-4B-Bass model on each difficulty tier.
23

---

Our results shown in Figure demonstrate clear inverse correlation between difficulty score and model accuracy: the model achieves the highest accuracy on problems with the lowest difficulty scores (Quintile 1), and its performance degrades as the difficulty increases, reaching the lowest accuracy on the highest difficulty tier (Quintile 5). This performance trend validates the effectiveness of our concept dependency graph-based difficulty metric in capturing the relative hardness of mathematical problems.
Table Definition of difficulty quintiles based on concept dependency graph scores. Each quintile groups problems whose scores fall within the specified range.
Quintile
Difficulty Score Range
Quintile (Easiest)
2.0 4.0
Quintile
4.0 4.0
Quintile
4.0 6.0
Quintile
6.0 10.0
Quintile (Hardest)
10.0 20.0
Model Accuracy Across Difficulty Quintiles
80
Trend
72.29%
Accuracy
65.06%
Accuracy (%6
20
30
86%
20
(Raslest)
Q3
(Nardestt)
Difficulty Quintile
Figure 4. Zero-shot performance of the Owen3-4B-Basse model across difficulty quintiles. Accuracy decreases as problem difficulty increases, validating our proposed difficulty metric.
D
Tag Clustering Details
We summarize the tag information identified in both the MATH dataset (Table 7) and the AIME dataset (Tables and 9).
E
Examples of Decomposed Data
We upply data decomposition method to the S1 dataset and demonstrate the two lwyelayer recursive decomposition process for a sample data point in Table 10.
Table Decomposed data sample. Black text denotes original data; yellow text represents the first level decomposition; and blue text represents the second level of decomposition.
24

---

(Question "Given rational number, write it as fraction in lowest terms and calculate the product of the resulting numerator and denominator. For how many rational numbers between 0 and will 20! be the resulting product?",
Solution: the rational number between 0 and be r. When written as a fraction lowest terms, let it be 3. where and b are positive integers, ged(a, b) 1, and 0 < a < b. We are given that the product of the numerator and denominator is 201, so b 20!. We need to find the number of pairs (a, b) such that a x b 201, ged(a, b) = 1, and a < b.Let N = 20!. The prime factorization of N PI P2 .. Pk where P; are distinct prime numbers. The number of distinet prime factors of 20! is the number of primes less than or equal to 20. These primes are 2, 3, 5, 7, 11, 13, 17, 19. Thus, there are 8 distinct prime factors, so k = 8.Since X b = N and ged(a, b) 1, each prime power factor P, of N must be entirely a factor of or entirely factor of h.e number of ways to distribute the prime power factors between and is 2 where k is the number of distinct prime factors of N. Each distribution corresponds to a unique pair (a, b) such that a X N and ged(a, b) = 1.The number of such pairs is 2*(N), where ((N) is the number of distinct prime factors of N Since 20! is not a perfect square, for every pair (a. b) such that a X b = 20! and ged(a, b) = 1, we have a # b. The pairs can be grouped into (a. b) and (b, a), where # The number of pairs with < b is equal to the number of pairs with b < a. a.The total number of pairs 2 The number pairs with a < is x 2 2002011-1 The number of distinct prime factors of 20! is (=20!) 8. The number of pairs (a, b) such that x = 20!, ged(a. b) I, and a < b is 28-1 = 2 = 128. Each such pair (a. b) corresponds to unique rational number 5 between O and I in lowest terms, with the product of the numerator and denominator equal to 20n. Answer: The final answer 128 ")
Solution:
ReasoningStep: "Let the rational number between and be in lowest terms with ged(a, b) < a b, and a b = 20!".
Tag:"Reducing a fraction to lowest terms", Question:") II the context of finding rational numbers r = in lowest terms with a = 20!, suppose
you start with the unreduced fraction 201/7' Reduce this fraction to lowest terms.",
Solution:
ReasoningStep: "We have the fraction F 201/7'
Since 20! contains 72 (because [20/7] = 2), the
number 20 is still divisible by 7.",
of a prime in factorial (Legendre's formula)", Question:" "In the context of reducing the fraction 7/(201/7) use Legendre's ' formula to find the exponent of the prime 7 in the prime factorization of 20!.",
Solution:" 1. Legendre's 's formula states that the exponent of a prime p in ! is given by the sum of
[n/p*] ork 1, 2..* > n.
2. Here, n=20 and p=7:
For k= |20/7] 2
For k=2: [20/49] 0
Higher powers of exceed 20, so their terms are zero.
3. Summing these gives 2 + 0 = 2."
25

---

Table 7. Summary of Cluster Tags. Sample count represents the number of mathematical problems associated with each cluster tag. Keywords are derived from the tags within each cluster.
Domain
Iguuestioss
Keywords
Equation Manipulatin
17
Equutiio Manipulatio. Equation Simplification Equation Selving, Equation Suberaction, Lincar Equation, Lincar Equation in Two Variables, Lincar Equations, Polynomial Equation Sulving. Polynomial Simplifica- tion, Quadratic Liquttinn Raticnal Equation. Rational Equation Solving, Rearaaggnng Equations, Simplifying Expressions, Simultatevus Equations, Simultaneous Equations Solving, Sulving Rational Equations. Suboti- uution Suhtraction, Variable substitution
Eliminatioa Method
14
Eliminzion Method, Equition Solving: lsclating Variables, Litear Equation Solving, Solving Linear Equa-
Substiution Method
Addition
Addition. Addition Incegers, Arithmetii Addition, Arithmetic Operations, Coturar Addition, Digit Sum, Fraction Addition Integer Addition, Muskiplicttion Place Value Addition, Summution
Divisibility
11
Divisibility Divisibility Rules, Division Property of Equality, Polynomial Division
Clearing Fractions
Cleuring Practions, Commen Denominator Calculation, Fraction Mustiplicatoo, Fraction Simplification Fraction Subtraction, Partial Fraction Decomposttion, Reciprocal Calculation Simplitying Fractions, Sub- mrating Fractious with Comman Desominator
Artthmetic Sequence
7
Anshmeri: Sequence. Arithmetic Sequence Formala, Arithmetic Subtraction, Counting Integers in Arithmetic Sequence. Modular Arihmmetic, Sum of Sequence, Sum un Arihmettc Series Factoring Polynomials, Factoring Quadiatic Equations, Factoring by Grouping, Factocing by grouping, Polynonual Expansion, Prime Pactorization Binomial Cocticient, Binomial Coefficient Calculation, Combinaionn Formula, Combinations Calulationn Combinatocial Conating, Saa of coeffiients
Factoring Rolysomials
Binomial Cootiiient
6
Complessentay Counting
Complementary Counting. Counting Principle, Comnting Principles, Couning Rows, Inclusion Exclusion
Principle
Combining Like Terms
6
Combining Like Terms
Cross Multipiccttonn
Cross Mullipiicatoo, Multiplication Principle. Prime Musiiplicatio, Scalar Multiplication Division, Division of Equations, Division of constanes, Long Division Absolune Valae, Absolune Value Calculation, Absolate Value Equatioo, Absolute Value Equation Salving, Absolute Value Equations, Magnitude of Complex Number, Magnitude of Vector Isclating Viriables, Isolating Variable
Divisian Alsolute Value
Isolating Variables
Calculation
GCD Calculation, GCD Progerty
Cuachy-Scbwarz Inequality
Caudty Schwarz Inequality, Compound Inequaiitiss Incqualtiess Inequality Manipulation, Linear Incquality Simpliiiiction
Ahitude Trinngees
Altinde in Trinngle, Altitdde of Trinngk, Samilar Trinngles, Trinngle Construction Stars and Bars, Exponcariation. Logarithm Power Rule. Modular Expencatiation Square Koot Calculation, Squaring number, Squaring both sades Orler Operations, Orler Flement Ideatifying Partillll Lines, Intersctionn Paraliel Lines Polygons, Slope Parallel Perpendicular Lines, Stope of Pespendicular Lines
Stars and Bars
Rars Mathod
Exponcaziation
Square Koot Calculation
Order Operations
Idcntitying Panillll Lines
Porperdicnlar Slepes
2
Perpendicular Slopes, Slope a Line, Vertical Tangent Line Quadatic Equation Identification, Quaditic Komulaa Combinatorial Placement
Quadanic Equation ldemification
Combinstorial Placerent
Selving Lines Inequalities
2
Selving Linear Inequalities, System of I Lincar Fquitions Cyclic Distanee Calculation, Distance Focmula Faler's Robation Triangle Centers, Faler's Arc 1 Length Caleslation, Prsimeter Calculation Angle Bisector Theoeem, Perpendicular Bisector of a Choed Royex' Theoram, Conditional Probuhility
Cyclic Distance Caluuiation
2
Ealer's Rolation Trianglo Centers
Are 1 Length Caleslation
Angle Bisector Theorem
2
Boyes' Theorem
Distriburive Property
Distritutive Progerty Flour Function
Floor Function
Finding Zeroa of Funcsioon
Finding Zeros of Function
Factorial Calesiation
Factorial Calesiation
Pigechhole Principle
Pigecnhole Principle
Cunclusion Verifiestion
Conclusion Verificstion
Change of Base Forarala
Charge of Base Formula
Logrrithm Properties
Logarihm Properties, Logerihhiic liemity, Logurihhmic Properties Logarilhmic Recipeocity
Summstion fimite
Summation of finite
Comples Number Magnitude
Comples Number Magniude.. Complex, Number Scaling, Dut Product Magnitude, Modulus of Complex. Number, Vecior Magnitude Calculation
Place Value
Place Value
Remainder
Remainder
Lifting The Exponeni Lemma (L.TE)
Liflnng The Exponeni Lemma (L.TE) Orler of an Flement in Modilar Arithmeric, Order of an element modulop
Deter of on Flement in Modular
Arilhmeii
Primitive Root Calculation
Primitive Root Calculation
P adicc Valaaion
p pddic Valation, v, (p adic valuation)
Diagonal of Reciaagular Prisas
Diagonal of Reciaagular Prisas Calculation
Calculation
Pythogorean Thoorasn
Pythogorzan Thooees, Pyrhagorean Thenrem in Lateriar Angle of Regular Polygos
Interiar Angle of Regular Polygos
Pairing Elements
Pariing Elements
Power Point
Power Point
Bipurttt Graph Matching
Hiparttte Graph Matching
Permutation Calculation
Permutaion Calculation
Prime Mentification
Prime Mestiiication Prime Numter Identification, Prime Number Multiplication
Perimeeer Calculation
Perimeter Calculation
Triangle Side Chasafication
Triangle Side Chassiication
Countroxample
Countrrxxupplc
Made Calculation
Made Calculation, Mode Identification
Serting Numbers
Serting Numbers
Area a
Area a Clicle, of Circle Calculation
Cross- Section of Sphere
Cross- Sectioe a Sphere, Cross sections of sphtres L.C Calenlation
L.C Caleulation
Propoction
Proportion, Propection Solving,
Proportion
Kadius a sphere
Kadius of sphere
Volume of Totrheedren
Volume of Tetrahedron
Noutoa's Sums
Neuton's Sums
Viet''s Fermulas
Vietn's Formulas, Victa's formolas
26

---

Table 8. Summary of Cluster Tags (Sample Count 2 3). Sample count represents the number of mathematical problems associated with each cluster tag. Keywords derived from the tags within each cluster.
Domain
Neguesiioss
Keywards
Combinatorial Probubulitt
100
Combinatorial Probuhility, Counting & Prohability
Bosic Couning Principle
46
Bisic Counting Principle. Counting Principle, Counting Principles, Pundumental Cousting Principle. Feade- mental Ptinciple Counting, Mullipication Principle of Counting
Combination Calculation
Combinatica Calcuation. Combination Emumeration Combination Formala, Comhination Selection, Coan- hinatica Subtraction, Contonation and Permutation Calculation, Combinations, Combinations Calculation, Combinaions Formula. Counting Combinations
Factorial
31
Factorial, Pactorial Cakulation, Factoril Division, Factorial Expansion, Factoridl Manipulation, Factorial Mutiplicatin. Factorial Propecties, Factorial Simplitiation, Factorial simgliiiation
Combinstovial Counting
Combinatorial Counting. Combitutoria Foumeration, Combinatorial Exclusion, Comhinatorial Ressoning, Combinatoies, Counting, Counting Arranemeess, Counting Multiplss Conating Sabsets
Binomial Coctiicient
29
Binomial Coetfeiian. Hinomial Coefficies: Calculation. Hinomiza Coeficient Formula, Hinomial Coefficient Multiplicttonn Bincnial Crefficier Simplification Rinomial Coefficienss Binomial Coefficians Calculation, Binomial Expession, Biuomial Probability Formols, Bisomial Thearem
Fraction Conversion
28
Fraction Convesion, Fraction ideatifcation, Fraction Multiplicaton, Fraction Subtraction, Multiplizaiinn of Fractions, Percesrage to Fraction Convession Simplifying Fractions Lineu Equation Evaluction, Division Simplification Division of Fractions, Fraction Division, Fraction Multipication and Simplificatio, Fraction Simplification Ratio Simplifiction Simplifying Ratos
Linea Equation Evaluation Division Simplification
36
Equation Solving, Linear Expression Evaluation, Solving Lincar Equations
34
Addition
23
Addition, Addition of lntegers, Addiion of integers Artthmetic Addition, Integer Addition
Arifhmetic Operations
Ariihmeric Operations, Rasic Arithmrti Subtraction, Mulliplication Mulliplication Integers, Multiplica- tion integers
Divisian Propecty Equality
21
Division Propesty Equality
Esprweent Simplifcation
21
Espreient Simplifcatiie, Exponenittivn Prohabiliry Calculation, Conditional Prohabiliry Caleslation, Probubility Probubility Colculation Equation Substiution, Subsiiution, Substittionn Mathol, Variable Substitution
Prohubility Calculation
Equation Subsiiution
18
Cieuiar Permutations
17
Ciecular Permurations, Cyclic Permutations, Permutztion, Permutstioos. Set Subtraction, Subtraction
Set Subtraction
13
Division
Division. Division Algorithm, Long Division
GCD Calculation
GCD Caltulation
Independent Prohability Multiglicatian
11
Indapendent Prohability Multipication, Muhtplication Rule for Independent Events, Multipication Ruie for Protabilties, Probuhility Muliplication Rule taolating Variables, Isolating varisile
Ioolatnng Varisbles
Cumbinatons with Repetition
Cumbinuions with Repetition, Permutations and Combinations, Permutations with Repetition Comning Exchusion, Exclusion Principle, Inclusion- Exclusion Principle Couning and Summaricn, Summation
Couning Exclusion
Couning and Summurion
Adding Fraciios
Adding Fractions, Adding Fractions Like Denuminaters, Adding Fractions with Unlike Denominators Addition of Fractions, Arilhmeric with Fractions, Common Denominator Addition, Fraction Addition, Mahhiplying Fractions
Prime Faciorization
8
Prime Factorisation
Combining Torms Place Value Arithmetic Sequence
Combining
Place Value, Place Value Identification
Arithmetic Sequence, Arilhmetic Sequence Formula, Arithmetic Sequence ldentificution, Arithmetic Se- quence Sum, Arihhmeric Sequence Sum Formuln, Arilhmeric Sequence Summation, Arithmetic Sequences, Arithmetic Sequences Counting. Couming Terms un Arithmetic Sequence Equation Simplificaioo, Linear Equation Simplification, Pulynomial Simplification
Equation Simplification
Conditional Probubility
Conditional Probubility
Permutaioo with Restrictions
Permotation with Restrictions, Permutations with Restrictions, Permulations with restrictions
Divisibilit for
Divisibility for Divisiblity Rules Factoring by grouping Factorizatian
Factoring by grouping
Complement Rule
Complement Ruls, Complement Rule in Probability
Multiplication Principle
Muliipication Principle
Artthmetic Series Foruula
Arithmetic Formula, Arithmetic Series Formula. Arithmetic Series Summation, Arithmetic Sum Calculatio. Summution Arilhmetic Series
Order of Operations
Orler of Operation
Equation Balancing
Equation Balancing
Binomial Probubility
Binomial Probubility
Couning Integers
Counting Integens, Counting Integers in Ronge
Power Calculation
3
Power Caluulation
Discriminant Caleulation
Discriminant Calesiation
Identifying Coefficcents in Quadratic
Idemiiyyng Coefficients in Quadratic Liquation, Quuddratic Cootiicents Idestification
Equation
Complementary Couning
3
Cumplementary Couning
Distributive Propenty
Distribtive Progerty
Combination Symmetry
Combination Symmetry, Combinatorial Symmetry. Symmetry Counting Anca Calculatóo, Area Calculation a Square, Area Ratios, Area of a Rectangle Calculation, Area of Square Calculation, Area of a Trinngle Calculation Arta a Right Triangle, Arcu a Triungle. Triangle Area Formula Counting Outcomes, Eaumentinn Outeones Independent Events, Independent Events Probability, Independent Events Probability Calculation, Probubilitt of Independent Events
Area Calculatin
Area of Right Triungle Counting Outeomes
Independent Events
Scalar Multplication
Scalar Multiliictton
Case Analyss Common Denominatur Calculation
Case Analysa
Commun Denominator Calculaion. Common Denominater Comversion, Finding a Common Denominator,
Subtracting Fractions with Comman Denominator, Subrracting Fractions with Common Denomiattors Finding the Least Common Multiple (LCM), LCM Calculation, Leust Common Multiple (LCM) Calculation
Finding the Least Comman Muktiple
(LCMM
Factoring Common Factor
3
Factoring Common Factor, Pactoriug Out Common Faztors, Finding Factors
Countng Even Numbers
3
Counting Even Number, Counting Odd Numbers, Even Numbers idcntiacation, Even and Odd Numbers, Ideniiyaag Even Numbers
Modular Arithmetic
Modilar Arithmetic, Modulo Operation
27

---

Table 9. Summary of Cluster Tags (Sample Count < 3). Sample count represents the number of mathematical problems associated with each cluster tag. Keywords derived from the tags within each cluster.
Equestions
Keywords
Probubilty Distribution
Probability Distribution
Range Sums for Dice Rolls
Range Sums for Dice Rolls, Sum Two Dice Rolls
Unidomu Probabilay Distribution
2
Unidoma Probability Distrbution
Properties Plasonic Solids
Properties Platonic Selids, Symmetry of Platonic Solids
vonstamis
Permutations of Mullisets
Permutations of Mullisets
Digit Fisation Positiona Nocation
2
Digii Fixation Positional Notation, Digit Placement Muttiples Idcutiication
Multiples Idcunfication
Formula
Distnce Formala, Hocizantal
Calculation
Graphing Incqualites
Graphing Inequalites, Graphing Lincar Inequalities Linear Inequality Gruphing Intersection of Lines and Curves, Line Intersectio
interiection of Lines and Curves
2
Isosceles Right Trinngle
Isosecles Right Triangle, Isosccles Trinngle Properties Probubility
Probubility Combined Ev
Events, Probubility Single
Combinatocial Selection
2
Combinatorial Selection, Subset Selection
Subset Identifcation
2
Subset Identifiationn
Cumplementary Probability
Cumplemenizry Probability
Probability Addition
Probability Addition Kule, Probability
Factor Pairing
Factur Pairing, Factur Paits Ideatifcatioon
Finding Maltiples
Finding Muliiples, Multigles of Number
Prime Mdeniiication
Prime Ideuniication, Prime Number idemiiccttion Prime Compasite Numbers Ideuification Expected
Expected
Additio: and Suborction Properties
2
Additica and Subtmation Proyerties Equality
Eqqulitt
Multipiiation
Muttipiicaion, Multipication
Numbers
Scries, Formula, Geometric Series Summation, Infinit Geometri: Series Formila, Sum Infinite Geometric Series
Series Focuula,, Geometric Identification, Geametric Series Sum
Piscal's Triongle Conctraction
2
Piscal's Triangle Condtrcctoon Piscal's Trizagle Row Sum
Independent Probability
Iadependent Prabability
Exponestation Practions
Exponestation
Simpli fying Rational Expressions
2
Simplitying Raricnal Expressions
Pascal's Mdeniiy
2
Piscal's Identiyy Pascal's Triangle
Summation Series
Summution Senes, Summation Sequence Intersection Sess, Interiection
Intersection
Set Union
2
Set Lnion, Set Union Cardinality
Percentige Calcolation
2
Percentage Calculation, Percentage Converson, Peroentage Decimal Cieversion Symmetry Probability
Symmetry Probubiliry
Couninn Positions
Counting Posiiins, Counting Rectangles Couming Squares Conating
Grid
Purity
2
Purity
Recurresee Relation
Recurresce Relation,
Relitions
Permutation
Peneutation
Digit Pairing for Sum
Digit Pairing for Sum, Digit Sum Calcolation, Pairing Numbers for Fixed Sum
Permutation Calculation
Permutation Calculution
Cyelic Number
Cyclic Number
Cyclic
Bipostie Gimph
Bipartile Graph, Bigortite Graph Caloring
Digii Consiraints
Digit Comerrinns, Digit Restriction Digit Sum Constraints, Single-dight constraint Countngg Leap Years, 1 zap Year Calculation
Couning Leap Years
Minimum Value Calculation
Minimum Value Calculation
Rigoohhole Principle
Pigocebole Principle
Rungo Calculation
Runge Calculation
Burnside's Lemma
Burncik's Lemma
Polyhedron Properties
Polyhedron Properies, Properties
Rotational Symmetry
Ratarional Symmotry
Rotational Symmetry of Polyhedra
Rutational Symmetry of Polyhedra, Symmetry Polybedri
Stope Caleulation
Slope Caleulation
Pair Counting
Puir
Pairwise Sum Calculation
Pairwise Sum Calculation
Division of Even Numbers
Division of Even Numbers
Prequency Distriuttion
Frequency Distribution
Cundiional Statements
Cunditonal Stttements
Couning
Comning
Period Calculation
Period Calculation
Time Interval Calculation
Time Interval Calculation
Unit Conversion
Unit Conversion
Factoring Quadratic Expressions
Factoring Quddrtic Expeessions
Simulaanous Equations
Simultanou Equations
Comming Cyclic Quadriaterals with
Comntnng Cyelic Quadrilaterals with Integer Sides
Integer
Countng Reciangles
Comnting Reciangles
Hockey Stick Idemitty
Hockey Stick Ideniiyy
Perimeter Calculation
Perimeter Calculation
Properties Quadilatenis
Properties Quadilaterals, Symmetry Quadriatterals
Properties of Square
Properlies of Square
Stan and Bars Method
Stm and Bans Method
Triangk frequality
Triangle Irequlity
Distance Comparison
Cumparison
Absolute Value Simplificatin
Absclute Value Simplification
Subset Defintion
Subset Definition
Equivalent Fractions
Equivalent Fractions
Inequliiy Compasison
Inequality Comparison
Seguential Multaplication
Sequential Multapicctionn
Angle Meusurement Radiums
Angle Mensuremen, Radiams, Radian- Degree Convension Length Culculation. Length Formula
Length Calculation
Azc Measure
Arc Measure
Central Angle Theocent
Central Angle Theorem
Circumference Circle
Ciccumference Circle
Property Addition
Property Addition
Factorial Decomposition
Factorial Decompostion
Matris Endexing
Matri Indexing
Probability with Replacement
Probability with Rolaccmmnt

---

ReasoningStepp"Compuue ged(7, 201/7). Because 20! contains 72, dividing by 7 leaves factor of 7, so ged(7, 20!/7) = 7.",
TGg:"GD calculation using prime factorization",
Question:"In reducing the fraction one over (20! divided by to lowest terms, compute ged(7, 20! 77) by using prime factorization.".
Solution:"1. We need the exponent of the prime 7 in 20!.
Use the formula for exponent of prime p in n!: vy(20!) [20/7] [20/49] +
3. Thus 20! has factor 72.
When we divide 20! by 7, the exponent of (201/7) 2 ==1.
So (201/7) contains exactly one factor 7, and the number also contains one factor of 7.
6. Therefore ged(7, 201/7) = =
BeasoningStte:""Diiid numerator and denominator by 7: Numerator: 7 = 1. Denominator: (201/7) + 201/(79) = 201/49.",
Tag: "Canclinn common factors in a fraction",
Question:" II the context of the original problem, you encounter the fraction 20//7 By cancelling the common factor in the numerator and denominator, what the resulting simplified denominator?",
201/7" 2. Observe that both the numerator and denominator contain the factor 7. 3. Divide numerator and denominator by 7: New numerator: 7 7 = 1. New
Solution:"1. 1. Start with the fraction 7
denominator: (201/7) 7 201/(7²) 201/49, Thus, after cancellation, the fraction becomes 5. The question asks for the simplified denominator, which 201/49."
201/49
}
ReasoningStep: "eence the fraction in lowest terms is 201/40
RRe:uinnn fractions lowest terms",
"Redue "Reduce the fraction 7 divided by (20! divided by 7) to lowest terms.",
Solution:"1. Start with the unreduced fraction: 7 / (20! 77).
Rewrite as a single fraction: 7 7 20! 49 / 20!.
Observe that 20! contains the factor 72 = 49, (49, 20!) 49.
4. Divide numerator and denominator by 49: Numerator: 49 49 Denominator: 20! + 495. Hence the fraction in lowest terms / (20! 49).
Compute 20! = 2 432 902 008 176 640 000, then divide by 49 to get 49 651 061 391 360 000.
7. Final reduced form is 1 49651 161 11611913300 3913660000."..
29

---

ReasoningStep: Set N 20!. Its prime factorization has & 8 distinct primes (2, 3, 5,7, 11, 13, 17,
19).",
Tag:"Prime Factorization",
Question: many distinct prime numbers appear in the prime factorization of 20!?",
Solution:
ReasoningStep:"BB By definition, 20! = 1x2x3x... x20, so every prime 20 divides 20!.",
Tag:" Deffiniinttiin of factorial",
Question:" 'In the definition of factorial, 20! is written as the product 1x2x3x... X20. How many integers are multiplied together in this product?",
Solution:"Bi By definition, n! 1x2x3x.. which multiplies all integers from 1 up to n. For 20, the integers are 1, 2, 3, ..., 20. Counting these gives 20 numbers being multiplied.",
ReasoningStep: "The primes less than equal to 20 are 2, 3, 5, 7, 11, 13, 17, and 19, and there are no others that range.".
Tag:"Primality Testing",
Question:" "araoa::AAmnn Among the integers from I through 20 inclusive, how many are prime numbers?",
Solution: 1. By definition, prime number is integer greater than that has no positive divisors other than 1 and itself.
2. We test each integer from 1 to 20: is not prime by definition. -2 is divisible only by 1 and 2 prime. 3 is divisible only by and 3 - prime. -4 is divisible by 2, 4 - not prime. 5 is divisible only and 5 prime. is divisible by 2, 3, 6 - not prime.
is divisible only by and - prime. 8 is divisible by 1, 2, 4, 8 - not prime. 9 is divisible by 3, 9 - not prime. 10 is is divisible only by and 11 prime. 12 divisible by 2. 3.4, 6, 12 - not prime. - 13 is divisible only by 1 and 13 - prime. 14 is divisible by 1, 7, 14 not prime. 15 divisible by 3, 15 15 not prime. 16 divisible by 1, 2, 8. 16 - not prime. 17 is divisible only by and 17 - prime. -
divisible by 1, 2, 10 not prime.
18 is divisible by 1, 2, 3, 6, 9, 18 not prime. 19 is divisible only by and 19 prime. 20 divisible by 2, 4, 5, 10, 20 not prime.
The primes this range are 2, 3, 5, 11, 13, 17, and 19.
4. Counting them gives a total of 8 primes.",
ReasoningStep:""hhrreoo,, the prime factorization of 20! includes exactly these distinct primes.". Tag:"Prim: Factorization",
"heen prime factorizing 20!, we include every prime number that is less than or equal to 20. How many distinct prime numbers appear the prime factorization of 20! ?",
Question:"
Solution:"I. A prime divides 20! exactly it is 20, since 20! is the product all integers from to 20.
The primes 20 are 2, 3, 5, 7, 11, 13, 17, and 19.
3. Counting these gives 8 distinct primes in the factorization of 20!." }.
30

---

ReeasoningStpp" Since "Since ged(a, b) = 1, each prime power Pf of must go entirely to a or b, giving 2 pairs b).".
Fundamaanttttall Theorem of Arithmetic",
Question:"Let be 20! and write in its prime factorization N = PP P2 Pk
Using the Fundamental Theorem of Arithmetic, in how many ways can you choose pair of positive integers (a, b) such that ab N and ged(a, b) 1 by assigning each prime power p entirely to cither a or 6?",
Solution: ReasoningStep:" BBy the Fundamental Theorem of Arithmetic, 20! factors into prime powers corresponding to each prime p 20.".
Tag:" "Prime Factorization".
Question:" "In the prime factorization of 20!, what is the exponent of the prime 2?",
Solution:" 1. By Legendre's formula, the exponent of prime p n! [n/p] [n/p²] [n/p']+ until the terms are zero.
2. For p= 2 and 20: [20/2] 10]20/4] 5|20/8] 2|20/16] [ (and all higher powers give zero)
3. Sum these: 10 + 5 2 = 18.",
[ReasoningStep."" "The primes 20 are 2, 3, 7, 11, 13, 17, 19, so there are k = distinct prime powers in the factorization of 20!.".
primes up to a given limit",
Question:"In the prime factorization of 20!, how many distinct prime bases (i.e., prime numbers 20) appear?",
Solution:"
1. To find the distinct prime factors of 20!, we list all prime numbers that are less than or equal to 20.
2. A prime number is number greater than 1 with positive divisors other than and itself.
Checking the integers from up to 20, the primes are: 2, 3, 5, 11, 13, 17, 194, Count these primes: there are 8 of them.",
31

---

(ReasoningStep:"TT form pair (a, b) with N and ged(a, = 1, each prime power P. must go entirely to a or entirely to b.",
TFnndmmnamt Theorem of Arithmetic",
Question: "Let N = 20! and write its prime factorization as N = =P PP" Theorem of Arithmetic, to form pairs of positive integers (a, b) with ab N and ged(a, b) 1, each prime power P must munttrentire you assign it to either y yroaorentire enoi ytob. www.arinneepiimpppcccceo
.....PP P& . By the Fundamental
how many ways can
that ged(a, b) 1?",
Solution:" The Fundamental Theorem of Arithmetic ensures N's prime factorization is unique.
2. To keep gad(a. b) you cannot split p between and b.
Therefore, for that prime power, you have exactly choices: Assign P to (so exponent in e,, iiniiiiii, -rassiggn to b (so exponent in a is 0, in is es).
4. No other distributions are allowed without violating ged(a, b) = 1.",
(Reasoning Step: Step:" of the prime powers has choices, so by the multiplication principle there are 2 2 ways.",
Tag: Fugdamnanttt Counting Principle",
"nesston:"Supposs" Suppose the prime factorization of 20! contains 8 distinct prime powers. you assign each prime power entirely to either integer integer b, in how many ways you make these assignments by the Fundamental Counting Principle?",
Question:"
Solution:"1. There are 8 prime powers to assign.
Each prime power has 2 independent choices: assign
b.
3. By the Fundamental Counting Principle, multiply the number of choices for each prime power: Total ways = 2+2.... 228 times) 2 = 256.".
}.
(ReasoningStep:"As As 20! is not a perfect square, * b, so half the pairs have < b, yielding 2 = 128.".
TTg:"Diviss Pairing Principle",
Question:" Usiig the Divisor Pairing Principle, in how many ways write 20! as product ab of two positive integers with gcd(a.))= 1 and a<b?",
32

---

Solution:
[ReasoningStep:" "The prime factorization of 20! involves exactly k 8 distinct primes (, 11, 13, 19).",
Tag:"Prime factorization",
Question: "In the prime factorization of 20!, how many distinct prime factors does it contain?",
Solution: By definition, 20! = 1-2 3 .... 20.
2. Every prime 20 divides one of the factors in the product.
3. The prime numbers less than equal to 20 are 2, 3,5, 7, 11, 13, 17, and 19.
4. There are such primes.",
}.
[ReasoningStep:" have ab = 20! and ged(a, b) 1, each prime's entire power in 20! must go either or b.".
Prime Factorization",
Question:" "In the prime factorization of 20!, what is the exponent of the prime 3?",
Solution:"1. 1. By unique prime factorization, the exponent of a prime p in n! is given by summing ffrk)fork- 2 luntilp >
2. For p=3 and n=20: [20/3] 62|20/9] 22/20/27] O(topphere)
Sum of these + 8.",
}.
ReasoningSSeep:"herrrfo there are 2 = 2 = 256 unordered assignments of prime-powers to (a, b).".
Tag: "Fundamental Counting Principle",
The prime factorization of 20! involves 8 distinct Prime-power factors. Suppose each entire prime-pove power factor must be assigned either to integer a or to integer Using the Fundamental Counting Principle, in how many ways these prime powers be distributed between and b?",
Question:"
Solution: There are 8 distinct prime-power factors in 20 (for primes 2, 3, 5, 7, 11, 13, 17, 19).
For each prime-power factor, we have exactly 2 choices: assign it to a or assign it to b.
3. By the Fundamental Counting Principle, the total number of ways to make all choices is 2 x 2 x
x 2 (8 factors) 28.
4. Compute 2 256.".
(ReasoningStep:: 'Since 20! not perfect square, no assignment yields a so exactly half of these yield < b, giving 256/2 = 128.".
hag::Syymeery argument in combinatorial counting".
Question:" "Supoose there are 256 ordered pairs of positive integers (a,b) such that ab = 20! and ged(a.b) 1. Using symmetry argument, how many of these pairs satisfy < b?".
Solution:"1. We are given that there are 256 ordered coprime factor pairs (a.b) with ab 20!.
2. For each ordered pair (a,b), there is a corresponding "swapped" pair (b,a).
3. Because 20! is not perfect square, no pair has b; every pair is distinct from its swap.
4. Thus the 256 ordered pairs split evenly into two groups: those with b and those with a
By symmetry, the number with < b half of 256, namely 256/2 128.".),
33


Reviews:
[{"review_id": "dTaybwu6ki", "reviewer": "Reviewer_A4Mp", "summary": "This paper introduces DECOMPX, a novel curriculum learning approach that recursively decomposes complex problems into simpler, more learnable components. \nThe authors propose a teacher-student framework where a powerful teacher model (GPT-4o) generates easier versions of examples by reasoning step-by-step. \nThe method also includes a novel scoring system for data difficulty based on structural complexity and conceptual depth, enabling curriculum construction over the decomposed data. \nExperiments on MATH and AIME datasets demonstrate improved performance compared to standard training baselines.", "questions": "* Could the authors provide more details on the specific prompts used for the teacher model during \"Step Extraction\" and \"Concept Tagging\"? How is the initial set of concept tags defined or discovered, and is there any pre-defined ontology or does it emerge purely from the teacher model's responses?\n* The description of training sets is somewhat vague. Could the authors clarify if Table 1 and Table 2 both utilize AIME2024 as the training set for their respective models, or if Table 1 uses the MATH dataset's training portion? Why DECOMPX use less data than SFT baseline in Table1 but more data in Table2. A detailed introduction of the training data for each experiment would be beneficial.\n* How is it guaranteed that every extracted \"step\" will always have a well-defined, verifiable answer? Some intermediate steps in a human-like reasoning chain might involve logical inferences, variable assignments, or reflections that do not directly yield a answer that can be \"solved\" or \"verified\" independently as a sub-problem. How does the current verification mechanism handle such cases, or are these types of steps implicitly ignored?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The core idea of recursively decomposing complex problems into simpler sub-problems, coupled with extracting and organizing \"concept tags\" into a dependency graph, is novel.\n* Models trained with DECOMPX exhibit better generalization, particularly on the AIME2025 dataset which represents an out-of-domain distribution shift from AIME2024.\n* The proposed scoring system, combining structural complexity and conceptual depth, provides a principled way to quantify problem difficulty within the decomposed dataset.\n\nWeaknesses:\n* The experiments appear to primarily focus on the decomposition of relatively short CoTs, lacking an in-depth exploration of how the recursive decomposition method performs and scales with long CoT solutions, which are common in complex mathematical reasoning problems and large reasoning models like DeepSeek-R1 and Qwen3.\n* While the paper highlights improvements on AIME, the AIME2024 training set used consists of only 30 problems. This is an extremely small dataset for fine-tuning language models in mathematical reasoning, making the observed performance gains potentially sensitive to this limited data size. The generalizability of conclusions drawn from such a small training set to broader mathematical reasoning tasks remains a concern.\n* The comparison with MetaMath-Aug, particularly with only 114 data points for AIME, appears to be a relatively weak baseline. The paper could benefit from comparing against a wider range of more recent and robust data augmentation methods in mathematical reasoning.", "was_revised": true, "final_justification": "Most of issues are resolved, including the clarification of long CoTs, the relative weak baseline, the detailed definition of \"step\", and other details. I choose to increase my score to 4.", "rebuttal": "We thank the reviewer for the detailed review and questions. We appreciate the recognition of the novelty of our recursive decomposition framework, the effectiveness of the concept tag dependency graph, the principled difficulty scoring system, and the improved generalization performance, especially under the OOD setting.\n\nBelow, we address each concern and question raised:\n\n**W1**\n\n> The experiments appear to primarily focus on the decomposition of relatively short CoTs, lacking an in-depth exploration of how the recursive decomposition method performs and scales with long CoT solutions, which are common in complex mathematical reasoning problems and large reasoning models like DeepSeek-R1 and Qwen3.\n\nThank you for raising this point. We interpret “long CoTs” in two ways and address both:\n\n* Long gold solutions for the original problems: Our method assumes access to a high-quality reference solution, which is standard in mathematical-reasoning benchmarks. On datasets such as AIME - representative in both problem complexity and solution length - our recursive decomposition performs reliably. \n\n* Long, model-generated traces (e.g., DeepSeek-R1, Qwen3): The framework is model-agnostic. We can control the length of each generated token number.\n\nOur experimental setup follows prior work and targets benchmarks commonly used. We agree that pushing to even longer-horizon tasks, e.g., automated theorem proving or high-dimensional geometry, would be valuable. While beyond our current scope, we view this as a natural extension and are happy to incorporate specific cases (e.g., very long traces from R1/Qwen3) if suggested. If this does not fully capture your concern, we welcome clarifications or pointers to target datasets/trace lengths you would like us to evaluate.\n\n**W2**\n> While the paper highlights improvements on AIME, the AIME2024 training set used consists of only 30 problems. This is an extremely small dataset for fine-tuning language models in mathematical reasoning, making the observed performance gains potentially sensitive to this limited data size. The generalizability of conclusions drawn from such a small training set to broader mathematical reasoning tasks remains a concern.\n\nWe would like to clarify that our goal is to demonstrate how a more effective data curriculum enabled by recursive decomposition can amplify a limited supervision signal for a complex problem into a more comprehensive and gradual training set.\n\nTo ensure a fair comparison, all methods start from the same original data (the 30 AIME2024 problems), serving as prior knowledge. We control the total number of training tokens  by adjusting training epochs inversely proportional to the ratio of token counts between different methods. This ensures that all approaches receive an equal compute budget. Therefore, the observed performance gains of our method are not due to more data or compute, but rather the improved structure and utility of the augmented data. The improved generalization to AIME2025 underscores the effectiveness and robustness of our curriculum design.\n\n**W3:**\n\n> The comparison with MetaMath-Aug appears to be a relatively weak baseline. The paper could benefit from comparing against a wider range of more recent and robust data augmentation methods in mathematical reasoning.\n\nWe conducted a baseline experiment of direct distillation from the same teacher model (GPT-4o) and the same original data (MATH training subset and AIME 2024, respectively). The comparison results shown in the table demonstrate the advantages of our method over direct distillation.\n\nTo further address this concern, we conducted an additional baseline using the MuggleMath dataset (147,787 examples). Evaluated on the same benchmarks, this baseline underperforms our approach despite leveraging substantially more training data and richer prior knowledge from the seed datasets (MATH and GSM8K). These results further support the advantage of our structured, decomposition-based curriculum.\n\n\n| Model (based on Qwen2.5-1.5B) | MATH 500 |\n| --- | --- |\n| SFT-Direct Distillation | 49.6 ± 2.2 |\n| SFT-MuggleMath | 50.4 ± 2.2 |\n| SFT-DecompX (Ours) | 50.8 ± 2.2 |\n| SFT-DecompX + Curriculum (Ours) | 51.6 ± 2.2 |\n\n| Task (based on Qwen3-4B-Base) | AIME 2025 |\n| --- | --- |\n| SFT-Direct Distillation | 6.67 ± 4.6 |\n| SFT-MuggleMath | 6.67 ± 2.2 |\n| SFT-DecompX (Ours) | 13.3 ± 6.3 |\n| SFT-DecompX + Curriculum (Ours) | 16.7 ± 6.9 |\n\n\n**Q1**\n> How is the initial set of concept tags defined or discovered, and is there any pre-defined ontology or does it emerge purely from the teacher model's responses?\n\nTo maintain the general applicability of our method, we deliberately avoided pre-defined ontologies; all concept tags are generated dynamically from the teacher model’s responses.\n\n> Could the authors provide more details on the specific prompts used for the teacher model during \"Step Extraction\" and \"Concept Tagging\"? \n\n- **Step Extraction Prompt:**\n\n```Python\nprompt = f\"\"\"\nSplit the following solution into at most {max_steps} reasoning steps.\nEnsure the steps are balanced in length and complexity.\nOnly split when the step introduces a new **mathematical operation**.\nIf the step is already atomic, do not split further.\n\nFormat your response as:\nStep 1: [first step content]\nStep 2: [second step content]\n...and so on up to at most Step {max_steps}.\nDo not include any additional text or explanation.\n\nSolution:\n{solution}\n\"\"\"\n```\n- **Concept Tagging Prompt:**\n\n```Python\ntag_prompt = f\"\"\"\nGiven the following reasoning step from solving a math problem:\n\n\\\"{step}\\\"\n\nOutput only one precise, specific high-school level mathematical concept (tag) relevant to this step.\nThis concept should represent the core mathematical knowledge required to understand and perform this step.\n\nImportant requirements:\n1. DO NOT use broad categories like \"Algebra\", \"Geometry\", or \"Number Theory\".\n2. Identify the exact skill or technique being used (e.g., \"Pythagorean Theorem\", \"Quadratic Formula\", \"GCD Calculation\").\n3. The concept should be atomic and specific enough to fit into a skill dependency graph.\n\"\"\"\n```\n\n**Q2: Clarification on Training Data (Tables 1 & 2):**\n\nFor Table 1, the baseline `SFT (full dataset)` refers to the original MATH dataset training portion (7500 examples), as explicitly mentioned in our paper (Line 215). Appendix 1 further details our balanced sampling subset for another baseline `SFT`. Our baseline `SFT-MetaMATH-Aug` and `DecompX` methods are generated from this subset (360 examples), chosen due to the extensive computational requirements of decomposing each problem. In Table 2, the AIME2024 dataset (initially 30 problems) was expanded into 385 subproblems using our decomposition method, surpassing the number of data points in the SFT baseline.\n\n**Q3: Ensuring Verifiable Answers for Extracted Steps:**\n\nSubproblems and answers are not directly generated from random steps; instead, they are generated based on concept tags that ensure each subproblem is associated with a specific mathematical operation from the step extraction. In the Appendix 4 of our submission, we listed the tag clusters identified in both the MATH and the AIME dataset for your reference, where there are no tags like `logical inferences`, `variable assignments`, or `reflections`.\n\nThe subproblem generation prompt explicitly requires questions to yield a \"specific numerical answer,\" ensuring solvability and verifiability:\n\n```Python\ncombined_prompt = f\"\"\"\nYou are helping to decompose and solve a math problem.\n\nOriginal problem:\n\\\"{problem}\\\"\n\nGiven the following reasoning step from solving the original problem:\n\\\"{step}\\\"\n\nand the relevant mathematical concept:\n\\\"{tag_topic}\\\"\n\nYour tasks:\n1. Generate a self-contained and standalone high school-level math **question** that:\n    - Is grounded in the **context of the original problem**.\n    - Asks specifically ONLY about the mathematical operation or concept in this individual step: {tag_topic}\n    - Does NOT include or reference information from earlier or later reasoning steps\n    - Requires a **specific numerical answer**.\n2. Solve the question with a detailed step-by-step **explanation** based on this reasoning step and original context.\n3. Output only a final **numerical answer**.\n\nStrict Format:\nQuestion: [your generated math question]\nExplanation: [step-by-step reasoning]\n|||\nAnswer: [final number only]\n\"\"\"\n```\n\n\nA verification process further ensures the correctness of generated subproblems, filtering out those that fail verification by generating and verifying repeatedly until passing, within a maximum retry limit. \n\n```Python\nfrom math_verify import parse, verify\nif answer:\n    try:\n        parsed_answer = parse(answer)\n        direct_answer_prompt = f\"\"\"\nSolve this question step by step.\n\n{question}\n\nProvide only the final numerical answer (e.g., 7.5, 4, 5/324), no explanation.\n\"\"\"\n        direct_answer = call_gpt(client, deployment_name, direct_answer_prompt).strip()\n\n        parsed_direct_answer = parse(direct_answer)\n        is_verified = verify(parsed_answer, parsed_direct_answer)\n```\n---\nWe thank the reviewer again for the constructive comments. We would be more than happy to further discuss if there are any remaining concerns or questions!"}, {"review_id": "Eh6MSW7GIV", "reviewer": "Reviewer_eiT7", "summary": "This paper proposes a curriculum learning approach built upon the concept of distillation/data augmentation. Applied to math problem solving, the approach entails decomposing math problems into subproblems based on reasoning steps generated by LLM. These subproblems are generated recursively and organized into a tree structure, tagged with difficulty levels using structural and conceptual features. Empirical results show marginal improvements over alternative methods on the training set but demonstrate significant gains on unseen datasets (AIME 2025) with encouraging generalization promise.", "questions": "1. Have you analyzed failure cases, particularly on challenging problems like AIME, to better understand the limitations of your approach?\n\n2. Have you considered human evaluation to validate the generated tags and difficulty levels, ensuring they align with human intuition?\n\n3. How well do you expect your method to generalize beyond procedural domains like math? do you have insights on what types of tasks is it best suited for and what are not suitable for?", "limitations": "The limitation was lightly discussed. I wish the authors could also discuss the social impact of this work. While it is interesting to design AI to be as smart as possible, the authors could also discuss how this approach can be used to improve math education, for example, using the approach to automatically generate scaffolding, determine the difficulty level, and tag the problem domains are very interesting contributions to math education and beyond.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Quality: This paper is well written, and the presentation of the proposed methods is both clear and sound, supported by solid experimental results. The work would be further strengthened by a deeper analysis of the results—for example, by examining failure cases to better understand the limitations of the approach. Additionally, incorporating human validation of the generated tags and difficulty levels would further enhance the credibility of the decomposition approach. \n\nClarity: Overall, the authors have done a decent job in presenting their ideas. The paper is clearly written, and the inclusion of illustrative examples and detailed algorithm descriptions supports comprehension. The use of a case study to demonstrate the decomposition process is particularly appreciated, as it offers concrete insight into how the proposed method operates in practice. \n\nSignificance: The paper presents a knowledge distillation approach to curriculum learning, demonstrated through the task of mathematical problem solving. While the authors suggest that the method can be generalized, a more detailed discussion on its broader applicability would be valuable. For instance, the approach appears well-suited for procedure-based problem solving, but its effectiveness in other domains remains unclear. Additionally, the definition of difficulty in the paper seems primarily tied to the number of steps required to complete a task. However, in some cases, difficulty also involves the need for non-obvious insights—such as re-framing a problem to allow for an elegant, minimal solution. This may help explain why, despite significant efforts (including the proposed enhancements), performance on solving new AIME problems remains relatively low.\n\nOriginality: The paper is situated in the literature of data augmentation/distillation with LLM as well as curriculum learning. The related work is detailed and relevant. The paper mentioned that the idea of curriculum learning with synthetic data is not entirely novel. I wish the authors could articulate better the contribution of this paper in the context of other similar methods, such as Tiny Stories, understanding it is from different domains.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive and thoughtful feedback and address their specific questions and suggestions below:\n\n**Q1**\n\n> Have you analyzed failure cases, particularly on challenging problems like AIME, to better understand the limitations of your approach?\n\nAs stated in our paper’s limitations (Section 5), the limitations of our approach are inherently bounded by the capabilities of the teacher model. Specifically, when the teacher model fails to conduct the Step Extraction in Section 3.1, the whole decomposition process is less effective. \n\nHowever, we would also like to point to some successful cases in Appendix 5 that illustrate how our method effectively deals with problems requiring \"non-obvious insights\". For instance, in the whole-prime-power assignment scenario (ab=N with gcd(a,b)=1), the key leap is recognizing that each prime power must be wholly assigned to either a or b; our Concept Dependency Graph captures this high conceptual depth (linking Fundamental Theorem of Arithmetic → coprimality → binary counting), so the curriculum surfaces the insight early. Similarly, for non-square symmetry halving (counting ordered factor pairs (a,b) of non-square N), the crucial step is seeing that a fixed point (a=a) occurs only for perfect squares; by tying parity tests to symmetry arguments, our graph again flags the step as concept-heavy despite its short reasoning chain, ensuring the curriculum allocates it appropriate emphasis. \n\n**Q2**\n\n> Have you considered human evaluation to validate the generated tags and difficulty levels, ensuring they align with human intuition?\n\nYes. We appreciate the reviewer’s suggestion regarding human evaluation. Our manuscript includes illustrative examples in Table 3, Appendix 2, and Appendix 5, showing strong alignment of our automatically generated tags and difficulty levels with human intuition. Additionally, Appendix 3 provides further validation through zero-shot performance analysis of the Qwen3-4B-Base model across difficulty tiers.\n\nWe would like to highlight that, given our method's generic and automated nature, it uniquely demonstrates broad applicability without relying on domain-specific prior knowledge or human expert intervention. Thus, our approach significantly reduces human cost by enabling effective generation of simpler problems independently of domain experts.\n\n**Q3**\n\n> How well do you expect your method to generalize beyond procedural domains like math? do you have insights on what types of tasks is it best suited for and what are not suitable for?\n\nTo explore generalization beyond purely procedural domains, we conducted additional experiments in the coding domain. We used the competitive programming dataset CodeForces-CoTs for training and HumanEval as our evaluation benchmark, and Qwen2.5-1.5B-Instruct as the base model. Our results demonstrate significant performance improvements using our approach:\n\n| Model | HumanEval pass@1 |\n| --- | --- |\n| Qwen2.5-1.5B-Instruct | 34.15 ± 3.71 |\n| Qwen2.5-1.5B-Instruct-SFT | 35.37 ± 3.74 |\n| Qwen2.5-1.5B-Instruct-DecompX (Ours) | 42.68 ± 3.87 |\n\nThese results underline the broader applicability of our decomposition method in domains that require structured reasoning, such as programming and symbolic reasoning. Conversely, less structured tasks or tasks demanding high levels of creativity or subjective interpretation (e.g., narrative storytelling or literary analysis) might benefit less from our structured curriculum approach.\n\n**Weakness**\n> I wish the authors could articulate better the contribution of this paper in the context of other similar methods, such as Tiny Stories, understanding it is from different domains.\n\nTinyStories highlights that data quality often outweighs data quantity for small language models. However, unlike our approach, TinyStories does not incorporate difficulty-based curriculum construction. Moreover, our method provides a generic synthetic data generation framework, contrasting with the dataset-specific focus of TinyStories.\n\n**Limitation: Social Impact and Educational Potential**\n\nFollowing the reviewer’s insightful suggestion, we will further emphasize the educational potential and social impact of our method, particularly in math education. Our approach can facilitate automatic scaffolding, dynamically build knowledge graphs, and design better curricula by precise difficulty assessments, thereby enhancing personalized learning experiences and supporting equitable educational practices. We will add this discussion to the revised manuscript.\n\n---\n\nWe thank the reviewer again for their constructive comments. We would be more than happy to further discuss if there are any remaining concerns or questions!"}, {"review_id": "9f7OqiQBVk", "reviewer": "Reviewer_UED6", "summary": "The authors propose a new method for generating curriculum (for curriculum learning), in which the teacher model recursively breaks down task into increasingly simple components for the student model to learn the curriculum in increasing difficulty. A new scoring system is proposed for measuring data difficulty during curriculum construction. Experiments are conducted on math datasets.", "questions": "Besides the issues noted weaknesses section, it would also be good to see if this method offers improvement on non-math datasets. The results in this submission currently run a bit thin.\n\nCould you also explain how you implemented Line 8 of Algorithm 1 in your experiments (i.e., what auto-rater you used and how consistency is checked)?\n\nAn interesting but not necessary thing would be to see if the proposed method can still offer big improvement when the teacher and student models have similar size, which could show some promise of self-improvement.", "limitations": "Limitations have been sufficiently discussed in the conclusion section.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThe writing is clear. The proposed method is convincing. I appreciate the case study in the submission.\n\nWeaknesses:\nI don’t think the experiment results are strong enough to demonstrate the superiority of the proposed method yet. First of all, only one other method is compared with in the experiments. More importantly, while I understand the reason behind some of the current choices of # data in Table 1 and 2, it would make more sense to have this number leveled across DecompX and comparison experiments. For example, both DecompX and DecompX+Curriculum use 4500 in Table 1 while MetaMATH-Aug only uses 2638 and SFT only 360. This makes the higher score of DecompX less convincing.", "was_revised": true, "final_justification": "The rebuttal from the authors have cleared some of my previous questions on some choices made in the experiment setup.", "rebuttal": "Thank you for your valuable feedback and constructive suggestions. We appreciate your recognition of our clear writing, convincing methodology, and the illustrative case study provided. \n\n**W1**\n\n> Only one other method is compared with in the experiments.\n\nWe conducted a baseline experiment of direct distillation from the same teacher model (GPT-4o) and the same original data (MATH training subset and AIME 2024, respectively). The comparison results shown in the table demonstrate the advantages of our method over direct distillation.\n\n| Model (based on Qwen2.5-1.5B) | MATH 500 |\n| --- | --- |\n| SFT-Direct Distillation | 49.6 ± 2.2 |\n| SFT-MuggleMath | 50.4 ± 2.2 |\n| SFT-DecompX (Ours) | 50.8 ± 2.2 |\n| SFT-DecompX + Curriculum (Ours) | 51.6 ± 2.2 |\n\n| Task (based on Qwen3-4B-Base) | AIME 2025 |\n| --- | --- |\n| SFT-Direct Distillation | 6.67 ± 4.6 |\n| SFT-MuggleMath | 6.67 ± 2.2 |\n| SFT-DecompX (Ours) | 13.3 ± 6.3 |\n| SFT-DecompX + Curriculum (Ours) | 16.7 ± 6.9 |\n\nTo further address this concern, we conducted an additional baseline using the MuggleMath dataset (147,787 examples). Evaluated on the same benchmarks, this baseline underperforms our approach despite leveraging substantially more training data and richer prior knowledge from the seed datasets (MATH and GSM8K). These results further support the advantage of our structured, decomposition-based curriculum.\n\n**W2**\n\n> It would make more sense to have this number leveled across DecompX and comparison experiments.\n\n\nThe discrepancy in the number of data points used between DecompX and other methods (e.g., MetaMATH-Aug and SFT) arises naturally due to our recursive decomposition process, which inherently generates multiple subproblems from each original problem. Importantly, our experiments were structured to provide a fixed knowledge budget. Thus, even though we used more decomposed samples, each smaller dataset (e.g., MetaMATH-Aug and SFT) received proportionally increased training epochs to maintain a fair comparison. This design choice demonstrates our method's effectiveness in enhancing performance without introducing new knowledge or data. \n\nTo ensure a fair comparison, all methods start from the same original data (the 30 AIME2024 problems), serving as prior knowledge. We control the total number of training tokens by adjusting training epochs inversely proportional to the ratio of token counts between different methods. This ensures that all approaches receive an equal compute budget. Therefore, the observed performance gains of our method are not due to more data or compute, but rather the improved structure and utility of the augmented data. The improved generalization to AIME2025 underscores the effectiveness and robustness of our curriculum design.\n\n**Q1: Generalization to Non-Mathematical Datasets**\n\nThank you for raising this valuable point. We have indeed explored our method's effectiveness beyond mathematics, applying it to a coding domain using the competitive programming dataset Codeforces-CoTs and the HumanEval benchmark. The base model was Qwen2.5-1.5B-Instruct. Our method resulted in significant performance improvements:\n\n| Model | HumanEval pass@1 |\n| --- | --- |\n| Qwen2.5-1.5B-Instruct | 34.15 ± 3.71 |\n| Qwen2.5-1.5B-Instruct-SFT | 35.37 ± 3.74 |\n| Qwen2.5-1.5B-Instruct-DecompX (Ours) | 42.68 ± 3.87 |\n\nThese results clearly demonstrate our method’s potential in non-mathematical domains.\n\n**Q2: Implementation of Line 8 (Algorithm 1)**\n\n> Could you also explain how you implemented Line 8 of Algorithm 1 in your experiments (i.e., what auto-rater you used and how consistency is checked)?\n\nWe clarify that the step employs an auto-solver (rather than an auto-rater). This refers to the process of generating a standalone answer to the subproblem and checking its consistency, as explained in Lines 112–117 of the main paper. Specifically, for each generated subproblem, the teacher model solves it twice: Once with access to the original context from which the step is derived; Once without any context, relying solely on the generated question. We then compare the two answers. Only when the answers match (verified by `Math-Verify`) do we accept the subproblem. Otherwise, we retry the process with small rephrasings until consistency is achieved or a retry budget is exhausted (Those that fail verification will be filtered out). It provides a reasonable and scalable mechanism to ensure basic correctness and filter out obviously invalid generations.This step is crucial to ensure that the generated subproblem is genuinely solvable and independent, thereby maintaining the reliability and trustworthiness of the decomposed dataset.\n\n**Q3: Teacher-Student Model Size Similarity:**\n\nThis is indeed an insightful suggestion. Our experiments indicate that smaller models struggle to produce clean and structured decompositions. Exploring the \"weak-to-strong\" generalization, where smaller models help train similarly-sized or even larger models, represents an exciting avenue for future research. We acknowledge this challenge and intend to pursue it in subsequent studies.\n\n---\n\nWe thank the reviewer again for their constructive comments. We would be more than happy to further discuss if there are any remaining concerns or questions!"}, {"review_id": "2fsFwtr9o4", "reviewer": "Reviewer_nBar", "summary": "This paper introduces a curriculum learning strategy where a \"teacher\" Large Language Model (LLM) recursively decomposes complex mathematical problems into simpler, hierarchically structured sub-problems. Each sub-problem is tagged with a concept and verified. A novel scoring system quantifies the difficulty of these sub-problems based on their structural complexity (number of sub-steps) and conceptual depth (derived from a \"Concept Dependency Graph\" that maps prerequisite skills). Smaller \"student\" LLMs are then trained on this decomposed dataset in stages of increasing difficulty. Experiments on MATH and AIME datasets show that this approach significantly improves the performance and generalization of student models compared to standard training and other data augmentation methods. The method's success hinges on the teacher model's capability.", "questions": "1. This method involves a more complex procedure than direct distillation and underperforms compared to distillation using a reasoning model. What advantages does it offer over directly distilling with a reasoning model?\n\n2. Is this method effective for reasoning models? Can it further enhance their performance?\n\n3. How dependent is this method on the capability of the Teacher Model? Can it still achieve good results when the Teacher Model itself performs poorly?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Novel Methodology: The paper proposes a novel \"reverse curriculum generation\" approach that recursively decomposes complex problems into simpler, learnable components using a teacher-student framework. This hierarchical decomposition is a key innovation.\n\n2. Sophisticated Difficulty Scoring: It introduces a unique system for measuring data difficulty based on both \"structural complexity\" (number of sub-problems) and \"conceptual depth\" (derived from a \"Concept Dependency Graph\" of skills). This allows for more granular curriculum construction.\n\n3. Dataset Insights: A byproduct of the method is the potential to create a \"cartography\" of a dataset, offering insights into skill coverage and relationships between concepts.\n\nWeaknesses:\n\n1. Reliance on a Capable Teacher Model: The entire framework heavily depends on the availability and proficiency of a strong \"teacher\" LLM (like GPT-40) that can accurately reason, decompose problems, and generate correct sub-tasks. The quality of the generated curriculum is tied to the teacher's capabilities.\n\n2. Incomplete Evaluation Scope: The paper primarily evaluates the method's effectiveness on base student models (Qwen2.5-1.5B and Qwen3-4B-Base). It does not include experiments applying the DECOMPX training methodology directly to reasoning models of comparable sizes to see if similar or enhanced gains are achieved on reasoning models.\n\n3. Limited Comparison with State-of-the-Art Distillation: This approach shares some similarities with knowledge distillation, as both leverage capability transfer from larger models to enhance smaller ones. However, experimental results demonstrate that neither the 4B nor 1.5B parameter models can match the performance of the DeepSeek-R1-Distill-Qwen-1.5B model, which is directly distilled from Deepseek-R1.", "was_revised": true, "final_justification": "This paper explores an interesting direction: leveraging a strong teacher model to decompose datasets in a way that facilitates learning for smaller models. While the authors present promising results and have addressed some concerns in the rebuttal, I still have the following reservations:\n\nGeneralizability remains unclear. The paper does not evaluate whether the method improves performance for reasoning-capable student models, nor does it test on out-of-distribution datasets (e.g., training on a subset of MATH and evaluating on Math500, or training on AIME 2024 and testing on AIME 2025).\n\nEffectiveness at scale is uncertain. Although the authors show improvements over direct distillation in small-scale experiments, their best-performing model underperforms compared to existing strong baselines such as DeepSeek-R1-Distill. It remains unclear whether the proposed method can match or surpass such baselines in large-scale training settings.\n\nOverall, I appreciate the novelty and the direction of the work, but these concerns limit my enthusiasm for a strong recommendation.", "rebuttal": "We thank the Reviewer for the insightful questions and comments. We appreciate the recognition of the novelty of our decomposition-based curriculum learning approach, the principled difficulty scoring mechanism, and the useful insights enabled by our dataset analysis framework. We address each question and concern below:\n\n**Q1 & W3**\n\n> What advantages does it offer over directly distilling with a reasoning model?\n\nWe appreciate the opportunity to clarify the advantages of our method compared to direct distillation.\n\nFirst, given identical original data (same prior knowledge) and the same teacher model, our proposed dataset decomposition approach outperforms direct distillation. Furthermore, our approach is compatible with existing distillation methods, enabling further performance improvements. \n\nWe conducted a baseline experiment of direct distillation from the same teacher model (GPT-4o) and the same original data (MATH training subset and AIME 2024, respectively). The comparison results shown in the table demonstrate the advantages of our method over direct distillation. \n\n| Model (based on Qwen2.5-1.5B) | MATH 500 |\n| --- | --- |\n| SFT-Direct Distillation | 49.6 ± 2.2 |\n| SFT-DecompX (Ours) | 50.8 ± 2.2 |\n| SFT-DecompX + Curriculum (Ours) | 51.6 ± 2.2 |\n\n| Task (based on Qwen3-4B-Base) | AIME 2025 |\n| --- | --- |\n| SFT-Direct Distillation | 6.67 ± 4.6 |\n| SFT-DecompX (Ours) | 13.3 ± 6.3 |\n| SFT-DecompX + Curriculum (Ours) | 16.7 ± 6.9 |\n\n**Q2 & W2**\n\n> However, experimental results demonstrate that neither the 4B nor 1.5B parameter models can match the performance of the DeepSeek-R1-Distill-Qwen-1.5B model, which is directly distilled from Deepseek-R1.\n\nWe respectfully point out that the comparison is not fair for three reasons. **(1) Training data scale and prior knowledge.** DeepSeek-R1-Distill-Qwen-1.5B is trained on ~800k SFT samples curated with DeepSeek-R1, which injects substantially more prior knowledge than our setup. In contrast, our training data are purely derived from a small MATH training subset and AIME 2024 via decomposition, without access to large external corpora. **(2) Different base models.** The base for DeepSeek-R1-Distill-Qwen-1.5B is Qwen2.5-Math-1.5B (math-specialized), whereas our 1.5B and 4B students are base models (e.g., Qwen2.5-1.5B Base, Qwen3-4B Base) without math specialization; this materially advantages the distilled model. **(3) Compute and reproducibility.** The distilled training set for DeepSeek-R1-Distill-Qwen-1.5B is not publicly released, making it impossible to match compute/data budgets or to run controlled re-trainings under identical conditions.\n\nCrucially, our goal is different and complementary: we show a cost-effective pathway to improve small models’ reasoning by structuring limited supervision. Training R1-distilled models at the 800k-example scale is not inexpensive; our method offers orders-of-magnitude lower data/compute while achieving strong gains and better generalization from easy-to-hard progression.\n\nFinally, our approach is compatible with direct distillation rather than a replacement. Decomposition can augment distillation by (1) filtering/structuring teacher signals into prerequisite-aligned sub-tasks, and (2) staging training by difficulty to stabilize optimization for small models. We view combining R1-style distillation with our decomposition-driven curriculum as a promising direction and are actively exploring it.\n\n> Is this method effective for reasoning models? Can it further enhance their performance?\n\nThank you for your question. Given time constraints and the significant cost of evaluating reasoning models, we are currently actively conducting experiments on reasoning models, to evaluate the effectiveness of our method. We will update our results and provide full metrics as soon as the experiments are complete.\n\n**Q3 & W1**\n\n> How dependent is this method on the capability of the Teacher Model? Can it still achieve good results when the Teacher Model itself performs poorly?\n\nWe appreciate this important question and would like to clarify both the scope and contributions of our work.\n\nOur method is designed to leverage the compositionality of language reasoning to enhance small models’ learning, especially in challenging reasoning tasks. A key motivation for our approach is the observation that small models often struggle to learn complex reasoning patterns from randomly shuffled data. Our recursive dataset decomposition explicitly addresses this by breaking down complex problems into hierarchical sub-problems, enabling a more controlled and interpretable learning curriculum.\n\nAs stated in our paper’s limitations (Section 5), we assume access to a reasonably capable teacher model to generate valid decompositions. This assumption is shared with many prior works in knowledge distillation. Importantly, our method includes built-in verification steps and retry mechanisms to ensure the correctness of generated subproblems, mitigating the risk of low-quality outputs from imperfect teacher models.\n\nWhile the method’s current implementation relies on a relatively strong teacher (e.g., GPT-4o), we view exploring the weak-to-strong generalization regime, where smaller or less capable models can still serve as effective teachers for equally-sized or even larger students, as an exciting direction for future work. We acknowledge this as a meaningful challenge and intend to explore it in subsequent research.\n\nWe also emphasize that our approach offers distinct and non-replaceable advantages over traditional distillation methods. Specifically, it enables (1) granular control over the learning trajectory of small models, (2) interpretability via explicit sub-task structuring, and (3) curriculum-guided training, which is especially critical for reasoning tasks where intermediate skills are essential.\n\nIn practice, small base models are highly desirable due to their lower cost and hardware efficiency, and are increasingly deployed in real-world scenarios. Our framework provides a principled way to improve their reasoning capabilities via structured and concept-aware training data—an area where direct distillation alone often lacks fine-grained control or gradual guidance.\n\nIn summary, while our method benefits from a strong teacher model, its design offers robustness and generalizability, and opens up promising future directions for more flexible teacher-student configurations.\n\n---\n\nWe thank the reviewer again for their constructive comments. We would be more than happy to further discuss if there are any remaining concerns or questions!"}]

Author rebuttal:

Meta-review: This paper received three borderline accept and one accept rating. The authors raised some concerns, which were mostly addressed in the rebuttal by authors. The reviewers appreciate, the novelty of the method, the clear writing, and the insightful discussion. The AC agrees that the paper should be accepted.
Decision: Accept (poster)