SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
Authors: Yanwei Ren
Haotian Zhang
Fuxiang Wu
Jiayan Qiu
Jiaxing Huang
Baosheng Yu
Liu Liu
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models (LLMs)
Monte Carlo Tree Search (MCTS)
Data Augmentation
Large Language Models (LLMs)
Sibling Guidance
Primary area: applications

Abstract: Enhancing large language models by simply scaling up datasets has begun to yield diminishing returns, shifting the spotlight to data quality. Monte Carlo Tree Search (MCTS) has emerged as a powerful technique for generating high-quality chain-of-thought data, yet conventional approaches typically retain only the top-scoring trajectory from the search tree, discarding sibling nodes that often contain valuable partial insights, recurrent error patterns, and alternative reasoning strategies. This unconditional rejection of non-optimal reasoning branches may waste vast amounts of informative data in the whole search tree. We propose SIGMA (Sibling Guided Monte Carlo Augmentation), a novel framework that reintegrates these discarded sibling nodes to refine LLM reasoning. SIGMA forges semantic links among sibling nodes along each search path and applies a two-stage refinement: a critique model identifies overlooked strengths and weaknesses across the sibling set, and a revision model conducts text-based backpropagation to refine the top-scoring trajectory in light of this comparative feedback. By recovering and amplifying the underutilized but valuable signals from non-optimal reasoning branches, SIGMA substantially improves reasoning trajectories. On the challenging MATH benchmark, our SIGMA-tuned 7B model achieves 54.92\% accuracy using only 30K samples, outperforming state-of-the-art models trained on 590K samples. This result highlights that our sibling-guided optimization not only significantly reduces data usage but also significantly boosts LLM reasoning.

SIGMA: Refining Large Language Model Reasoning via Sibling-Guided Monte Carlo Augmentation
Yanwei Ren 1.2 Haotian Zhang!. Fuxiang Wu Jiayan Qiu
Jiaxing Huang Baosheng Yu Liu Liu
SScooo of Artificial Intelligence, Beihang University Hangzhou International Innovation Institute, Beihang University
3 Shenzhen Institutes of Advanced Technology. Chinese Academy of Sciences Univversity of Leicester
Nanyang Technological University
Abstract
Enhancing large language models by simply scaling up datasets has begun to yield diminishing returns, shifting the spotlight to data quality. Monte Carlo Tree Search (MCTS) has emerged as powerful technique for generating high-quality chain-of-thouge data, yet conventional approaches typically retain only the top- scoring trajectory from the search tree, discarding sibling nodes that often contain valuable partial insights, recurrent error patterns, and alternative reasoning strate- gies. This unconditional rejection of non-optimal reasoning branches may waste vast amounts informative data in the whole search tree. We propose SIGMA (Sibling Guided Monte Carlo Augmentation), novel framework that reintegrates these discarded sibling nodes to refine LLM reasoning. SIGMA forges semantic links among sibling nodes along each search path and applies two-stage refine- ment: a critique model identifies overlooked strengths and weaknesses across the sibling set, and a revision model conducts text-based backpropagation refine the top-scoring trajectory in light this comparative feedback. By recovering and amplifying the underutilized but valuable signals from non-optimal reasoning branches, SIGMA substantially improves reasoning trajectories. On the challeng- ing MATH benchmark, our SIGMA-tuned 7B model achieves 54.92% accuracy using only 30K samples, outperforming state-of-the-arr models trained on 590K samples. This result highlights that our sibling sibling-uudeed guided optimization not only sig- nificantly reduces data usage but also significantly boosts LLM reasoning.
Introduction
Scaling laws show that large language model (LLM) performance typically improves as model size and training data increase [23]. However, recent studies show that simply adding more common training data leads to diminishing returns, especially on complex reasoning tasks [38, 50]. This has led to shift toward structured supervision, where detailed annotations help the model solve problems step by step [29, 12, 50, 55, 3]. Among these methods, chain-of-thought (CoT) expla- nations have proven especially effective [50], helping LLMs break down and solve reasoning tasks in a more organized way. Large-scale CoT datasets, such as DART-Math (~-590K samples) and MMIQC (~2.3M samples), have demonstrated strong effectiveness in improving mathematical rea- soning. Recent studies further show that even medium-sized models can achieve competitive per- formance when trained on such datasets [46, 31]. As illustrated in Figure 1, high-quality datasets can substantially enhance LLM reasoning capabilities while requiring far fewer training samples.
Corresponding author: liuliubh@buse Code is available at https: /////ggtttbb// wonm/frank1100845//1111
cn.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Performance MATH
60
9ISMAJ0K
MuthFusion (80K)
RFT [500K]
DARTMath (3BOK)
GPT-4
Accuracy
50
(788K)
KM/GC (E200K)
45
+16.8
Methfuusoon
40
+10.7
+9.1
35
DespBoskWathBase-TP
Data size
Figure I: An illustration the reasoning performance fully fine-tuned DeepSeckMath-77 7B models on the MATH benchmark dataset. The models trained different datasets, with training data sizes ranging from 15K to 2300K.
However, constructing such datasets demands substantial computational resources and human ef- fort, underscoring the need for more efficient methods generate high-quality data. A promising direction is to automatically synthesize CoT examples using tree- ree-based search strategies. Recently, Monte Carlo Tree Search (MCTS) has been adopted for this purpose [6], where a language model explores a branching structure by proposing and scoring candidate reasoning steps. After sufficient exploration, MCTS selects single high-reward path as the final solution. While effective, this approach discards all sibling nodes-alternative reasoning steps that were explored but ultimately not selected. These discarded paths often contain informative signals, such as partially correct rea- soning steps, recurring error patterns, and alternative reasoning strategies. Ignoring them limits the potential to extract comprehensive and structured feedback from the search process.
To address the above-mentioned issue, we introduce new framework that reuses this lateral in- formation improve the selected reasoning path. We propose SIGMA (SIbling Guided Monte Carlo Augmentation), which augments MCTS- reasoning by treating sibling nodes sources of symbolic feedback. At each step along the selected path, SIGMA compares the chosen node to its siblings to derive natural language critiques. These critiques function as approximate gradi- ents-textuul signals indicating how the reasoning step should be revised. revision model then uses these signals to refine each step in the path. SIGMA integrates the strengths of search-based exploration and feedback-basee optimization. Inspired by recent advances in symbolic supervi- sion via LLM-generated feedback [58], it enables reasoning refinement without additional rollouts or ground-truth labels. By systematically incorporating information from all nodes produced dur- ing search, SIGMA produces higher-qualit reasoning paths while maintaining computational ef- ficiency. Experiments show that this refinement significantly improves downstream performance, offering data-efficient alternative to large-scale dataset construction.
Our main contributions this paper are as follows:
We introduce SIGMA, new framework that improves the selected reasoning path by incorporating sibling nodes discarded during MCTS. We aim to develop principled data- synthesis framework that systematically exploits previously overlooked information to hance the reasoning capabilities of larage language models.
We leverage only the existing information from the MCTS search tree to perform sym- bolic optimization over reasoning paths, requiring no additional rollouts or external reward models. This design enables seamless integration into existing data generation pipelines.
Our produced SIGMA-15K outperforms all 30K-scale baselines, and SIGMA-30K re- mains better or competitive with 60K-scale methods across multiple base model. Our results demonstrate the effectiveness of the proposed framework, which leverages sibling nodes to optimize reasoning paths.

---

2
Related Works
Math Data Synthesis Large math corpora such as WIZARDMATH and METAMATH have pushed chain-of-thought supervision into the millions, while multiple studies report that accuracy plateaus once data volume exceeds a few million examples [33, 56, 57, 5, 47]. To break this ceiling, re- cent work explores three complementary directions: difficulty-aware sampling, which allocates more generation or retention budget to problems deemed hard via pass-rate variance or confidence scores [45]; tool-augmented generation, which weaves external library calls into the reasoning trace so the model offloads symbolic or numeric sub-tasks [48, 16]; and feedback driven filterin,, which keeps only traces that satisfy verifier or preference signals, prioritising step-level quality over raw quantity [28, 30, 54, 61, 60]. Building difficulty sampling. DART-Math adapts the sam- pling budget on-the-fly, giving extra dnrection-samplnng trials to harder questions [46]; while Math- Fusion fuses multiple source problems into sequential, parallel, and conditional aneega-pompps" to promote relational reasoning [37]. However, these methods seldom exploit the internal structure of incorrect or discarded traces, missing opportunities to learn from intermediate reasoning failures.
Language-Model Feedback Optimization Recent work has begun to close the feedback loop by letting a model use its own outputs as training signals. The idea traces back to SELF-REFINE, where a model iteratively critiques and rewrites its answers to improve without new data [34]. DSPY generalises this to full pipelines, compiling graph of text transformations whose prompts tomatically tuned [24]. Reflection then becomes explicit: SELF-RAG inserts special tokens so the model learns when retrieve and how critique its draft [2], while RL CONTEMPLATION treats the model's s own evaluations as a reward, removing the need for external labels [36]. In mathematics, -HEEHERDD builds fully automatic, step-level reward pipeline that verifies and reinforces reasoning without human annotations [49]. The paradigm is unified by TEXTGRAD, which back- propagates natural-language feedback through generative pipelines ranging from code molecular design [58]. Extensions already leverage this principle to repair code via divide-and-conquer con- sensus [7] and to scale automated process verifiers for math reasoning [41]. Yet most frameworks lack mechanisms to propagate fine- ame-grained feedback across alternative reasoning paths, limiting their ability to exploit structured variation during search.
Math Reasoning with Monte-Carlo Tree Search As single-path CoT prompting reaches its lim- its, growing body of work turns to branching search, letting the model explore multiple par- tial solutions before committing. Tree-structured exploration offers an alternative to single-chain prompting. The idea first appeared in plicit breadth- or depth-first search over "thoughts" [55]. Subsequent work replaced exhaustive search with Monte-Carlo Tree Search (MCTS): REST-MCTS* performs process-reward-guuiee rollouts and iteratively retrains the policy and value heads [59]; ALPHHAMATH-ALMOOST-ZZAE adds a learned value head to steer step-level beam search without human labels [6]: MULBERRY runs collective MCTS across multiple models to build a 260k multimodal rre-oo-reasoning cor- pus [53]: VERMCTS couples MCTS with a lightweight verifier to synthesise step-correct programs for proofs [4]; and peference-learning
which framed reasoning
variants treat each rollout as a pairwise comparison and fine- tune via DPO [52]. While these approaches lift accuracy by sampling and selecting stronger reason- ing paths, they also incur heavy inference cost and depend on auxiliary reward models or external tools, which complicates deployment. Moreover, current methods discard sibling branches after selection, underutilizing the diverse intermediate reasoning states already produced during search.
3 Method
In this section, we introduce the proposed SIGMA framework, a two-stage approach that integrates MCTS-based CoT data generation with sibling-level refinement to improve the quality of selected candidate reasoning paths. As shown in Figure 2, the process begins with an MCTS- -ased reasoning engine that explores the space of multi-step CoT paths and selects an optimal path of depth D based on high-reward feedback. This selected path is then refined iteratively, one step at a time, using feedback from its sibling nodes. At each depth, we first compare the chosen node with its siblings to identify discrepancies that serve as feedback. A critique model then acts symbolic gradient oracle, generating directional cues in natural language rather than numerical gradients. These textual gradients are used to revise the candidate path through a textual gradient descent (TGD) process.
3

---

The following subsections detail the generation of candidate CoT paths using MCTS, the construc- tion of sibling guidance, and the refinement of these paths based the guidance.
Sibling Guided Monte Carlo Augmentation
Sibling Nodes
Selected Nodes
Depth
Sibling Nodes
A1
A2
82
A2
B3
Depth
Critique Model
CLEM
81
82
83
A2
Depth
C2
C3
C4
C3
Revision Model RIL
B3
Original Reasoning Path
SIGMA Refined Reasoning Path
Figure 2: The main SIGMA framework for enhanced CoT data generation.
3.1 MCTS Reasoning Path Selection
We leverage MCTS as tree search procedure to select a promising reasoning path. Starting from the initial state (the problem statement), MCTS iteratively builds search tree of possible next reasoning steps (nodes), simulating outcomes and using those simulations to decide which branches expand. We denote the finally selected path as = {n("), p(()) .....P? .P!)) where p(d) is the index of the branch chosen at depth d {1,2, .... D} and D is the largest depth. Each simulation (rollout) of MCTS reaches depth D, obtains terminal reward R (e.g. correctness or utility of the final answer), and then backpropagates this reward up the tree. During backpropagation, each node on the path of the simulation updates its value estimate V, to reflect the new outcome, For example, on each depth d, one common update is an incremental average:
V Vn
RR V.,).
N, +1
where Nn is the visit count for node This recursively propagates the simulation's result to all ancestor nodes, akin to Bellman backups in dynamic programming. Additionally, MCTS uses a selection policy to balance exploration and exploitation. We adopt the Upper Confidence bound for Trees (UCT) criterion to select the child of node by UCT:
In N N.
C = arg max UCT(n.J), where UCT(n, + jEC(n)
(2)
V. is the current value of the child c, N: denotes the counts visits of the child c, C(n) denotes the set of all child nodes of node and cp is exploration constant. This UCT formula encourages traversal of high-value branches (V, large) while occasionally exploring less-visited siblings (via the second term). After sufficient number of simulations, the search converges high-value path. Finally, MCTS selects only the top path: T (the sequence of branch choices (1:D) is returned as the candidate solution CoT with highest estimated reward. We next describe how we refine this path based feedback.
using sibling- bbased
3.2 Sibling Guidance to Refine Reasoning Path
illustrated the left panel of Figure To capture latent supervision from nearby alternatives, we define symbolic loss over sibling nodes. At each depth d (1...., D), we compute discrepancy
ا(ام between the selected node (d) and its siblings, defined as the children of p(d-1) excluding p (d) itself. For simplicity, we omit the subscript "(d)" Let Tp be the textual content of the selected step,
and (eS()))))))) the texts of siblings, where S(p) denotes the set of all sibling nodes that share the same parent as the selected node p. This restriction to siblings under shared parent ensures that all

---

Sibling Guidance
CoT Refinement
You are part of an optimization system that improves original reasoning paths. Your task to creatively and critically refine the original reasoning peth according the key point
System prompt:
System prompt: are brillient math-solution reviewer Your evaluate the sibling nodes and Identrfy both positive and negative aspects the sibling nodes in order further improve the Original path.
Critique Model
Cous + (T, (T, S(p)))
TGD step!T, G) Ruad (Tp, G)
Original path Sibling
Evaluation:
Step 0-1:...
0-2-1c..;
Key points
>2((0-))Step:)
A (10 (ttpp:
(0 (O-t)St
Step
Step 0-2:....
Step 0-2-3:...
0-2-2:...
For Step 0-2...
For Step 0-2-3...;
Revision
Refine
(ttep-))teppoh
OCas 0T,
Ca (T. S(p))
Al Step Ba Step 2:
(0-222))tepp))
Key
Originel path (10=2) ((0-2-3)Stmp:)
used improve
Original path:
4
Step
Step 0-2-3:..;
Figure 3: Visualization of the gradient computation process involving sibling nodes and the original optimal path. Node identifiers follow hierarchical format: for instance, 0-2 denotes the second node at depth I, and 0-2-3 denotes the third child of node 0-2 at depth 2.
candidates are conditioned on the same local context, preserving alignment in the partial reasoning path. As a result, observed differences can be reliably attributed to step-level content rather than upstream variation.
Lixt = 4 (Tp, (T's
S(p))),
where & is symbolic operator that compares the selected content to its siblings, which synthesizes the comparisons between the selected node p and all its siblings s E S(p). Unlike scalar loss values, Luext is structured natural language output that consolidates multi- culti-perspective judgments into unified textual feedback. This provides a contrastive signal at each decision point, where sibling steps act localized references that reveal potential omissions errors.
Since fiest is non-differentiable the conventional sense that operats over natural language rather than continuous parameters, we introduce Critique Large Language Model CLLM that serves as symbolic gradient oracle. Given (Tp: (e)))))))) CLLM produces natural language critique that suggests how to revise Tp to reduce the loss.
G 0Leett 8T,
CLIM (Tp, (T, S(p)))
The output G not numerical gradient, but symbolic directional cue encoded in natural language. For instance, CLLM may produce message such as: "Add justification for this computation." " or "Avoid ambiguity present in sibling s." These critiques serve as surrogate gradients for optimizing textual reasoning through revision.
3.3 Revision of Original Reasoning Path
Figure 2 presents textual example of a candidate CoT data alongside its refined version generated by the proposed SIGMA framework. Given the textual gradient G, we update the corresponding reasoning step by applying single iteration of Textual Gradient Descent (TGD). In classical gra- dient descent, a parameter x is updated as x <- T 7) 0T./Ox, where 1j the step size. our case, there is explicit numerical step size arithmetic subtraction. Instead, we employ a Revise Large Language Model RELM that plays the role of an optimizer. Given the current text T, and its corresponding gradient G defined in (4)(4), the revise LLM RILM generates an improved version Л intended reduce the loss Liett- This update is abstracted as TGD.step(-) and defined follows:
/ + TGD.step G) = FRIM((T) G).
(5)
That is, T, revised based on the critique G. For instance, G indicates that an arithmetic step lacks justification, the revise model RLLM will attempt to incorporate the necessary rationale, yiclding
5

---

From MATH
Selected MCTS path Sibling MCTS nodes Critique feedback Optimized reasoning steps
What the remainder when 7^2010 is divided by 100?
Critique Model
Revision Model
Answer:49
Step 1: Apply ELier's Theorem reduce the exponent. Euler's totient function gives up(000) 40, su: 2010 7a12010 mod 40) 7-10 [moc 190)
Depth
The selected node relies on pattern observation, while sibling node applies Euler's theorem for principled reduction. Lmprove by grounding the cycle pattern in Euler's theorem to make it generalizable.
Euler's theorem, (K(00)==40 Reduce
detect periodicity from Xon mod 100
exponent so 2>2010
7010 mad 100
Depth 2
7^4=1modi00
= 7>2010
Manually compute
The selected node performs explict expansion, whereas sibling node simplifies with moduler rules, Replace exponentiation symbolic nodular reduction streamiine reasoning.
Step 2: Compute intermediate powers We compute: 7- 2401 2401 mod 100
(7^4)-502 x 7*2
powers: Reduce: 7~2010 (7~8)~251
+ Reduced to computing 7 mod 100
Depth 4
mod 1000>251 x 49
Compute (5764801
Compute giant lest- two two-digits: 01~251 x49 enswer
The selected nade completes computation decimal multiplication, sibling stays fully med space.
Step Final
The remainder 7^2010 answer Is $495
mod 100 + D1>1
by
49 = 49 = The answer
49,
is $195
Original MCTS
Critique
SIGMA Path
Figure 4: A textual example of the proposed SIGMA framework including three aspects: Original MCTS, critique feedback, and SIGMA path.
more complete Tр. This procedure is applied iteratively in step-wise manner. At each depth d, a single textual gradient update is applied to Tyg0, while keeping all other steps fixed. Sequential updates across d = to D transform the original path T into a revised chain T1, which exhibits greater coherence and robustness. Each update is localized and incremental, resembling a single coordinate-we wise descent step in a high- sigh-dimensional optimization problem. After D such updates, the procedure effectively completes one full pass of coordinate descent over the chain-of- of-thouggt.
The sibling-guided TGD process may be viewed as iterative refinement, where each reasoning step is adjusted in light of its siblings, ultimately yielding more coherent global solution. Notably, our method operates in a model-agnostic manner and does not require access to the internal gradients or parameters of the underlying LLM. All updates are conducted in the space of natural language, mediated through CLIM and RELM- The final output is a refined chain-of-thouuht that preserves the structural strengths discovered by MCTS while improving local reasoning quality through targeted textual feedback. For instance, Figure 4 showcases the SIGMA framework's 's workflow using the mathematical problem: "What's the remainder when 72010 is divided by 100". In addition, some illustrative examples are provided in Appendix F,By refining the chosen MCTS path with step- specific critique feedback. SIGMA path improved the rigor and logical consistency though shares the same answer as the original path.
4 Experiments
4.1 Experimental Setups
Reasoning Path Selection. We adopt the Qwen2. 5-Math-7B as the generation model to construct
search trees based on prompts from MATH [20] and GSM8K [13]. At each node, we decode n = candidate completions and select the highest-scoring one from k: = 5 samples, ranked by hog-probability, with a maximum tree depth of 16. To enhance diversity, we generate two MCTS datasets using decoding temperatures of 0.4 and 0.7, each contributing 15K examples to form a combined 30K training set. From each search tree, we extract the path with the highest cumulative Q-value. At every depth d, we retain the selected node p(d) and up to two sibling nodes S(p(d) from the same parent node to construct a candidate set for step-wise refinement.
Original Reasoning Path Refinement. For each step, we construct a loss&critique prompt that includes the textual content of the selected node Ty60 and its sibling nodes ())( This prompt is fed into the critique model to evaluate and compare the reasoning quality among the candidates, producing a textual gradient ((d) that highlights potential improvements. A separate revision prompt

---

is then constructed using both T,00 and G(d), which is passed to the revision model to generate refined version of the reasoning step. This booling-guided refinement is applied independently at each depth d, resulting in local improvements without altering the overall structure of the reasoning path. We use GPT-4o-mini -222--0----022---11- as both the critique model and the revision Model.
4.2 Implementation Details
We fine-tune three representative base models: math-specialized language model, DeepSeekMath- 7B [42], and two general-purpose models, LLaMA3-8B [17] and Mistral-7B-v0. [22]. We full- finetuned and evaluated those base models on 4 x H100 GPUs. To evaluate both effectiveness and generalizability, we conduct experiments using the SIGMA-refined 15K and 30K training subsets. For each model, we search for the optimal combination of learning rate and batch size, keeping all other hyperparameters fixed. We follow the official DART-Math evaluation protocol [46] and reuse their publicly released test scripts evaluate all models under a zero-shot greedy decoding setting. For each base model, we report performance on both in-domain (ID) and
(OOD) mathematical reasoning tasks. The ID evaluation includes GSMSK [13] and MATH [20]. The OOD evaluation include four benchmarks: College.aath [44], which contains 2,8 18 university- level problems spanning seven mathematical domains; DeepMind Mathematics [40], curriculum- aligned suite of 1, 1,000 problems designed for students up to age 16; OlympiadBench-Man which consists of 675 Olympiad-level problems from international contests; and TheoremQA [9], which evaluates symbolic reasoning using theorem statements from various STEM disciplines.
[19],
We compare SIGMA-tunee models against wide range of competitive baselines, including both instruction-tuned and reinforeement-pptimizedd approaches. These include MetaMath [56], Wizard- Math [33]. MMIQC [31], MathScale [44], RefAug [60]. DART-Math [46] and MathFusion [37]. For DeepSeekMath-72 -B, we additionally include its official supervised variant [42]. Base models that are fine-tuned directly on MATH and GSM8K without augmentation are reported under the
Standard setting. Most baseline results are taken from MathFusion [37] or DART-Math [46], where
applicable.
4.3 Main Results
Table summarizes the performance of base models fine-tuned SIGMA datasets (SIGMA- 15K and SIGMA-30K), alongside models fine-tuned on large-scale datasets generated by various baseline methods. methods..uuthermoor Furthermore, we present extended analysis of performance across six different benchmarks in Appendix B, offering a complete view of the evaluation results.
SIGMA-15K Beats All 30K Models. With only 15K training examples, SIGMA-15K outper- forms all prior methods trained on 30K samples across all three model backbones. For example, on DeepSeckMath-7B. SIGMA- 15K achieves 47.0 average accuracy, exceeding the best 30K baselines such as MathFusion-DSMatthh -B (45.7) and MathFusion-DSMann DSMath-7B (Sequential) (45.7). Similar trends are observed on Mistral-7B (31.6 vs. 29.9 from MathFusion) and LLaMA3-SB (36.1 vs. 35.6 from MathFusion), demonstrating SIGMA's superior data efficiency.
LLaMA3-8B Series. The upper part of the table presents the performance of the LLaMA3-8B model series. Controlling for identical data budgets (30K samples), our SIGMA-8B-30K model achieves an overall average of 37.7, 1-pint absolute gain over the MathFusion-30K baseline. Notably, SIGMA-30X outperforms MathFusion-30K on every benchmark, with the largest improve- ment on the DeepMind set (+5.5). Even when using only 15K samples, our SIGMA-15K model still achieves a 36.0 average, 0.4 points higher than MathFusion-30K trained with double the data. Moreover, scaling to 60K samples yields significant boost to 40.2 average accuracy, surpassing DART-Math-6000 (37.6) and establishing a new best performance within this family. These results highlight SIGMA's strong data efficiency and scalability.
The second part of Table reports fine-tuning results on the Mistral-7B- v0.1 backbone. With a 30K sample budget, SIGMA-7B-30K achieves 34.9, a 5.0-point gain over MathFusion-30k (29.9), and outperforms it on every task, most notably + +14.5 on DeepMind. Even using just 15K samples, SIGMA-7B-15K scores 31.6, still 1.7 points above MathFusion-308 When scaled to 60K samples, SIGMA-7B-60X reaches 36.9, outperforming the DART-Math-60K model (32.9) and demonstrating consistent improvement with data scaling.
Mistral-7B Series.
7

---

Table 1: Performance comparison across base models and training strategies. Results are reported 0). All scores were obtained
exact-match accuracy under 0-shot greedy decoddnng(eemperttue from the first attempt. Arrows indicate accuracy changes relative to the baseline in blue background. Some results are quoted from MathFusion [37] and are quoted from DART-Math [46].
In-Domain
Out-of-Doman
Model
Samples
AVG
MATH GSM8K College
DM
Olympiad Theorem
LLAMA3-SB (General Base Model)
Llama3-8B- Hama3-8B-MettMaall
400K
32.5
77.3
20.6
35.0
5.5
13.8
30.8
Llama3-8B-PP 8---FT
590K
39.7
81.7
23.9
41.7
9.3
14.9
35.2
Maima--B-DARTT-MA
590K
46.6
81.1
28.8
48.0
14.5
19.4
39.7
Llama3-8B- -amaa-8B-MMIQC
2.3M
39.5
77.6
29.5
41.0
9.6
16.2
35.6
Jlama--SGGMA-88-1555
15K
36.0
82.074.1
24.2
42.0
10.5
26.015.036..11.
Llama--SIGMMA-9B-3000
30K
40.812.0
79,
26.310.8
47,515.5
12.7
19.112.1
37.712.1
MathFusion (Sequential)
30K
38.8
77.9
25.1
42.0
12.6
17.0
35.6
MathFusion (Conditional)
30K
34.7
76.9
21.2
27.4
11.9
15.5
31.3
MathFusion (Parallel)
30K
38.1
75.4
25.5
41.9
11.9
18.9
35.3
Llama3-8B-Metall
60K
28.7
78.5
19.7
31.3
5.3
16.1
29.9
60K
24.4
69.7
13.4
30.9
5.2
10.6
25.7
Llama3-8B-RefAuu
60K
20.3
68.6
15.5
29.1
5.5
13.0
25.3
Llama3-8B-DA
60K
39.6
82.2
27.9
39,9
12.9
22.9
MathFusion-LLama3-88
60K
79.2
27.9
20.0
JJama3-SIGMMA-5B-660000
60K
44.9
82.413.2
49.215.8
21.31
40.211.2
Mistral-7B-vO.1 (General Base Model)
Mistral 7-MeetMath
400K
29.8
76.5
19.3
28.0
5.9
14.0
28.9
Mistral-7B- WizardMath-VL.1
Haarrll-B-WizzrMatthe
418K
32.3
80.4
23.1
38.4
7.7
16.6
33.1
Mistral-7B-RFT
590K
38.7
82.3
24.2
35.6
8.7
16.2
34.3
Mistral-7B-DARR 11B-DART-Math
590K
45.5
81.1
29.4
45.1
14.7
17.0
38.8
Mistral-STGNAA-7-1335
15K
30.0
75.3114
20.811
7.7
16.31u.8316611
Mistral-SIGMAA-B-300E
30K
35,512.8
78.614.7
13.233.
14.5
11.1118
18.012.534.915.
MathFusion (Sequential)
30K
32.7
73.9
18.9
29.3
9.3
15.5
29.9
MathFusion (Conditional)
30K
26.3
73.0
15.6
21.4
7.3
12.8
26.1
MathFusion (Parallel)
30K
30.9
75.1
20.9
26.5
11.0
15.2
29.9
Distral-B-MMIIOS
60K
17.3
61.4
11.1
13.5
5.0
5.9
19.0
Histral-7B-RefAAu
60K
17.4
63.1
12.5
18.1
3.9
11.1
21.0
Distral-IB-MetaMMan
60K
22.7
70.8
14.1
27.2
5.0
12.2
25.3
60K
34.1
77.2
23.4
18.2
32.9
MathFusion-Mistiee
Mistral-7B
60K
41.6
79.8
24.3
13.6
18.1
Mistral-SIGMMA7B--600
60K
79.2
46.116.9
19.211.1
36,910.8
DeepScekMath-TT
-ilatt--ppcializedd Base Model)
DeepSeekMath-7B-RU8
590K
53.0
88.2
41.9
60.2
19.1
27.2
48.3
DeepSeekMath-7B-DD
S-----ART---AAE Math
590K
53.6
86.8
40.7
61.6
21.7
32.2
49.4
DeepSeekMath-7B-Innstee
780K
46.9
82.7
37.1
52.2
14.2
28.1
43.5
DeepScckMath-7--MMIII
2.3M
45.3
79.0
35.3
52.9
13.0
23.4
41.5
DeepSeeNMath:SSCMAA-PBBBB
15K
52,212.3
81.1
37.4
64.5
20.3
N6.11.347.0113
DeepSekNaath-SSGMMA-PB...B
30K
54.915.0
82.2
36.7
67.212.
21.6
26.613.8
48.212.5
MathFusion (Sequential)
30K
49.9
76.6
38.8
64.6
21.6
22.8
45.7
MathFusion (Conditional)
30K
48.5
74.6
37.0
55.2
19.3
19.0
42.3
MathFusion (Parallel)
30K
50.9
76.7
38.9
62.2
19.0
23.8
45.3
D M-MMIQC
60K
26.3
60.6
19.2
41.5
10.4
6.8
27.5
60K
33.1
71.6
26.2
35.4
10.5
14.0
31.8
MerrSeeeLMatt-77-MMeaaaaaaa
60K
40.0
79.0
33.2
45.9
9.5
18.9
37.8
DeepScel.auth-7--DARR---.
60K
$1.4
82.9
39.1
62.8
21.0
27.4
47.4
MathhFussorr-DeepSceettttth--
60K
53.4
39.8
65.8
24.6
47.5
DeepSsekkNattssSICAAAAAPP-a-a
60K
56.513.1
81.7138
37.2
68.412.6
22.5
29.314.749.3113

---

Table 2: Ablation study on 15K datasets. Results reported exact-match accuracy under O-shot greedy decoding(temperature 0). SIGMA outperforms both the unrefined MCTS best-path and GPT-4o-mini CoT (Blackbox) baselines under identical training settings.
Model
# Samples MATH GSM8K College
DM
Olympiad Theorem
AVG
DeepSeekMatth BB-Blackbox
15K
50.5
77.3
38.6
62.0
19.1
25.8
45,5
PerpSeekMath-7B-MCCC
15K
40.7
58.3
30.2
43.7
18.1
10.7
33.6
PeepSeckMatt-SSIGMA-BBB.B..
15K
52.211.7
81.113.8
37.4
64.512.5
20.3112
26.1
46.91144
Llama3-8B-Blackben
1SK
34.5
77.9
21.1
39.9
11.1
16.6
33.5
Llama3-8B-MCH
15K
27.8
40.2
18.3
12.6
23.6
Jama3-SIGMA-8B-I555
15K
36.011.5
82.014.1
24.213.1
42.012.1
10.5
22.015.4
36.112.6
Mistral-7B-Blacke -B-Blackbox
15K
31.2
20.6
37.7
10.5
6.1
29.0
Mistral-7B-MCTT
15K
21.7
45.6
20.1
25.4
11.1
8.3
22.0
Mistral-SIGMMA7B-11555
15K
30,011.2
75.317.
20.810.2 39,5118
7.703.4
16.318.0
31.612.6
Table 3: Comparison different Critique-and- and-Revision (C&R) contigurations under 15K training samples. Results are reported pass @ I mathh accuracy under 0-shot greedy decoding (tem- perature = 0). (C&R) denotes using the model as a Critique and Revision model.
C&R Model Method
MATH GSMSK College DM Olympiad Theorem AVG
Llama3-8B Series
Qwen2.5-785 7B Instruct (C&R)
35.3
77.1
21.7
39.5
9.6
16.9
33.3
Qwen2.5-72B InItruttt (C&R)
36.3
78.9
23.2
39.4
18.0
34.3
GPT4o-mini (C&R)
36.0
82.0
24.2
42.0
10.5
22.0
36.1
MathFusion (30K Avg.)
37.2
76.7
23.9
37.1
12.1
17.1
34.1
Mistral-7B-vd. Series
Qwen2.5-7BB 7-0.5---B-InttrcC (C&R)
28.8
79.3
18.4
33.3
13.5
Qwen2.5-722 Been2.5-72B-Bie Ins-Intructt (C&R)
30.6
76.8
21.1
35.3
6.8
15.6
31.0
GPT4o-mini (C&R)
30.0
75.3
20.8
39.5
7.7
16.3
31.6
MathFusion (30K Avg.)
30.0
74.0
18.5
25.7
9.2
14.5
28.6
DecpSeckMathh7BB- B-Base Series
Qwen2.5- 5-w.---BBBB- 5--5---Bs-stttt Instruct (C&R)
49.8
80.6
38.3
60.6
18.4
24.1
45.3
Swen2.5-72B-Instruu (C&R)
51.3
81.9
39.9
64.8
17.5
23.8
46.5
GPT4o-mini (C&R)
52.2
81.1
37.4
64.5
20.3
26.1
47.0
MathFusion (30K Avg.)
76.0
38.2
60.7
20.0
21.9
44.4
DeepSeek.Math-75 Series. Our DeppSeeMaatt-STMMA-77---- model achieves a 48.2 average across six benchmarks, 2.5 points higher than MathFusion-30K (45.7) and even 0.7 points above MathFusion-DSMATTHH trained on 60K samples (47.5). SIGMA-7B-155. still attains 47.0, 1.3 points above ting new state-of-the-ee among all tested models. These results show that with just 30K-60K synthetic examples, our SIGMA pipeline not only outperforms all MathFusion variants regardless of data budget, also scales effectively with larger synthetic datasets.
Further scaling to 60K examples yields 49.7 average accuracy. set-
Overall, with a comparable or smaller number of lokens(Detaile) token-level statistics are provided in Appendix D), SIGMA datasets yield consistent and significant improvements across all model backbones and data budgets. Even small-scale SIGMA sets (15K) outperform large-scale baselines (30K-60K), while scaling 60K examples further extends the advantage, highlighting the robust- ness and scalability of our synthetic data generation strategy. Beyond mathematical reasoning tasks, we further evaluate the applicability of the SIGMA framework on non- mathematical domains. reported in Appendix C, the results demonstrate that our approach exhibits strong generality across diverse tasks.
4.4 Ablation Studies
To better assess the impact of SIGMA's sibling-guided refinement, we conduct two controlled ab- lation experiments across all three base models: DeepSeckMath-7B, LLaMA3-8B, and Mistral-7B-

---

v0.1. In all settings, the number of training samples is fixed at 15K, and training configurations remain identical SIGM
Comparison to vanilla MCTS paths. We construct a baseline by extracting the selected best path from each MCTS tree used in SIGMA-15K, without applying any refinement. These paths are di- rectly used as training targets, forming the MCTS- dataset. As shown in Table 2, SIGMA significantly outperforms vanilla MCTS across all models: +13.3 on DeepSeekMath-777 (46.9 vs. 33.6), +12.5 on LLaMA3-SB (36.1 1 v.s 23.6), and +9.6 on Mistral-7B (31.6 v.s 22.0). These improve- ments highlight SIGMA's ability densify the learning signal from existing MCTS trees without additional generation.
Comparison to black-box CoT generation. To further isolate the effect of sibling- based refine- ment, we compare our SIGMA-15K against BLACKBOX-15K a dataset generated by prompting GPT-4o-min1-2024--0022
with standard prompt obtain Chain-of-Though (CoT) outputs for each problem in the SIGMA-15K query set. Despite sharing the same generator model, SIGMA con- sistently outperforms black-box generation on MeepSeekMath (+1.4), LLaMA3 (+2.6), and Mistral (+1.6) in average accuracy. Notably, SIGMA achieves this without relying on full-solution sampling, instead leveraging internal structural comparisons among sibling nodes to guide refinement.
Replacing the Teacher Model. To examine the generality of SIGMA's Critique-and-Rent Revisiioo (C&R) pipeline. replace the teacher model used in refinement with several alternatives, including Quen2. S5-BB-Instruce Qwen2. IInstuuc,,, and EPT-4o-mini. As reported in Table 3, all teacher models substantially enhance student performance compared the MathFusion (30K Avg.) baseline, even under the same 15K data budget. Notably, GPT-4o-mini (C&R) achieves the strongest results across all three backhones, reaching 47.0 average accuracy DeepSeekMath-78. 36.1 on LLaMA3-SB, and 31.6 on Mistral-7B. These findings indicate that SIGMA's refinement process is highly transferable: larger or more capable teachers yield proportionally better student performance, yet even smaller open-weight models (e.g., Qwen2.5-7B) maintain strong gains. This demonstrates that SIGMA's C&R mechanism is not tied to a specific teacher model and can gener- alize across instruction tuned LLMs with varying scales.
These results confirm that SIGMA enhances the supervision quality of MCTS data by extracting and reusing informative contrastive signals from partial rollouts. Even under identical data budgets and generator capabilities, SIGMA yields superior downstream performance.
5 Conclusion
In this work, we focus on enhancing the utility of MCTS-generatee reasoning traces by revisiting the discarded sibling nodes. We propose SIGMA, a refinement framework that leverages sibling comparisons and gradient-like language model feedback to improve the quality of step-by-step rea- soning data. By applying SIGMA to MCTS paths, we construct compact yet high-quality datasets (SIGMA- 15K,SIGMA--30 and SIGMA-60K), which consistently outperform much larger base- lines across multiple base models. Despite using only one-fourth to one-half the training volume of typical instruction-tuning datasets, SIGMA consistently improves mathematical reasoning perfor- mance across model families and task domains.
Limitations. While our framework demonstrates promising results, several limitations remain. First, the use of GPT-4o-mini as the backbone model may restrict overall performance due to its relatively limited capabilities compared to larger models. Second, we focus on full fine-tuning of 7B-scale models and do not explore larger models, which could potentially yield stronger results.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (62441617, KZ37 773313 XZ37145801 and the Research Funding of Hangzhou International Innovation In- stitute of Beihang University (Grant No. No.20024K095),,
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal. Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprini xxXiv23033087774 2023.
[2] Akari Asai, Zeqiu Wu, Yizhong Wang. Avirup Sil, and Hannameh Hajishirzi. Self-RAG: Learning to re- trieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2024.
[3] Maciej Besta, Nils Blach, Ales Kubicck, Rohert Gerstenherger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Nicwiadomski, Piotr Nyezyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682-17690, 2024.
[4] David Brandfonbreneer, Simon Henniger, Sibi Raja, Tarun Prasad, Chloe R Loughridge, Federico Cas- sano, Sabrina Ruixin Hu, Jianang Yang. William E. Byrd, Robert Zinkov, and Nada Amin. VerMCTS: Synthesizing multi-step programs using a verifier, a large language model, and tree search. In The 4th Workshop on Mathematical Reasoning and Al at NeurlPS'24 24, 2024.
[5] Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, arXiv preprint arXiv:2407.11787, 2024.
[6] Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: Process supervision without process. The Tiihtt- Annual Conference on Neural Information Processing Systems, 2024.
[7] Jingchang Chen, Hongxuan Tang, Zheng Chu, Qianglong Chen, Zekun Wang, Ming Liu, and Bing Qin. Divide-and-conquer meets consensus: Unleashing the power of functions in code generation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[8] Justin Chih Yao Chen, Zifeng Wang. Hamid Palangi, Rujun Han, Sayna Ebrahimi, Long Le, Vincent Perot, Swaroop Mishra, Mohit Bansal, Chen- Yu Lee, et al. Reverse thinking makes Ilms stronger reason- ers. arXiv preprint arXiv:241 1. 19865, 2024.
[9] Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma. Jianyu Xu, Xinyi Wang, and Tony Xia. TheoremQA: A theorem-drrven question answering dataset. In The 2023 Conference Empirical Methods in Natural Language Processing, 2023.
[10] Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeyhi, Bryan Catanzaro, and Wei Ping. Acereason nemotron: -eeeetsonnnnmmotoon: Advancing math and code reasoning through reinforcement learning. arXiv preprint 250.::::55056644000. 2025.
[11] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.088 055455557, 2018.
[12] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthiss Plappert, Jerry Tworek, Jacoh Hilton, Reichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.144 2021.
[13] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, al. Training verifiers solve math word problems, arXiv preprint arXiv:21 10. 14166, 2021.
[14] Ganqu Cui, Lifan Yuan, Zefan Wang. Hanbin Wang. Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Pracess reinforcement through implicit rewards. arXiv preprint 2025.
[15] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use laptop? a question answering benehmark with implicit reasoning strategics. Transactions of the Association for Computational Linguistics, 9:346-361, 2021.
[16] Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang. Minlie Huang, Nan Duan, and Weizhu Chen. ToRA: A tool- integrated reasoning agent for mathematical problem solving. In The Twelfth Inter- national Conference Learning Representations, 2024.
11

---

[17] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama herd of models. arXiv preprint urXiv:240..21783, 2024.
[18] Xinyu Guan, Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun. Yi Zhu, Fan Yang. and Mao Yang. math: Small Ilms can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501.04511 2025.
[19] Chaoqun He, Renjie Luo. Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu. Xu Han, Yujic Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientifie problems. pages 3828 3850, 2024.
[20] Dan Hendrycks, Collin Burs, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang. Dawn Song, and Iacob Steinhardt. Mcasuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
[21] Cheng- Yu Hsich, Chun-Liang Li, Chih-Kuan Ych, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen- Yu Lee, and Tomas Pfister. Distilling step-by-step outperforming larger language mod- els with less training data and smaller model sizes. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8003-8017, 2023.
[22] Alhert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William Sayed. Mistral 7b, 2023.
[23] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint 083162001.08361, 2020.
[24] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan A. Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines, In The Twelfth International Conference on Learning Representations, 2024.
[25] Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 141.6644,,6900 2014.
[26] Chen Li. Weiqi Wang. Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng. Common 7b language models already possess strong math capabilities. arXiv preprint 0tti::20030047066 2024.
[27] Liunian Harold Li, Jack Hessel, Youngjae Yu, Xiang Ren, Kai-Wei Chang, and Yejin Choi. Symbolic chain-of thoughttt distillation: Small models can also "think" bep-bb-sstep. (C), pages 3665-2673 2023.
[28] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024.
[29] Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146 2017.
[30] Zhan Ling. Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. De- ductive verification of thain-of-thought reasoning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[31] Haoxiong Liu, Yifan Zhang, Yifan Luo, and Andrew C Yao. Augmenting math word problems via iter- ative question composing. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 24605-24613. 2025.
[32] Zihan Liu, Yang Chen, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Acemath: Advancing frontier math reasoning with post-training and reward modeling. arXiv preprint arXiv:2412.15088 2024,
[33] Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang, Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint 095583,,, 2023.
12

---

[34] Aman Madaan, Niket Tandon, Prakhar Cupta, Skyler Hallinan, Luyu Gao. Sarah Wiegreffe. Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang. Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann. Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refine- ment with feedbbcc In Thirry- Conference on Neural Information Processing Systems, 2023.
[35] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg. Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. In International Conference on Learning Representations, 2018.
[36] Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. Language model self-improvement by reinforcement learning contemplation. In The Twelfth International Conference Learning Representations, 2024.
[37] Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li, Honglin Lin, Chenlin Ming. Xin Gao, Conghui He, and Rui Yan. Mathfusion: Enhancing mathematic problem-solving of Ilm through instruction fusion. arXiv preprint arXiv:2503.1021 2025.
[38] Jack W Rae, Sehastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young. et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint 1r 1446, 2021.
[39] Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong Hc. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 3505-3506, 2020.
[40] David Saxton, Edward Grefenstette, Felix Hill, and Pushmect Kohli, Analysing mathematical reasoning abilitics of ncural models. In International Conference Learning Representations, 2019.
[41] Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng. Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for LLM reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
[42] Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning open lan- guage models. arXiv preprint arXiv:2402.0 03300, 2024.
[43] Alon Talmor, Jonathan Herzig. Nicholas Lourie. and Jonathan Berant, Commonsensequ: question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Compututional Linguistics: Human Language Technolo- gies, Volume 1 (Long and Short Papers), pages 4149-4155, 2019.
[44] Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. Mathscale: Scaling instruction tuning for mathematical reasoning. In Forty-first International Conference on Machine Learning, 2024.
[45] Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Pong, Yunjie Ji, Han Zhao, and Xian- gang Li. Deepdistill: Enhancing IIm reasoning capabilities via large-scale diffieulty-grades data training. arXiv preprint arXiv:2504.1 17565, 2025.
[46] Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He., DART-math: Difficulty-wware rejection tuning for mathematical problem-solving In The Thirty-eighth Annual Conference on Neural Information Prucessing Syxtems, 2024.
[47] Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Git- man. Openmathinstract-2: Accelerating AI for math with massive open-source instruction data. In The Thirteenth International Conference on Learning Representations, 2025.
[48] Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. Matheoder: Seamless code integration in lims for enhanced mathemat- ical reasoning. In The Twelfth International Conference on Learning Representations, 2024.
[49] Peiyi Wang, Lei Li. Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms steep without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 9426-9438 2024.
[50] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837. Curran Associates, Inc., 2022.
13

---

[51] Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models monsense models. Proceedings of the 2022 Conference of the North Americun Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies, pages 4602-4625, 2022.
[52] Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Yen Kan, Timothy Lillicrap. Kenji Kawaguchi, and Michael Shich. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint arXiv:2455.0045/ 2024.
[53] Huanjin Yao, Jiaxing Huang. Wenhao Wu, Jingyi Zhang. Yibo Wang, Shunyu Liu, Yingjie Wang. Yuxin Song, Haocheng Feng. Li Shen, et al. Mulberry: Empowering milm with ol like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319 2024.
[54] Jiarui Yao, Yifan Hao, Hanning Zhang, Hanze Dong, Wei Xiong, Nan Jiang, and Tong Zhang. Optimizing thain-of-thoughy reasoners via gradient variance minimization in rejection sampling and rl. arXiv preprint arXiv:2505.02399 02391,, 2025.
[55] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Thirry-seventh Conference on Neural Information Processing Systems, 2023.
[56] Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Zhang, James Kwok, Zhenguo
Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfth International Conference on Learning Representations, 2024.
[57] Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXi::2308.001 01825, 2023.
[58] Mert Yuksekgonul, Federico Bianchi, Joseph Bocn, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback. Nature, 639:609- 616, 2025.
[59] Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mets*: Llm self-training via process reward guided tree search. Advances in Neural Information Processing Systems, (1664755-67772 2024.
[60] Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. The Thirteenth International Confer- ence on Learning Representations, 2025.
[61] Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu. Wei Gao, and Min Lin. Chain of preference optimization: Improving chain-of-thought reasoning in LLMs. In The Thirty-eighth Annual Conference on Neural Information Processing Syxtems, 2024.
14

---

A Evaluation Result on Qwen2.5-Math-7B
Table 4: Comparison of SIGMA and other math reasoning models based on Qwen2.5Math-
Results are reported aspass @1 exact-match accuracy under 0-shot greedy decoding (temperature 0) across six mathematical reasoning benchmarks.
Model
# Samples
MATH
GSMSK
College
DM
Olympiad
Theorem
AVG
Estar@Greedy
747K
78.40
89.70
52.50
47.10
Farus-2-7B-PRIMA
230K
53.88
42.99
20.97
47.30
22.22
18.62
34.33
3.2M
40.52
73.39
32.93
54.70
11.41
13.63
37.76
2.5M
82.16
95.15
46.80
82.00
41.60
40.75
64.74
AceMath-7B-Instruu
2.3M
75.40
58.38
45.82
59.70
39.85
30.75
51.65
DART-Math-601
60K
63.14
91.28
47.23
74.90
26.81
35.38
56.46
MathFusion-660
60K
56.14
59.59
37.08
67.70
41.19
27.87
48.26
SIGMA-60K (ours)
60K
79.92
89.23
45.17
88.80
43.41
47.00
65.59
To further verify the broad applicability of the SIGMA framework beyond DeepSeck-Math-T7 model. As shown in Table 4, SIGMA-60K
conduct additional experiments on the achieves average accuracy of 65.6, outperforming all baseline models by clear margins across six benchmarks.
Compared with Rstar[ [8], SIGMA-60K achieves similar results on MATH and GSM8K while show- ing stronger robustness on more complex tasks such as DeepMind and Theorem. Relative to Eurus- 2-7B-PRIME(14 and
SIGMA surpasses them by +31.3 and +27.8 points in average accuracy respectively, despite using far fewer training samples. When compared to the mraehh.5math-ing instruct base model, SIGMA-60K still gains +0.9 overall, with particularly large improvements on DeepMind (+6.8) and Theorem (+6.2). Over AceMath AccMMath-7--Instruelpp IB-Inssruct((3], the aver- age gain reaches +13.9, showing consistent advantage across all tasks. Furthermore, SIGMA-60K outperforms both DART-Math--a
and MathFusion-60K by +9.1 and +17.3 in average accuracy, respectively, achieving the best results on every individual benchmark.
Overall, these results confirm that SIGMA's improvements are not limited to a specific base model: even when applied to Qwen2.5Math-77B the framework maintains strong generalization and delivers substantial gains across diverse mathematical reasoning tasks.
B Analysis of Performance Across Six Different Benchmarks
In Appendix B, We provide a benchmark level breakdown across six diverse mathematical reason- ing tasks (GSM8K[13), MATH[20), College[44], DeepMind[40], Olympiad[ 19), and Theorem(9)) highlighting SIGMA's 's robustness and its ability to generalize across varying levels of problem dif- ficulty.
To systematically quantify how each data construction technique impacts performance across our six benchmarks: MATH, College. DeepMindd Olympiad, and Theorem. We present differ- ent benchmarks' analysis comparing SIGMA with MetaMath [56], MMIQC [31], RefAug [60], DART-Math, and MathFusion (Figure 5). We highlight both absolute and relative gains, as well as consistency across the three backbone models, to demonstrate SIGMA's robustness and general applicability.
60K-Sampled Datasets Comparsion Across Six Benchmarks. GSMSK: SIGMA consistently outperforms MathFusion across all base architectures. Furthermore, it matches the performance of DART-Math, and remarkably, all three models converge to nearly identical accuracies when fine-tuned with our method. MATH: On the more demanding MATH benchmark. SIGMA deliv- ers substantial gains over DART-Math for every backbone. In particular, it also exceeds MathFu- sion's results on both LLaMA and DeepSeekMath, underscoring its robustness on complex arith- metic reasoning. College: In the College evaluation, SIGMA yields pronounced improvements for Mistral and DeepSeekMath. This demonstrates the strength of our data generation in enhanc- ing performance on intermediate-difficulty problems. DeepMind: SIGMA outstrips all competing cearoaches-innlute DART-Math and MathFusion wide margin on DeepMind. The consis- tent uplift across every model highlights its effectiveness on advanced reasoning tasks. Olympiad:
15

---

GSMBK
MATH
College
Mitral
Mistal
40
MEstal
DeepSookMMath
DeepSeekharhh
75
70
63
Retkig NottRWieeSSGMA
MeaatshMMINC
Reisag
DeepMind
Olympiad
Theorem
LLaMA
LLaMA
LlaMA
DespleslMtt
DeepleekMath
DeepSeckMlth
90
40
30
MMICC RelAgg
DART MathFaicESOMA
MeaMashMM3CCC RefAag DART MatFFssceSSOMMA
MMIQU Refsu DART MahFisieSIGMA
Figure 5: Performance comparison of models fine-tuned 60k-sample datasets generated by dif- ferent methods, evaluated across six benchmark tasks. Different colored dots represent different methods
GSMBK
MATH
College
LL-MA
LLMMA
Mistrl
Masul
Molnd
35
75
23
DeepMind
Olympiad
Theorem
10
12aM
LLMMA
ILaMA
Midrrl
Matul
Motid
25
DeepSeckltoh
25
Depfockhain
20
30
Sequestal
Retlug
Rethug
Seppertial
Figure 6: Performance comparison of models fine-tuned on 30k-sample datasets generated by dif- ferent methods, evaluated across six benchmark tasks. Different colored dots represent different methods.
For the Olympiad benchmark, SIGMA falls slightly short of MathFusion on LLaMA and Mistral but outperforms it on DeepSeekMath. Crucially, it still achieves clear, uniform improvements over DART-Math across all three architectures. Theorem: On Theorem, SIGMA secures a clear advan- tage over MathFusion, confirming its ability to generate data that strengthens formal and symbolic reasoning.
30K-Sampled Datasets Comparsion Across Six Benchmarks. As shown in Figure 6, GSM8K: SIGMA surpasses all baselines on GSM8K when fine-tuned with 30 K samples, demonstrating consistent superiority across multiple base models. MATH: On MATH benchmark, SIGMA deliv- ers marked gains over every competing method, showing stable uplift across different base models. College: In College evaluations, SIGMA yields notable improvements for each architecture, demon- strating that our data construction effectively enhances general-purpose purpose models' ability for reasoning challenging mathematical problems. DeepMind: SIGMA delivers substantial gains over MathFu- sion on DeepMind tasks, underscoring its strong adaptability to DeepMind dataset. Olympiad: For Olympiad problems, SIGMA maintains consistent gains over DART-Man across all backbones and
16

---

GSMBK
MATH
College
ELANA
Uttt
23
60
10
Tetheg
Supartil
Paatd Condeced SOMAAS
Refug
Supertal
Porld
Cediioall SKGMAIER
ReDup
Seperral
Puodll
Credtioul SKGMAIRK
DespMiid
Olympind
Thoorem
ELANA
Doplishhiat
52
22.5
15
42
17.5
12.5
Retheg
Segariil
Pauill
Cootiied SIOMAS
Sepeetal
Prodld
Conliiool SIGMA1KK
Rechan
Seperiid
Peald
Goalizal SKMMAFFK
Figure 7: Performance comparison of models fine-tuned 15k-sample datasets generated by our 55GMA15K and 30k-sample datasets generated by other methods, evaluated across six benchmark tasks. Different colored dots represent different methods.
narrows the gap with MathFusion, confirming robust performance on competition-level questions. Theorem: On the Theorem dataset, SIGMA achieves uniform improvements over every baseline, illustrating that its advantages in theorem reasoning persist regardless of dataset size.
SIGMA-15K vs. 30K-Scale Methods: Performance Across Six Benchmarks. As shown in Figure 7, GSM8K: With 15K training samples, SIGMA still leads all augmentation strategies GSM8K, delivering uniformly higher accuracies across LLaMA, Mistral and DeepSeekMath and narrowing performance disparities among them. MATH: Even reduced scale, SIGMA achieves clear improvements on MATH, outperforming each baseline and preserving its edge on all three backbones. College: In the College evaluation, SIGMA demonstrates pronounced advantage, boosting each architecture's 's capability to tackle -nterueddate-difficuuly tasks despite the limited training data. DeepMind: SIGMA adapts effectively the DeepMind dataset, surpassing Math- Fusion and other approaches across all backbones, which underscores its resilience on complex reasoning challenges. Olympiad: On Olympiad questions, SIGMA outperforms DART-Math for every base model and approaches MathFusion's ' standard, confirming its steady competitiveness on high-difficulty problems even at reduced scale. Theorem: For the Theorem benchmark SIGMA sus- tains its lead over all competing strategies, demonstrating that its strengths in reasoning persist when trained on only fifteen thousand instances.
C Evaluation on Commensense Reasoning Tasks
To further evaluate the generalization capability of the SIGMA method on text-based reason- ing tasks, we compared the performance of SIGMA framework against several baselines (Zero- shot, SKD[51, 27]. Distill Step-by-Step(211, Rephrase Question[56). Question Aug[26], An- swer Aug[56], and Revthink(8]) on three commonsense reasoning datasets: CommonsenseQQ (CSQA)[43),, StrategyQA (SQA)[15], and ARC-Challenge (ARC) Following the experimental setup of Revihink, we fine-tune - and Gemma-7B-Insinn with data generated by SIGMA. All results are reported as pass@ exact-match accuracy under O-shot greedy decoding (temperature 0). Detailed results are presented in Table 5.
using the LoRA
As clearly shown in Table 5, the SIGMA method significantly outperforms all baseline methods across all three datasets (SQA, CSQA, ARC) and also achieves the highest average accuracy on models. Specifically, on Mistral-7B, SIGMA
both Mistrra-77B-Instrrutt-0... and Gemma-7B-Instmn achieves an average accuracy of 75.74%, showing noticeable improvement over the second-best method, Revthink (+0.66). The advantage of SIGMA even more pronounced on Gemma-7B, where it reaches an average accuracy of 73.63%, considerably surpassing Revthink(+2.36).
17

---

Table 5: Performance comparison of different reasoning methods Ba-nna---Innsruet.. Results are reported as pass@1 exact-match accuracy under O-shot greedy decoding (temperature = 0).
and
Mistral-77BInsttrutt--0. Instruct--tu....
Gemma- Mommaa-BB-ao Bariaa--B--nsttti Instruct
Methods
SQA
CSQA
ARC
AVG
SQA
CSQA
ARC
AVG
Zero-shot
53.89
62.57
73.68
63.38
56.33
66.26
68.34
63.64
SKD
63.76
71.86
74.66
70.09
56.77
72.48
73.29
67.51
Distill Step-hy-Sten
64.19
71.92
75.32
70.48
56.77
73.01
72.92
67.57
Rephrase Question
65.07
70.19
74.51
69.92
54.15
70.22
72.37
65.58
Question Aug
65.07
72.23
73.32
70.21
57.21
68.11
72.74
66.02
Answer Aug
66.38
69.12
76.77
70.76
57.21
73.01
73.92
68.05
Revthink
70.97
75.76
78.50
75.08
64.19
74.53
75.09
71.27
SIGMA (Ours)
71.62
76.30
79.30
75.74
67.40
75.40
78.10
73.63
These results strongly demonstrate the effectiveness of the SIGMA method. Fine-tuning with our proposed approach significantly enhances the models' performance purely text-based common- sense reasoning tasks, validating the method's strong generalization ability across different models and tasks.
D Computation and Data Efficiency Analysis
Table 6: Comparison of synthesis costs for different methods. GPU cost, API cost, and total cost
measured in USD.
Method
GPU Type
GPU Hours
Price
API Cost
Total Cost
DART-Math
A100
3840
0.9
3456 106.4
SIGMA (GPT-4o-mini C&R)
RTX 4090
168
0.35
47.6
SIGMA (Qwen2.5-7B-Insee) as C&R)
RTX 4090
200
0.35
70.0
We further analyze the synthesis cost and data efficiency of the SIGMA framework. As a data- efficient approach, maintaining low synthesis and computational cost is central SIGMA's prac- ticality and scalability. Unlike previous data generation pipelines. SIGMA does not require any additional model training during synthesis. We use (wen2.5-Mathh 5-Mttt-B for MCTS generation, publicly available pretrained model from HuggingFace, requiring approximately 42 GPU hours on an RTX 4090 for generating full MCTS trees for 155-sa dataset.
For the refinement phase, we employ GPT-4o-mini to process the 15K MCTS trees, resulting in total prompt token count of 33.6M and completion token count of 11.4M, with API cost of 11.7 USD. Processing 60K examples with GPT-4o-mi -iniiiiiii costs only 47.6 USD. A detailed compar- ison of synthesis costs shown in Table Compared DART-Math, which requires 3,840 GPU hours on A100s ($3,456 total), SIGMA reduces the total cost by more than 30x. When replacing GPT-4o-mini with the open- opighweight Qwen2. the total cost further decreases to only $70, while maintaining comparable refinement quality. These results highlight SIGMA's efficiency and flexibility in reducing both computational and monetary overhead. In addition to computational efficiency, we analyze the token usage of SIGMA and sev-
as the Critique and Revision model,
Table Token-level comparison across datasets. SIGMA achieves higher performance with com- parable or smaller token budget.
Dataset
Tokens (LLaMA3-8B tokenizer)
DART-Math
190,441,618
MMIQC
94,093,794
RefAug
54,228,133 136
MetaMath
110,278,040
MathFusion
40,437, 234
SIGMA
44,064,782
18

---

eral baseline datasets to assess data efficiency. All token counts are measured using the LLaMA3-8B tokenizer. As shown in Table LGGM--15K contains 11.5M tokens, while SIGMA-60K contains 44M tokens comparalee to MathFusion's 's 60K dataset (40M tokens) yet achieving substantially higher accuracy. This demonstrates SIGMA's superior token-level data efficiency: with similar token budget, it provides denser supervision and stronger generalization across benchmarks.
E Training Setup
All fine-tuning experiments carried out on four NVIDIA H100 GPUs with DeepSpeed2 ZeRO [39] optimizations, operating in mixed precision [35] (FP16) to maximize memory efficiency and throughput.
The computation sequence token length was fixed 4096 to capture long range mathematical rea- and used Fraddmmtt.ccmmllaiioo.sss.
soning. We set per_ seeviccetrrinnbatcllee accumulate gradients.
We used the AdamW [25] optimizer with weight decay of 0.01. A cosine decay learning rate scheduler was applied with a linear warmup the first percent of total training steps (varmup_ratio=O. 03) to prevent instability. All models were trained for 3 epochs.
Initial learning rates tuned per model:
DeepSeekMath-7B: 5 x 10-3
Mistral-7B: 10-6
LLaMA3-8B: 10
We logged metrics regular intervals during training and validation. Random seeds were fixed and model checkpoints were saved after each epoch to ensure reproducibility.
F
Examples
In Appendix F, we present detailed examples and Q&A examples to provide deeper insight into the mechanism by which the SIGMA method refines reasoning data. Appendix F.1 illustrates how SIGMA refines reasoning steps on both geometry and algebra problems. Appendix F.2 presents additional SIGMA-refined reasoning paths for questions from MATH and GSM8K datasets.
F1
Detailed examples
Example 1. Geometry
The altitude the hypotenuse of triangle with angles of 30 and 60% is units. What is the area of the triangle, in square units? Express your answer in simplest radical form.
Question:
#Picture below is compiled from Asymptote code in question line.
3 units
Answer: The area is
6v3 square units.
19

---

MCTS Selected Path: (0-1): Step: Calculate the length the hypotemuse:
hypotenuse -
(0-1-2): Step: Calculate the length of the base:
base hypotenuse
(0-1-2-1): Step: Calculate the length of the altitude:
altitude = 3
(0-1-2-1-1); Step: Calculate the area of the triangle:
area = 0.5 base altitude
The answer is:
4.5
Step-by-Step Sibling Comparison and Critique: Step
[abbing(-2)ssttep:alallllll the length of the hypotemuse using the altitude and the 60-degree angle. We use the relationship in a 30"-60"-90" triangle. Since the altitude from the right angle to the hypotenuse is 3 units, the hypotenuse is: hypotemuse = 2v3.
Oi Calculate the length of the legs of the triangle. Bastggge..iingg triangle ratios again, the longer leg (opposite the 60º angle) is:leg dength=3vl
Evaluation: Step 0-0 incorrectly assumes a leg length without relating it to the given altitude; step 0-2 correctly uses the 360-60-990" property to link the altitude and hypotenuse; step 0-1 computed hypotenuse but resulted in wrong value.
Step
Sibling(--10)ssteecaalllual the length of the side opposite the 30-degree angle:side 30 hypotenuse /
Bbbling(tl--Ixstappcoommpp the base using the Pythagorean
theorem: base Vhypotenuse altitude"
Evaluation: Step 0-1-0 conflates the 30 as opposite side with the base without justification; step 0-1-1 applies Pythagorean theorem correctly but adds unneces- sary complexity; step 0-1-2 halves the hypotenuse that only holds isosceles right triangle.
Step
Calullllaatee the length of the height height=3 Sibling(0-1---2):steey Compute the altitude using triangle tegs:altittue==1ees (leg1 leg2) / hypotenuse
Evaluation:Step 0-1-2-0 merely restates the given;step 0-1-2-1 correctly extracts the altitude;step 0-1-2-2 applies geometric formula but assumes leg values.
Step 4:
Sibinng(-1224440::speecaalllllld square: base * altitude 2
Sbblnng(+122211::ssee:.c.allll answer: 0.5 base altitude =
6
Evaluation: Step 0-1-2-1-0 calculate the square; step 0-1-2-1-1 calculate the area of the triangle; step 0-1-2-1-2 compute the answer.
Key points:
20

---

For step 0-1, explicitly derive the hypotenuse length from the altitude to the hy- potenuse using the correct geometric relationship a 30-60-90 triangle.
For step 1-2 form and solve the altitude equation for that parameter.
For step 0-1-2-1, substitute back to obtain all side lengths.
For step 0-1-2-1-1, keep the square computing logic still.
Optimzed Output:
Step 1: Assign variables the sides of the 30-60-90 triangle.
Let the shortest leg (opposite 30%) be z. Then the hypotenuse is 2x, and the longer leg (opposite 60%) is zv33. Thus, the legs are I and vv and the hypotenuse is 2r. Step 2: Use the given altitude length to solve for We are told the altitude to the hypotenuse is units, So:
3 IV3
x 2 6
v3
v3
Rationalizing the denominator:
6
v3
6v3
2/3.
V
v3
3
Step 3: Calculate the lengths of all sides using x = 2/3.
Short leg (opposite 30"): x = 2v33. 2/3 v 2x
Longer leg (opposite 60%): xv
Hypotenuse: 2r = 2 x 2 4v3. Step 4: Calculate the area of the triangle.
The area of right triangle is half the product of its legs:
1 Area 2
x (short leg) (long leg) 1
1
2v3
6 2
12v3 6v3.
The answer is: (6v3)
Example 2. Algebra
Question:
Four positive integers A, B, C and D have a sum of 36. what is the value of the product A B x C D)?
A + 2 = B - 2
D 2
Answer: The correct product is 3840
MCTS Selected Path:
(0-2): Step: Define the variables A, B, C, and D.
(0-2-2): Step: Write the system of equations:
A B + C + D 36,
A+2 B 2,
= C,
C D
2
(0-2-2-1): Step: Solve the system to obtain
A 8, B=8,
C=
= 16.
21

---

(0-2-2-1-0); Step: Compute the product:
A 16 4096.
The answer 4096 (Incorrect)
Step-by-Step Sibling Comparison and Critique: Step 1:
Si De ine
variables:A
C
36
Sibling(--):SteepDDees
sum: 36
Evaluation: Step 0-1 correctly begins by defining the total sum constraint explic- itly; step 0-1 gives clear definition of total sum; step 0-2 omits in this step.
Step Sibling(0-2-0::Stt::D:efef iD equations A C * 2 - 2; B C 2 + 2;
D
+2
D A * (ieeDi:stee:Deefiee Define relationships:A
4; C
2;
Evaluation: Step 0-2-0 expresses all variables in terms of single variable C, which provides clearer structure; step 0-2-1 give recurrent definition of relative variables;step 0-2-2 instead defines relationships isolated pairs.
Step Sibling((-222-0):sttpp:so)) equations:A B + C D 36 ;
B ; 2
Sibling(0---222:.ttt:::oo Solvv equations:5 C
36
Evaluation: Step 0-2-2-0 captures the equality logic via C 2 = D/2 correctly; step 0-2-2--2 express all variables using C; step 0-2-2-1 misrepresents this logic using B C' and C D/2.
Step 4: 2agiing((22-2l11::sst::.a.allll requirement:
A
B C
4096
Sibinng(D2221112sstpp::claluluu answer: A * C D 4096
Evaluation: Step 0-2-2-1-1 calculate the required output: step 0-2-2-1-2 calculate the answer; step 0-2-2-1-0 compute the product.
Key Points:
For step 0-2, explicitly state constraints in initial setup.
For step 0-2-2, express all variables using a single reference variable.
For step 0-2-2-1, correctly encode and solve the system with logical relationships.
For step 0-2-2-1-0, keep the computing logic.
Optimized Output:
Step 1: Introduce common value l: that
D =
+ = B - = 2C =
and
A+ B C = 36.
2
Step 2: Express each variable in terms k:
- 2,
B = k + 2,
C
D 2k:
2
then substitute into the sum:
9
(k ( 2) + (k: + 2) + $ + 2k 36
36
1: 8.
22

---

Step 3: Determine the four integers:
- = 10,
C= 4, D = 16,
which indeed satisfy both the sum and the four-way equality. Step 4: Compute the product:
A Bxxxx=x==0 10x 16 = 3840.
The answer is: 3840
F.2
Q&A examples
Example
Mari made 4 more than five times as many decorated buttons as Kendra. Sue made half as many as Kendra. Sue made buttons. How many did Mari make? Answer:
Question:
Step 1: Sue made 6 buttons, Kendra made x 6 12. Step 2: Mari made more than 5x Kendra's ' count: 5 x 12 + 4.
Step 3: Compute x 12 60, then add 4 to get 64.
Step 4: Therefore, Mari made 64 buttons.
The answer is: 64
Example
Question: When three standard dice are tossed, the numbers a, b, e are obtained. Find the probability that
(a 1)(b 1)(c--1((c-- - #1
Answer:
Step 1: Total outcomes = 6 = 216.
Step (a () 1)(c 1) 0 iff least one die shows 1.
Step 3: Favorable: none is - 5 125 outcomes.
Step 4: Probability 216
125
Step Hence the probability is 125
216
125
The answer is:
216
Example
Question:
What is the minimum value of y y 3x + 6x 9?
Answer:
Step Rewrite or note vertex of parabola 3(x² + 2xx) + 3(x + 1)2
Step 2: Since (r + 1)2 2 0, minimum occurs at 1. 6
Step Substitute
1: y
Step 4: Therefore, the minimum value is 6.
The answer is: 6
Example
Question: You are given a sequence 58 terms P n where P is the product all primes 61 and
2, 2.3...., 59. Let N be the number of primes in this sequence. Find N.
23

---

Answer:
Step 1: If is prime (17 values), P + n (mod n) so composite.
Step 2: Remaining 58 17 41 terms with composite may be prime.
Step 3: Known result for Euclid-type numbers gives exactly 16 primes among them.
Step 4: Thus N = 16.
The answer is: 16
Example
Question: A rectangle has length 40 m and width 20 m. A similar smaller rectangle has half the length and width. What the area of the smaller rectangle?
Answer:
Step Scale factor z. so smaller sides are 20 and m.
Step Area 20 x 10 200 m
Step 3: Alternatively area factor ( of 800 gives 200.
Step 4: Therefore, the area 200.
The answer is: 200
Example
Last year, Isabella took 7 math tests with distinct integer scores 91-100. Each running average after tests was an integer. She scored 95 on the 7th test. What was her 6th test score?
Question:
Answer:
Step 1: Let Sp, be sum after tests; S, (mod n).
Step 2: From Sy = S₂ + 95 = 0 (mod 7) = S6 = 3 (mod 7).
Step Also Sg (mod 6). so S6 24 (mod 42).
Step 4: In allowable range 546 s So 600, the valid sum is 570, giving 86 570 S5 100.
Step 5: Therefore, the 6th score is 100.
The answer is: 100
Example
4x 12
Question:
Let p, q be distinct solutions to
= x + If > q. find p - q.
x² + 2x
15
Answer:
Step Domain excludes 5. Multiply by -222r-1555 15 2 34.² 2-
15x 18 0.
Step 2: Rational root r = is extraneous. Factor gives (x 3) + 7x + 6) = 0.
Step 3: Solve 22 7x
-1, 1,
Step 4: Thus p 11, = 6, sop 5.
Step Therefore, the difference is 5.
The answer is:
Example
Question: A vampire needs 7 gallons of blood week, sucking 2 pints person. How many people per day must he feed on?
Answer:
Step 1: 7 gal x 8 pt/gal 56 pints/week.
Step 2: At 2 pt/person, he needs 56/2 28 people/week.
Step 3: Dividing by 7 days gives 28/7 = people/day.
Step 4: Therefore, he must feed people each day.
24

---

The answer is:
4
Example
Question: Marla is mixing lilac paint 70% blue, 20% red, rest white. If she adds 140 of blue, how many ounces of white does she add?
Answer:
Step 1: WhiteYx=1000-70+2)11000
Step 2: Total T: 0.77 140 T 200 oz.
Step 3:
White amount 0.1 x 200 = 20 oz.
Step 4: Therefore, she adds 20 ounces of white.
The answer is: 20
Example 10
Question: By partial fractions,
B
C
D
E
x(x 1) 2)(x 3)
x+1
1+2
2
3
for some constants A, B, C, D, E. Find A+ B B+C + D E.
Answer:
Step 1: Multiply both sides by z(x + 2)x 3)(x + 4) to get
1 = A(x+1)(x
+ +3)(+ +4)
+ +)((++1](2+)(((+++
+ 1)(x++1c+++)r++++
Step 2: Evaluate at the roots:
x 0: 1 24A = 21
I = 11 1 = -63 = B 6
I = 2 1= 4C = C 1
= 33 1 == -6D = D= 6'
4: 1 24F
= E 1
24'
Step 3: Sum the constants:
A + B C+ ++ 24
24
4
24
6
24
4
+
1- 4+6 1446-441 4+1
21
21
0.
Step 4: Therefore, the sum is zero.
The answer is:
0
Example 11
Question: Let P be a point outside of circle O. segment is drawn from such that is tangent to circle O at T. Meanwhile, a secant from P intersects O at A and B, with PA PB. If PA and PT AB PA, then what is PB?
Answer:
Step 1: By the Power of Point, PT = PA PB.
Step 2: Since AB PB - P A, the condition PT AB - AB-PA gives PT (PB (PB-PA)-
25

---

PA PB - 2PA. with PA 3 have PT -
Step 3: Substitute into the power relation: (PB 6)² =
Step 4: Expand: PB 12PB 36 = 3PB, hence PB 15PB 36 0.
Step 5: Solve via the quadratic formula:
15 + 225 144
15 9
PB
2
2
yielding PB = 12 or PB 3.
Step 6: Discard PB = 3 since PB > PA = 3, so PB = 12.
The answer is: 12
26

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the ap- proach. For example, a facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
27

---

Justification:
No need.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proof's provided in appendix or supplemental material. Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yss]
Justification:
Section 4
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
28

---

Answer: [Yes]
Justification: Github
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips) cc/
ceciooooolliyiy oolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.c.. cccccc/ppa http:pptlllluuoooee / issiissin onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer:
[Yes]
Justification: Appendix and Section 4 Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
29

---

is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproducc the experiments?
Question:
Answer: [Yes]
Justification: Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : //werr///ee.p///eeeeeeeee cc httppp/iiiccthccccuueeeeeeeeeeee
Answer:
[Yes]
Justification:
We have read the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Not related.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
30

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not related.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: Not using existing assets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the pack- age should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documenta- tion provided alongside the assets?
31

---

Answer:
[NA]
Justification: No new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text instructions given to participants and screenshots, applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: Not related.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: yes approve
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
32

---

Justification: Section 4
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy for what should or should not be described.
33


Reviews:
[{"review_id": "IPRhcoQPhr", "reviewer": "Reviewer_F8sN", "summary": "This paper presents SIGMA, an enhanced MCTS framework that refines model-generated reasoning traces by leveraging sibling information in the search tree. Given a standard MCTS that explores the space of multi-step CoT paths and selects an optimal path, the SIGMA framework additionally refines the CoT on every node of the selected path, by considering the discrepancies between the selected node and discarded sibling node. For every refinement, a critique language model is used to suggest how to revise and a revise language model takes that suggestion and generates an improved version.\n\nThe author applied this framework to the math domain, with a setup that prompts and final answers are given, and the goal is to refine the reasoning traces to construct better training data. The training set is based on prompts from MATH and GSM8K datasets, and Qwen2.5-Math-7B is used to generate the original reasoning paths. Then, GPT-4o-mini is used to conduct the refinement. The resulting datasets are compared to existing math datasets of similar size or larger size, demonstrating better performance in terms of training math models.", "questions": "- What is the reasoning behind choosing Qwen2.5-math for generating the original CoT and GPT-4o-mini for the refinement?\n- For the training experiments, why not finetune Qwen2.5 model using the resulting data? It might be intriguing to see the model's self-improvement using this framework.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper addresses a timely topic of enhancing the MCTS framework for generating more effective CoT reasoning paths. The proposed improvement incorporates more information from sibling nodes looks meaningful and promising.\n- I found the formulation of the critique model and refine model under the textual gradient descent very intriguing and inspiring. This may be another good attempt of the [textgrad](https://arxiv.org/abs/2406.07496) framework.\n- The experiment results demonstrated better performance of the resulting math datasets.\n\nWeaknesses:\n- My major complaint is about the evaluation setup. The resulting datasets in this paper is based on CoT reasoning generated with Qwen2.5-Math model and then refined by GPT-4o-mini. The other existing datasets for comparison were generated with different models (e.g., MathFusion was generated using GPT-4o-mini only?). I believe that the base model makes a considerable difference here, and it's not clear to me how the proposed refinement contributed to the final improvement.", "was_revised": true, "final_justification": "I have read the rebuttal and other reviews. \n\nThe new results provided in the rebuttal answered my original questions and concerns, making it a sound extension to the MCTS framework, especially for generating targeted CoT data. \n\nWith its current technical novelty and contribution, I would like to keep my original recommendation.", "rebuttal": "We appreciate the reviewer for the thoughtful feedback，recognizing (i) novelty of incorporates more information from sibling nodes (ii) inspring attempt that formulating critique model and refine model based on textual gradient descent. We've addressed your concerns below.\n> The other existing datasets for comparison were generated with different models (e.g., MathFusion was generated using GPT-4o-mini only?).I believe that the base model makes a considerable difference here, and it's not clear to me how the proposed refinement contributed to the final improvement.\n\nThank you for your attention to the impact of the generation model and the effectiveness of the proposed refinement. I address this concern below.\n\nSwitching refinement model to weaker open-source LLMs resultd in a comparable performance compared with closed-source model as refinment models.Under Qwen-2.5-72B-instruct，the results achieves 34.3 on Llama3-8B，which is still comparable；31 on mistral，and 46.5 on deepseekMath-7B,which remains state-of-the-art with other methods.\n\n| **15K Samples** | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Llama3-8B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 35.3  | 77.1  | 21.7  | 39.5  | 9.6  | 16.9  | 33.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 36.3  | 78.9  | 23.2  | 39.4  | 9.8  | 18.0  | 34.3  |\n| GPT4o-mini as Critique and Revision Model | 36 | 82 | 24.2 | 42 | 10.5 | 22 | 36.1 |\n| MathFusion（30K Average Score） | 37.2  | 76.7  | 23.9  | 37.1  | 12.1  | 17.1  | 34.1  |\n| **Mistral-7B-v0.1 Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 28.8 | 79.3 | 18.4 | 33.3 | 6.7 | 13.5 | 30 |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 30.6 | 76.8 | 21.1 | 35.3 | 6.8 | 15.6 | 31 |\n| GPT4o-mini as Critique and Revision Model | 30 | 75.3 | 20.8 | 39.5 | 7.7 | 16.3 | 31.6 |\n| MathFusion（30K Average Score） | 30.0  | 74.0  | 18.5  | 25.7  | 9.2  | 14.5  | 28.6  |\n| **DeepSeekMath-7B-Base Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 49.8  | 80.6  | 38.3  | 60.6  | 18.4  | 24.1  | 45.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 51.3  | 81.9  | 39.9  | 64.8  | 17.5  | 23.8  | 46.5  |\n| GPT4o-mini as Critique and Revision Model | 52.2 | 81.1 | 37.4 | 64.5 | 20.3 | 26.1 | 47 |\n| MathFusion（30K Average Score） | 49.8  | 76.0  | 38.2  | 60.7  | 20.0  | 21.9  | 44.4  |\n| **Qwen2.5-Math-7B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 74.0  | 89.9  | 48.5  | 83.1  | 39.3  | 40.6  | 62.6  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 76.9  | 91.2  | 48.6  | 87.0  | 39.6  | 39.0  | 63.7  |\n| GPT4o-mini as Critique and Revision Model | 79.9  | 89.2  | 45.2  | 88.8  | 43.4  | 47.0  | 65.6  |\n\nIn Section 4.4, we conducted ablation studies that mesures the contribution of refinement by testing models fine-tuned by datasets with same problems sets but different answer synthesis methods.The results are shown in Table 2 at page 9 of our paper.\n - Lines with suffix \"SIGMA\":Qwen2.5-Math-7B is used to generate the original MCTS.GPT-4o-mini is used to conduct the refinement.\n - Lines with suffix \"MCTS\":Qwen2.5-Math-7B is used to generate the original MCTS.For each question,extract best tracjectroy based on Q-value and discard all other nodes.\n - Lines with suffix \"Blackbox\":For each question,generate step-by-step answer with standard COT prompt by GPT-4o-mini-2024-07-18.\n\nThe results proves that (i)Improvement of refinement(SIGMA vs MCTS) (ii)Refine from siblings outperforms oracle model(SIGMA vs Blackbox)\n\n> What is the reasoning behind choosing Qwen2.5-math for generating the original CoT and GPT-4o-mini for the refinement?\n\nThank you for the insightful question. Our method is not restricted to specific choices of generation or refinement models. A core focus of our work is data efficiency. In early-stage testing, we experimented with several math-specialized models including Mathstral and DeepSeek-Math. We ultimately selected Qwen2.5-Math-7B for CoT generation due to its strong MCTS generation speed and token efficiency,that it produces fewer tokens per problem on average,reducing the overall computational cost of both generation and refinement.\n\nFor the refinement model, GPT-4o-mini was chosen as a cost-effective option. However, our method does not depend on closed-source models. We have shown performance on different critique models above.\n\nWe will clarify these design choices and their implications in the camera-ready version.\n> For the training experiments, why not finetune Qwen2.5 model using the resulting data? It might be intriguing to see the model's self-improvement using this framework.\n\nThank you for your valuable suggestion. In the main experiments, we did not finetune Qwen2.5-Math-7B because existing strong baselines such as DART and MathFusion and other baselines have not reported results on this model. To ensure fair and verifiable comparisons, we focused on base models with publicly available results.\n\nTo further assess the self-improvement ability of our method, we additionally conducted finetuning experiments on Qwen2.5-Math-7B.SIGMA remains state-of-the-art performace on Qwen2.5-Math-7B, reaching 79.92 for MATH, 89.23 for GSM8K and 65.59 on all datasets average,surpassing other methods such as Eurus and OpenMath.The results are shown in the following table.\n| Qwen2.5-Math-7B (Fine-tuned) | # Samples | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Rstar@Greedy | 747K | 78.40  | 89.70  | **52.50** | – | **47.10** | – | – |\n| Eurus-2-7B-PRIME | 230K | 53.88  | 42.99  | 20.97  | 47.30  | 22.22  | 18.62  | 34.33  |\n| OpenMath-Nemotron-7B | 3.2M | 40.52  | 73.39  | 32.93  | 54.70  | 11.41  | 13.63  | 37.76  |\n| qwen2.5math-instruct | 2.5M | 82.16  | 95.15  | 46.80  | 82.00  | 41.60  | 40.75  | 64.74  |\n| AceMath-7B-Instruct | 2.3M | 75.40  | 58.38  | 45.82  | 59.70  | 39.85  | 30.75  | 51.65  |\n| DART-Math-60K | 60K | 63.14  | 91.28  | 47.23  | 74.90  | 26.81  | 35.38  | 56.46  |\n| MathFusion-60K | 60K | 56.14  | 59.59  | 37.08  | 67.70  | 41.19  | 27.87  | 48.26  |\n| SIGMA-60K | 60K | 79.92  | 89.23  | 45.17  | **88.80** | 43.41  | **47.00** | **65.59** |\n\n**Final Note:** Thank you for your thoughtful feedback on refinement effectiveness, model variation, and self-improvement on Qwen2.5. We hope our clarifications and results have addressed your concerns, and would appreciate your consideration in raising the score."}, {"review_id": "9ze7ON1dZr", "reviewer": "Reviewer_Aiin", "summary": "This paper introduces SIGMA (SIbling-Guided Monte Carlo Augmentation), a novel framework designed to enhance the reasoning capabilities of large language models (LLMs) by refining Monte Carlo Tree Search (MCTS)-generated reasoning paths. Traditional MCTS-based approaches select a single high-reward reasoning path and discard alternative (sibling) paths, potentially losing valuable information. SIGMA addresses this by:\n\n1. Reusing sibling nodes from the MCTS tree to generate natural language critiques of the selected reasoning steps.\n\n2. Using these critiques as symbolic gradients to iteratively refine the reasoning path via a Textual Gradient Descent (TGD) process.\n\n3. Employing a critique model and a revision model (both LLMs) to perform this refinement without requiring additional rollouts or external supervision.\n\nThe authors demonstrate that SIGMA significantly improves the quality of reasoning traces and downstream performance on mathematical reasoning benchmarks, outperforming larger datasets and state-of-the-art baselines with fewer training examples.", "questions": "1. What is $n$ & $k$ as mentioned in Section 4.1 in the context of the algorithm given in Section 3? There seems to be some notation abuse here since in Section 3 $n$ is used to denote node and in Section 4.1 it is used to denote candidate completions.\n\n2. In line 280 you say, \"Despite sharing the same generator model...\". How is the generator model the same for black-box CoT generation and fine-tuning using SIGMA when the former uses GPT-4o-mini (line 279) and the latter uses Qwen2.5-Math-7B?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n**Quality**: The method is well-motivated and addresses a clear inefficiency in current MCTS-based reasoning pipelines. Empirical results show consistent and significant improvements across multiple benchmarks and model backbones (DeepSeekMath-7B, LLaMA3-8B, Mistral-7B).\n\n**Clarity**: The paper is clearly written, with intuitive figures (e.g., Figures 2–4) that illustrate the SIGMA pipeline. The methodology is well-structured, with detailed explanations of each component (MCTS, critique generation, revision).\n\n**Significance**: SIGMA offers a data-efficient alternative to large-scale dataset construction, which is valuable given the high cost of data generation. The framework is model-agnostic and could be extended to other domains beyond math reasoning.\n\n**Originality**: The idea of treating sibling nodes as sources of symbolic feedback and using them for gradient-like refinement is novel to the best of my knowledge.\n\n### Weaknesses:\n\n1. The key point about refining the selected step $T_p$ based on the unselected siblings $T_s$ feels a little counterintuitive. If the siblings were not selected because they were expected to yield inferior results by the MCTS process then why would refining the content of $T_p$ to be closer to the content of $T_s$ be beneficial?\n\n2. The experiments are restricted to mathematical reasoning tasks. While the authors claim generalizability, this is not empirically validated.\n\n3. Given the extra effort involved in preparing the data using SIGMA and the fact that MathFusion-60K gives better results for two (LLaMA3 & Mistral) out of the three models considered, it isn't clear if we should go with SIGMA or just use a larger dataset.", "was_revised": true, "final_justification": "My main concern with this paper was about the refinement of the reasoning path based on the unselected siblings because it seemed to me that the approach was going to bring the step closer to the unselected siblings. However, the authors have explained that this will not happened as the LLMs used for critiquing and refining the step are prompted to incorporate the benefits of the alternate reasoning paths and to learn from the mistakes in them. On re-reading the relevant section of the paper I am also convinced about this.\n\nThe authors have also responded to my other concerns by adding additional results for non-mathematical tasks, pointing me to the results for their approach with large datasets and providing clarifications where the explanation in the paper was not clear. \n\nWhile I am still not 100% convinced that this method will always be better than simply using more data, I believe it is an important contribution that will be especially useful in settings where data is scarce. Therefore, I am increasing my rating to Accept.", "rebuttal": "We appreciate the reviewer for the thoughtful feedback，recognizing (i) data-efficiency for dataset construction (ii) well-structured methodology.We've addressed your concerns below.\n> The key point about refining the selected step $T_p$ based on the unselected siblings $T_s$ feels a little counterintuitive. If the siblings were not selected because they were expected to yield inferior results by the MCTS process then why would refining the content of $T_p$ to be closer to the content of $T_s$ be beneficial?\n\nThe central contribution of our work lies in how to extract constructive revision signals for the selected step $T_p$ from its suboptimal siblings $T_s$. Our method does not refine the content of $T_p$ to be closer to the content of $T_s$. Instead, we first apply a critique model to analyze $T_s$, and then use a revision model to improve $T_p$ based on the critique outcomes.\n\nThis design enables two key benefits:\n– Alternative reasoning paths in $T_s$ are leveraged to enrich the revision of $T_p$ without copying them directly.\n– Mistakes in $T_s$ are explicitly identified and used to detect and correct similar issues in $T_p$, rather than introducing the same errors into $T_p$.\n\nWe will clarify these mechanisms in the camera-ready version.\n> The experiments are restricted to mathematical reasoning tasks. While the authors claim generalizability, this is not empirically validated.\n\nWe appreciate the reviewer for the helpful suggestion.We agree that evaluating the generalizability of SIGMA beyond mathematical reasoning is important. Owing to its design—which relies entirely on textual signals for refining reasoning paths—SIGMA holds strong potential in non-mathematical domains as well.\n\nFollowing the experimental setup of RevThink[1], we applied SIGMA to CommonsenseQA, StrategyQA, and ARC-Challenge. We conducted LoRA finetuning on Mistral-7B-Instruct-v0.3 and Gemma-7B-Instruct to assess SIGMA’s cross-domain generalization ability. SIGMA outperformed RevThink and other baselines, achieving state-of-the-art performance(75.74 AVG on Mistral-7B-Instruct-v0.3 and 73.63 AVG on Gemma-7B-Instruct) on these benchmarks.\n\nDetailed results are provided in the table below.\n\n| Methods | Mistral-7B-Instruct-v0.3 |  |  |  | Gemma-7B-Instruct |  |  |  |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  | SQA | CSQA | ARC | AVG | SQA | CSQA | ARC | AVG |\n| Zero-shot | 53.89 | 62.57 | 73.68 | 63.38 | 56.33 | 66.26 | 68.34 | 63.64 |\n| SKD | 63.76 | 71.86 | 74.66 | 70.09 | 56.77 | 72.48 | 73.29 | 67.51 |\n| Distill Step-by-Step | 64.19 | 71.92 | 75.32 | 70.48 | 56.77 | 73.01 | 72.92 | 67.57 |\n| Rephrase Question | 65.07 | 70.19 | 74.51 | 69.92 | 54.15 | 70.22 | 72.37 | 65.58 |\n| Question Aug | 65.07 | 72.23 | 73.32 | 70.21 | 57.21 | 68.11 | 72.74 | 66.02 |\n| Answer Aug | 66.38 | 69.12 | 76.77 | 70.76 | 57.21 | 73.01 | 73.92 | 68.05 |\n| Revthink | 70.97 | 75.76 | 78.5 | 75.08 | 64.19 | 74.53 | 75.09 | 71.27 |\n| SIGMA(Ours) | 71.62 | 76.3 | 79.3 | 75.74 | 67.4 | 75.4 | 78.1 | 73.63 |\n\nWe will include the discussion on generalization and the corresponding experimental results in the camera-ready version. \n> Given the extra effort involved in preparing the data using SIGMA and the fact that MathFusion-60K gives better results for two (LLaMA3 & Mistral) out of the three models considered, it isn't clear if we should go with SIGMA or just use a larger dataset.\n\nThe scalability of SIGMA to larger datasets is essential to its general applicability. In the main text, we emphasized its data efficiency by demonstrating that SIGMA achieves comparable or even superior performance with only half or fewer training samples compared to other methods, and in Table 1 we present results comparing the SIGMA-30K series against the 60K MathFusion dataset to further illustrate this point.\n\nTo further address this concern, we include in Appendix A.2 additional results of SIGMA trained on the full 60K dataset. SIGMA continues to outperform all baselines under this setting, achieving state-of-the-art results. This provides important empirical evidence of its effectiveness at scale.\n\nWe will incorporate this finding into the camera-ready version. We appreciate the reviewer for the valuable suggestion.\n\n> What is $n$ & $k$ as mentioned in Section 4.1 in the context of the algorithm given in Section 3? There seems to be some notation abuse here since in Section 3 $n$ is used to denote node and in Section 4.1 it is used to denote candidate completions.\n\nIn Section 4.1, the setting of $k = 5$ and $n = 3$ refers to the sampling of 5 candidate outputs for each non-terminal MCTS node, from which the top 3 are selected as its child nodes. This is consistent with the use of $n$ in Section 3, where it denotes a node in the MCTS tree. The term \"candidates\" in Section 4.1 refers to possible nodes considered for inclusion in the best trajectory.\n\nWe appreciate the reviewer for pointing out this ambiguity. We will revise the notation and clarify the explanation in the camera-ready version to avoid confusion.\n>In line 280 you say, \"Despite sharing the same generator model...\". How is the generator model the same for black-box CoT generation and fine-tuning using SIGMA when the former uses GPT-4o-mini (line 279) and the latter uses Qwen2.5-Math-7B?\n\nIn Section 4.4, we conducted ablation studies that mesures the contribution of refinement by testing models fine-tuned by datasets with same problems sets but different answer synthesis methods.The results are shown in Table 2.\n - Lines with suffix \"SIGMA\":Qwen2.5-Math-7B is used to generate the original MCTS.GPT-4o-mini is used to conduct the refinement.\n - Lines with suffix \"MCTS\":Qwen2.5-Math-7B is used to generate the original MCTS.For each question,extract best tracjectroy based on Q-value and discard all other nodes.\n - Lines with suffix \"Blackbox\":For each question,generate step-by-step answer with standard COT prompt by GPT-4o-mini-2024-07-18.\n\nThe results proves that (i)Improvement of refinement(SIGMA vs MCTS) (ii)Refine from siblings outperforms oracle model(SIGMA vs Blackbox)\n\nThe statement “Despite sharing the same generator model...” was intended to highlight the contrast between SIGMA and the Blackbox variant, where SIGMA achieves superior performance despite refining from sibling traces instead of using an oracle CoT generated directly by the same backbone used for refinement.\n\nWe appreciate the reviewer’s correction and will revise the wording in the camera-ready version to improve clarity.\n\n[1]Chen, J. C. Y., Wang, Z., Palangi, H., Han, R., Ebrahimi, S., Le, L., ... & Pfister, T. (2024). Reverse thinking makes llms stronger reasoners. arXiv preprint arXiv:2411.19865.\n\n**Final Note:** Thank you for your detailed feedback on sibling-based refinement, generalization beyond math, and scalability. We hope our empirical results and clarifications have addressed your concerns and kindly invite you to consider raising your score."}, {"review_id": "VPkHnryp4e", "reviewer": "Reviewer_5rCb", "summary": "In this paper, the authors introduced a framework for refining the reasoning capabilities of large language models (LLMs) by leveraging sibling nodes in Monte Carlo Tree Search (MCTS), called SIGMA. SIGMA reintegrates discarded sibling nodes to enhance reasoning paths through a two-stage refinement process: a critique model discovers strengths and weaknesses across sibling sets, and a revision model refines the top-scoring trajectory based on the findings from the first step. The method improves reasoning trajectories, achieving state-of-the-art performance on the MATH benchmark with only 30K samples.", "questions": "The authors need to answer all the weak points listed above.", "limitations": "Please check the weak points listed above.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1.The core idea of leveraging sibling information for refinement is not entirely novel. Similar concepts have been explored in other domains, such as symbolic supervision via LLM-generated feedback [43]. The paper does not provide a clear distinction or significant advancement over these existing methods.\n2.The paper lacks a formal theoretical analysis of why the proposed method works and its potential limitations. A deeper understanding of the underlying mechanisms and potential failure modes would strengthen the paper\n3.The performance of SIGMA is highly dependent on the quality of the sibling nodes generated during MCTS. If the sibling nodes are not well-formed or fail to capture robust reasoning behaviors, the transferred feedback may not provide significant benefits. \n4. While SIGMA avoids full MCTS fine-tuning on large models, it still requires access to a reasonably strong small expert model trained with MCTS, which can be computationally expensive to obtain. The paper does not provide a detailed analysis of the trade-offs between the computational cost of training the small expert models and the benefits gained from using SIGMA.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for the thoughtful feedback，We've addressed your concerns below.\n>1.The core idea of leveraging sibling information for refinement is not entirely novel. Similar concepts have been explored in other domains, such as symbolic supervision via LLM-generated feedback [43]. The paper does not provide a clear distinction or significant advancement over these existing methods. \n\nOur primary motivation is to improve data efficiency by reusing sibling trajectories from the MCTS process, which are typically discarded in prior work. TextGrad [43] serves as a key component in realizing this goal, but our contribution lies in how we apply it to exploit the structure of MCTS reasoning trees.\nUnlike previous methods that focus solely on the best trajectory, our approach systematically leverages feedback from suboptimal siblings to revise and improve selected steps. This enables more effective utilization of existing data, addressing the trade-off between accuracy and test-time scaling cost. Our experiments demonstrate that this strategy leads to consistent performance gains across multiple models and settings, validating its effectiveness.\n>2.The paper lacks a formal theoretical analysis of why the proposed method works and its potential limitations. A deeper understanding of the underlying mechanisms and potential failure modes would strengthen the paper \n\nWe appreciate the reviewer’s comment regarding the need for a deeper understanding of SIGMA's underlying mechanisms and its potential limitations.\n\nSIGMA builds on the structural alignment of MCTS-generated reasoning trees, where each selected step $T_p$ and its sibling steps $T_s$ originate from the same parent and thus represent semantically parallel alternatives. This setting enables meaningful counterfactual supervision: by applying a critique model to $T_s$ and using its output to guide the revision of $T_p$, SIGMA transforms discarded reasoning attempts into constructive feedback. This aligns with principles of contrastive learning and error-based refinement, contributing to SIGMA’s high data efficiency.\n\nHowever, a potential failure mode arises when the input problems are too simple or formulaic, such that the resulting reasoning trajectories are dominated by non-critical steps (e.g., routine calculations or definition lookups). In such cases, SIGMA still performs step-level refinement uniformly, which may lead to unnecessary processing of low-impact steps. Moreover, language models may fail to produce substantive critiques for these trivial steps, making the refinement less effective and reducing the marginal utility per token. This can weaken SIGMA’s data-efficiency advantage in low-complexity settings.\n\nNevertheless, because SIGMA refines each step individually and siblings share local context, critical steps are still improved using relevant sibling information. Thus, when refinement truly matters (i.e., on non-trivial reasoning segments), the method remains robust and effective.\n\nWe will include this discussion of SIGMA’s underlying mechanism and its potential failure modes in the camera-ready version.\n>3.The performance of SIGMA is highly dependent on the quality of the sibling nodes generated during MCTS. If the sibling nodes are not well-formed or fail to capture robust reasoning behaviors, the transferred feedback may not provide significant benefits. \n\nThe central contribution of our work lies in how to extract constructive revision signals for the selected step $T_p$ from its suboptimal siblings $T_s$. Our method does not refine the content of $T_p$ to be closer to the content of $T_s$. Instead, we first apply a critique model to analyze $T_s$, and then use a revision model to improve $T_p$ based on the critique outcomes.\n\nThis design enables two key benefits:\n– Alternative reasoning paths in $T_s$ are leveraged to enrich the revision of $T_p$ without copying them directly.\n– Mistakes in $T_s$ are explicitly identified and used to detect and correct similar issues in $T_p$, rather than introducing the same errors into $T_p$.\n\nWe will clarify these mechanisms in the camera-ready version.\n\n>4. While SIGMA avoids full MCTS fine-tuning on large models, it still requires access to a reasonably strong small expert model trained with MCTS, which can be computationally expensive to obtain. The paper does not provide a detailed analysis of the trade-offs between the computational cost of training the small expert models and the benefits gained from using SIGMA.\n\nWe appreciate the reviewer’s concern regarding the computational cost of SIGMA. As a data-efficient method, maintaining low computation cost is central to its practicality and effectiveness.\n\nWe clarify that SIGMA does not require any additional model training during the data synthesis phase. We use Qwen-2.5-Math-7B for original MCTS generation,which is a publicly available pretrained models from HuggingFace. Following the MCTS data generation procedure of AlphaMath [6], using Qwen-2.5-Math-7B to generate full MCTS trees for a 15K dataset requires ~42 GPU hours on RTX 4090.Generating MCTS for 60K dataset requires ~168 RTX 4009 GPU hours.\n\nFor the refinement phase, we use GPT-4o-mini to process the 15K MCTS trees.The total prompt token count is 33.6M and the total completion token count is 11.42M, resulting in a total cost of 11.7 USD.Processing 60K MCTS on GPT-4o-mini costs 47.6 USD.Detailed comparison for synthesis cost are listed below. The GPU price data are from vast​.​ai.\n\n| Method | GPU Type | GPU Hours | Price per hour(USD)  | API Cost（USD） | Total Price(USD) |\n|:---:|:---:|:---:|:---:|:---:|:---:|\n| DART-Math | A100 | 3840 | 0.9 | – | 3456 |\n| SIGMA(GPT4o-mini as Crtique and Revision model) | RTX 4090 | 168 | 0.35 | 47.6 | 106.4 |\n| SIGMA(Qwen2.5-7B-Instructas as Crtique and Revision model) | RTX 4090 | 200 | 0.35 | – | 70 |\n\nCompared to alternative data generation pipelines, SIGMA offers a competitive balance of low computational cost, high data efficiency, and improved overall accuracy. We will include detailed computational cost analysis in the camera-ready version.\n**Final Note：** We sincerely thank the reviewer for the thoughtful and constructive comments. We have revised the paper to clarify the novelty, mechanisms, and cost-efficiency of SIGMA, and hope the additional discussions address your concerns. If our responses have addressed your concerns, please kindly consider raising the score."}, {"review_id": "oIdve2Qz7g", "reviewer": "Reviewer_Jhvz", "summary": "The paper presents a framework that reuses the discarded alternative generations (sibling nodes) produced during LLM test-time search via Monte Carlo Tree Search (MCTS). These sibling nodes were discarded in prior work [1]. The framework instead leverages a critique LLM (GPT-4) to provide feedback on these sibling nodes, which is then used by a revision LLM (also GPT-4) to rewrite the generation step into a chain of thought, thereby improving the quality of supervised training data.\nThis process enables the authors to generate smaller but higher-quality training data compared to using MCTS alone. The approach is evaluated on three 7–8B parameter models across several math reasoning datasets and benchmarks, including Vanilla MCTS, Random-Sibling MCTS, Self-Refine, Tree-of-Thoughts-20, direct CoT SFT, and MathFusion-60K.\n\n[1] Chen, G., Liao, M., Li, C., & Fan, K. (2024). AlphaMath Almost Zero: Process Supervision without Process (arXiv:2405.03553). arXiv. https://doi.org/10.48550/arXiv.2405.03553", "questions": "How would SIGMA’s performance degrade when you change the critique and revision LLMs to weaker open-source models?\nHave you measured how the number of sibling nodes affects SIGMA's performance?", "limitations": "None", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strenghts:\n\nTest-time search in LLMs and self-refining reasoning LLMs are highly relevant topics, and this paper is well motivated and well written.\n\nWhile I find the methodological contribution narrow as it mainly relies on using all generations and as such modest, the repurposing of the otherwise discarded siblings is carried out methodologically sound. The evaluation on different baselines (Vanilla MCTS, Random-Sibling MCTS, Self-Refine, Tree-of-Thoughts-20, direct CoT SFT, MathFusion-60 K) is comprehensive.\n\nThe incorporation of critique and refiner models is a simple approach to improve certain siblings and can be easily applied.\n\nWeaknesses:\n\nAs Mentioned already in the strengths, the novelty lies in a narrow domain, using all generations for learning as an addition to previous work using MCTS. There is work in that direction I find not adressed in this submission, such as [2], that in a similar manner use generations for each token in a prompt for supervised training. While the authors add a trick with the critique- and refinement LLMs, this shows some overlap. As such it further limits novelty in my view.\n\nWhile evaluation across benchmarks is comprehensive, I find evidence for the generalizability across models and domains could be more robust:\n1. The approach is limited to the math domain, so I am unsure how well it generalizes to more open domains such as commonsense reasoning.\n2. While it is evaluated across model families, the evaluation is limited to similar sizes and doesn't use specialized models, as [1] does.", "was_revised": true, "final_justification": "The authors provided many experiments underlying their claim, so I suggest accepting the paper.", "rebuttal": "We appreciate the reviewer for the thoughtful feedback，recognizing (i) our methods are well motivated and can be easily applied (ii) experiments and evaluations are comprehensive. We've addressed your concerns below.\n>There is work in that direction I find not adressed in this submission, such as [2], that in a similar manner use generations for each token in a prompt for supervised training. While the authors add a trick with the critique- and refinement LLMs, this shows some overlap. \n\nWe appreciate the reviewer for highlighting related work such as Self-RAG [2]. While both methods leverage partial generations for supervision, there are key differences that distinguish SIGMA.\n\n(i) Self-RAG evaluates token-wise fragments and discards suboptimal completions, selecting only one high-quality continuation. In contrast, SIGMA explicitly retains and reuses sibling answers that are typically discarded, using them as the basis for critique generation. This reuse of suboptimal trajectories is a central motivation of SIGMA and contributes to its data efficiency.\n\n(ii) In Self-RAG, the selected fragments are sequential and share a context-dependent relationship. In SIGMA, by contrast, the selected node $T_p$ and its sibling nodes $T_s$ are parallel completions generated from the same parent node during MCTS. This structural alignment ensures that the critique model can extract meaningful comparative signals from $T_s$ to revise $T_p$, rather than introducing unrelated information.\n\nWe will include a discussion of this distinction in the camera-ready version.\n\n>The approach is limited to the math domain, so I am unsure how well it generalizes to more open domains such as commonsense reasoning.\n\nWe appreciate the reviewer for the helpful suggestion.We agree that evaluating the generalizability of SIGMA beyond mathematical reasoning is important. Owing to its design that relies entirely on textual signals for refining reasoning paths, SIGMA holds strong potential in non-mathematical domains as well.\n\nFollowing the experimental setup of RevThink[1], we applied SIGMA to CommonsenseQA, StrategyQA, and ARC-Challenge. We conducted LoRA finetuning on Mistral-7B-Instruct-v0.3 and Gemma-7B-Instruct to assess SIGMA’s cross-domain generalization ability. SIGMA outperformed RevThink and other baselines, achieving state-of-the-art performance(75.74 AVG on Mistral-7B-Instruct-v0.3 and 73.63 AVG on Gemma-7B-Instruct) on these benchmarks.\n\nDetailed results are provided in the table below.\n\n| Methods | Mistral-7B-Instruct-v0.3 |  |  |  | Gemma-7B-Instruct |  |  |  |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|  | SQA | CSQA | ARC | AVG | SQA | CSQA | ARC | AVG |\n| Zero-shot | 53.89 | 62.57 | 73.68 | 63.38 | 56.33 | 66.26 | 68.34 | 63.64 |\n| SKD | 63.76 | 71.86 | 74.66 | 70.09 | 56.77 | 72.48 | 73.29 | 67.51 |\n| Distill Step-by-Step | 64.19 | 71.92 | 75.32 | 70.48 | 56.77 | 73.01 | 72.92 | 67.57 |\n| Rephrase Question | 65.07 | 70.19 | 74.51 | 69.92 | 54.15 | 70.22 | 72.37 | 65.58 |\n| Question Aug | 65.07 | 72.23 | 73.32 | 70.21 | 57.21 | 68.11 | 72.74 | 66.02 |\n| Answer Aug | 66.38 | 69.12 | 76.77 | 70.76 | 57.21 | 73.01 | 73.92 | 68.05 |\n| Revthink | 70.97 | 75.76 | 78.5 | 75.08 | 64.19 | 74.53 | 75.09 | 71.27 |\n| SIGMA(Ours) | 71.62 | 76.3 | 79.3 | 75.74 | 67.4 | 75.4 | 78.1 | 73.63 |\n\nWe will include the discussion on generalization and the corresponding experimental results in the camera-ready version. \n\n>While it is evaluated across model families, the evaluation is limited to similar sizes and doesn't use specialized models, as [1] does.How would SIGMA’s performance degrade when you change the critique and revision LLMs to weaker open-source models? Have you measured how the number of sibling nodes affects SIGMA's performance?\n\nWe appreciate the reviewer for the valuable suggestions regarding evaluation across model strengths and sibling node configurations.\n\nTo further assess the self-improvement ability of our method, we additionally conducted finetuning experiments on Qwen2.5-Math-7B.SIGMA remains state-of-the-art performace on Qwen2.5-Math-7B, reaching 79.92 for MATH, 89.23 for GSM8K and 65.59 on all datasets average,surpassing other methods such as Eurus and OpenMath.The results are shown in the following table.\n| Qwen2.5-Math-7B (Fine-tuned) | # Samples | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Rstar@Greedy | 747K | 78.40  | 89.70  | **52.50** | – | **47.10** | – | – |\n| Eurus-2-7B-PRIME | 230K | 53.88  | 42.99  | 20.97  | 47.30  | 22.22  | 18.62  | 34.33  |\n| OpenMath-Nemotron-7B | 3.2M | 40.52  | 73.39  | 32.93  | 54.70  | 11.41  | 13.63  | 37.76  |\n| qwen2.5math-instruct | 2.5M | 82.16  | 95.15  | 46.80  | 82.00  | 41.60  | 40.75  | 64.74  |\n| AceMath-7B-Instruct | 2.3M | 75.40  | 58.38  | 45.82  | 59.70  | 39.85  | 30.75  | 51.65  |\n| DART-Math-60K | 60K | 63.14  | 91.28  | 47.23  | 74.90  | 26.81  | 35.38  | 56.46  |\n| MathFusion-60K | 60K | 56.14  | 59.59  | 37.08  | 67.70  | 41.19  | 27.87  | 48.26  |\n| SIGMA-60K | 60K | 79.92  | 89.23  | 45.17  | **88.80** | 43.41  | **47.00** | **65.59** |\n\nFor the critique and revision LLMs, GPT-4o-mini was chosen as a cost-effective option. However, our method does not depend on closed-source models. To support this, we have included additional results using, Qwen2.5-72B-Instruct, and Qwen2.5-Math-7B-Instruct as critique and revision LLMs to synthesise 3 new 15K datasets, and SFT on models in our main experiment.Switching refinement model to weaker open-source LLMs resultd in a comparable performance compared with closed-source model as refinment models.Under Qwen-2.5-72B-instruct，the results achieves 34.3 on Llama3-8B，which is still comparable；31 on mistral，and 46.5 on deepseekMath-7B,which remains state-of-the-art with other methods.\n\n| **15K Samples** | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Llama3-8B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 35.3  | 77.1  | 21.7  | 39.5  | 9.6  | 16.9  | 33.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 36.3  | 78.9  | 23.2  | 39.4  | 9.8  | 18.0  | 34.3  |\n| GPT4o-mini as Critique and Revision Model | 36 | 82 | 24.2 | 42 | 10.5 | 22 | 36.1 |\n| MathFusion（30K Average Score） | 37.2  | 76.7  | 23.9  | 37.1  | 12.1  | 17.1  | 34.1  |\n| **Mistral-7B-v0.1 Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 28.8 | 79.3 | 18.4 | 33.3 | 6.7 | 13.5 | 30 |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 30.6 | 76.8 | 21.1 | 35.3 | 6.8 | 15.6 | 31 |\n| GPT4o-mini as Critique and Revision Model | 30 | 75.3 | 20.8 | 39.5 | 7.7 | 16.3 | 31.6 |\n| MathFusion（30K Average Score） | 30.0  | 74.0  | 18.5  | 25.7  | 9.2  | 14.5  | 28.6  |\n| **DeepSeekMath-7B-Base Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 49.8  | 80.6  | 38.3  | 60.6  | 18.4  | 24.1  | 45.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 51.3  | 81.9  | 39.9  | 64.8  | 17.5  | 23.8  | 46.5  |\n| GPT4o-mini as Critique and Revision Model | 52.2 | 81.1 | 37.4 | 64.5 | 20.3 | 26.1 | 47 |\n| MathFusion（30K Average Score） | 49.8  | 76.0  | 38.2  | 60.7  | 20.0  | 21.9  | 44.4  |\n| **Qwen2.5-Math-7B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 74.0  | 89.9  | 48.5  | 83.1  | 39.3  | 40.6  | 62.6  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 76.9  | 91.2  | 48.6  | 87.0  | 39.6  | 39.0  | 63.7  |\n| GPT4o-mini as Critique and Revision Model | 79.9  | 89.2  | 45.2  | 88.8  | 43.4  | 47.0  | 65.6  |\n\nFor the influence of the number of sibling nodes, We have conducted additional experiments to study the effect of varying the number of sibling nodes used during refinement.We choose DeepSeekMath-7B and Llama3-8B as Math-Specialized Base Model and General Base model.Lines with suffix MCTS are extracted from Table 2 in Section 4.4, page 9 of our paper.The results shows clearly monotonic increase trend when raising numbers of siblings(33.6->46.5->47.0 for DeepSeekMath and 23.6->35.0->36.1 for Llama3-8B) \n\n| Model(15K Samples) | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|---|---|---|---|---|---|---|---|\n| Math-Specialized Base Model |  |  |  |  |  |  |  |\n| DeepSeekMath-7B-MCTS(0  Sibling) | 40.7 | 58.3 | 30.2 | 43.7 | 18.1 | 10.7 | 33.6 |\n| DeepSeekMath-7B(1 Sibling) | 50.7  | 81.3  | 35.1  | 65.5  | 19.6  | 26.9  | 46.5  |\n| DeepSeekMath-7B(2 Siblings) | 52.2  | 81.1  | 37.4  | 64.5  | 20.3  | 26.1  | 47.0  |\n| General Base Model |  |  |  |  |  |  |  |\n| Llama3-8B-MCTS(0  Sibling) | 27.8 | 40.2 | 18.3 | 34.3 | 12.6 | 8.5 | 23.6 |\n| Llama3-8B(1 Sibling) | 78.9  | 36.1  | 23.6  | 40.3  | 11.6  | 19.3  | 35.0  |\n| Llama3-8B(2 Siblings) | 36.0  | 82.0  | 24.2  | 42.0  | 10.5  | 22.0  | 36.1  |\n\n[1]Chen, J. C. Y., Wang, Z., Palangi, H., Han, R., Ebrahimi, S., Le, L., ... & Pfister, T. (2024). Reverse thinking makes llms stronger reasoners. arXiv preprint arXiv:2411.19865.\n\n**Final Note：** We thank the reviewer for recognizing the motivation and completeness of our method and experiments. We have added comparisons with Self-RAG, evaluated generalization to commonsense domains, and analyzed the effects of model strength and sibling count. We hope these additions fully address your concerns."}, {"review_id": "lBeLpaAckq", "reviewer": "Reviewer_UihT", "summary": "The authors propose SIGMA, a refined version of MCTS that uses an oracle LLM prompted with sibiling node reasoning traces to generate feedback to refine the selected reasoning path. SIGMA is used to generate low sample reasoning datasets for fine-tuning which is empirically validated with three model variants. The authors find that SIGMA outperforms other methods in terms of fine-tuning sample efficiency and is competitive with much larger datasets such as DART-math.", "questions": "### Key questions to improve my score:\n* What is the synthesis cost of SIGMA vs. other baselines such as DART-Math or MathFusion? Is SIGMA more optimal in terms of number of samples vs. cost to generate than the studied baselines? \n* What is the total number of completion tokens in SIGMA vs. the other baselines? When we analyze the main results in terms of accuracy vs. completion tokens, do the main findings hold? Ultimately, our fine-tuning fine-tuning cost will scale with total tokens, not samples. \n* How does Qwen-2.5-Math-7B perform when fine-tuned on the SIGMA dataset? Does SIGMA enable self-distillation? \n* Some outputs from $\\mathcal{C}\\_{LLM}$ do not actually appear to contain a critique. Eg., from the geometry question: “Evaluation: Step 0-1-2-1-0 calculate the square; step 0-1-2-1-1 calculate the area\nof the triangle; step 0-1-2-1-2 compute the answer.”. $\\mathcal{C}\\_{LLM}$ appears to simply be describing the sibling node approaches but doesn’t offer any specific critiques. Could the authors speculate on whether the more robust critiques could be extracted from $\\mathcal{C}_{LLM}$  and if so, how?\n\n### Minor issues, suggestions, typos\n* Exploring additional oracle models, candidate generation models, and tasks would make the work more impactful. Given that SIGMA produces natural language feedback, it may be good fit for tasks where traditional verifiers are not available (eg., creative writing). \n* Consider my comments regarding the overall presentation above. I believe the work would benefit from improvements in this area.\n* Typos:\n\t* L145: “To”\n\t* L148: “Subscript” -> “Superscript”\n\t* L159: “operats”", "limitations": "More discussion on the synthesis cost and total number of tokens in SIGMA datasets is required to fully understand the impact and limitations of the work.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n* The proposed method is intuitive and well motivated. \n* It appears the use of sibling guidance in MCTS is novel in the studied setting. \n* A diverse set of models and math datasets are empirically studied. \n* The result is significant and represents a valuable contribution to an important area of study. Improving sample efficiency is a long-standing challenge of the field. \n* The proposed method is automated and scalable, potentially enabling the curation of large, high quality datasets. \n\n## Weaknesses\n* My primary concerns regarding this work are related to its motivation, some absent analysis, and presentation.\n* The synthesis cost of SIGMA is not discussed. While I am convinced that SIGMA can be used to generate high-quality training samples, it’s unclear what the cost is to generate the dataset. This information would help motivate the selection of using SIGMA to generate a bespoke dataset versus using a larger dataset such as DART-math. \n* The total tokens in the SIGMA dataset is unclear. Since we backpropagate across all completion tokens during fine-tuning, an important but absent discussion is the total number of tokens across SIGMA and baseline datasets. \n* Only GPT-4o-mini is explored as $\\mathcal{C}\\_{LLM}$ and $\\mathcal{R}_{LLM}$. \n* Qwen-2.5-Math-7B is used as the candidate generator to construct search trees. One particularly interesting analysis, which was not included, is to fine-tune the same model on the SIGMA dataset to observe if SIGMA can be effectively leveraged in the self-distillation setting. \n* Presentation concerns:\n\t* Authors should cite sections of the appendix in the main body where relevant. For example, when discussing training hyperparameters that are not explicitly stated in the main body. \n\t* The 60k SIGMA results presented in Appendix A are the most significant, I suggest moving these to the main body in lieu of Table 1. Highlighting that at 60k SIGMA enables higher quality models than even DART-Math or models trained on the full MathFusionQA dataset would improve the impact of the work. \n\t* Overall I find the formalization of Textual Gradient Descent to be more confusing than illustrative. Equation 5 can simply be re-written as $\\tilde{T}\\_p = \\mathcal{R}\\_{LLM}(T_p, \\mathcal{C}_{LLM}(T_p | T_s \\in \\mathcal{S}_p) )$ without requiring us to stretch the definitions of terms such as gradient, loss, and optimizer to contexts beyond which they normally apply in ML literature. I understand that the authors are following the lead of [43] here; however, I believe the method could be more plainly stated without losing meaning and would improve the overall clarity.", "was_revised": true, "final_justification": "My original key concerns regarding the synthesis cost of SIGMA and the ability to apply the method to a self-distillation setting were adequately addressed by the rebuttal provided by the authors. The abation using different models for $\\mathcal{C}\\_{LLM}$ and $\\mathcal{R}_{LLM}$ have also improved my confidence that the proposed method is generalizable.", "rebuttal": "We appreciate the reviewer for the thoughtful feedback，recognizing (i) The proposed method is intuitive and well motivated, automated and scalable (ii) contribution in improving sample efficiency.We've addressed your concerns below.\n>The synthesis cost of SIGMA is not discussed...\n\nWe appreciate the reviewer’s concern regarding the synthesis cost of SIGMA. As a data-efficient method, maintaining low synthesis cost and computation cost is central to its practicality and effectiveness.\n\nSIGMA does not require any additional model training during the data synthesis phase. We use Qwen-2.5-Math-7B for original MCTS generation,which is a publicly available pretrained models from HuggingFace. Following the MCTS data generation procedure of AlphaMath [6], using Qwen-2.5-Math-7B to generate full MCTS trees for a 15K dataset requires ~42 GPU hours on RTX 4090.\n\nFor the refinement phase, we use GPT-4o-mini to process the 15K MCTS trees.The total prompt token count is 33.6M and the total completion token count is 11.42M, resulting in a total cost of 11.7 USD.Processing 60K MCTS on GPT-4o-mini costs 47.6 USD.Detailed comparison for synthesis cost are listed below. The GPU price data are from vast​.​ai.\n\n| Method | GPU Type | GPU Hours | Price per hour(USD)  | API Cost（USD） | Total Price(USD) |\n|:---:|:---:|:---:|:---:|:---:|:---:|\n| DART-Math | A100 | 3840 | 0.9 | – | 3456 |\n| SIGMA(GPT4o-mini as Crtique and Revision model) | RTX 4090 | 168 | 0.35 | 47.6 | 106.4 |\n| SIGMA(Qwen2.5-7B-Instructas as Crtique and Revision model) | RTX 4090 | 200 | 0.35 | – | 70 |\n\nCompared to alternative data generation pipelines, SIGMA offers a competitive balance of low computational cost, high data efficiency, and improved overall accuracy. We will include detailed computational cost analysis in the camera-ready version.Moreover, thanks to the flexibility of the SIGMA framework, we can also substitute the proprietary GPT‑4o‑mini with an open‑source 7B model, reducing costs by over 30% while incurring only a 4.6% drop in performance.\n\n>The total tokens in the SIGMA dataset is unclear...\n\nWe appreciate the reviewer’s detailed concern for computational cost for training that dives into token-level.\n\nTo support the claim of SIGMA’s data efficiency, we report the total number of tokens used in SIGMA-15K, SIGMA-60K, and other baseline datasets, all measured using the LLaMA3-8B tokenizer. SIGMA-15K contains 11.5M tokens,It is worth noting that, with a comparable number of tokens per question, DeepSeek-7B-SIGMA (15K) outperforms MathFusion trained on 30K examples. while SIGMA-60K contains 44M tokens, which is roughly comparable to the token count of MathFusion’s 60K dataset, SIGMA achieves state-of-the-art performance, highlighting its efficiency in data usage.\n\n| Dataset | Tokens (Llama3-8B tokenizer) |\n|:---:|:---:|\n| DART-Math | 190441618 |\n| MMIQC  | 940937974 |\n| RefAug  | 54228136 |\n| MetaMath | 110278040 |\n| MathFusion | 40437234 |\n| SIGMA | 44064782 |\n\nMoreover, the low cost of data synthesis in SIGMA makes it scalable to larger datasets without prohibitive resource demands. We will include detailed token statistics in the camera-ready version.\n>Only GPT-4o-mini is explored as $C_{LLM}$ and $R_{LLM}$.\n\nThank you for the insightful question. For the $C_{LLM}$ and $R_{LLM}$, GPT-4o-mini was chosen as a cost-effective option. However, our method does not fully rely on closed-source models. To support this, we have included additional results using Qwen2.5-72B-Instruct, and Qwen2.5-Math-7B-Instruct as $C_{LLM}$ and $R_{LLM}$ to synthesise 3 new 15K datasets, and SFT on models in our main experiment.Switching $C_{LLM}$ and $R_{LLM}$ to weaker open-source LLMs resultd in a comparable performance compared with closed-source model as $C_{LLM}$ and $R_{LLM}$.Under Qwen-2.5-72B-instruct，the results achieves 34.3 on Llama3-8B，which is still comparable；31 on mistral，and 46.5 on deepseekMath-7B,which remains state-of-the-art with other methods.\n\n| **15K Samples** | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| **Llama3-8B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 35.3  | 77.1  | 21.7  | 39.5  | 9.6  | 16.9  | 33.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 36.3  | 78.9  | 23.2  | 39.4  | 9.8  | 18.0  | 34.3  |\n| GPT4o-mini as Critique and Revision Model | 36 | 82 | 24.2 | 42 | 10.5 | 22 | 36.1 |\n| MathFusion（30K Average Score） | 37.2  | 76.7  | 23.9  | 37.1  | 12.1  | 17.1  | 34.1  |\n| **Mistral-7B-v0.1 Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 28.8 | 79.3 | 18.4 | 33.3 | 6.7 | 13.5 | 30 |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 30.6 | 76.8 | 21.1 | 35.3 | 6.8 | 15.6 | 31 |\n| GPT4o-mini as Critique and Revision Model | 30 | 75.3 | 20.8 | 39.5 | 7.7 | 16.3 | 31.6 |\n| MathFusion（30K Average Score） | 30.0  | 74.0  | 18.5  | 25.7  | 9.2  | 14.5  | 28.6  |\n| **DeepSeekMath-7B-Base Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 49.8  | 80.6  | 38.3  | 60.6  | 18.4  | 24.1  | 45.3  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 51.3  | 81.9  | 39.9  | 64.8  | 17.5  | 23.8  | 46.5  |\n| GPT4o-mini as Critique and Revision Model | 52.2 | 81.1 | 37.4 | 64.5 | 20.3 | 26.1 | 47 |\n| MathFusion（30K Average Score） | 49.8  | 76.0  | 38.2  | 60.7  | 20.0  | 21.9  | 44.4  |\n| **Qwen2.5-Math-7B Series** |  |  |  |  |  |  |  |\n| Qwen2.5-7B-Instruct as Critique and Revision Model | 74.0  | 89.9  | 48.5  | 83.1  | 39.3  | 40.6  | 62.6  |\n| Qwen2.5-72B-Instruct as Critique and Revision Model | 76.9  | 91.2  | 48.6  | 87.0  | 39.6  | 39.0  | 63.7  |\n| GPT4o-mini as Critique and Revision Model | 79.9  | 89.2  | 45.2  | 88.8  | 43.4  | 47.0  | 65.6  |\n\nWe are deeply grateful for your insightful comments on self-distillation. Taking your suggestion to heart, we used Qwen2.5-7B-Instruct — a model with the same parameter size and comparable performance to Qwen2.5-Math-7B — as both the critique and revision model to synthesize 15K data points. To our delight, we found that despite Qwen2.5-7B-Instruct being less capable than GPT4o-mini, the fine-tuned model exhibited only a very minimal performance drop on this dataset (from 65.6 to 62.6).\n\n>Qwen-2.5-Math-7B is used as the candidate generator to construct search trees. One particularly interesting analysis, which was not included...\n\nThank you for suggesting exlporation on self-distillation setting. In the main experiments, we did not finetune Qwen2.5-Math-7B because existing strong baselines such as DART and MathFusion and other baselines have not reported results on this model. To ensure fair and verifiable comparisons, we focused on base models with publicly available results.\n\nTo further assess the self-distillation ability of our method, we additionally conducted finetuning experiments on Qwen2.5-Math-7B.SIGMA remains state-of-the-art performace on Qwen2.5-Math-7B, reaching 79.92 for MATH, 89.23 for GSM8K and 65.6 on all datasets average,surpassing other methods such as Eurus and OpenMath.The results are shown in the following table.\n\n\n| Qwen2.5-Math-7B (Fine-tuned) | # Samples | MATH | GSM8K | College | DM | Olympiad | Theorem | AVG |\n|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Rstar@Greedy | 747K | 78.40  | 89.70  | **52.50** | – | **47.10** | – | – |\n| Eurus-2-7B-PRIME | 230K | 53.88  | 42.99  | 20.97  | 47.30  | 22.22  | 18.62  | 34.33  |\n| OpenMath-Nemotron-7B | 3.2M | 40.52  | 73.39  | 32.93  | 54.70  | 11.41  | 13.63  | 37.76  |\n| qwen2.5math-instruct | 2.5M | 82.16  | 95.15  | 46.80  | 82.00  | 41.60  | 40.75  | 64.74  |\n| AceMath-7B-Instruct | 2.3M | 75.40  | 58.38  | 45.82  | 59.70  | 39.85  | 30.75  | 51.65  |\n| DART-Math-60K | 60K | 63.14  | 91.28  | 47.23  | 74.90  | 26.81  | 35.38  | 56.46  |\n| MathFusion-60K | 60K | 56.14  | 59.59  | 37.08  | 67.70  | 41.19  | 27.87  | 48.26  |\n| SIGMA-60K | 60K | 79.92  | 89.23  | 45.17  | **88.80** | 43.41  | **47.00** | **65.59** |\n\n>Some outputs from $C_{LLM}$ do not actually appear to contain a critique.$C_{LLM}$ appears to simply be describing the sibling node approaches but doesn’t offer any specific critiques. Could the authors speculate on whether the more robust critiques could be extracted from $C_{LLM}$ and if so, how?\n\nThis is an important question and directly relates to the effectiveness of the SIGMA method.\n\nIn most mathematical problems, there exists a distinction between critical step（logical steps that directly determine correctness) and non-critical steps(routine computations or definition lookups). To ensure that all potentially critical reasoning steps are considered, SIGMA applies step-level refinement throughout the entire MCTS trajectory. This full-chain processing inevitably includes some non-critical steps that may not benefit significantly from refinement. Occasionally, LLMs may fail to generate meaningful critiques for these simpler steps.That's why even if the refinement of simple steps may not directly improve final accuracy, including them is necessary to ensure robustness across the entire reasoning paths.\n\nThe processing of entire reasoning paths does not compromise the overall effectiveness of SIGMA. Because our refinement operates at the step level, and each selected step $T_p$ shares the same parent as its sibling steps $T_s$, any critical step will be refined using other steps that are equally critical. This guarantees that when refinement matters most, the available sibling signals are relevant and high quality.\n\n We will clarify this point in the camera-ready version.\n \n**Final Notes:** We appreciate for recognizing the motivation and efficiency of our method. We've added clarifications on synthesis cost, token usage, and open-source/self-distillation results. We hope these updates address your concerns and highlight SIGMA's practicality."}]

Author rebuttal:

Meta-review: This paper introduces SIGMA, a data-efficient framework for refining chain-of-thought (CoT) reasoning paths generated by Monte Carlo Tree Search (MCTS). The main contribution is a two-stage refinement process: instead of discarding non-optimal sibling nodes in the search tree, leveraging them to provide corrective feedback. A "critique" model first identifies the strength and weakness of the chosen reasoning step relative to its siblings, and generate a "symbolic gradient" in natural language. A "revision" model then applies this feedback to improve the original step. This method systematically extracts valuable signals from the entire search tree, significantly enhancing the quality of the final training data. The work is relevant to NeurIPS, addressing the critical challenge of improving data quality for complex LLM reasoning tasks.

Most reviewers were positive about the paper, with scores leaning towards acceptance: 5, 5, 5, 4, 3.
Also, confidence level is generally high due to the clarity of the proposed method and the strength of the empirical results.

Reviewers pointed out the novelty and intuition of the core idea (reusing discarded sibling nodes). The data efficiency demonstrated in the experiments is quite impressive and highlighted as a major strength. The comprehensive evaluation across multiple base models and benchmarks was also commended.

One concern raised is this framework relies on a capable proprietary model (GPT-4o-mini) for the critique and revision steps. This dependency could limit the accessibility of the method and raises questions about whether the performance gains are partly attributable to the knowledge transfer from this "teacher" model.

The method is presented as a heuristic approach. The analogy to "Textual Gradient Descent" is intuitive but not theoretically rigorous. However, the overall methodology is sound, well-motivated, and clearly described.

The empirical evidence is a key strength of this work - the authors show that models trained on small, SIGMA-refined datasets (15K-30K samples) consistently outperform models trained on much larger datasets from prior work. The well-designed ablation studies effectively isolate the benefits of the sibling-guided refinement process over both vanilla MCTS paths and standard black-box CoT generation.

Furthermore, the paper provides sufficient detail regarding the experimental setup, models, and hyperparameters, suggesting that the results are reproducible. This work has the potential for significant impact on how the community approaches data synthesis for LLMs. By demonstrating that substantial gains can be achieved by more intelligently processing the outputs of existing generation methods, this paper offers a compelling alternative to simply scaling dataset size.

Overall, this is a nice paper with a novel, well-executed idea and compelling empirical results. The method's demonstrated ability to create high-quality training data with remarkable efficiency is a significant contribution. While the reliance on a proprietary teacher model is a valid concern, the novelty of the framework and the strength of the results make it a clear accept. The work will be of great interest to the NeurIPS community and is well-suited for discussion in a poster session.
Decision: Accept (poster)