Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
Authors: Xuemiao Zhang
Can Ren
Chengying Tu
Rongxiang Weng
Shuo Wang
Hongfei Yan
Jingang Wang
Xunliang Cai
Venue: ICLR 2026 Poster (2026)
Keywords: large language models
reasoning potential
long chain of thought
reasoning pattern
challenging mathematical reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose CoTP to select data rich in high-value reasoning patterns to greatly expand the reasoning potential of the 85A6B MoE foundational model, thus achieving a 9.58% improvement on AIME 2025&2024 and raising the upper bounds of RL by 7.81%.

Abstract: Recent progress in large reasoning models for challenging mathematical reasoning has been driven by reinforcement learning (RL). 
Incorporating long chain-of-thought (CoT) data during mid-training has also been shown to substantially improve reasoning depth. 
However, current approaches often utilize CoT data indiscriminately, leaving open the critical question of which data types most effectively enhance model reasoning capabilities.
In this paper, we define the foundation model's reasoning potential for the first time as the inverse of the number of independent attempts required to correctly answer the question, which is strongly correlated with the final model performance.
We then propose utilizing diverse data enriched with high-value reasoning patterns to expand the reasoning potential. 
Specifically, we abstract atomic reasoning patterns from CoT sequences, characterized by commonality and inductive capabilities, and use them to construct a core reference set enriched with valuable reasoning patterns. 
Furthermore, we propose a dual-granularity algorithm involving chains of reasoning patterns and token entropy, efficiently selecting high-value CoT data (CoTP) from the data pool that aligns with the core set, thereby training models to master reasoning effectively. 
Only 10B-token CoTP data enables the 85A6B Mixture-of-Experts (MoE) model to improve by **9.58\%** on the challenging AIME 2024 and 2025, and to raise the upper bound of downstream RL performance by **7.81\%**.

Published as a conference paper at ICLR 2026
EXPANDING REASONING POTENTIAL IN FOUNDATION MODEL BY LEARNING DIVERSE CHAINS OF THOUGHT PATTERNS
Xuemiao Zhang 1.3+
Can Ren 1.3+ Chengying Tu
Rongxiang Weng 37 Shuo Wang' Hongfei Yan2.4 Jingang Wang' Xunliang Cai²
Peking University School of Computer Science, Peking University MMeituan
4 State Key Laboratory of Multimedia Information Processing, Peking University !zhangxuemias, yanhf @anhf edu. en wengronngxiangeeaalll tuchengying, 24C1210098 www.stu.uuu.ee....mmeeeeeee..eee pku.edu: en (anggshuu81, wangjingang02, ca saunliangg @me tuan. com
ABSTRACT
Recent progress in large reasoning models for challenging mathematical reason- ing has been driven by reinforcement learning (RL). Incorporating long chain- of-thought (CoT) data during hid-training has also been shown to substantially improve reasoning depth. However, current approaches often utilize CoT data indiseriminately. leaving open the critical question of which data types most ef- fectively enhance model reasoning capabilities. dation model's reasoning potential for the first time as the inverse of the num- ber of independent attempts required to correctly answer the question, which is strongly correlated with the final model performance. We then propose utilizing diverse data enriched with high-value reasoning patterns to expand the reason- ing potential. Specifically, we abstract atomic reasoning patterns from CoT se- quences, characterized by commonality and inductive capabilities, and use them construct a reference set enriched with valuable reasoning patterns. Fur- thermore, we propose a granulaaatyr algorithm involving chains of reasoning patterns and token entropy, efficiently selecting high-value CoT' data (CoTP) from the data pool that aligns with the core set, thereby training models to master rea- soning effectively. Only 10B-token CoTP data enables the 85A6B Mixture-of- Experts (MoE) model to improve by 9.58% on the challenging AIME 2024 and 2025, and to raise the upper bound of downstream RL performance by 7.81 %.
this paper, we define the foun-
INTRODUCTION
Recent progress in large reasoning models (LRMs) for challenging mathematical reasoning has largely been driven by post-training optimization, particularly via RL frameworks that improve
mublem--sllving abilities through exploratory feedback (Zeng al. 2025 Zheng et al. 2025
Schulman et al. 2017; Shao et al. (2024a). Empirical studies (Chen et al. 2025 Yue et al. 2025
Liu et al. 2025a Zhao al. 2025b; Wen et al. 2025) have revealed some critical insights that the parameter space of the foundation model inherently contains latent pathways for challenging reasoning and RL training effectively operationalizes the explicit manifestation of these implicit ca- pabilities. In other words, the reasoning capability learned in foundation models directly influences and limits the upper bounds of RL performance, with certain open-source foundation models like Llama (Grtttafion et al.) 2024) displaying unstable RL performance, emphasizing the urgent need for a thorough exploration foundation model reasoning capabilities.
Recent studies suggest that blending question-answe (QA) data with thain-of-thoure (CoT) during the mid-training stage, especially long-CoT samples, can markedly enhance the depth of reasoning
"Equal contribution.
1 'Corresponding author.
'The core code and dataset available whttpsss/githhbb.com/r33/11111 9

---

Published as a conference paper at ICLR 2026
Potential
Performance
Normal/Impartont Patter
Theeretical Optinum
Orocle
Foundation
Think
P7
P8
Idealization
Approximate.
t-o Training bata
Recall times replacement)
Framework
Set
Core
copy
Mcnuel Selection
Set
Times
Recall
Recall Nearest
Collect QA
Arnotction
by
/No Araner
Source Data Pool
Source Data Pool
(a) The CoTP framework that selects long-CoTT data similar to the core set based on the chains of reasoning patterns and token entropy.
(b) Graph of core set reason- ing pattern chains.
Figure 1: Illustration of the CoTP framework. The left figure shows the overall process of the CoTP framework, while the right figure shows the graph of reasoning chains. The patterns with higher TF-IDF weights are important, while the remaining patterns are considered normal. The CoTP framework selects the minimum distance chain from the source data pool.
needed to tackle challenging problems (Wang et al. 2025b; Zhang et al. 2025b; Tu et al. 2025) Prevailing approaches primarily focus on augmenting datasets with challenging problems gener- ate long-Co'I' trajectories through knowledge distillation. However, these methods typically employ CoT data in coarse-graine goarse-rrained manner, lacking thorough investigation into the inherent paradigmatic characteristics and deeper essence of reasoning within CoT sequences for nuanced and differenti- ated applications. Therefore, elucidating the reasoning paradigms that can significantly expand the reasoning potential of foundation models represents pivotal research direction.
Building on these insights, our research focuses on how to effectively enrich reasoning patterns during mid-training to expand the reasoning potential of foundation models. We first theoretically define the reasoning potential of the foundation model and demonstrate that expanding this poten- tial is equivalent to reducing the average number of reasoning attempts needed to correctly answer then abstract atomic reasoning patterns from CoT sequences, characterized by commonality and inductive capabilities, and these to construct core reference set enriched with valuable reasoning patterns. We propose a dual-granularii algorithm using weighted Dynamic Time Warping (DTW) to select long-Cof data exhibiting high-value reasoning patterns similar to those in the core set based on both the chains of reasoning patterns and token entropy.
the question.
Extensive experiments demonstrate that mid-training just 10B high-value reasoning data can significantly expand the reasoning potential of the 85A6B Sesture-offFxperr foundational model (MoE-6B-85A) and substantially raise the upper bounds of RL performance. In summary, our con- tributions are as follows:
1. We theoretically define the reasoning potential of the foundation model for the first time as the inverse of the number of independent attempts required to correctly answer the question and abstract reasoning patterns that exhibit commonality and inductive capabilities from CoT data, guiding the construction of a high-value core reference set.
2. We propose a dual-gra granularaullritty algorithm involving chains of reasoning patterns and token entropy, which efficiently selects CoT data with high-value reasoning patterns that aligns with the core reference set from the data pool, to effectively enrich reasoning patterns and expand the reasoning potential of foundation models.
3. We construct a long-CoT reasoning dataset, CoTP, which enables the 85A6B MoE model to maintain general performance while achieving 9.58 % improvement on the challenging AIME 2024 and 2025 and raising the upper bound of downstream RL performance by 7.81 %.
2

---

Published as a conference paper at ICLR 2026
2 СотP: HIGH- VALUE REASONING PATTERNS FOR EFFICIENT REASONING
Our research aims train the foundation model Mo learn a wide variety of high-value reason- ing patterns during the mid-training stage, thereby expanding the reasoning potential of Mo and incentivizing RL performance. We begin by theoretically defining and analyzing the reasoning po- tential of the foundation model. By abstracting atomic reasoning patterns from CoT sequences, we construct a core reference set enriched with valuable reasoning patterns to approximate the oracle reasoning data. As depicted in Figure Ia our CoTР framework effectively selects long-CoT data similar the core set from the source data pool using chains of reasoning patterns and token entropy, aided by a dual-granularity algorithm involving weighted DTW.
2.1 PROBLEM FORMULATION
We theoretically define and analyze the model potential. Unlike conventional deterministic model evaluation, we assess the model potential by adopting the sampling mode inference multiple times to capture the stochastic nature of model performance. The model potential is then defined as follows:
Definition 1 (Model Potential) For given model M in sampling decoding mode and question qs we define the model potential & (M, q:) as the probability that the model generates the correct answer for question qe when sampling from its output distribution:
(MM, = P[fm(g)) = =ai],
(1)
where (m) denotes the sampled output and of is the correct answer. The overall model potential (=))
(M,, Devot) is defined as the expected potential on an evaluation dataset Deval = {(g, Nouel;
Novns
(MM, Denat) R (gg.a) ())
(M,,
Neval
There is fundamental relationship between model potential and expected reasoning cost.
Corollary Let K; denote the furst-pasaage time for question qu representing the number of inde- pendent attempts required to solve qs Suppose each attempt is an independent Bernoulli trial with success probability (M, qs), that K; Geom ( q:)). Then,
(M,. = [[K]]
(3)
In other words, the model potential is the inverse of the expected first-passage time and smaller K; indicates higher model potential.
To achieve lower K for any given question, a promising approach is to extensively expose the model to a wide range of effective reasoning paradigms and enable to internalize them. This insight motivates us to construct ideal oracle dataset for training LLMs, which consists of samples that exhibit high reasoning efficiency, i.e., those with low-K K characteristics.
Objective. Formally, assume there exists an ideal oracle training dataset Doracte that enables the foundation model to achieve maximal reasoning potential. Our goal to select training subset Dirain containing M samples from given source dataset D Donurce {(jj, Cj. Ujj { to min- imize the gap between the reasoning potential of the model trained on Dirain and that trained on D Di Doracte'
Dt = arg max
( Devat) - (MM, Devat)l
(4)
D
Di arg
min
((MD, D Denat) (MM bradte Devat)],
(5)
DCD Dorreee D
where MD denotes the model trained on dataset D, cj and a, are the CoT and answer for question Gj and denotes metadata labels such subject and difficulty.
3

---

Published as a conference paper at ICLR 2026
2.2 CORE SET APPROXIMATES ORACLE
Since it is difficult to determine Döracle we propose approximating it using a carefully constructed reference core consisting of CoT data rich in diverse high-value reasoning patterns, shown in Figure Tb
This core set will guide the selection of Diraan from D Peource-
We propose capturing the reasoning properties of each CoT sample at two granularities: using pat- tern chains to capture highly abstract reasoning paradigms (Chen et al., 2025), and using entropy chains to capture token features with high reasoning gain (Wang et al. 2025a Cui et al. 2025).
Definition 2 (Reasoning Pattern and Pattern Chain) A reasoning pattern is an atomic cogni- rive operation representing a fundamental reasoning step applicable across diverse problem do- mains (see Figure 5). And pattern chain C = [p), p2.,. p2....,Pr. is an ordered sequence of reasoning patterns extracted from a CoT sequence (see Figure
We employ Deepseek- V3 (Liu et al. 2024) to annotate the reasoning pattern chain for each CoT sequence (detailed in Appendix E The entropy of each token in the CoT sequence is computed as h4 = - -Evev (e(w) log (v), where pe(v) denotes the probability of token v at position t. Consequently, the entropy chain is represented as H. = h/,,, hu,.hh.,hhp Illustrative examples of annotated reasoning pattern chains and entropy chains for CoT data are provided in Appendix G.3
Formally, we define extraction functions & : C- P and : - IRT"" mapping each CoT sequence to its pattern chain f() and entropy chain n()). (1) To construct the core set, first filter questions from the source dataset D Peource, annotated with difficulty levels and problem types. Questions are randomly sampled to match the expected distribution. We then employ multiple strong reasoning models to independently generate answers, using majority voting to determine the correct ones, and filter out low-quality data, such as unsolvable questions, yielding = {9I ........ 9 (2) For each
qs Q, we employ strong reasoning model to generate CoT sequences denoted as { We extract the pattern chains {(c,,) and assess the importance of each pattern Pr for the question g. using the TF-IDF weighting scheme:
w(pk 9:: Q) TF(pk 9:) X IDF(PE. Q).
where the calculation details for TF and IDF are provided in Appendix B.I (3) From the remain- ing CoT sequences with correct answers, according to the importance scores of different patterns, manually select those exhibiting distinctive, high-importance patterns construct the core set Crore = {() Ci, as ()) 1= (see Figure Ib). where each instance is accompanied by its pattern importance weights Ω [w<Pk q: 2)) for Pk E E(c)).
2.3 SELECT TRAINING DATA WITH VALUE REASONING PATTERNS
Given Ceore {lg ci, af. (f) we construct a training dataset Dirain of size T by selecting instances with similar CoT sequences from the source dataset DD source = {af aj. aj, {)));= For analytical convenience, we set T with O source instances. We formulate this as an assignment problem with capacity constraints. Let D E RC*N where Diy denotes the distance between CoT sequences of and oz. We seek to optimize a binary matrix S E {0, 1)&xN, where Siz = 1 signifies the assignment of source instance j to core instance i. The objective is given by:
o. where T N, ensuring each core instance is associated
N
min S
Dig
Sis
i=1
j=
N
S4 = Vi Σ
Σ S. ( Sny E (0.1)
s.l.
1=1
1=1
The distance Dij is the weighted sum of pattern chain and entropy chain distances, with E [0, 1]:
Dij = \ dpatem (c]), E(c))) + (1 - X) (entropy(y1())))) ((()))((())))))
(8)
For distance computation, we employ DTW for both dimensions:
d(z,)) v) WeightedDTTW (1:, y, w. 8).
(9)
4

---

Published as a conference paper at ICLR 2026
We set the parameters follows: for the pattern chain distance, E(c)) y E { Q4 and 8 dagram (sce Figure 4); for the entropy chain distance, n(()), y n/c: w
1, and
dahs. The distance computation and WeightedDTWW are detailed in Algorithms 2 and
To efficiently solve this optimization problem, we reformulate it standard linear assignment problem by replicating each core instance o times, resulting in an expanded cost matrix of size t N with entries Drj- The optimal assignment is then obtained using the Hungarian algorithm Kuhn 1955; Mills-Tettey et al. 2007). This transformation guarantees optimality, as each source instance is assigned to most one core, and the replication ensures that each core receives exactly assignments (see Appendix [B.2] for proof). The overall data selection procedure of our CoTP framework summarized Algorithm II Note that it is domain-agnostic and theoretically appli- cable to any scenario decomposable into atomic reasoning patterns (see Appendix F33 for pattern visualization in STEM domains).
Algorithm/ Hungarian Data Selection Algorithm
Input: Source dataset D poot; Core Set Coore's Weight parameter À Output: Final selected dataset D Destect
1: Initialize Deetent +- 0. D E RiPpoal x x|Ccoral
2: for each (qf. cj. az. {j) E D poot, == to do
3: dpattem WeightedDTW (E(C(((()))))))))) (E(c)),E(C))) S2 dingram // refer Algorithm 3 and
4: dentropy WeightedD (()(c))))(C)) 1, refer Algorithm
5: i[ + A dpattm + (1 A) dentropy
6: assignment, t Hungarian (D,) Input: cost matrix; Output: optimal assignment
7: Deelect & [l ct ag,()) ez)= E selected indices from assignment,) }
8: return Dxclett
EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Data Construction. To construct a high-quality reasoning data pool, we integrate diverse math-
ematical QA datasets as follows and conduct rigorous n-gram deduplication: (1) OpenRI-Math-
220k (Hugging Face, 2025) (OpenR1-Man) 1-Math) comprises 220k math problems, each expanded with two to four reasoning traces generated by DeepSeek-RI (Guo al. 2025): (2) AM--eepSeek--
R1-Distilled (Zhao et al. 2025a) (AM-Distilled) focuses on general reasoning tasks, with detailed thinking traces; (3) BoostQA (Zhang et al., 2025b) consists of large-scale QA pairs of different difficulty levels. We specifically select high-difficuly H4/H5-level questions from BoostQA and employ DeepSeek-RI to generate long reasoning COT, maintaining consistency with OpenR -Math and AM-Distilled, with a maximum output length set to 32k tokens. In industrial production, the use of multiple reasoning models encouraged to enrich the variety of reasoning patterns. We construct unanswered data, annotating chains of
the LongCoTPool (see Figure Lap by excluding truncated reasoning patterns, and ensuring no overlap with the core set. Following Shao et al.] (2024b), we employ exact 10-gram matching and embedding- based similarity filtering to mitigate contamination of questions and answers originating from benchmarks.
Training Details. We conduct mid-training experiments using the 85A6B MoB (Jiang al. 2024) foundation model, which is pre-trained on 14T-token corpora. During the mid-training stage, the model decays on mixture of 30B-token specialized experimental reasoning data and general- domain data, KnowEdu (Zhang et al., (2025)) at 1:2 ratio. The reasoming data follows a structured format of (question) ) (((oor answer with final answers encapsulated in boxedd() In scaling ex- periments, the data volume is expanded to 60B tokens while maintaining the same data blend ratio. We further conduct SFT using the same collected dataset of 60k long-CoT entries to enhance the models' capability to generate long-CoT sequences. This step is crucial for ensuring fair compar- isons across models, as it prevents underestimation of models that initially lack the ability to produce long- CoT outputs (see analysis in Appendix D.2). Moreover, the SFT stage facilitates smooth transi- tions to RL by establishing a robust foundation in CoT reasoning for effective rollouts. Importantly, apply the same SFT data across all models to eliminate any variances and uphold consistency
5

---

Published as a conference paper at ICLR 2026
Table 1: General performance and average pass @ accuracy (%) of models. The best and second- best are in bold and underlined, respectively. Abbreviations: Beyond BeyondAIME.
Dataset
General
ATME2025
AIME2024
HMMT2025
Beyond
MATH500
AVG.
KnowEdu
64.39
0.33+0035
1.22+0068
5.10+1.40
0.00+0000
45.80+4.37
10.49+1.36
BoostQA
63.29
0.52+0.46
1.46
4.06
0.40
54.00
12.09+1.45
JiuZhan33.0
64.32
0.83+0057
1.25+0.71
1.92+
0.10+0.20
56.00-:4336
12.22
MegaMathQA
64.79
0.211:0.29
2.19:10.33
4.17
0.40-10.39
51.80:14.38
11.75::1.45
OMInstruct-2
66.24
3.44
8.02.11.73
5.42.11.44
1.20.10.67
72.20:13.93
18.06
M-enRI
66.58
23.96:1:2.74
29.6912.92
16.04.12..34
9.10:11.79
87.80.12.87
33.32.12.53
AM-Distilled
67.97
23.12++2270
25.52+22.79
18.02:+2.46
8.30
87.20
32.43+22.52
LongCoTPool
65.95
21.89::2.46
24.90
15.63-42.31
7.90+
85,40++..10
31.14
CoTP (Ours)
66.08
28.02:12.88
37.9213.09
20.7312.558
10.20:1..8
90.8012.54
37.5312.59
in evaluation conditions. Furthermore, we conduct experiments verify that the expanded reason- ing potential of the foundation model can enhance downstream RL performance. Specifically, we apply the same experimental settings to foundation models trained on different datasets to evaluate their subsequent RL performance, where the RL algorithm adopts GSPO (Zheng et al. 2025). The detailed setup is provided in Appendix C.I
Evaluation. We conduct comprehensive evaluation of mid-trained models to assess their capa- bility sustain general performance after exposure reasoning-intensivve data, including bench- marks such as MMLJU (Hendrycks et al.] (2021a). CMMLU (Li et al. (2024), C-Eyal (Huang et al,
2023). WinoGrande (Sakaguchi et al./ (021). HellaSwag (Zellers et al.) 2019
ARC-C (Clark
eet al. 2018). BIG-Bench (Suzgun et al., 2023) and DROP (Dua et al.
2019). Furthermore,
conduct assessments of the SFT models on downstream challenging mathematical reason- ing tasks to examine the reasoning potential of the foundation models. The benchmarks include AIME 2025 & 2024 (MAA), HMMT 2025 (HMMT), BeyondAIME (ByteDance- Seed, 2025). and MATH500 (Hendrycks et al. 202 each repeated multiple times for statistical robustness (detailed in Appendix C.2). Pass @ (Chen et al. 2021) curves are drawn to provide a detailed visualization of model performance dynamics with increasing attempts k. For the RL stage, the same benchmarks are employed to evaluate the enhancements in reasoning capabilities.
Baselines. Baselines can be divided into two categories, which differ in mid-training data with the same other settings. The first paradigm evaluates the 30B-token general corpus KnowEdu (Zhang et al. 2025b), high-quality knowledge-rich educational dataset. The second paradigm assesses the QA blend following the same 1:2 blend ratio between QA data and KnowEdu. BoostQA (Zhang al. 2025b) contains QA data without CoT sequences. The short-CoT QA datasets include Ji- uZhang3.0 (Zhou et al., (204)), MegaMathQA, a QA subset from SrggaMatth-Synthher (Zhou et al. 2025), and OpenMathInstruct-22 (Toshniwal et al. 2024) (OMInstruct-2), The long-CoT QA datasets include OpenR Mathh, AM-Distilled, and our curated data pool, LonoC.TPool. (detailed in Table
3.2 MAIN RESULTS
Only 10B high-value reasoning data selected by CoTP can significantly improve multiple chal- lenging mathematical reasoning tasks by an average of 6.39 % reaching SOTA results and sub- stantially raising the upper bound of RL performance. As depicted in Table | CoTP not only sustains performance across general benchmarks, but also surpasses baselines multiple challeng- ing mathematical reasoning benchmarks, with average gains of 4.21% 1 compared to OpenR1-N0 -Math and 5.10% compared AM-Distilled. More detailed discussions on general performance in Appendix[D.. It offers an average improvement of 6.39% over LongCoTPool with a remarkable 9.58% enhancement on AIME 2025 & 2024. Furthermore, LongCoTPool, shows a slight decline in average performance relative to OpenR I-ath and A.M-Distilled, indicat- ing that simple mixing of reasoning data might be insufficient to expand the model's reasoning po- tential. In contrast, CоТР, curated from LongCoTPool, achieves optimal performance, underscoring the importance of selecting high-value reasoning data and the advantages of our CoTP framework.
shown
mixed data pool,

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of the performance of models mid-trained on different datasets.
AIME 2025
AIME
2024
HMMT 2025
BeyondAIME
MATH500
AVG.
Dataset
SFT
RL
SFT
RL
SFT
RL
SFT
RL
SFT
RL
SFT
RL
KdowEdu Lono
0.33
0.31
1.22
0.83
5.10
1.25
0.00
0.00
45.80
44.60
10.49
9.40
21.89
31.88
24.90
44.38
15.63
35.31
7.90
16,40
85.40
90.20
31.14
43.63
CcTP (Ours)
28.02
40.81
37.92
58.65
20.73
41.35
10.20
23.20
90.80
93.20
37.53
51.44
AIME 2025
BeyondAIME
As depicted in Figure in the pass@k evaluation, models mid-trained CoT datasets, particularly CoTP, show consis- tent and progressive improvement as the number of attempts k increases, along with superior performance as the RL training steps increase, consistent with findings from OctoThinker (Wang et al.) 2025b). This growth highlights the panded model potential during the mid- training stage (see Table D which can substantially raise the upper bound of RL performance. This demonstrates that the improvements introduced dur- ing mid-training can be effectively car- ried over to RI., rather than being prema- turely acquired during mid-training and thereby diminishing the distinct benefits typically observed in RL, which would otherwise result in no significant differ- ence in final model performance. Our CoTP dataset stands out by exhibiting superior mathematical reasoning capabil- ities, with average improvement of 7.81% % over LongCoTPool and 42.04% over KnowEdu. This highlights its ef- ficacy for expanding challenging mathe-
Coverage (pass.0
04
0.4
0.3
0.2
0.0
0.-
16
32
128
16
64
Number of Samples Number of Samples k
Knew
MegaMaati3A
PoPol
BA
OpenRt-Atah
CHFP
40-
Accuracy (%)
30
20-
M
100
200
300
100
200
300
RI Steps
Steps
Figure 2: The comparison of pass @k and RL perfor- mance across different datasets.
matical reasoning potential, thereby incentivizing RL performance.
CoTP exhibits exceptional scalability in challenging mathematical reasoning tasks. As illus- trated in Figure 3 (detailed in Appendix [D3). when sealed to 60B tokens-witt additional data incorporated by relaxing the similarity threshold the model sustains its upward performance tra- jectory, achieving a 4.72% average improvement on AIME 2025 & 2024 compared to the results 30B tokens. This suggests that CoTP effectively expands the model's potential in challenging mathematical reasoning tasks even at larger data volumes while maintaining general performance, demonstrating its continuous effectiveness in recalling high-value reasoning data.
3.3 ABLATIONS
We conduct ablation studies to analyze the impact of various components in the CoTP framework. For efficiency and rapid validation, these ablation experiments are performed on the 12B-token QA blend, maintaining the same 1:2 ratio between QA data and KnowEdu and and using the same experimental settings as in the main experiments. The results are shown in T
Entropy-based selection enhances the quality of reasoning data accessed. By employing a reference model to perform offline evaluation of the information content of each token within the CoT reasoning data (Wang et al., (025a), entropy-based selection enables more fine- ane-grained capture of token-level reasoning paradigms. The utility of this approach is further illustrated by the
7

---

Published as a conference paper at ICLR 2026
AIME 2025
AIME 2024
HMMT 2025
Beyond AIMondAMME
Accuracy (%)
30
20
10
10
0
0
0
20
40
0
20
40
60
0
20
40
0
20
40
Training Tokens (B)
Training Tokens (B)
Training Tokens (B)
Training Tokens (B)
KnowEdu
LongCoTPool
CoTP
Figure Scalability of data volume examining the SFT performance of models miid-trained on datasets of varying volumes. The dashed lines represent the performance of each dataset configured under the 30B-token setting.
Table 3: Ablation results. CoTP uses n=/ or 2 for n-gram setting, 2=088 for entropy. And w/o entropy denotes X=l.
Dataset
AIME 2025
AIME 2024
HMMT 2025
BeyondAIME
MATH500
AVG.
CoTP
21.8412.68
25.76:1..70
13.9012.27
6.70=
85.2013.25
30.68:12.449
w/o entropy
21.46+2..3
25.63++2.78
11.88+2.066
6.30
84,40+3.118
29.93++2.49
i-grram n=2
18.3342.48
25.42.12.77
11.56:12.04
6.50
83.80113.23
29.1242.41
importance
19.6912.54
24.1712.772 13.65.1..19
6.60
84.2011.20
29.66.12.44
which provides deeper insights into the structural nuances captured. This enhanced quality correlates with improved model performance, underscoring the pivotal role of entropy-based selection in fostering advanced reasoning capabilities.
entropy visualization in Appendix G.2
For n-gram pattern similarity, n=] or 2 yields superior outcomes, providing more balanced evaluation of pattern similarity. This configuration integrates the broader contextual encapsula- tion of n=1 with the detailed specificity of n=2, offering a comprehensive representation of pattern alignments. Notably, unlike English, each character in Chinese carries intrinsic semantic meaning and we demonstrate that Chinese pattern representation exhibits clear distinctions between unrelated pattern pairs under the character-level n-gram method (detailed Appendix F22). which makes more suitable for calculating distances between pattern entries in Algorithm Consequently,
employ Chinese for annotating reasoning pattern chains.
Importance scores play a critical role the CoTP framework. The model lacking importance weighting shows poorer performance, underscoring the significance of pattern importance scores. highlights the distinction of normal and important reasoning patterns, due to their different contri- bution reasoning potential.
4 ANALYSIS
Our CoTP-trained model demonstrates an enhanced mastery over broader array of correct and key reasoning patterns. To validate whether CoTP enables models to learn key reasoning patterns closer to the ideal ones, conduct comparative analysis of reasoning patterns exhibited by models trained on various datasets, taking AIME 2025 & 2024 as examples. Specifically, we consider the model trained on the oracle set as the ideal model, whose generated reasoning traces encompass comprehensive patterns. Our core set serves as an approximation of this oracle. Through the data selection algorithm, we select training data that aligns with the core set distribution. In
8

---

Published as a conference paper at ICLR 2026
Prebiem Decoms
Enumension ITten
Mas Antienetic
Min Bearch
Mot H Flaing
Soxtion Churel Lilinn)
Min Seluton
Veri calion
1760
17031
Post-raning Cbben
Lifing
0.060
0033
3,30
Delete
roart Optirnll
on
Pro-triining
Rarfernce Chain
Figure 4: Illustration of DTW alignment analysis on the pattern chain similarity matrix.
Coperiiang Set Enuegration Vodu lar Subetitutionggt laffl
Conclusion Optimization
Condittion Check
can carting
Currrrrrenn enrrency conversion ion Fracsson Constructian
Geonetry opttetration
Inverse Mapping
hoppurs
Step beduct Scoss Validation Output Inteerra reggration ion
Probl em & Goal Genera aomerallzaan Lization Gedlutte
grroup
Factor Liatiioon io malpca Functionn Inversion
Linea Combinat
Linear Traesfore
explacenee
seu eddetes Sin at Case Veri iarttiiiaioo
teltt Reverse Ver ifriiiie inffiiciionn
Metho 6 Brin
2pr pack
Ingquality Parsing Crass Bultiblication
Pr Popprtiio tiontio Reasoning Median Ca
Calculat Lining
Tescing Re lnnegration
Concept Revi wwwwwweee Knowled edge
Ret Rivvl
and
Param arameter Substi bobssiittte tution
w Proding Cennrszon converston boace tartition
Placmuss innegration Group Transform....
Baxis Pr iniiciple
Conclus ion Output Goal Conf ma tion
Concapt in
Result Confir Confirmati tion
(a) Reasoning patterns with high importance scores.
(b) Reasoning patterns with low importance scores.
Figure 5: Examples of reasoning patterns with different levels of importance.
this analysis, use the strong reasoning model DeepSeek-RI R1 a proxy for the ideal model. appearing in correct reasoning pattern chains.
Correct patterns are defined as those previously As illustrated in Table 4 (detailed in Ap- pendix [.T.). CoTP not only increases the num- ber of correct patterns utilizes but also shows an enhanced overlap with the key patterns mas- tered by DeepSeek-R1, with an increase of 7.25% compared to LongCoTPool. This ad- vancement suggests that our training method- ology effectively improves the model's capa- bility to solve challenging reasoning tasks by systematically integrating and aligning with the reasoning paradigms established by high- performance models like DeepSeek-R1.
Table 4: A comparative analysis of reasoning pat- terns on AIME 2025 & 2024.
Dataset
Patterns
Correct
Key
KnowEdu
4296
147
116 (1.79%)
LongCoTPool
3407
2246
1345 (20.73%%
CoTP
3970
3226
1815 (27.98%)
Our CoTP-trained model shows notable convergence to the reasoning paradigms of the ref- erence model, evidenced by a reduction in pattern chain distances. By conducting a compre- hensive pattern distance analysis of models before and after mid-training against DeepSeek-RI on the core set, we quantify these alignment enhancements. As depicted in Figure (6a] we employ the DTW algorithm (see Figure 4 calculate the reasoning pattern chain distances between the CoT responses from both the model instances before and after midtraining and those from DeepSeek-RI within each question, then compute the Wasserstein distance between these cross-model distance distributions for each question, and finally average the results across all questions. The average dis- tance between the model before mid-training and DeepSeek-R1 is 0.51, whereas our C-rr-trained model achieves a reduced distance of 0.35, marking 31 ..4% improvement. Notably, CoTP acquires sophisticated reasoning patterns (see Figure 5a). These advanced reasoning patterns reflect expert- level problem-solving approaches and underscore that CoTP effectively facilitates the acquisition of sophisticated reasoning capabilities critical for solving challenging mathematical problems.
The composition of CoTP closely matches the distribution the core set. As shown in Fig- ure 6b the KL divergence between the problem type distributions relative to the core set decreases from 0.18 (LongCoTPool) to 0.04 (CoTP), demonstrating improved alignment. Furthermore, as il- lustrated in Figure [cc| the token length distribution of CoTP aligns more closely with the core set, which generally features longer token lengths, than with Tonol. This alignment highlights the effectiveness of our CoTP framework targeting the core set distribution.

---

Published as a conference paper at ICLR 2026
hong
After middrriiing
Core
Sumber
Percenaggy
Problem
Core Set lorgCot
Nurra
Chee
Distance Reference Nadill Car
Percentage 193
Noken Leagth
(a) Reasoning chain distance distri- butions relative to DeepSeek-RI.
(b) Distribution of problem types in (c) Distribution of token lengths in different datasets.
different datasets.
Figure 6: Distribution analysis.
Effective reasoning patterns not only guide the steps of chain-of-thought but also facilitate deeper introspective processes. We investigate the relationship between reasoning patterns and reflection and examples in Appendix illlustaae how reflection (Shah et al.] 2025) be seam- lessly integrated into established reasoning pattern chains, showcasing the multifaceted character- isties of introspective processes. Our detailed examination reveals that pattern chains inherently encompass reflective characteristics. Reflection may be encapsulated within discrete patterns, pro- viding precise cognitive insights, or may appear as recursive loops in pattern chains, indicating iterative thought refinement. This analysis to investigating reasoning and reflection indicates their intertwined nature.
5 RELATED WORK
Endowing LLMs with reasoning capabilities remains challenging, particularly in the realm of com- plex mathematical reasoning. Recent advancements in RL algorithm optimization strategies have significantly enhanced the performance LLMs on downstream challenging mathematical reason- ing tasks (Chen et al.] 2025 Yue et al. 2025 Liu et al.) 2025a). These improvements are largely driven by exploration- feedbbcck mechanisms that systematically elevate the problem-solving skills of these models. Additionally, they have unveiled the inherent constraints on RL performance imposed by the reasoning capability reserves of the foundation models. Studies investigating the interaction between the foundation models and RL performance (Gandhi al. 2025; Liu et al. 2025b Wang cct al., 20255), such as those related to cogmitive behaviors that facilitate self-improvement and in- sights from training paradigms like R1-Zero, have demonstrated that different foundation models exhibit diverse initial reasoning behaviors. These behaviors critically influence their ability for self- improvement, thereby manifesting varied RL scaling characteristics.
Among these studies, Octokhinker (Wang et al.) 2025b) has validated that integrating CoT QA data, particularly long-CoT samples, during the mid-training stage significantly enhances the rea- soning capabilities required for complex groblem-solving This approach offers a promising avenue for enhancing the reasoning capabilities of foundation models. Notable large-scale open-source long-CoT QA datasets include OpenRI OpenRR1MMath-25 -Mathh-220 (Hugging Face (2025) and AMM-eepSeek-RL Distilled (Zhao al., 2025a), with their long-Co'I responses predominantly generated by DeepSeek- R1. Such developments underscore the importance of strategically aligning mid-training data with downstream tasks as a potent strategy for expanding the reasoning proficiency of LLMs.
CONCLUSION
In conclusion, our proposed CoTP framework presents significant advancement in expanding LLM reasoning potential. By defining reasoning potential as the inverse of the number of independent at- tempts required to correctly answer the question, we have proposed a framework that constructs high-value reasoning data. Our dual-granularity algorithm. which leverages reasoning patterns and token entropy, efficiently selects valuable CoT data similar to the core reference set, thereby enrich- ing reasoning patterns in foundation models. Through the construction of the CoTP dataset, we have enabled the 85A6B MoE foundational model to achieve a 9.58% improvement on the challenging AIME 2025 & 2024 and to raise the upper bound of downstream RI. performance by 7.81 %.
10

---

Published as a conference paper at ICLR 2026
7 ETHICS STATEMENT
Our research adheres to the ICLR Code of Ethics. We have carefully read the ethical guidelines and ensured that our research does not present ethical concerns.
REPRODUCIBILITY STATEMENT
We provide comprehensive details throughout various sections, including the main paper, appendix, and supplementary materials, ensure reproducibility. We ensure all experiments and analyses are clearly described, with additional details provided in the appendix. The source codes of the algorithm are provided in the supplementary materials. Explanations and complete proofs of the theoretical results are also included in the appendix. Details about the experiments, including the datasets and the computing infrastructure, are provided in both the appendix and the supplementary materials.
REFERENCES
Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. Matharena: Evaluating Ilms on uncontaminated math competitions. arXiv preprint arXiv:2505.23260 2025.
ByteDance-Seed. Beyondaime:
Advancing math reasoning evaluation beyond high https:
school olympiads. BeyondAIME 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. arXiv preprint arXiv:2107.03374. 2021.
Xingwu Chen, Tianle Li, and Difan Zou. On the mechanism of reasoning pattern selection in reinforcement learning for language models. arXiv preprint SEiv:2506.046993 2025.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv: 1803.05457 2018.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22617, 2025.
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp- 2019.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cogni- tive behaviors that enable self aaf-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01 03 2025.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21778 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in lims through reinforce- ment learning. Nature, 645(808 (000008))63336333 2025.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the Interna- fional Conference on Learning Representations (ICLR), 2021a.
11

---

Published as a conference paper at ICLR 2026
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 2021b.
HMMT. Harvard- mathematics tournaments (HMMT), 2025. URL https: www.hmmt.
org/
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, al. C-eval: multi-level multi-discipline chinese eval- uation suite for foundation models. Advances in Neural Information Processing Systems, 36: 62991-63010, 2023.
Hugging Face. Open rl: A fully open reproduction of deepseek-rl, January 2025. URL https: h//ithub.comm///// i ngface/
Albert Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bam- ford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 2024.
Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.
Haonan Li, Yixuan Zhang, Fajri Koto. Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. Cmmlu: Measuring massive multitask language understanding in chinese. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 11260-11285 1285, 2024.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. 2024.
Zichen Liu, Changyu Chen, Wenjun Li, Tianyu Pang. Chao Du, and Min Lin. There may not be aha moment in ri-zero-like training pilot study. https::/oatln 1lm.lmm..mmm.mm.mmmm...mm...mm.m not inn. on. site/ zero 2025a. Notion Blog.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl ri-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025b.
MAA. American invitational mathematics examination (AIME). Mathematics Competition Series, 2025. URL/https: httt::::/mmaaoroei www./orgggaaaao.ooooo..o.a..aaa. math cowpetitionssttttcccccc..oooctee
Ayorkor Mills-Tettey, Anthony Stentz, and M Bernardine Dias. The dynamic hungarian algorithm for the assignment problem with changing costs. Robotics Institute, Pittsburgh, PA, Tech. Rep. CMU-RR-7RR-07227.7 7, 2007.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. Communications of the ACM, 64(9)999-106, 2021.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.0633 2017.
Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. Rethinking reflection in pre- training, arXiv preprint arXhv:2504.04000 2025.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.033300 2024a.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Rumxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li. et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv: 03300, 2024b.
12

---

Published as a conference paper at ICLR 2026
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery. Quoc Le, Ed Chi, Denny Zhou, et al. Challenging big bench tasks and whether Mhain-of-thougee can solve them. In Findings of the Association for Computational Lin- guistics: ACL 2023, pp. 13003-13051, 2023.
Shubham Toshniwal, Wei Du, Ivan Moshkov, Branislav Kisacanin, Alexan Ayrapetyan, and Igor Gitman. Openmathinstruct-:: Accelerating ai for math with massive open-source instruction data. arXiv preprint arXiv:2410.01 2024.
Chengying Tu, Xuemiao Zhang, Rongxiang Weng, Rumei Li, Chen Zhang, Yang Bai, Hongfei Yan, Jingang Wang, and Xunliang Cai. A survey on Il mid-training. arXiv preprint arXiv:2510.23301 2025.
Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning. arXiv preprini (etii::25000019999 2025a.
Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. Octothinker: Mid-training incentivizes reinforcement learning scaling. arXiv preprint arXiv:2506.20512 2025b.
Xumeng Wen, Zihan Liu, Shun Zheng, Zhijian Xu, Shengyu Ye, Zhirong Wu, Xiao Liang, Yang Wang, Junjie Li, Ziming Miao, et al. Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base llms. arXiv preprint arXiv:2506. 2025.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does inforcement learning really incentivize reasoning capacity in Ilms beyond the base model? arXiv preprint arXiv:2504. 13837, 2025.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a ma- chine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, 2019.
Aohan Zeng, Xin Lv, Qinkai Zheng. Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and et al. Glm-4.5: Agentic, reasoning, and coding (arc) foun- dation models. arXiv preprint arXiv:2508.065 06470,,.677,, 2025.
Xuemiao Zhang, Can Ren, Chengying Tu, Rongxiang Weng, Hongfei Yan, Jingang Wang, and Xunliang Cai. Linkqa: Synthesizing diverse qa from multiple seeds strongly linked by knowledge points. arXiv preprint 1508::2508.011111 2025a.
Xuemiao Zhang, Chengying Tu, Can Ren, Rongxiang Weng, Hongfei Yan, Jingang Wang, and Xunliang Cai. Large-scale diverse synthesis for mid-training. arXiv preprint arXiv:2508.01326, 2025b.
Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, and Xiangang Li. 1.4 million open-source distilled reasoning dataset to empower large language model training. arXiv preprint arXiv:2503.19633. 2025a.
Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. Echo chamber: RI post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912. 2025b.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071 2025.
Fan Zhou, Zengzhi Wang, Nikhil Ranjan, Zhoujun Cheng, Liping Tang, Guowei He, Zhengzhong Liu, and Eric P Xing, Megamath: Pushing the limits of open math corpora. arXiv preprint arXiv:2504.02807 2025.
Kun Zhou, Beichen Zhang, Zhipeng Chen, Xin Zhao, Jing Sha. Zhichao Sheng, Shijin Wang, Ji- Rong Wen, et al. Jiuzhang3. O: Efficiently improving mathematical reasoning by training small data synthesis models. Advances in Neural Information Processing Systems, 37:1854-1889, 2024.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
We utilize LLMs solely to refine our manuscript. Specifically, LLMs are employed to identify and correct emrors in spelling, grammar, punctuation, and formatting the sentence level.
B MATHEMATICAL FORMULATIONS AND ALGORITHMS
B.1 PATTERN IMPORTANCE CALCULATION VIA TF-IDF
We employ TF-IDF weighting scheme to quantify the importance of reasoning patterns, balancing pattern frequency within individual questions against their global rarity across the dataset.
The importance of pattern ps for question ge is calculated using a TF-IDF weighting scheme:
w(pk 9: Q)
TF(pk, 9:) X IDF(Pk, Q)
count(pu: in U2=1 E(G,))
TF(pk:
Σ, count(p in Uj=1 E(c,,))
(10)
O
IDF(pa, 2) = =log 11g Pk: - E(c,g)
21
where E(cr,) represents the pattern chain extracted from the j-th CoT data of qr.
B.2 EQUIVALENCE TO PERFECT BIPARTITE MATCHING
We establish the theoretical foundation for our assignment optimization by proving the equivalence between the
assignment problem and the perfect bipartite matching problem.
Theorem The capacity-come constraiied assignment problem in Eq. is equivalent minimum weight perfect bipartite matching problem.
Proof. We construct balanced bipartite graph G' (U' UV, E') where both partitions have size N. The left partition U' = {wu,k i E [1, k E [1,o]] { : [1, - L [) contains t replica nodes (each core instance replicated times) plus (N- dummy nodes. The right partition V = {u> e [1, NJ} contains the original N source instances.
Edge weights are defined as:
w(uik, UU) = Dij, [e [I w Uddummy & U2) = M, We [1,N - ol, [1.N]
(11)
(12)
where M maxi.j Dij is sufficiently large constant.
The transformed problem seeks minimum weight perfect matching in G¹:
N-to
min X
Σ
D Xiks
Σ
M Xduumy, f.j
N
s.t.
Xu.t 1,
Yu E
(13)
/=1
Σ Xu.i 1, [jE[L,N]
KEt!
Xn.5 {0,1) 1}
We establish equivalence by showing that feasible solutions correspond bijectively with identical objective values.
Given any feasible solution S to the original problem, we construct a perfect matching X as follows. For each core i, let J, {j Sig be its assigned sources with [J]] Order J; arbitrarily
14

---

Published as a conference paper at ICLR 2026
11 3.....J )o") and set X (. for h E [1, This matches exactly o sources replica nodes. The remaining (N (-0) unmatched sources are matched to dummy nodes with cost M. The objective value is:
+ (N (N- Doj (N o0 - D.S (.K.
-) O)
(14)
i.j
Conversely, given any minimum weight perfect matching X in G', we construct S by setting Stj there exists such that X,s.j max,, D any optimal perfect matching will minimize the use of dummy nodes. Specifically, exactly (N - t- o) dummy nodes must be matched (due to perfect matching constraints), and the optimal solution will match all o replica nodes distinct sources to minimize total cost. Therefore, each core has exactly o matched replicas, giving Σ, Sij = o, and each source matched to at most one replica, giving E, Sis < 1. The objective values satisfy:
for replica node ug and Stj
otherwise. Since M >
DIS. DDIXin 1
(15)
This bijective correspondence between feasible solutions with proportional objective values estab- lishes that optimal solutions of both problems correspond exactly.
B.3 ALGORITHM
We implement two complementary distance metrics for measuring similarity between reasoning patterns and pattern chains: character n-gram cosine distance for lexical similarity (detailed in Al- gorithm 2) and weighted dynamic time warping for structural alignment (detailed in Algorithm 3).
Algorithm Character N-gram Cosine Distance
Input: String a; String b; Maximum n-gram length n
Output: N-gram cosine distance dogram (a, b)
1: a <- normalize(a): b < normalize(h) (b)
2: Initialize frequency maps F, 0, Fb -
3: for k = 1 to do:
4: for i
to a|
do: Pa lall: + - F. Palaji :
5: for = -1 to b k + do: Folbli : i k 1]]
Polbli: -
6: dot < E, Falg] Folg]
7: ||/,]|2 < E, F.[g]; |661 + Eg 2
8: if FF.12 or F
0 then return 0.0
9: return 1.0 dot
VIP..1P-1E.11
C EXPERIMENTAL SETUP
C.I TRAINING DETAILS
We use H800 mid-train our pre-trained 85A6B Mixture-off Mixture-of-Experts foundation model. which is pre-trained on 14T-token corpora. The model decays 30B tokens of specialized experimental reasoning data and general-domain data, KnowEdu (Zhang et al., 2025b a) a 1:2 ratio, using the WSD scheduler with the learning rate initialized lc-4. The reasoning data follows vanilla struc- tured format of {question) ) |nfcot_ansser with final answers encapsulated in \boxed{) notation. In scaling experiments, the mid-training token size is further expanded to 60B while maintaining the same data blend ratio.
Furthermore, utilize identical SFT data across all models, thereby eliminating the introduction of additional variables and ensuring consistency in evaluation conditions. The SFT stage takes a batch size of 256 and epoch size of 3, processing inputs up to 16,384 tokens to accommodate extended
15

---

Published as a conference paper at ICLR 2026
Algorithm 3 Weighted Dynamic Time Warping Distance
Input: Sequence x
(Z)). Sequence y = (y11...,, vm); Weight vector w
(wi..... :;). Distance function 8
Output: Distance value
1: n 0 or m = 0 then return 1.0
2: Initialize matrices D, W p(+t))x(m+1) with zeros
3: for i 1 to n do:
4: D:.0 f- 1; + w1 6(2,, Wi, < Wi-1,0 + w1
5: for 1 to m do:
6: Do.j f- Do.j-1 + wy ((x1,1)): Wo. f- Wo.-1 w.
7: for i = 1 to n do:
8:
for j =
m do:
9:
d E 6(24, 84))
10:
D; E Di.j and 1-j-- 2 D;-1, then
11:
(Derev: W prev) f- (Di-1.j11, W ii----)
12:
elif Di S D5 then
(prev W prev) t (Dij-1: -1)
14:
else (Dpee, W (ev) f (Di-1j) I,) W;-
Di, Dreev + w d: W,j W prev + Wi
Dm.. if W n,m else 0.0
16:
return W Wa.m
reasoning sequences. optimized by the Adam algorithm and initializes the learning rate at 5e-6 with cosine decay to zero.
The RL stage runs 300 steps for all models and applies the GSPO algorithm (Zheng et al. 2025). Its hyperparameters include global batch size of 512, rollout batch size of 256, and single epoch, optimized by the Adam algorithm with a constant learning rate of 1.0e-6. The generation max sequence length is set to 32,768, with the sample number at 16, temperature and top-p both at 1, and GSPO-specific clip range between 0.0003 and 0.0004.
C.2 EVALUATION
We conduct an evaluation of the SFT and RL performance on challenging mathematical reasoning tasks, similar Balunović et (2025). For statistical robustness, we use 32 repetitions for AIME and HMMT, and 10 repetitions for BeyondAIME, once for MATH500.
C.3 DATASETS
Table 5 shows the comparison with open-source CoT QA datasets.
Table 5: Comparison with open-source CoT QA datasets.
Dataset
Target Domain
CoT
Date
JiuZhang3.0
Mathematical Reasoning
Short-CoT
2024 May.
OpenMathInstruutt22
Mathematical Reasoning
Short-CoT
2024 Oct.
MegaMathQA
Mathematical Reasoning
Short-CoT
2025 Apr.
OpenR M-Math-220k
Mathematical Reasoning
Long-CoT
2025 Feb.
AM- AM-DecpSeck-a -ooopSeek-RI-issilllle
General Reasoning
Long-CoT
2025 Mar.
CoTP
Mathematical Reasoning
Long-CoTT 2025
16

---

Published as a conference paper at ICLR 2026
D RESULTS
D.1 GENERAL PERFORMANCE
Table shows the detailed comparison of general performance, corresponding the general eval- uation results in Table T in Section 3.2 Our CoTP achieves better general performance than KnowEdu and LongCo]Pool, demonstrating that CoTP effectively expands the reasoning potential of foundation models in challenging mathematical reasoning tasks while maintaining general per- formance. Note that the performance gaps between CoTP and OpenRI-Men -Math or AM-Distilled stem from the heterogeneous nature of LongCoTPool, which integrates OpenR1-Math, AM-Distilled, and BoostQA, with BoostQA exhibiting relatively weaker general performance. Since CoTP is selected from LongCoTPool, inherits these performance gaps. However, this does not affect objective and the main conclusions of our experiments.
Table General performance comparison of models mid-trained on different datasets. The best and second-best are in bold and underlined, respectively. Abbreviations: W.G. WinoGrande, H.S. HellaSwag, BBH = Big-Bench.
Dataset
MMLU
CMMLU
C-Eval
W.G.
H.S.
ARC-C
BBH
DROP
AVG.
KnowEdu 69.83
75.59
74.30
58.00
50.50
83.50
48,86
54.50
64.39
BoostQA
69.97
75.50
73.25
54.00
44.00
84.00
47.09
58.50
63.29
JiuZhang3.0
68.71
74.94
74.51
61.50
50.50
85.00
45.37
54.00
64.32
MegaMathQA
68.80
75.43
73.71
62.00
47.50
85.00
45.85
60.00
64.79
OMInstruct-2
68.76
75.39
74.29
62.00
50.00
85.50
49.45
64.50
66.24
O-enR1-Math
69.28
76.05
74.23
61.50
55.50
87.50
49.54
59.00
66.58
AM-Distilled
69.65
75.73
74.14
61.50
63.00
85.00
50.75
64.00
67.97
LonoCoTPooo CoTP (Ours)
69.72
76.08
73.89
57.00
53.00
87.00
49.00
61.87
65.95
69.54
75.67
74.33
59.50
55.50
83.50
49.11
61.50
66.08
D.2
PASS @KK K CURVES
As illustrated Figure the pass@k curves for both base and SFT models on the CoT dataset show stable increase as & becomes larger. In contrast, for the KnowEdu corpus, the pass @ k values are notably lower without the application of SFT, but they improve once SFT is applicd. This indicates that SFT ensures the model has foundational ability to produce long CoT outputs, contributing to a more equitable evaluation across different datasets. Given the consistent relative trend observed before and after SFT, we exhibit our main results the SFT models to ensure reliability and fairness in comparison.
D.3 SCALING DETAILS
The specific accuracy values at 60B tokens in the scaling experiments are presented in Table
Table Accuracy of different models 60B tokens in the experiments of data volume scale.
Dataset
AIME 2025
AIME 2024
HMMT 2025 BeyondAIME MATH500
KnowEdu
0.1010.20
0.9410.61
3.65#1.18
0.30+0.34
42.0014.33
PonoCoTPool
27.88+2.67
32.62+22.3
18.22+2.24
10.01+1.66
90.56+2.87
CoTP (Ours)
31.52+2.87
43.8543.16
22.6041.68
14.00+2.11
91.00+2.51
D.4 TRUNCATION RATIO
The truncation ratios are shown in Table g
We observe that the truncation rate is notably lower in long CoT datasets, and the truncation rate decreases further in RI compared SFT.
17

---

Published as a conference paper at ICLR 2026
AIME 2025
AIME 2024
BeyondAIME
0.3
04
Coverage (pass
0.4
0.3
0.5
0.3
0.2
0.2
0.3
0.1
0.1
0.1
0.
16
64
128
4
32
64
128
16
64
128
Number of Samples k
Number of Samples k
Number of Samples k
KnowEdu Base
LongCoTPool Base
BBse
CoTP Base
KnowEdu SFT
LongCoTPool SFT
CoTP SFT
Figure 7: @ass@k curves of base and SFT models.
Table 8: Comparison of truncation ratio (%).
AIME 2025
AIME
2024
HMMT 2025
BeyondAIME.
Dataset
SFT
RL
SFT
RL
SFT
RL
SFT
RL
KnowFdu LonoCoTPool
82.34
44.69
85.68
45.05
85.05
48.78
80.75
45.67
70.89
2.24
73.18
2.94
82.24
1.61
82.70
0.90
CoTP (Ours)
61.61
5.76
57.11
5.23
71.51
4.61
75.34
3.82
D.5 RESPONSE LENGTHI
As depicted in Figure the response lengths during RL training exhibit different characteristics. Initially, models that have undergone SFT produce relatively longer responses. As the training progresses, the response lengths for LongCoTPool and CoTP stabilize and converge, whereas the lengths for KnowEdu continue to fluctuate significantly. This suggests that the RL. training with KnowEdu may be prone to instability or collapse.
AIME2025
AIME2024
HMMT2025
BeyondAIME
MATH500
Response Length (K)
10-
100
300
100
200
300)
100
200
100
100
200
300
1000
200
Training Step
Training Step
Training Step
Training Step
Training Step
Knowldd
LongCoTTPoo
C6TP
Figure 8: Response length (k) at the RL stage.
D.6 EVALUATION CHART ON MORE BENCHMARKS.
Figure 9 and Figure I0 present the p evaluation curves on the AIME2024 and Math500 benchmarks, respectively, illustrating the performance trajectory throughout the RL training pro-
18

---

Published as a conference paper at ICLR 2026
AIME 2024
AIME 2024
60
0.8
Coverage (pass @kk
Accuracy (%)
50
0.6
40
30
2.5
0.0
32
64
128
0.0
Number of Samples k
100
200
300
RL Steps
Edu
MegaMarhQA
PonoC
BocsiQA
OpaaRI-Malh
COTP
LongCoTPool
Figure 9: AIME2024.
MATH500
MATH500
95
Accuracy (%)
Accuracy (%)
80
90
60
40
50
Ww
20
40
60
100
200
300
Training Tokens (B)
RL Steps
KnowEdu
LongCoTPool
CoTP
KnowEdu
LongCoTPool
CoTP
Figure 10: MATH500.
cess. These results serve as a complement to Figure Section 3.2 results in Figure 2
and are consistent with the
E REASONING PATTERN CHAIN ANNOTATION
Prompt for CoT pattern chain extraction annotation
Task Objective:
Systematically explore and summarize the Chain-of-Thought (CoT) processes employed by mainstream LLMs in reasoning tasks, analyzing the core reasoning patterns embedded within these processes.
Analysis Instructions:
Please conduct an in-depth examination of the reasoning paths taken by various Al models reasoning tasks, demonstrating how different models approach and solve problems. Your goal is to summarize and categorize the general thinking patterns reflected in these
19

---

Published as a conference paper at ICLR 2026
reasoning processes, to help understand the essential characteristics CoT reasoning in large models.
Analysis Steps:
For each reasoning process, please clearly identify the following elements:
1. Use of keywords and high-frequency phrases
2. Logical structure and organization of argumentation Techniques strategies used solve the problem
4. The manner in which reasoning steps are unfolded
Classification Requirements:
Based on the following commonalities, accurately categorize similar reasoning processes into one more general reasoning patterns:
Lexical pattern (organization and use of common terms and phrases)
2. Logical framework (structure of argumentation and reasoning flow)
3. Solution pathway (methods and paths to reach conclusions)
Important Notes:
You are required to summarize " "general thinking patterns for problem solving, " not specific problem types.
Each pattern should be applicable to any problem scenario, limited particular type of task.
Focus on the thinking method itself, rather than specific solution steps or answer content
Illustrative Examples:
You may categorize as follows:
Knowledge retrieval-based reasoning
Reasoning combined with verification
Step-by-step deductive calculation
Detailed stepwise derivation
Attention Points:
Precisely categorize the above reasoning processes into one or more patterns (>=1), defining each category based on its shared characteristics, explaining role in reasoning for the given case, and providing examples.
Briefly explain your analysis and classification criteria first, then output detailed annotation for each reasoning pattern in the JSON format below. The "name" field for each pattern must output in Chinese.
The "pattern chain" " field outputs a list, where the element order represents the sequence of patterns used in this CoT solution, e.g., [1,2,3,4). If necessary, the reasoning pattern chain may contain loops.
Output atomic patterns only (no pattern should contain words like "and", "or", etc.).
Output Format: json
pattern list":
("id":1, "name": "", "description": n", "features "",
sample-input, flow "",
corresponding CoT content": ["" " ""]. "common elements"; "",
typical. expreessoopesss ; ["" ""]
{ ["id":2, " "name": "" "description ". "features": ""
sample input_flow flow "role the.iinttiii..ommm wwwiiittiscssee... mhss"
corresponding.Com CoT__ontent"""" [=", " ""], "common _elioontseees: "", axpical.expression ["", ""]
20

---

Published as a conference paper at ICLR 2026
J.
"how Co'T utiiiizes attlizes.pptteerss... in ww.thhs.comm casee" {
"process,deseee "onnesssdescriptionn.. "..
ceattrm_chain" [].
J.
Reasoning process to be analyzed: (extracted responses)
F ANALYSIS
F.1 REASONING PATTERNS ACROSS DIFFERENT MODELS
In the comparative analysis of reasoning patterns across different models on AIME 2025 & 2024, the number of patterns is defined as size of the entire of reasoning patterns. Correct patterns refer the set composed of patterns that appear in correct reasoning chains, while key patterns represent the overlap between the correct pattern sets and those mastered by DeepSeek-RI. Notably, DeepSeek- RI has mastered 6,487 correct patterns, and the percentage indicates the proportion of the overlap within this set.
F.2 PATTERN REPRESENTATION ANALYSIS
We conduct comparative analysis of the pattern representations in Chinese and English. We sam- ple pattern pairs from the CoTP dataset, manually selecting unrelated pairs that do not share core mathematical concept words. For example, "Problem Understanding and Information Extraction" versus "Trigonometric Identity Transformation" are considered unrelated because the former per- tains to problem comprehension, while the latter involves specific mathematical technique. In contrast, "Verification and Confirmation" and "Verification and Adjustment" are related, as both concern verification processes. We evaluate the similarity distributions of both related and unrelated pattern pairs in Chinese and English representations using the n-gram method.
As shown in Table and Table TO most pattern pairs in English identified either moderately similar or highly similar, making it difficult to differentiate between unrelated patterns, while Chi- nese offers superior ability distinguish between patterns. Compared with Chinese, English tends to display more misjudgment cases, as illustrated in Table IT]
These results demonstrate that Chinese pattern representation exhibits clear distinctions between un- related pattern pairs under the character-level n-gram method, which is beneficial for the consistency and controllability of large-scale extraction of patterns. Even there slight noise or drift between patterns in Chinese, such as small amount of character interference, this method is essentially insensitive to these variants and can still identify semantically similar patterns.
Table 9: The proportion of different similarity ranges of Chinese and English
Similarity Range
Chinese
English
Low (<0.3)
88.2%
1.4% 83.2%
Medium (0.3-0.7)
11.2%
High (>007)
0.6%
15.4%
21

---

Published as a conference paper at ICLR 2026
Table 10: For the average similarity of unrelated pattern pairs, Chinese can score these pairs with low similarity, whereas English tends to misjudge them as similar.
Language
Average Similarity
Chinese 0.035
English
0.541
Table 11: Typical misjudgment cases with high similarity in English but low in Chinese.
Chinese Pattern
Chinese Simi- larity
English Pattern Pair
English Simi- larity
Root Cause
Pair
MOARSAE
0.000
Problem Understanding
0.762
High repetition suffixes "-tion", "-ation"
E vy E
and Information Extraction Trigonometric Identity Transformation
#
0.000
Mathematical Modeling and Equation Construction Logical Elimination Method
0.760
Overlapping character-level suffxees
N w : 3
0.000
Verification and Correction Trigonometric Identity Transformation
0.758
Common suffix interference
F.3 REASONING PATTERN VISUALIZATION IN DIFFERENT DOMAINS
To demonstrate the domain-agnose nature of our CоТР framework, we visualize the reasoning patterns with high and low TF-IDF importance scores in Figure across across four representative STEM domains: physics, chemistry, biology, and computer science. These visualizations are generated from the TF-IDF weighted pattern distributions extracted from our training data.
These visualizations demonstrate that our pattern-bat battern-based representation successfully captures domain-specific reasoning strategies across diverse STEM domains, supporting the claim that CoTP is applicable to any scenario that can be decomposed into atomic reasoning patterns.
G CASE STUDY
In practice, we found that, unlike English, each character in Chinese possesses inherent semantic meaning, which makes Chinese particularly suitable for calculating distances between pattern entries using the ngram-cosine algorithm (see Algorithm[2), Consequently, we employ a annotation prompt extract pattern chain features from the CoT data, and require the annotation
(see Appendix model to output the patterns in Chinese, as described in Section 2.2
For clarity in the presentation of experimental results, illustrated in Figure 5 have translated the pattern entries into English. The original Chinese-to-English mapping of pattern expressions is provided in Table [2
Table 12: Mapping of English and Chinese expressions in reasoning models.
Chinese Pattern
English Pattern
Chinese Pattern
English Pattern
Parameter Substitution
te L
Data Conversion
Group Factorization
Geometry Optimization
Median Calculation
Piecewise Integration
Proportion Reasoning
Ha
Fraction Construction
Linear Combination
#
Term Calculation
22

---

Published as a conference paper at ICLR 2026
Table 12: (continued)
Chinese Pattern
English Pattern
Chinese Pattern
English Pattern
Step Function
- H
Space Partition
Formula Substitution
fi #
Percentage Conversion
Group Transform
Recursive Expansion
Inverse Mapping
JT. H24
Set Comparison
Function Inversion
Term Replacement
Direction Mapping
#4
Conditional Probability
Data Sorting
Exponential Matching
Currency Conversion
Relation Analysis
Convex Hull
Function Decomposition
Inequality Parsing
34 #6
Path Enumeration
Modular Substitution
A it it
Term Counting
Basic Principle
Euler Line Property
Linear Transform
2 T
Common Factor
Cross Multiplication
Exclusion Method
Set Enumeration
Polynomial Root Analysis
Hint Utilization
3 JJ & Of
Multi-solution Filter
HAULTI yC
Area Matching
Exponential Adjustment
Operation Simplification
Conjugate Root ID
Logic Reasoning
Problem Decomposition
H
Math Tools
i /1 12
Boundary Check
N ME
Numeric Verification
# 3
Conclusion
Logic Deduction
Goal Clarification
Multi-method Check
Math Concepts
Result Integration
Condition Check
Problem Abstraction
Concept Definition
Knowledge Retrieval
Generalization
Case Verification
X HE
Reverse Verification
G.1 RELATIONSHIP BETWEEN REASONING PATTERNS AND REFLECTION
The cases that describe the relationship between the reflection and the chains of reasoning patterns are shown Table 13, including the key features and descriptions of the reflection process LLM reasoning.
G.2
TOKEN ENTROPY
Figure 12 and Figure 13 present two examples of entropy-granular labeling of CoT data utilizing the open-source Qwen-7B model. High-entropy tokens frequently reflect the model's underlying rea- soning process. Our framework exploits the fine-grained token-level entropy distribution to identify nuanced reasoning patterns within the model's CoT data.
G.3
PROBLEM, REASONING CHAIN, AND EXTRACTED PATTERNS
Table 14: Examples of questions, CoT sequences, pattern chains, and entropy chains.
Case 1
Question
23

---

Published as a conference paper at ICLR 2026
Table 14: (continued)
Imagine as 2015 fractions:
3
4
2014
2015
2016
2016
2015
2014"
4
3
2
Is it possible to choose three of these fractions with a product equal to 1?
Pattern Chain
Pattern Recognition and Structural Analysis + Equation Establishment and Transformation + Symmetry Analysis and Special Value Verification + Proof by Contradiction and No-Solution
Exploration
Chain-of-Thought
Okay, let's see. The problem is asking if it's possible to choose three fractions from the given 2015 fractions such that their product equals 1. The fractions are structured in a specific way: starting from 2/2016, then 3/2015, 4/2014, and so on, up to 2016/2. Each numerator increases by 1 while the denominator decreases by 1. First, I need understand the pattern here. The first fraction is 2/2016, the next is 3/2015, then 4/2014, continuing until 2016/2. So, the nth fraction would be (n+1)/(0117 - n), right? Let me check. For n=1, (1+1)/(017 1) 2/2016, which matches. For n=2, 3/2015, which also matches. So yes, each fraction can be represented as (k+1)/(2017 - k) where k ranges from I to 2015. Alternatively, maybe another way to index them? Let me see. If we let the numerator be m, then m starts at 2 and goes up to 2016, and the denominator is 2016 (m 2) = 2018 m. Wait, because when numerator 2, denominator is 2016; when numerator is 3, denominator 2015, which is 2016 -1, so denominator for numerator is 2018- m. Because 2018- 2 2016, 2018 3 = 2015, etc. So each fraction is m/(2018 m). where m ranges from to 2016. That seems correct.
So the fractions can be written as m/(2018- m) for m 2, 3, .... 2016. So there are 2015 fractions total because 2016 2 + 1 = 2015.
We need to find three fractions such that their product is 1. Let's denote the three fractions ml/(2018 ml), m2/(2018- m2), m3/(2018- m3). Then their product is: (ml x m2 x m3) / [20018 ml (2008 m2)(2018- mm3]] =
Therefore, the equation simplifies to: m1 x m2 x m3 = (2018- m2) m2) m3) So, we need three numbers ml, m2, m3 (each between and 2016 inclusive) such that the product of the three m's 's equals the product of (2018- m) for each m.
This seems like a symmetrical equation. Maybe we can look for some symmetry here. Let's consider that for each m, (2018- m) is also a term in the fractions. For example, if m is 2, then 2018 = 2016, which is the denominator of the first fraction. Similarly, if m is 2016, then 2018 2016 2, which is the denominator of the last fraction. So each fraction m/(2018 m) has a reciprocal fraction (2018- m)/m. However, note that when m is 2018 m, which would be when 1009, then the fraction is 1009/1009 1. But looking the original fractions, starting from 2/2016 up to 2016/2. the middle fraction would be when m is around 1009. Let me check: since there are 2015 fractions, the middle one is position 1008, but maybe is 1009. Let's compute 2018- - m = 2018 1009- = 1009, so yes, when m=1009, the fraction is 1009/1009=1. So that fraction is so multiplying by that fraction doesn't 't change the product. But the problem is asking for three fractions, so even if one of them is 1, the other two need to multiply to I as well. But 1 only present once (if m=1009 is allowed). Wait, m ranges from 2 to 2016, so m=1009 is included. So the fraction 1009/1009 is part of the list. So if we include that, then maybe we can have two fractions that are reciprocals. For example, take 2/2016 and 2016/2. Their product is (2x2016)(0166x))
1. Then if we include 1, the product would be 1 1= Wait, but we need three fractions. So 2/2016 x 2016/2 x 1009/1009 = So that's three fractions. But hold on, 2/2016, 2016/2, and 1009/1009. Are these all in the list? Yes. The first fraction is 2/2016, the last is 2016/2, and the middle one is 1009/1009. So their product is 1. So that would be a valid triplet. But wait, but the problem says "three of these fractions". So if 1009/1009 is allowed, then yes. But is 1009/1009 in the list? Let's check. When m=1009, demominator is 2018 1009 = 1009. So yes, that's the fraction 1009/1009. So that's in the list. Therefore, the triplet is 2/2016, 2016/2, and 1009/1009. So their product is (2 x 2016 x 1009)(2016 x 2 x 1009) = 1. So yes, that works.
24

---

Published as a conference paper at ICLR 2026
Table 14: (continued)
Wait, but the problem is stated as: Is it possible to choose three of these fractions with product
equal to 1? ...
Token Entropy
{Okay: 1,0781;, {:: 0.0693), let: 1.3125). {'s: 0.1729). { see: 2.7188). {.: 0.7969), { The:
1.8203). { problem: 2.0938). { is: 1.9375), { asking: 1.0859). { if: 1.0938).
J. { it: 1.2656), f's:
0.5078). possible: 0.0173), to: 0.0376), choose: 1.1641), { three: 0.0491). fractions:
0.8320). from: 0.2393), the: 0.1699). given: 1.2109), {: 1.0313), (2: 0.3691), (0:
0.0021). (1: 0.0044), (5: 0.3613), { fractions: 0.3047), { such: 1.4297], that: 0.0027), {
their: 0.16703...
Case 2
Question
Let a, b, be positive integers, and the quadratic equation ax² + bæ + c = 0 has two real roots whose absolute values are both less than Find the minimum value of a + b c.
Pattern Chain
Condition Analysis and Constraint Extraction Mathematical Derivation and Inequality Analysis Systematic Enumeration and Verification - Two-Root Case Analysis
Chain-of-Thoughh
Okay, so I need find the minimum value of + + e where a, b, are positive integers, and the quadratic equation az² + bx + c = 0 has two real roots whose absolute values are both less than 1/3. Hmm, let's start by recalling some properties of quadratic equations and their roots.
First, for a quadratic equation to have real roots, the discriminant must be non-negative. The discriminant
b2 -4cc. So, D 2 O. That gives us one condition: b² 2 4ac.
Next, the roots have absolute values less than 1/3, Let me denote the roots
and 72. Then [ri] < 1/3 and [r2l < 1/3. Since the quadratic ax² + bur + c, by Vieta's formulas, we have: r1 72 bb/a and r1 r2 ccaa
Given that the absolute values of the roots are less than 1/3, we can derive some inequalities. Let's think about the implications. Since both roots real and their absolute values are less than 1/3, they must lie in the interval (-1/3, 1/3). So both roots are between -1/3 and 1/3.
But since coefficients a, b, с are positive integers, let's consider the signs of the roots. If the quadratic equation has two real roots, and a, b. are positive, then the quadratic opens upwards (since a > 0). The sum of the roots is -b/a, which is negative because b and a are positive. The product of the roots is c/a, which is positive because e and a are positive. Therefore, both roots must be negative. So, actually, the roots are in the interval (-1/3, 0). That simplifies things bit because we don't 't have to consider positive roots. Both roots are negative, so their absolute values are just their magnitudes.
Therefore, we can say that the roots r1 and 72 satisfy - -1/3 < ri S r2 < 0. Let me rephrase Vieta's formulas in terms of magnitudes. Let me set Tı = -s and r2 = -t where and are positive numbers less than 1/3. Then:
F1 + 72 = - (s + t) = - b/aa = s + == b/a and r1 x =2=2==== c/a So and are both in (0, 1/3). So
have s + t = b/a and st = c/a. Then, since a, b, e are positive integers, we need to find a, b, c such that these equations hold with and t 1/3. Our goal is to minimize
b + Since a. b, are positive integers, the minimal value is likely be small, so maybe a, b, c are small integers. But let's 's think through systematically. Given that s and t are both less than 1/3, so S +t t 2/3 and st (1/3) 1/9. But 8 + 1- = b/a and st =0==0// c/a.
So from st = c/a, since st < 1/9, then c/a < 1/9 + c < a/9. But since c is a positive integer, this gives that must be at least 10 for to be at least I (since 1 < a/9 + a 9). Wait, but is that correct? Let's see: If c/a < 1/9, then e < a/9. Since e must be at least 1, then 1 < < a/9 + > 9c. But since c is at least 1, must be greater than 9. So the minimal possible a is 10? Wait, but maybe e can be and a = 10, but then == 1, a= 10. Then < 10/9 x 1.11, so that would satisfy 1 < 10/9. But I is not less than 10/9, it's equal? Wait, no. Wait, c is less than a/9. So 1 < a/9 = a > 9....
Token Entropy
25

---

Published as a conference paper at ICLR 2026
Table 14: (continued)
{Okay: 0.8789). {.: 0.1445). { so: 1.6484). ). I: 1.4453), { need: 1.9688), J. to: 0.0149), { find:
0.8711), { the: 0.3203]. minimum: 0.3379). ). f value: 0.1523), f of: 0.0513), { a: 0.9844). + 1.9609). 1. b: 0.0005). +: 0.0004), { c: 0.0007). where: 1.4141), { a: 0.9961), 3, {.r 0.0122].
b: 0.0011), ). {: 0.0127). { c: 0.5039). { are: 0.0103), { positive: 0.1445), { integers: 0.0110).
(:: 0.7500), and: 0. 0.1797), the: 0.2676), { quadratie: 0.3613), (equation: 0.2051)...
Case 3 Question
Symmetric about the y-axis, passing through the focus of the parabola y² 4x. and divided into two ares with a length ratio of 2 by the line y = x, the equation of the circle is
Pattern Chain
Knowledge Retrieval and Verification - Symmetry Analysis Equation Construction and Solu- tion- + Geometric Condition Transformation + Vector Analysis - Multiple Solution Verification
ofain-hf-Though
Okay, let me try to work through this problem step by step. So, the problem is about finding the equation of a circle that's symmetric about the y-axis, passes through the focus of the parabola y² = 4r, and is divided into two ares by the line = x with a length ratio of 1:2. Hmm, that seems bit involved, but let's break it down.
First, let's recall some basics. The parabola y² = 4r. The standard form of a parabola that opens to the right is y² 4ax, where (a, 0) is the focus. Comparing that to the given equation, 4a = 4, so a = 1. Therefore, the focus of this parabola is at (1, 0). So the circle we're looking for passes through (1, 0). Got that.
Next, the circle is symmetric about the y-axis. That means if (I,, y) is on the circle, then (-x, v) also on the circle. So the equation of the circle must be of the form (x h)2 + (y k)²
r2, but since it's symmetric about the y-axis, h must be O. Because if the center is (h, k), then symmetry about the y-axis implies that / = -h, so
O. Therefore, the equation simplifies to x² + (v kj)² = 2². So the center is at (0, k) and radius r. So we need to find k and r.
We know the circle passes through the focus (1, Let's plug that point into the equation to get an equation relating k and Plugging in
1, y 0: 1 + (0- - k)² = yd + 1 + k22 = r2. That's ' one equation. Now, the other condition is that the line y divides the cirele into two arcs with length ratio of 1:2. Hmm, okay. So the circumference of the circle is 2 and the line y = x intersects the circle two points, dividing it into two arcs. The ratio of their lengths is 1:2, so the shorter are is (1/3) x 2Tr and the longer arc is (2/3)= x 2TV. So the central angles corresponding to these ares should be 120 degrees and 240 degrees, since the ratio 1:2 corresponds to angles of 120 and 240 (which sum to 360). So the angle between the two intersection points as viewed from the center of the circle should be 120 degrees. Alternatively, maybe the line central angle corresponding to each are, since are length is proportional to the central angle. So if the ratio is 1:2, the angles are 0 and 20, where 0 + 20 360, so 0 = 120. Wait, no. Wait, 1:2 ratio in terms of length. So total circumference is 3 parts, so each part is 120, 240. So the central angles are 120 and 240, so the central angle between the two intersection points is 120, right?...
cuts the circle such that the ares are in 1:2 ratio. To find the
Token Entropy
(Okay: 1.1641), J. {:: 0.1143), { [let: 1.1484), { me: 0. { try: 2,5000). { to: 0.3301 }. workk: 1.7969), { through: 1.0313). this: 0.5078). problem: 0.9023). { step: 0.4648). { by: 0.6523).
{ step: 0.0007}. {.: 1.3359]. { So: 1.8438). {.: 1.0781]. ). the: 1.3672). { problem: 2.5938].
is: 2.5938), { about: 1.5391), { finding: 0.7813), { the: 0.4668). { equation: 0.0344), { of:
0,0060). a: 0.0527], circle: 0.0757). { that: 0.8398), ('s: 1.2188). (symmetric: 0,2373), {
about: 0. 1...69....
26

---

Published as a conference paper at ICLR 2026
Aames shil Aa
Object Lifecycle Analysis
Metaphor Interpreer www.rrreatte aation
Kinetic iic Energy Theorem Application
Sereery
Logical Equival ence T form
Force Balance Analysis
Prenise IdentIrication
Dimensional Analysis Reasoning Modeling
Argument Structur geran Pars serrsing ing
Di Differential Equatiion.Modellnee
Tex tual I Eviden ce extrraction
Periodic
Nar parrative Coherence Check
Relative Motion Analysis
Claim Justifin pettiiicatton
Nernst Equation Application Parametris Model ingeeliiiaatiiiitti Effect Analysis
Semantic Ambiguity Resolution Contextual Inference
(a) Physics: high importance scores.
(b) Physies: low importance scores.
Chemical Equation Balancing
Ditiiuu scoveyy
Pragmatic Implication
Lectron manciple
Chatalier Principle Application Electonhaaai an Base Titration Calculation
Terv rio Leessoning ng
Ixpectied Dertiin
Topic Cont ty Tracking
Speech Act ilassification
Relation Typing
Buffer Solution Ana
Analysin
vositti peely sctera Potrollyttee beterniwsiee
Presuppos tion
wereii.oooooeess. Chemical cal Equilibrium Analysis Nol Wolecular Orbital Theory Applicat Lombee berp
zure Ioichiooetricg
Redox React ion Analysis
Vodeling Enthalpy
Change Calculation
Deixis Interpretation
Mon,
Andosmat
uin Solie
Anaphora Resolution sing
Infor nation Structur
Melt: Scope besduriion
Discourse Structure Analysis
implicature Derivation
(c) Chemistry: high importance scores.
(d) Chemistry: low importance scores.
Pheno type Ra Ratio Derivation
Phrasal Verb Analysia ieflection) Paratign Application
Lexical Semantic Typing
&i Componua ford parsing
Lon
Genetic Mechanism_ ..t.smi.aaiingggaa
Selectional Restriction Check
Genotype Combinatio
Anall Analysis
Syntactic Dependency Parsing
Metabolic Pathway Tracking Meiosis Process Deduct tion inheritance Pattern Confirman t
Idiom Interpretation case assigeenn
Morphological
Decomposition
Gamete Fowmat a0 Analysis
Consti tuency Structure Analysis
Doression
Genotype Parsing
Coll oca tiatiion Watoonnaattaeec Pattern Recognition
wir ommiaaio tHe t
(e) Biology: high importance scores,
(f) Biology: low importance scores.
Greedy Algorithm Application meadu
Kalaga Pupule attepersatonn Dio the oo Wi eiw Ana lysis
State Transi tion Tracking Cyclic Pa ttern Reco
Plot Structure Decomposition Miagent
Secolutiin Potters tdoriiiiaatto
Foreshadowing Dctection
Narrative ive Arc Identi conntifian fication dratatic trany hacogiition Flastnacs Interpreiation
Loop Detection Termination
Setting Description Inter pre paeeataaatt tition
erternes bettu woudile eexplexxty Ana inalysis
Tosiepion
Markoy Chain
State Modeling
Tone Asses sment Theme trantion....
Recurs
Decompos tion Analysis i
thone
Symbolism Recognition
feait the bevoarie
Irqpety
Dyn maam Progr angiig Model eelll Ling
Character Motivation Analysis
Srep contitio Paam Tock Pask Twdx bollistes testuttis
(g) Computer science: high importance scores.
(h) Computer science: low importance scores.
Figure 11: Examples of reasoning patterns across four STEM domains. For each domain, the left column (red) shows patterns with high importance scores, while the right column (blue) displays patterns with low importance scores.
27

---

Published as a conference paper at ICLR 2026
Table 13: The relationship between the reflection and the chain of reasoning patterns.
Key Features
Description
Chains of Reasoning Patterns
Self-supervision and Self-correction
The model can actively review its own output, discover and correct logical, factual or expressive errors. This self- monitoring ability the core of the reflection process.
Structural Analysis and Counting, Cross-validation and Error Correc- tion, Reaction Mechanism Deduction, Conditional Driven Path Selection, Structural Analysis and Counting, Reaction Mechanism Deduction, Cross-validation and Error Correc- tion
Multi-step Iteration
Reflection is not one-off event but can carried out multiple rounds. The model can repeatedly check and correct its reasoning process until it outputs a more reasonable and accurate result.
Formula Substitution Method, Equa- tion Simplification Method, Iterative Trial and Error Method, Verification Method
Explicit Reasoning Chain
During the reflection process, mod- els often explicitly review and analyze their reasoning steps (such "chain thought")) looking for rationality and potential problems of each step.
Pattern Recognition and Analogi- cal Reasoning, Construction of Re- cursive Relations, Exact Computation and Verification, Approximate Estima- tion and Boundary Determination, Er- ror Checking and Correction
Dynamic ment
Adjust-
The model will dynamically adjust the reasoning direction, supplement infor- mation or change the reasoning strategy based on the reflection results adapt to different problems and scenarios.
Variable Substitution and Recombina- tion, Step-by-step Decomposition and Simplification, Substitution of Known Results, Numerical Verification Confirmation, Multi-method Cross- verification
Uncertainty Identi- fication
During the reflection process, the model can identify uncertainties or knowledge blind spots in its reasoning and tempt make up for these deficiencics.
Knowledge Retrieval Reasoning. Un- certainty Processing Reasoning, Step- wise Computation Reasoning, Condi- tional Verification Reasoning
Enhanced
Inter-
Through reflection, the model put more detailed reasoning process and correction reasons, enhancing the interpretability and transparency of the answers.
Understanding of Physical Processes, Derivation Threshold Conditions,
pretability
Verification of Mathematical Calcula- tions, Interpretation and Application of Results
Recursive Reason- ing
Recursive reasoning means that after the model generates the output for the first time, will return certain key steps, concepts or assumptions con- duct reasoning again to verify or adjust the previous inference. This reflective process usually occurs in plex tasks, especially scenarios that require multiple rounds of reasoning.
Problem Decomposition and Sub-goal Setting, Mathematical Structure Ab- straction, Logical Equivalence Trans- formation, Special Case Verification and Induction, Combinatorial Count- ing Technique, Mathematical Struc- ture Abstraction, Logical Equiva- lence Transformation. Combinatorial Counting Technique
28

---

Published as a conference paper at ICLR 2026
O let's see here. We have six points A, B, C, D, E, F on a straight line in that order
Point G is not on the line. We need to find the area of triangle BGE. First, maybe I
should sketch the points on a line to visualize their positions. Let me note down the
given distances: A 2 BD 22, CE = 31, DF = 33, AF 73. Also, CG 49 and DG = 39. Hmm. The coordinates of are unknown, but since it's s not on the line, we might need to assign coordinates to the points A through F first, then figure out G's coordinates using the distances CG and DG Then, once we have coordinates for B, G, and E, we can compute the area of triangle BGE. Right, coordinate geonetry approach makes sense here. Let's assign coordinates. Since all points are collinear, let's place them on the x-axis for simplicity. Let me let point be at coordinate 0. Then, since =F= 73, point F would be at coordinate 73. Wait, but if A is at and F is at 73, the other points B, C, D, E are in between. But need to figure out their positions based on given distances AC, BD, CE, DF. Let me denote the coordinates of A, B, C, D. E, as a, b, C, d, e, f respectively. Since they are in order A, B, C, E, F, on a straight line, their < e f. And we know = 0, f= 73. Given AC
coordinates should satisfy a a
26, 50 a 26 c 26, since a = 0. Similarly, BD = =Ddd a=d-ba :d -=d-b=22
2F, CE=e
31, DF
f- d 33. But wait, DF is given as 33. Since f= 73, then 73 d= 33 - d 73 3
= 49. So d 40. Then, BD d b= 22 -40 b0 22=
22= 18. Therefore, b F
18. Next, CE c 31. was found as 26, s0 26
Therefore. e 57.
Let's verify the positions: a = 0, b = 18, 26, d
the other distances if they make sense. =F=f a- 57.
57 B Let's check
73, correct. BD= 40 18 = 22, correct. CE 57- 26 31, correct. OF = 73 40 33, correct. So, the coordinates alongg the x-axis are: A: 0, B:18, C:26, D:40, E:57, F:7 Now, point 6 is not on this line. We know CG 40 and DG = 39. So, points C and D are on the x-axis at 26 and 40 respectively. G is somewhere off the x-axis such that the distance from C(26,0) to G is 40, and the distance from D(40, to G is 30. Therefore, we can consider G as
point with coordinates (x,y), y x 0, satisfying: v[(x- - 26) + y22] = 40 --> (x /
y2 = 1600 (1) and v[ 20 + y2] 30 --> (x - 40) + y22 = 900 ... (2) We
can subtract equation (2) from equation (1): [(x 26) + y2) - [(x 2 22 = I
600 - 900 (x 26) (x 40) = 700 Expanding both squares: (x22 -52x 676) (x2
-80x + 1600) = 700 Simplify: x²2 -2x + 676 xx2 +80x 16600 = 700 Combine like terms
( -52x +80x ) + (676 - 1600 ) = 700 28x 924 = 700 28x = 700 + 924 = 1624 x= 1624 /
28 = Divide numerator and denominator by 4: 406 /7 - 58.07 Wait, wait. Let me compute 28
x 58 = 1624. Yes, 28 x 58: 28-00-1400, 2-- total 1624. Therefore, x-58. Wait, but points D is at 40 on the x-axis, and E is at 57, F is at 73. So x-58 is between E and F? Hrm, but G could be there, off the line. But let's check. Wait, coordinates are x-58, so E iis at 57, so G is at 5- on the x- x-axis, but since y 2,, but the x- x-coordinate is 58 So the distance from C(26,0) to (58 sqrt( 26) y²2 = s3rrt + y² ) = sqrt
24 + y² ) which is given as 40. So, 1624 + y² =1600- y² = 576 - y= or y=-24. Similarly distance from 0) to (58, sqrt( (58-40) + y22 )= sqrt
y² )= sqrt + y
)=30 Therefore, 324- + y2-900 - y2-576, same as before. So y-24 or -24. Therefore, G is at (58,24) or (58,- 24)). Since area is absolute, it won't t matter if y is positive or So coordinates of G are (58,24) 24). Now, points B, G, E.
24)
negative. So take 24.
Coordinates: B is at (18,0), Gat (58, 24), E at (57.0) Wait, E is at 57 on the x-axis So E is (7,00) B is (18,0) G is (58,24) 24). To find the area of triangle BGE. Coordinates B: (18, 0) G: (58, 24) E: (57, 0) We can use the formula for the area of a triangle
given three coordinates. The formula is (x2 y3) + x2y yl) + x3 - y2))/21
Plugging in: xl = 18, 0 x2 = 58, y2 224 x3 =57, y3-0 Compute each term: 124
0) = 18-24 = 432 (8 0) 058 57 224==57*( (-24)= -1368 Sum: 432 +0 -1368 -93
Take absolute value and divide by 2: 936 /2-936/2-688 Therefore, area is 468.
Wait, is this correct? Let me check. Alternatvell,, since points B and are on the -xxis, and G is above, the base can be BE and the height is the vertical distance from G
to BE Let me check BE length. BE is from 18 to57 on axis, so length 57 18=39 The height is the y- y-coordinate of G, which is24. Therefore, area is (1/2) base (1/2) (1/2)* 39=24. Calculate: 39+24= (40- 1)=24=40=24 24-960-24-936 Then 936/2-468. Yes, same result So the area is 468. 5o maybe (468) is the answer. But wait, the problem states Find the area of triangle BGE". Let me double check. Coordinates:
(18,0),
24),
(57,0). If use the shoelace formula: List the coordinates in order: (18,0), (58, 24), (), (18,0) Compute sum of products down diagonals: 24 +58"0 +57*0 432 +0 +0 = 432 Sum of products up diagonals: 0*58 +24*57 +0'18 = 0 1368 +0=1368 Subtract the two 1432 -1368 1366//933 -936 Area- 936/2-468. So same answer. So that's 's confirmed. Therefore, the area is 468. **iinal *nswer** ** (46X) </think- We have six points \ A, B, C, D. E, \) and V lying on straight Line in that order, and point G \) not on the line. We need to find the area of triangle A( Atriangle BGE V. First, we assign coordinates to the points A, B, C, D. E, V) and A( ((F)) on the x-axis. Let N (AA)) be
Low (0.00)
High (4.72)
Example I of token entropy chain annotation for CoT data High-entropy tokens fre- quently indicate the underlying reasoning process of the model.
Figure 12:
29

---

Published as a conference paper at ICLR 2026
Okay, so I have this problem here: There exist real numbers x and y. both greater than 1, such that log base x of y'x equals log base y of (4)) equals 10. I need to find the product xy. Hmm, let's start by breaking down the given equations. the logarithm properties. The expression log base a of b can be rewritten using the power log a(b'c) = log a(b). So applying that here might simplify things.
First, let's recall
Starting with the first equation: log x(yx) 10. Using the power rule, this becomes x *
log x 10. Similarly, for the second equation: log ((x 10. Applying the power rule here gives 4y log y(x) = 10. Alright, so now we have two equations: 1) x log X
(y) = 10 4y * log y(x) 10 I know that log a(b) is the reciprocal of log b(a). That
's another logarithm property: log abb) =b)=1// log b(a). So, if I denote log x(y) as A,
then log y(x) 1/A. Let's assign variables to make this easier. Let me set A == log x(y
), so the first equation becones x A= 10. Then the second equation becones 4y * (1/A) =
10, because log y(x) 1/A. So, equations now: 1) * A= 10 2) (4y)/A= = 10 Now, maybe solve for one variable in terms of another. Let's solve the first equation for x:
10
/ A. Similarly, from the second equation: (4y)/A 10 = 4y = 10A = y (10A)/4 = y
52) So, now we have expressions for and y in terms of A: x= 10 / A y (5A)/2
Our ultimate goal is To find xy. Let's compute xy with these expressions: xy= (10 / A) (5A 2) (10 * 5A) (2A) (50A) (2A) = 25. Wait, the A terms cancel out here. So, xy = 25? But that seens too straightforward. Wait, but if the problem states that both log expressions equal 10 and by solving these equations we get xy 25. Maybe this is the case. Let me check again. But hold on, maybe there's ss something wrong here. Let's verify if x and y as functions of A satisfy the original equations. Let's s see, x 10/A, y SA
2. Let's compute log x and see if it's 10. First, log x(y'), Using the power rule:
x log x(y). But log x(y) is A, so x * A. X * A= (10/A) * A= 10. That checks out. Now
the other equation: log (4x)
4y log y(x). Log y(x) is 1/A, so (1/A). Substit
uting y 5A/2: 4 (5A/2) = (5/2) = 10. That also checks out. So even though A is involved in the expressions, when you plug everything back in, the A's cancel out, and both conditions give 10. Therefore, regardless of A (as long as A is positive, since x and are greater than 1), the product xy is 25. Wait, but A is log x(y). Since x and are greater than 1, what constraints does that put on A? Well, if x > 1 and y> 1, then log x(y) can be greater than or less than 1 depending on whether y is greater than x or not. For example, if y = x*, then log x(y) k. So, if k 1, then y > x, if k =1, y=x if k <1, then y < x. But here, since x and y are both greater than 1, but we can have different relations between x and y. But in our solution, xy=25, regardless of A. But
wait, does this mean that for any A (positive, as log is defined), x and y would adjust to satisfy the equations, and their product is always 25? But we need to ensure that x > and y=1. So Let s check what conditions on A will make x and y greater than 1. From x = 10/A >1, so 10/A >1 = A <10. Similarly, from y = 5N2 >1, so 5A/2 >1 = A> 2/5. So, A must be between 2/5 and 10 for x and y to be greater than 1. So even though A can vary in that interval, the product xy is fixed at 25. So the answer is 25. Wait, but let me think again. Since both the logs result in 10, which is a fixed value, even though variables are involved, the product is determined uniquely. Altematively, perhaps there's s a different way to approach the problem. Let me try changing the base of the logarithms to
see if we can relate x and y more directly. Let's first note the two original equations
1) log x(y")) = 10. As before, this is x * log x(y) = 10. 2) log (4x)) = 10. This
is 4y log y(x) = 10. Let's switch the base of logarithm to natural log to see if that helps, Recall that log a(b) = In(b)/In(a),
So, log x(y) = In(y)/Ln(x)) and log y(x) = In
(x)/In(y). So substituting into equation 1: x (In y / /In x) = 10 Equation 2: 4y *
In x / In y) = 10 Let's denote u= = In x and v= In Then equation becomes: x (v
u) = 10 Equation 2 becomes: 4y * (u / v) =10 But we can express x and y in terms of exponentials:
x= (u,,, e=
(v So substituting back into equation 1: eN * (v /
u) = 10 --2 eru v =10 Equation 2: 4 (v) * (u / v) =10 --> e'v * u =10 /4 ) v
2.5 v So now we have two equations: i) e'u =10 u 2) e'v * =2.5 v This seems a
bit complicated. But maybe we can relate them. Let's denote equations as: From 1) = (i u) / e'u Plug this into equation 2: e'v * =.5 (10 u e )
=2 u / e'u Cancel u (assuming u #0, which it isn't since x >1, so In = u >)) Therefore, evV =25 Leu = (u v)=25
e'v
u = Ln25 So, u+ = In25. But since u= In and v= In y,
then In x + In y= In25 = In(xy)=In25 = 2y=25. So, that gives us xy=25, regardless of the values of u and v. So that's a different way to look at it, and it shows that regardless of the specific x and y, their product must be 25. Therefore, the answer is 2 5. Wait, this is the same answer as before, but now through this method, which is more direct because when we add the exponents, it relates to the product of and y. So that method seems to get directly to the answer. Alternatively, when I used substitution with variable A, leading to 2y=25, but also checked constraints on A to ensure that and y are greater than 1, but regardless of those constraints, as long as they are met, the
Low (0.00)
High (4.47)
Figure 13:
Example 2 of token entropy chain labeling. High-entropy tokens frequently indicate the underlying reasoning process of the model.
30


Reviews:
[{"review_id": "m7c47aHbVE", "reviewer": "Reviewer_vJQb", "summary": "This paper presents a mid-training approach using CoT reasoning patterns. Through a process of selecting long-CoT data that is similar to the core set using similarity measures based on pattern & token entropy. The approach has dramatic empirical gains, advancing SOTA significantly in mathematical reasoning.", "questions": "1. As I understand from line 227, DeepSeek-R1 is the model used for reasoning trace generation. It seems using a variety of datasets is important. Do you think diversity of generative model would be helpful or harmful?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper presents an innovative and detailed approach for a well studied problem.\n* **Empirical Results**: Significant gains in the performance of models when using the proposed method of mid training. \n* **Innovative approach**: The proposed algorithm is clever, elegant and interesting to read. It is nice to see such a well motivated method work well empirically.\n* **Ablations**: The authors provide a very nice set of ablations that help to complete the understanding of the design space of the given approach, especially Table 3.", "weaknesses": "Mostly I believe this to be a strong paper, which has interesting and high quality results and methodologies.\n\nThere are a lot of nice ablations in the paper, but the results that I think may be missing are: connecting the core set similarity to downstream performance? We see only Figure 6 for LongCotPool and CoTP but not the Table 4 ablations? This will help to prove the claim?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JUJtnTrSTi", "reviewer": "Reviewer_zwnc", "summary": "This paper introduces CoTP, a data selection framework that works by expanding the dataset with the COT patterns to increase what the authors define as the model's reasoning potential. The model reasoning potential is defined as the inverse of the number of independent attempts required to correctly answer a question. Using CoTP data the authors are able to improve an MoE model by 9.58% on AIME dataset.", "questions": "The paper mentions a manual screening process for selecting high-value reasoning traces (line 165). Could the authors clarify what this process entails? Specifically, how were examples chosen, and what criteria or safeguards were used to minimize potential bias or subjectivity in the selection?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- **Novelty**: The paper presents an original and clearly articulated approach to expanding reasoning potential in foundation models. Both the methodological design and the underlying problem formulation demonstrate genuine novelty.\n\n- **Elegant Solution**: The combination of core-set construction with a dual-granularity selection strategy represents an elegant and well-motivated solution that advances beyond conventional Chain-of-Thought augmentation methods.\n\n- **Theoretical Rigor**: The framework is supported by a strong theoretical foundation and is further reinforced through detailed empirical analysis, illustrating both conceptual robustness and practical effectiveness.\n \n- **Comprehensive Empirical Validation**: The work provides extensive experimental results and ablation studies that empirically validate the proposed theoretical framework, showing consistent improvements across mathematical reasoning benchmarks.\n\n- **Reproducibility**: Code was added as supplementary materials, which enhances the transparency and reproducibility of the work.", "weaknesses": "- **Limited Evaluation Scope**: The experiments focus exclusively on mathematical reasoning tasks, leaving uncertainty about whether the proposed framework generalizes to broader domains such as logical, commonsense, or multimodal reasoning.\n- **Dependence on DeepSeek-R1**: The framework relies heavily on DeepSeek-R1 for generating Chain-of-Thought (CoT) traces, which is computationally expensive. It would be valuable to examine whether similar improvements can be achieved using smaller or alternative base models.\n- **Limited Cross-Model Evaluation**: The study evaluates the method on only a single MoE (mixture-of-experts) model configuration, providing little evidence of robustness across different architectures or model families. Broader testing across diverse model types would strengthen the general applicability claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fz2PwNqGHq", "reviewer": "Reviewer_1xzE", "summary": "The paper discusses which kind of chain of thought data expand the reasoning potential of a foundation model when used during mid-training and how they should be selected?\nThe authors annotate CoT chains with an atomic reasoning pattern and token wise entropy and weight patterns using TF-IDF to score them with a weighted DTW and use Hungarian algo to match each core instance with the closest candidate data point. With this, they use a pre-trained 85A6B base model and do a mid training with 10B tokens on selected COTP data. They achieved better performance on mathematical datasets. They further perform ablations to justify the importance of entropy based token selection.", "questions": "1. I want the authors to shed more light on the sensitivity of lambda? What happens when we change it?\n2. Also, I am not sure if Hungarian algo is truly needed here? What if we use kNN and why it will not work?\n3. Some more ablations in terms of using a random subset of longCoT vs COTP is needed to understand the true usefulness of COTP. \n4. Any comment on the generalization of the approach beyond Maths, beyond this specific MoE models (maybe to a dense model), beyond mathematical datasets, using a different model to annotate instead of Deepseek, non chinese prompts and so on. \n5. I am also wondering if the contamination test was only done for MathArena or for all the mathematical datasets?\n6. Can the approach be extended to other RL recipes beyond GSPO? \n7. I am also wondering if mid-training is the best place for such a recipe? I have seen some papers like SIKeD [1]  where they choose datasets during the fine-tuning stage (so in this case, post-training probably) and that improves the performance of the model. I am wondering if you want to comment on that ? \n\n[1] SIKeD: https://aclanthology.org/2025.findings-acl.513/", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper uses the reasoning potential of a language model to motivate a data selection process. This is very interesting and justifies the need for such a method to be used during mid-training. \n- The paper is clearly presented and each step in the process is well specified with neat plots and algorithms wherever needed. I would say the paper is easy to read and well structured. \n- The results looks promising with gains on mathematical datasets which justifies the need for selecting high quality CoT chains.", "weaknesses": "- I think the major drawback in the paper is the experimentation limited to a single model on a single domain. Moreover, the “atomic patterns” are produced by another external model like Deepseek and the prompt in Chinese (as provided in the Appendix) seems quite restricted to work on a specific kind of model with a specific kind of dataset. I am not sure how generalizable this approach can be based on the prompt used, model used and the dataset. Also, it might work because of the similarity of the teacher model used (Deepseek) alongside the student model 85A6B and might not work with different combinations. Also, the RL used is limited to one RL algo of GSPO and model limited to MoEs. \n- Also, some more ablations are needed to justify all the steps in the process used. For example, a sweep of lambda can be used to show the usefulness of entropy, Hungarian also seems a bit extra and what if we replace it with simple kNN, comparing COTP with a random subset of longCoT given its performance, and so on. This seems that the complexity of each step and all the choices made needs to be studied in more detail. \n- Finally, the use of geometric trails is missing from the constraint or the selection process for COTP it seems. Maybe I am missing something but the distance algo is not bounded by the geometric trials in any way and I am wondering if there is any use for geometric trials in itself? Is it more theoretical ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FMB7x2XV3q", "reviewer": "Reviewer_5gwE", "summary": "The paper introduces CoTP, a framework for expanding the reasoning potential of foundation models by selecting high value chain of thought (CoT) data. The authors define “reasoning potential” as the inverse of expected attempts to solve a problem, extract atomic reasoning patterns to build a core set, and use a dual granularity similarity measure (pattern chains + token entropy) with a Hungarian assignment algorithm to select training data. Experiments on an 85A6B MoE model show significant gains on challenging math benchmarks and improved RL upper bounds.", "questions": "1. Could the authors clarify how they mitigate potential bias and scalability issues in the core set construction process, and provide an analysis of the computational cost of their similarity matching pipeline at scale?\n\n2. The paper relies on reasoning patterns expressed in Chinese to compute distances between pattern chains. However, it is unclear whether there is empirical evidence demonstrating that this language‑specific representation is optimal or robust. Could the authors provide experiments or ablations to justify this choice?\n\n3. Moreover, when abstracting a large number of reasoning patterns from massive datasets, how do the authors ensure that the extracted patterns remain consistent, controlled, and aligned with the intended definition of “high‑value” reasoning steps? What mechanisms are in place to prevent noise or drift in the pattern inventory?\n\n4. The proposed CoTP framework shows consistent and significant improvements on mathematical reasoning benchmarks, yet it does not yield comparable gains on general‑domain tasks, sometimes even underperforming other baselines. Could the authors clarify why the method appears highly effective in math reasoning but does not extend to broader general benchmarks? Are there inherent limitations in the design of CoTP that make it domain‑specific?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Provides a formal definition of reasoning potential, linking it to expected attempts and offering theoretical grounding.\n\n2. Proposes a novel data selection method combining reasoning pattern chains and token entropy, operationalized with weighted DTW and Hungarian matching.\n\n3. Demonstrates strong empirical improvements on difficult mathematical reasoning tasks while maintaining general performance.", "weaknesses": "1. The construction of the core set heavily depends on strong model annotations and manual filtering, which may introduce bias, limit scalability, and lead to redundancy. \n\n2. The similarity computation (weighted DTW combined with the Hungarian algorithm) could be computationally expensive when applied to very large data pools, raising concerns about efficiency.\n\n3. The reliance on Chinese‑expressed reasoning patterns for distance computation lacks empirical justification, and the paper does not clearly explain how consistency and quality of the large set of abstracted patterns are maintained at scale.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)