R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?
Authors: Yi Lu
Jianing Wang
Linsen Guo
Wei He
Hongyin Tang
Tao Gui
Xuanjing Huang
Xuezhi Cao
Wei Wang
Xunliang Cai
Venue: ICLR 2026 Poster (2026)
Keywords: Large Reasoning Models
Long Horizon Reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: A scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon reasoning capabilities of LRMs

Abstract: Recent trends in test-time scaling for reasoning models (e.g., OpenAI o1, DeepSeek-R1) have led to remarkable improvements through long Chain-of-Thought (CoT). However, existing benchmarks mainly focus on immediate, single-horizon tasks, failing to adequately evaluate models’ ability to understand and respond to complex, long-horizon scenarios. To address this incomplete evaluation of Large Reasoning Models (LRMs), we propose R-HORIZON, a method designed to stimulate long-horizon reasoning behaviors in LRMs through query composition. Based on R-HORIZON, we construct a long-horizon reasoning benchmark, comprising complex multi-step reasoning tasks with interdependent problems that span long reasoning horizons. Through comprehensive evaluation of LRMs using the R-HORIZON benchmark, we find that even the most advanced LRMs suffer significant performance degradation. Our analysis reveals that LRMs exhibit limited effective reasoning length and struggle to allocate thinking budget across multiple problems appropriately. Recognizing these limitations, we use R-HORIZON to construct long-horizon reasoning data for reinforcement learning with verified rewards (RLVR). Compared to training with single-horizon data, RLVR with R-HORIZON not only substantially improves performance on the multi-horizon reasoning tasks, but also promotes accuracy on standard reasoning tasks (+7.5 on AIME2024). These results position R-HORIZON as a scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon reasoning capabilities of LRMs.

Published as a conference paper at ICLR 2026
R-HORIZON: How FAR CAN YOUR LARGE REASON- ING MODEL REALLY Go IN BREADTH AND DEPTH?
Yi L
Jianing Wang2*
Linsen Guo
Wei Hel
Hongyin Tang²
Tao Gui
Xuanjing Huang 1.3
Xuezhi Cao²
Wei Wang**
Xunliang Cai²
Fudan University Meituan LongCat Team
Shanghai Collaborative Innovation Center of Intelligent Visual Computing
https://gittee com www. meatonmmmtuan--aa -on
ABSTRACT
Recent trends in test-time scaling for reasoning models (c.g., OpenAI ol, DeepSeek- R1) have led to remarkable improvements through long Chain-of-Thought (CoT). However, existing benchmarks mainly focus on immediate, single-horizon tasks, failing to adequately evaluate models' ability to understand and respond to complex, long-horizon scenarios. To address this incomplete evaluation of Large Reasoning Models (LRMs), we propose R- E-HORIZON, a method designed stimulate long- horizon reasoning behaviors in LRMs through query composition. Based on E-HORIZON, we construct long-horizon reasoning Benchmark, comprising complex multi-step reasoning tasks with interdependent problems that span long reasoning horizons. Through comprehensive evaluation of LRMs using the R- HORIZON Benchmark, we find that even the most advanced LRMs suffer significant performance degradation. Our analysis reveals that LRMs exhibit limited effective reasoning length and struggle allocate thinking budget across multiple problems appropriately. Recognizing these limitations, we use R H-HRIIZON to construct long- horizon reasoning data for reinforcement learning with verified rewards (RLVR). Compared to training with single-horizon data, RLVR with R-HORIZON not only substantially improves performance the multi-horizon reasoning tasks, but also promotes accuracy on standard reasoning tasks (+7.5 on AIME2024). These results position R- H-HORIZON scalable, controllable, and low-cost paradigm for enhancing and evaluating the long-horizon reasoning capabilities of LRMs.
MATH500
AIME24
AMC23
RI-Qwen-32B -1-Qwen-7B
0.75
0.50
0.25
0.30
0.80
0.75
0.50
0.40
0.20
0.25
0.00
1.30
DespSeck-R1
0.95
0.80
0.30
0.90
0.60
0.85
0.50
16
Composed Query Num
Composed Query Num
Composed Query Num
Accumecy
Appected/Accraary
Figure 1: Actual versus expected accuracy of R RI-series models on R-HORIZON datasets.
"Corresponding authors. lygwjn@gmail.com gmail.com, tgui @ fudan.cdu.com wangwei432@ www.aaaddddemmmmmm..mmmmom

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent advances in reasoning-focusee language models, exemplified by OpenAl's ol (OpenAI et al., 2024) and DeepSeek-R1 (Guo et al., 2025), have established test-time scaling as a fundamental component for enhancing reasoning abilities in large reasoning models (LRMs). Specifically, test-time scaling enables long Chain-of- Thoinhtf-Thhuuht (CoT) (Wei et al., 2023) and induces sophisticated reasoning behaviors, leading to remarkable improvements reasoning (He et al., 2025b; Yu et al., 2025; Yue et al., 2025; Zeng et al., 2025), code generation (Luo et al., 2025a; Zeng al., 2025) and agentie tasks (Team et al., 2025b;a).
challenging reasoning tasks like mathematical
By continuously expending computational resources throughout the reasoning process, models with longer reasoning trajectories achieve superior performance various reasoning benchmarks (Muen- nighoff et al., 2025), yet this simultaneously exposes critical limitations in current training and evaluation paradigms. Existing training and evaluation datasets (Cobbe et al., 2021; Hendrycks et al., 2021; Jain et al., 2024) primarily confine themselves to the reasoning of isolated problems, focusing on immediate single-horizon tasks where questions and answers remain independent of each other. However, real-world scenarios often require an Al agent to reason, plan, and act over an extended series of steps, sometimes thousands or even millions, where inference must span across multiple sequential and potentially interdependent problems (Yao et al., 2024; Tao et al., 2025). Single-horizon evaluation paradigms cannot effectively assess the ability of a model to understand and respond to complex, multi-horizon tasks or scenarios that require a sequence of logical steps over a longer period of time. Moreover, conventional reinforcement learning (RL) typically focuses on single, isolated problems, preventing models from developing long- mon-hooizoon reasoning capabilities to tackle multiple problems through the RL process. The incomplete picture of training and evaluation paradigms raises fundamental question: How far can large reasoning models really go in breadth and depth?
In this study, we propose R- HORIZON, a simple yet effective method to stimulate long-horizon reason- ing behaviors in LRMs through query composition. This method aims to construct dependencies and concatenate existing single-horizon tasks, transforming isolated problems into complex multi-horizon reasoning scenarios. For instance, in mathematical tasks, we first extract key information from all problems, then establish dependencies by linking one problem's is answer to another problem's 's critical information, requiring models solve multiple problems sequentially to obtain all correct answers. To address the limitations of current training and evaluation paradigms, we leverage this method to establish an evaluation benchmark and training data to evaluate and enhance the long- horizon reasoning capabilities of LRMss.
We first establish R- R-HORIZON benchmark, which comprises 6 representative datasets across math- ematics, code generation, and agent applications (e.g. MATH500 (Hendrycks et al., 2021), ), Live- CodeBench (Jain et al., 2024), WebShaper (Tao et al., 2025)). Through evaluating 26 LRMs, we find that even the most advanced LRMs suffer significant performance degradation on R-HORIZON benchmark. The performance in multi-horizon reasoning scenarios falls substantially below the expected performance (Section 3.2) in Figure 1. Further analysis reveals critical limitations of current LRMs that contribute to the performance gap: (1) LRMs possess a limited effective reasoning length, with performance declining sharply once the thinking budget exceeds this threshold. (2) LRMs exhibit constrained reflection scope-L.MM often reflect within the current problem, failing to identify errors from previous questions. (3) The overthinking phenomenon (Chen al., 2025) in LRMs prevents the appropriate allocation of thinking budget when facing multiple reasoning problems.
To address the absence of long-horizon problems in current RL training data, we leverage R- HORIZON to reconstruct training datasets and design different reward functions, then conduct reinforcement learning with verified rewards (RLVR) with varying composed problems and reward schemes to investigate the impact of long-horizon reasoning data on the training process. By employing mainstream RLVR algorithms GRPO (Shao et al., 2024) with R- R-HORIZON, we observe that traditional RLVR provides limited improvements on multi-step reasoning tasks. In contrast, training with R-HORIZON data a highly efficient training approach that not only enhances single- problem performance more effectively but also rapidly improves performance multiple problems. Our analysis demonstrates that training with R- HORIZON also improves response length efficiency and thinking budget allocation. In summary, R- R-HORIZON mitigates the current limitations of long- horizon reasoning in training and evaluation paradigms- offering a scalable, controllable and low-cost path to improve and evaluate the long-horizon abilities of LRMs.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
2.1 TEST TIME SCALING IN LARGE REASONING MODELS
The success of OpenAT's ol introduced new scaling paradigm, test-time compute scaling, which improves performance through increasing inference computation (OpenAI et al., 2024). However, recent studies reveal that LRMs may generate verbose reasoning trajectories with marginal accuracy gains. Chen et al. (2025) reveals the "overthinking" phenomenon, showing that LRMs generate significantly more tokens than conventional LLMs on simple arithmetic tasks, with minimal increase in accuracy. To address this, Aggarwal & Welleck (2025) proposed cength-controlled policy opti- mization, providing precise control over the length of the reasoning trajectories during generation. Yang et al. (2025b) developed a thinking-optimal scaling strategy, allowing models to flexibly adjust their reasoning depth according to the available test-time compute budget. Recent studies have also focused on fine-tuning models to think efficiently according to task complexity (Hao et al., 2024; Liu al., 2025; Fang et al., 2025; Arora & Zanette, 2025; Zhang et al., 2025). In agentic tasks, overthinking also reduces performance while increasing inference costs (Cuadron et al., 2025). Although previous studies indicate that overthinking leads to computational inefficiency with limited performance gains, our findings reveal that prolonged reasoning substantially degrades performance on compound multi-step reasoning tasks.
2.2 EFFECTIVE REASONING LENGTHI OF LARGE REASONING MODELS
Recent studies explore the effective reasoning length of LRMs in mathematical benchmarks (i.e., GSM8k (Cobbe et al., 2021), MATH500 (Hendrycks et al., 2021) and AIME (AIME, 2024; 2025)). Su et al. (2025); Yang et al. (2025b); Wu et al. (2025b) investigate the relationship between reasoning length and accuracy. Su et al. (2025) finds that models fail to adaptively calibrate their response length according to the problem difficulty. Wu et al. (2025b); Ghosal et al. (2025); Chen et al. (2024) demonstrate the existence of an optimal CoT length beyond which performance degrades. By directly concatenating multiple independent questions, NEST (Pan et al., 2025) reveals that LRMs fail to keep their performance under multi-context stress. However, these tasks either focus on a single problem or concatenate independent problems without meaningful logical dependencies. In contrast, we design multi-dependent synthetic tasks to expose failure modes amplified by extended reasoning, consistent with findings that reasoning chains exceeding optimal length reduce accuracy. GSM-Infinite (Zhou al., 2025) builds dependencieSs within a computational graph across problems, but it mainly focus on long-context input. However, R-HORIZON focuses on scenarios with short inputs but long outputs with long CoT, which more closely match real reasoning scenarios.
R-HORIZON
We propose R- R-HORIZON, a method designed to stimulate long- norg-hoizzon reasoning behaviors in LRMs via query composition. As illustrated in Figure 2, R-HORIZON supports the concatenation of three types of expanded questions and can be employed in both the training and evaluation stages to enhance and evaluate the long-horizon capabilities of LRMs.
3.1 R-HORIZON DATASETS CONSTRUCTION
For mathematical tasks, adopt the sequentially composed concatenation to construct dataset of multi-step mathematical problems with explicit dependencies that enforce sequential solving. The construction pipeline consists of two stages: seed problem filtering and expanded problem composition. For code and agentic tasks, we provide the construction process in Appendix A.
Seed Problem Filtering Given an initial dataset D = {(g; a:) at)] where (ys; (z) is a pair of a question and an answer. We apply the following filtering criteria to obtain a seed set Dreed:
Daeed = ((g.a) E D [T()) 0 A a E Z}:
where 1(-)= = =extact_intt denotes extracting all integers appearing in the input text.
For each (q, a) E Dsced, we identify key variables from the extracted integers. We then employ model M to verify each interger m E Iqq) whether is a key variable:
(()) = {m [q)) Mg m) = 1}.
where Mg. m) = 1 indicates that removing m from q renders the problem unsolvable, Each filtered seed problem is then represented as triple (g, a, K(g)).
3

---

Published as a conference paper at ICLR 2026
Seed Question t
(b) Question Composition
(c) Expanded Question q())
Maryy
Asswer
Deperter Qvery
Real Anowe
O
Create Dipendoncy
Pribien 2o NOw!
Anawer 1:3
Asswer 1:3
Aonn 2:7
42
Bg
81
4
Pribien 2:3
2.1
Probiee 1: 1+2-
Podken ammde?
Directly Compose
Sequeniall Compose
Graphic Compose
Long Horizon Evaluation Benchmark
Long Horizon Reinforcement Learning
Composiio
Madiso
LiveCherellnch
Repear
Stop
Short Horizon Training
B
Easily Solve Long Horizen Tasks
Training Data
3p
((T)
WeisShaper
R-Horizon
AIME24
Trijectery T
Bg
Long Horizon Taakk
X
The R-HORIZON data composition pipeline is illustrated in (a)-(c). We leverage R- HORIZON to construct comprehensive long-horizon reasoning evaluation benchmark spanning tasks and generate multi-horizon training data for long-horizon reinforcement learning.
Figure 2:
Expanded Problem Composition Given seed problems with annotated key variables, construct dependency chains using Algorithm 1. The algorithm ensures that each modified problem d'+1 contains placeholder variable U,+1 that must be resolved through the dependency function fa;) 14+1, requiring the solution a, from the previous problem. The augmentation step prepends the dependency specification the problem statement, making the sequential constraint explicit. The final dataset D composed consists of problem sequences that enforce strict sequential solving.
Algorithm 1: Dependency Chain Construction
Input: Seed problems {(11, 41, Ki),..., (In, On Kn)} Output:
Composed problem Q Initialize Q + [q:]:
for i to
do
Select key variable m,+1 E K5+1 and create placeholder variable 8:413
Define dependency function fz) <- + (m++1 a;);
Substitute Mi+1 with 01+1 in gi+1 to obtain 4411
Augment 2/41 with dependency constraint U,-1 = f((a;);
Append 4:41 to Q:
end
return Q = (q1, Le: de..... ::))
3.2 R-HORIZON BENCHMARK
We use R-HORIZON to reconstruct existing evaluation datasets, combining different datasets through problem filtering and composition approaches, and design evaluation metrics for composed problems.
Evaluation Metrics R- H-HORIZON evaluates model performance by extracting all answers from
the model's response. Given composed problem sequence Q = (q), f0..z/..... .., dn)) E Deomposed. we extract the corresponding answer sequence A (āj, .2...., â,) from the model's response R. We use all-or-nothing scoring:
correct only if all sub-problems are solved:
[ if à; = a; for all (1.....n].
Acc( Q) = 0 otherwise.
We also propose a metric estimate its expected accuracy. For each (g. a) Dseed, the pass rate of these atomic problems to estimate the expected accuracy of composed problems through:
AcCespected ②
Pi
44)
4

---

Published as a conference paper at ICLR 2026
where p; is the pass rate of atomic problem 9:- We use model-based extraction to handle diverse
response formats (details in Appendix E.2).
3.3 REINFORCEMENT LEARNING WITH R-HORIZON
To investigate the formation of long-horizon reasoning capabilities and understand how multi-step dependent queries influence the reinforcement learning process, we employ R-HORIZON datasets as training data for reinforcement learning from verifiable rewards (RLVR). We follow Skywork- OR1 (He et al., 2025a) RLVR pipelines while utilizing our constructed training data.
Group Relative Policy Optimization (GRPO) We adopt GRPO (Shao et al., 2024) as our opti- mization algorithm, which eliminates the value function requirement of PPO (Schulman al., 2017) by computing advantages in group-relative manner. For each question g, the behavior policy Ths samples a group of G response candidates {ol,.... ..7) We use GRPO with token-level policy gradient loss, which optimizes the policy model by maximizing the following objective:
lodl
(+) = (o 1e D
1
min (r (reo 1- - -,,1 ec - BDD [To]] (nd
(5)
where Ti,e Roge (0.,..4) and are hyperparameters. Åi, is the advantage calculated based on the relative rewards of the outputs inside each group only, and DxL denotes the KL divergence between the learned policy and a reference policy Tee
no molo, 910),ct)
Reward Design
We design two reward schemes for multi-horizon training data:
{
1 if an = an and otherwise,
Rau f1 if â; = a, for all i E [I....,m}
Riss
(6)
otherwise.
Last-only reward R provides feedback on the final answer only, while all-correct reward Rail requires all intermediate steps to be correct. This distinction allows us to study how different reward function influence the development of long-horizon reasoning capabilities.
EXPERIMENT
4.1 EVALUATION SETUP
Datasets For mathematical tasks, we construct MATH500 (Hendrycks et al., 2021), AIME24 (AIME, 2024), AIME25 (AIME, 2025), and AMC23 (AMC, 2023) with multiple de- pendent queries, using n (1, 2, 8, 16, 20) for MATH500, n E (1, 4, 6,8) for AMC23 and E (1, 3, 4,5) for the more challenging AIME datasets. For code tasks, we reconstruct Live- CodeBench (v5) (Jain et al., 2024) with n E 11,2,3,4,55, For agentic tasks, we use Web- Shaper (Tao et al., 2025) with multi-round tool calls for web search (n {1, 3, 4, 5J). Details about dataset statistics, evaluation metric, models, and prompt examples are in Appendix E.I.
Models We select 25 advanced LRMs to perform evaluation on our built R-HORIZON benchmark. including the R1 R1-distill series models (Guo et al., 2025), Qwen series (Yang et al., 2025a) models, and Nemotron (Bercovich al., 2025) series models. Model details are in Appendix E. We set the generation length to 64k tokens to avoid truncation. More inference settings are in Appendix E.3.
4.2 EVALUATION RESULT
Performance Degradation as the Reasoning Horizon Increases As shown in Figure 3, we observe that models across different categories experience performance degradation as the reasoning horizon increases. Even the most powerful models, including DeepSeek-R1, Owen3-235B-A222 -28---22nn and 04-mini, suffer from severe performance degradation as the reasoning horizon increases. For instance, on AIME25, DeepSeek-RI drops from 87.3% (n 1) to 24.6% (n 5). Additionally, we find that larger models exhibit less degradation when confronting composed problems, while smaller models experience more severe performance degradation. For example, Ri-Qwen-7B drops from 93.6% (n = 1) to 0% (n = 16), which is 34.1 1. % more than the RI- R1-Qwen-32B model.
'The time ranged from August 2024 to May 2025.
5

---

Published as a conference paper at ICLR 2026
Math500
AIME24
AIME25
Gwen3-238-TTinkkng
99.6
99.3
91.2
97.
91.44
93.7
$1.3
5799
bi.5
63.2
92.
612
68.2
51.4
29.2
o4-Mimi
98.4
56.8
90.8
85.0
73.4
82.1
63.6
43.4
44.1
305
15.7
50.2
37.3
68.8
16.4
@eegSeek-81
99.0
58.6
96.8
91.2
82.8
89.6
16.1
46.4
66.1
523
81
56.2
41.1
67.0
26.6
Gemini-2.5PP
97.6
94.0
84.0
70.4
54.6
82.6
59.1
43.8
53.1
40.7
72.7
50.2
37.8
42.2
23.8
Qwen3-2355 (ann3-2350--0 instrutt-
98.8
96.4
98.4
90.0
70.2
82.3
$6.6
32.0
34.6
20,8
72.3
3622
19.4
17.6
6.0
Gemini-2.5-FT
98.4
92.5
88.2
87.8
45.6
74.3
46.1
20.1
15.4
12.0
53.7
22.2
9.0
9.9
2,4
Claude: Sonnet-4 4
85.5
29.3
82.2
69.4
50.8
46.3
34.1
32.8
31.0
8.9
70.5
21.8
21.7
22.3
2.7
0w0-328
97.8
64.2
B8.4
76.3
14.2
75.5
14.9
16.1
6.2
Qwen3-328
96.4
64.2
81.8
60.2
81.1
51.7
27.6
37.0
17.0
19.8
5.7
Skyeork-ORL-328
97.6
93.3
68.6
80.3
56.4
79.8
53.1
27.0
27.5
23.0
V1.5
40.6
14.5
21.1
6.0
R1-Uama-700
74.6
50.4
25.6
14.4
3.8
0.7
RI Rt-Qwen322
97.5
$3.9
65.4
38.0
5T.3
39.6
4.7
9.6
2.0
DAPO 328
54.2
23.0
2.6
0.1
0wen3-58
12.6
15.1
4.3
Namstron-Nano588
work-o01-18
Ri-mia
Eficient-R--1
76.2
45.2
28.2
6.0
0.0
0.0
R1-Qwen-78
0.0
75.8
39.0
0.0
45.7
5.0
0.0
0.3
0.0
Palaric-48-
97.0
83:0
23.6
7.1
7.9
0.4
Palaris-1.78
95-1
91.2
X3.0
46.4
6.6
49.5
2.8
4.6
2-2
8.9
0.0
N emmotron- Owen-1.38
914
24.4
22.6
0.0
50.7
2.9
0.0
B.0
0.0
DeepScale-1.00
874
77.6
14.0
0.4
0.0
37.5
3.5
0.2
0.0
0.0
31
0.0
0.0
0.0
R1 11-Qwen-1.58
874
56.6
10.4
0,0
0.0
10.4
144
0.0
0.0
0.0
2
4
16
3
5
AMC23
LiveCodeBench
WebSShaer
Owen3 2wem3-2359thannknn
100.0
97.5
91:1
V9.3
sn.
77.8
15.0
21.3
10.8
9.5
45.3
#1:33
16.77
$4.0
27.8
o4-Mini
100.0
91.6
90.6
86.4
80.3
76.3
39.8
20.1
13.3
10.6
83
82.6
64.4
70.3
81.4
8
872
80.9
80.9
89.7
75.
74.5
35.5
16.9
7.9
5.0
53.3
42.7
41.9
333
31.3
Gomini-2.5-Po
97.5
85.9
77.5
79.4
70.6
98.8
33.3
15.4
6.5
3.5
76.2
77.8
70.7
65.3
64.0
Qwen3-2358- -
963
30.6
87.2
78.4
(5.0
57.0
23.3
10.8
5.4
3.2
752
072
64.7
58.0
56.7
Gemini-2.5.FT
95.1
59.4
55.6
51.99
29.7
45.5
18.5
14
56.5
47.3
Sboude-Sonnet44 4
81.5
60.9
42.2
30.6
17.2
50.9
11.8
144
48.0
43.3
35.3
35.3
40.7
Ow0--20
97.8
81.9
73.8
32.0
59.5
30.0
Qwen3 328
99.4
24.1
67--7
4:3.8
29.0
19.1
Skywork-OR1 320
46.2
13.2
13.9
R1-Uama-708
:2.2
48.1
34.1
19.3
20.7
R2-Qwen-328
$0.0
$0.8
DMPO-Deen-328
12.0
12.7
Nemetron-Nard
Mkywork--R1-11
20.1
6.7
5.3
Laser-DE7B
Rimaa
47.7
15.2
0.9
38.0
8.5
5.8
Blfcient-R1-78-
47.2
3.4
2.2
36.6
5.3
R1-Qwen-78
48.8
6.2
0.3
0.0
39.1
D.D
3.3
2.0
2.7
2.0
0penihine
79.4
37.2
5.9
0.9
27.2
4.7
4,0
3.3
4.0
2.9
Palaris-48-
98.4
66.6
54.1
37.5
6.9
5/8
12.9
1.4
0.4
5E.0
27.3
20.7
19.3
Polaris-1.78
55.9
10.6
3.8
19.4
33.8
1.8
54.0
22.7
22.4
10.0
8.7
N Ammotron Dowe-1..5 86.2
35.3
0.9
0.0
8.0
27.2
0.0
0.0
0.0
0.0
0.0
1.3
0.0
0.0
0.7
DeepScale-1.5e 81.2
27.5
0.0
0.0
0.0
1866
0.4
0.0
0.0
0.0
1.4
2.0
3.3
2.0
2.0
A1-Qwen-1:58 75.9
20.5
0.0
0.0
0.0
16.3
0.0
0.0
0.0
0.0
1.6
2.1
0.0
3.7
0.0
2
2
5
Composed Query Num
Composed Query Num
Composed Query Num
Figure 3: Evaluation results of R- R-HORIZON Benchmark.
Same Degradation Trends Across Different Model and Task Categories We observe consistent degradation trends across tasks of varying difficulty and types. Models exhibit greater performance drops when facing more challenging tasks. For instance, Wwem3-235B-Thinkin drops from 93.7% (n = 1) to 69.2% (n = 5) on AIME24, but experiences a steeper decline from 92.3% (n = 1) to 29.2% (n 5) on AIME25, For code tasks, we find that the degradation trend more severe compared to mathematical tasks, with smaller models (7B) struggling to complete multiple code problems. For web search tasks, we observe that many trained reasoning models have lost their ability to call tools, resulting in poor performance.
4.3 REINFOR LEARNING WITH R-HORIZON DATASETS
Despite reinforcement learning bringing long CoT thinking capabilities models, current mainstream LRMs still cannot achieve good performance on R E-HORIZON Benchmark. We follow Skywork- OR1 (He al., 2025a) observe the changes in long-horizon reasoning capabilities of long CoT models before and after standard RI. in Appendix B. We find that training with only single-problem data leads to slow improvement in models' ability to handle composed problems. To investigate the impact of R-HORIZON data RL training, we construct composed training data through R-HORIZON based on the original math training datasets.
Training Setup We construct data pool D Dittered from Seywork-OR--BR training data using Problem Filtering (Section 3.1). To manage difficulty, we combine problems by pass rates, keeping AcCespected 0.25. We train RI R1-Qwen-7B and set maximum response length 40k prevent

---

Published as a conference paper at ICLR 2026
truncation and use the last-only reward Rust as default, which provides feedback on the final answer only. Details are in Appendix
(a:1)
A1ME2A (a)-2)
Reward
Traninng tom1)
0.35
Transng Dea cend)
0/00
Trannng Data tom1) Traniing
0,00
0.90
600
Acancy
0.75
6.55
0239
0.20
6:00
6465
0115
100
200
300
200
240
450
306
Training Steps
Training Stops
Training Skepe
Figure 4: Training curves comparing single and composed data on AIME24 avges and reward.
Training with R-HORIZON Datasets We train R1-Qwen-7B using both original data and 2-query composed data. As shown in Figure 4, compared to the original data, composed data significantly improves performance on composed problems (+17.4 on AIME24 (n=2)). Additionally, we find that training with composed problem data also substantially improves performance the original tasks (+7.5 on AIME24). During the training process, the reward for composed data gradually increases and surpasses the reward for the original data.
Impact Number of Composed Queries and Different Reward Schemes To further investigate the impact of the number of composed problems, we construct four types of training data based on the number of composed problems: composed problem counts of (1, 4, and a mixture of problems with counts 1, 2, 3, 4). We also study the effects of different rewards on composed data in Table 1.
Table 1: Results of different number of composed queries and reward function
MATH500
AIME24
ATME25
AMC23
Avg.
Model
Origin
n=8
Origin
n=2
Origin
n=2
Origin
n=2
Origin
Multi
RI RI-Qwen-7B
48.3
16.4
33.3
3.5
90.2
48.8
66.4
20.1
Naive Training Data (n=l)
95.6
8.4
57.9
16.7
47.9
5.1
95.9
55.0
74.3
21.3
w/ composed queries (n=2)
95.4
21.4
65.4
34.1
49.6
10.0
94.1
80.6
76.1
36.5
w/ composed queries (n=4)
94.6
50.6
62.9
34.8
45.4
8.1
91.9
79.1
73.7
43.2
w/ composed queries (mixed)
96.8
47.8
57.1
32.8
44.2
10.0
93.1
81.6
72.8
43.1
w/ Rai (n=2)
95.0
26.8
64.6
38.8
48.8
11.9
95.0
83.4
75.9
40.2
All models trained with composed data demonstrate significant performance improvements on composed problems. Moreover, composed data also substantially enhances performance on the original datasets. For instance, composed problems with n=2 yield the largest improvements ALME24 and AIME25. As the number of composed problems increases, models exhibit stronger capabilities in handling problems requiring more reasoning steps. Additionally, observe that using Ran as the reward function on training data with 2 composed problems outperforms Rias when confronting scenarios with multiple problems. More training dynamics provided in Appendix C.
5 ANALYSIS
Our analysis covers evaluation results of the R- R-HORIZON benchmark (Section 5.1) and RL training results using R-HORIZON datasets (Section 5.2). Ablation studies on evaluation metrics, dependency relationships, and problem difficulty ordering in Appendix D.
7

---

Published as a conference paper at ICLR 2026
5.1 EVALUATION RESULT ANALYSIS
Error Type Analysis We analyze the error types of the evaluation result in Figure 5. We find that as the number of problems increases, Problem Reasoning Errors increase rapidly. Adding simple dependencies between problems increases the overall reasoning difficulty, and the number of Dependency Reasoning Errors gradually increases with the number of problems, though the overall count remains relatively small. We observe that when facing multiple problems, models frequently terminate their responses prematurely, answering only subset of the problems.
Maat5to-Rl-Qweas 7B
Xa12-00-R 111-500-R1--wmn---33 22B
ADMEX-----QWeE.. (22tines)
AIMEL-RR-- 52B (3ttines)
80%
805
60N
6AA
40%
-0%
40%
40%
405
300
30%
15%
Componal Nez
Compondd Query Num
Compoaad
Compase Query Nan
Figure 5: Error type distribution across different query numbers. Four error categories: Problem Reasoning Error represents reasoning errors made by the model for specific problems; Dependency Reasoning Error indicates the model correctly solved previous problems but made errors when calculating the dependencies; Early Stop indicates the model prematurely terminated generation after solving previous problems; Output Truncation indicates generation exceeded token limit.
As shown in Figure 6, as the number of problems increases, the gap between the actual accuracy and expected accuracy of models becomes increasingly larger, indicating that models struggle to maintain their original performance as reasoning length increases. We observe that the error position of models gradually declines and stabilizes within certain range as the number of problems increases. Comparing RI RI-Qwen-71 and R RI-Qwen-32B, we observe that larger models can reason over longer contexts, and each model has its reasoning boundary, For example, the 7B model's s eтror range is (4-6k tokens) while the 32B model's 's error range is (8-10k tokens) for Math500.
Effectivv Reasoning Length of LRMs
20/00
MattScouttmer Postre)
1.0
AIMIAA
30,000
ADME34 (Enve Posike))
20,000
13:00
5,00
Query Nazi
Quecy Nam
Quay Sun
Query Nan
Expered
Sqpedeed
lapk
Figure 6: Analysis of accuracy and error position with R1-Qwen-7B and RI-Qwen-32B.
Reflection Frequency and Depth of LRMs Reflection refers to the model's self-reflective behav- iors such as "wait," " "but..." ". As shown in Figure 7, the reflection frequency models gradually increases with the number of problems and converges problems increases, the proportion problems involving long-range reflection also rises, yet we find that more than half of the problems lack any long-range reflection process, which indicates that LRMs' reflections are highly localized.
maximum value. As the number of
Thinking Budget Allocation of LRMs As shown in Figure 8, current models tend to allocate more tokens to early reasoning stages. Even DeepSeck-RI cannot effectively distribute the thinking budget reasonably to subsequent problems, indicating that current mainstream LRMs have not yet developed the capability to allocate thinking budgets according to reasoning horizon.
8

---

Published as a conference paper at ICLR 2026
RI-Qwes-7B
R1-Qwm-32B
DeepsoekeR1 R1
116
191
Re Andtoo Soo
60
20
Quary Num
Conpppoed Qucry Nur
Compore Nam
Long Retiectio
Retfectin Iroqpancc
Figure 7: Reflection analysis on MATH500 dataset. Reflection Frequency refers to the average number of reflections per question. Long Reflection Rate refers to the proportion of questions whose reflection range exceeds the current question. Shaded areas indicate standard deviation.
Rl-Qwan-1B
BI-Qwam-328
DeqpockkR
18
15
Compoal Quary Nar
Compesad Quecy Nar
Componl Nar
Prskiem
Pribicn
IrNNen
PriNian
Proskem
Figure 8: The thinking budget allocation for different query configurations (1-5 queries) across RI-Qwen-7B, RR-QQwen322B and Deepseck-RI models on AIME24 datasets.
5.2 ANALYSIS OF REINFORCEMENT LEARNING WITH R R-HORIZON
Analysis on Models Trained with Composed Data We analyze models trained with R-HORIzON data versus those trained with original data using RL, shown in Figure 9, In Figure 9 (a), we find that training with composed queries significantly improves model performance on composed tasks and can generalize longer reasoning horizons. Additionally, phenomenon. Models generate shorter responses when facing multiple problems compared to models trained on original data in Figure (b), and training with composed problems enables models to learn more reasonable token budget allocation in Figure 9 (d). These results demonstrate that training with composed data promotes efficient reasoning. which is consistent with the findings of training dynamics in Appendix C. We also provide case study in Appendix H to compare the reasoning behavior between standard training and training with R- R-HORIZON datasets.
find it alleviates the overthinking
In Figure 9 (c), as the number of problems increases, training with composed problems enables models to engage in longer reflections with increasing frequency, while the reflection frequency of models also increases more reasonably. This demonstrates that using R R-HORIZON facilitates longer-range reflection in models, thereby improving performance on long-horizon reasoning tasks.
(a) lc).ccurccy (Math500)
(b) Eiror Position (Mahh500)
(c) Reflection (Mart500)
(4) Joken Dudget (AIME24)
1.50
25
Luce Lenglectttt (4))
0.00
6.00
-0000
Compased Quary Nom
Cherpoord Query Nem
Compoeed Nan
Compond Query Nor
Pas
Tree
orlLan Rall
Ref. 2440000
Prckian
Sip.deec.... Ave
Inpeceed
ged Preperer
Figure 9: Analysis of reinforcement learning effects with single and composed datasets. (a) Math500 performance comparison, (b) error position analysis, (c) reflection analysis, and (d) token budget allocation across multi-horizon scenarios.
Analysis on Rollout Efficiency We compute the proportion of Solve None, Solve All, and Effective samples in each rollout batch in Figure 10. Models trained with compositional data using 2 and

---

Published as a conference paper at ICLR 2026
4 obtain, average, 20% more effective samples compared with 1. This indicates that the composed datasets indeed yield more balanced reward signals. By combining multiple problems, the model receives larger amount of effective training data, thereby improving rollout efficiency.
Eifecctive
(6))
Solve None (7)
Solve All (N)
100
20C
303
350
130
206
403
100
350
400
203
Triking Saps
Thiining Strp.
Triining
Figure 10: Rollout efficiency analysis across training steps for n=1, n=2, and n=4 composed queries, showing Effective, Solve None, and Solve All ratios.
CONLUSION
In this paper, we present R- R-HORIZON, a novel and efficient approach stimulating long-horizon reasoning in LRMs through query composition. By composing simple problems into sequential, interdependent tasks, R-HORIZON constructs multi-step reasoning datasets that serve dual purposes: evaluating LRMs' long-horizon reasoning capabilities and enhancing their complex reasoning abilities during training. Our method establishes a foundation for future advances in complex reasoning data synthesis and the development of models with robust long-hone fong-horizon reasoning capabilities.
ACKNOWLEDGMENTS
The authors wish to thank the anonymous reviewers for their helpful comments. This work was partially funded by National Natural Science Foundation of China (No. 62576106, 62476061, 62376061), Shanghai Rising-Star Program (23QA1400200), and Natural Science Foundation of Shanghai (23ZRI 4035000.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Pranjal Aggarwal and Sean Welleck. LI: Controlling how long reasoning model thinks with reinforcement learning, 2025. URL https://arxiv.tttt www.xiv.org/abaa wwt.:os/gaab....... 04655 04697.
AIME
American American Invitational Mathematics URL
invitational mathematics
examination
aime. 2024,
In Febru-
Examination
AIME
ary 2024.
https: /aas:://maaac.. saagg//athhooot conee-itmmttiiionss aime.
american-inviett
american-
invitational
AIME
American
invitational
mathematics
examination
In AIME 2025, Febru-
aime.
American Invitational Mathematics
Examination
ary 2025.
URL
https://maa org/ wwwt-maoteetiieeeeeeee competitios///////
american-
cea.ttiiiiceeaaiiiiooooo -ime-
AMC. American mathematics comperition ame. In American Mathematics Competition AMC, 2023. URL httpp::/maaa..... org. www.oeenettttooooooo//eeeee
/r amc/.
Chenxin An, Zhihui Xie, Xiaonan Li, Lei Li. Jun Zhang, Shansan Gong, Ming Zhong, Jingjing Xu, Xipeng Qiu, Mingxuan Wang, and Lingpeng Kong- Polaris: A post-training recipe for scaling reinforcement learning on advanced reasoning models.
thub.
/b/blogg/025/Polarr/ 2025. Accessed: 2025-07-01.
Daman Arora and Andrea Zanette. Training language models to reason efficiently, 2025. URL https://arxiv: org/oogg///ooooo
04463.
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran EI- El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen. Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng. Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Olivier Delalleau, Zijia Chen, Zhilin Wang. David Mosallanezhad, Adi Renduchintala, Haifeng Qian, Dima Rekesh, Fei Jia. Somshubra Majumdar, Vahid Noroozi, Wasi Uddin Ahmad, Sean Narenthiran, Aleksander Ficek, Mchrzad Samadi, Jocelyn Huang, Siddhartha Jain, Igor Gitman, Ivan Moshkov, Wei Du Shubham Toshniwal, George Armstrong, Branislav Kisacanin, Matvei Novikov, Daria Gitman, Evelina Bakhturina, Jane Polak Scowcroft, John Kamalu, Dan Su, Kezhi Kong. Markus Kliegl, Rabeeh Karimi, Ying Lin, Sanjeev Satheesh, Jupinder Parmar, Pritam Gundecha, Brandon Norick, Joseph Jennings, Shrimai Prabhumoye, Syeda Nahida Akter, Mostofa Patwary, Abhinav Khattar, Deepak Narayanan, Roger Waleffe, Jimmy Zhang, Bor- Yiig Su, Guyue Huang. Terry Kong. Parth Chadha, Sahil Jain, Christine Harvey, Elad Segal, Jining Huang, Sergey Kashirsky, Robert McQueen, Izzy Putterman, George Lam, Arun Venkatesan, Sherry Wu, Vinh Nguyen, Manoj Kilaru, Andrew Wang, Anna Warno, Abhilash Somasamudramath, Sandip Bhaskar, Maka Dong, Nave Assaf, Shahar Mor, Omer Ullman Argov, Scot Junkin, Oleksandr Romanenko, Pedro Larroy, Monika Katariya, Marco Rovinelli, Viji Balas, Nicholas Edelman, Anahita Bhiwandiwalla, Muthu Subramaniam, Smita Ithape, Karthik Ramamoorthy, Yuting Wu, Suguna Varshini Velury, Omri Almog, Joyjit Daw, Denys Fridman, Erick Galinkin, Michael Evans, Katherine Luna, Leon Derczynski, Nikki Pope. Eileen Eileen Long. Seth Schneider, Guillermo Siman, Tomasz Grzegorzek, Pablo Ribalta, Monika Katariya, Joey Conway, Trisha Saar, Ann Guan, Krzysztof Pawelec, Shyamala Prayaga, Oleksii Kuchaiev, Boris Ginsburg, Oluwatobi Olabiyi, Kari Briski, Jonathan Cohen, Bryan Catanzaro, Jonah Alben, Yonatan Geifman, Eric Chung, and Chris Alexiuk. Llama-nemotron: Efficient. reasoning models, 2025. URL https: //arri/aaiii./// org abs aaaaabbs//500 2500. 00949.
Qiguang Chen, Libo Qin, Jiaqi Wang, Jinxuan Zhou, and Wanxiang Che. Unlocking the capabilities of thought: reasoning boundary framework to quantify and optimize chain-of-thought, 2024.
URL https: /aap:v//////vv.///a org/ 2410. 05695.
Xingyu Chen, Jiahao Xu. Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang. Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of ol-like llss, 2025. URL https: // //arxiivooxx wwwp//////////aeeeeeeoeeeeeeeeea aww. 21187.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Refichiro Nakano, Christopher Hesse, and John

---

Published as a conference paper at ICLR 2026
Schulman. Training verifiers to solve math word problems, 2021. URLhttps://arxiv.oom 0tss/2110.14166
Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang. Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, lon Stoica, Ana Klimovic, Graham Neubig, and Joseph Gonzalez. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks, 2025. URL https: /ttpiaaxivvv..v///////.////a.ava org /2502 08235.
Gongfan Fang, Xinyin Ma, and Xinchao Wang. Thinkless: Lim learns when to think, 2025. URL https: / http//aaxvvvv..//// orgoor////ooooo. 0ttbs//505..3311
Soumya Suvra Ghosal, Souradip Chakraborty, Avinash Reddy, Yifu Lu, Mengdi Wang, Dinesh Manocha, Furong Huang, Mohammad Ghavamzadeh, and Amrit Singh Bedi. Does thinking more always help? understanding test-time scaling in reasoning models, 2025. URL https:
//a //arxiv.com //atxiv..rg//bllbbb abs/ 2506. 04210.
Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smymis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, and Ludwig Schmidt. Openthoughts: Data recipes for reasoning models, 2025. URL https: / arxiv org/ abs /.tp//55660000 04178.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space, 2024. URL https:
//ar
//arxiv. org. abs/ /atpb///21..666666 06769.
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xi- aoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li. Liang Zeng. Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. Skywork open https: ooperr/eereccoo---eoooreeee--eee notion. ittiiinnsiee///ee
reasoner series. -rr.raaeeerereeee--ooaaaaaaaa330
2025a.
Notion Blog.
Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning, 2025b. URL https: / arxiv. orgg abss 2504. 1456.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset, 2021. URL abss////////// 2103. 03874.
https: http://www..gga//.....////.oooot
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code, 2024, URL https: /aariviiiii.///.i.//aaaaaaaaavva wwweiiiiorrr/.ggoo 2403. 07974.
Wei Liu, Ruochen Zhou, Yiyun Deng, Yuzhen Huang. Junteng Liu, Yuntian Deng, Yizhe Zhang, and Junxian He. Learn to reason efficiently with adaptive length length-based reward shaping, 2025. URL https::/arret org/ abs /205 15612.
12

---

Published as a conference paper at ICLR 2026
Michael Luo, Sijun Tan, Roy Huang, Xiaoxiang Shi, Rachel Xin, Colin Cai, Ameen Patel, Alpay Ariyak, Qingyang Wu, Ce Zhang. Li Erran Li, Raluca Ada Popa, and lon Stoica. Deepcoder: A fully open-source 14b coder at 03-mini level. Fully-Open-Source- Notion Blog.
-ado-t--rrddd---5.ttoooooooooo www.:ooooiooooooooooooooeoo 2025a.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol -preeeww with a 1.5b model by scaling rl. wtt.:/aliin./teo1o1111000666666a http:w-wiihh-----BB-ooodb----b
2025b.
Notion Blog Accessed: 2025-02-12.
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URL httpt:p::/aaxii..roooo www.oor////////////////o..e..ee/ /2501. 19393.
Nvidia, Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H. Anh, Pallab Bhattacharya, Annika Brun- dyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, Sirshak Das, Ayush Dattagupta, Olivier Delalleau, Leon Derczynski, Yi Dong, Daniel Egert, Ellie Evans, Aleksander Ficek, Denys Fridman, Shaona Ghosh, Boris Ginsburg, Igor Gitman, Tomasz Grzegorzek, Robert Hero, Jining Huang, Vibhu Jawa, Joseph Jennings, Aastha Jhunjhunwala, John Kamalu, Sadaf Khan, Oleksii Kuchaiev, Patrick LeGresley, Hui Li, Jiwei Liu, Zihan Liu, Eileen Long, Ameya Sunil Mahabalesh- warkar, Somshubra Majumdar, James Maki, Miguel Martinez, Maer Rodrigues de Melo, Ivan Moshkov, Deepak Narayanan, Sean Narenthiran, Jesus Navarro, Phong Nguyen, Osvald Nitski, Vahid Noroozi, Guruprasad Nutheti, Christopher Parisien, Jupinder Parmar, Mostofa Patwary, Krzysztof Pawelec, Wei Ping. Shrimai Prabhumoye, Rajarshi Roy, Trisha Saar, Vasanth Rao Naik Sabavat, Sanjeev Satheesh, Jane Polak Scowcroft, Jason Sewall, Pavel Shamis, Gerald Shen, Mohammad Shoeybi, Dave Sizer, Misha Smelyanskiy, Felipe Soares, Makesh Narsimhan Sreedhar, Dan Su, Sandeep Subramanian, Shengyang Sun, Shubham Toshniwal, Hao Wang, Zhilin Wang, Jiaxuan You, Jiaqi Zeng, Jimmy Zhang, Jing Zhang, Vivienne Zhang, Yian Zhang, and Chen Zhu. Nemotron-4 340b technical report, 2024. URL https::///// / /ttxivavvvvvvv.////vv//ea/aaaaaa org/ abs/ / 2406 1 174.
OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameey, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph. Behrooz Ghor- bani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong. Elizabeth Prochl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter Evan Mays, Fan Wang. Felipe Petroski Such, Filippo Raso, Florencia Lconi, Foivos Tsimpourlas. Francis Song. Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclere, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O' Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, Ilya Kostrikov, Ilya Sutskever, Irina Kofman, Jakub Pachocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore, Jiacheng Feng, Jiahui Yu, Jiayi Weng, Jie Tang. Jieqi Yu, Joaquin Quiñoneeo Candela, Joe Palermo. Joel Parish, Johannes Heidecke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang, Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang. Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng, Linden Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tintor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yatbaz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan, Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek, Michele Wang, Michelle Fradin, Mike McClay, Mikhail Pavloy, Miles Wang,
13

---

Published as a conference paper at ICLR 2026
Mingxuan Wang. Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowd- hury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz. Gaon, Reah Miyara, Reimar Leike, Renny Hwang, Rhythm Garg, Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez. Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang, Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang, Taylor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi, Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong. Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojeiech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang. Zheng Shao, and Zhuohan Li. Openai ol system card, 2024. URL https: / arxiv. org. abs 2412 167.26720.
Zhuoshi Pan, Qizhi Pei, Yu Li, Qiyao Sun, Zinan Tang. H. Vicky Zhao, Conghui He, and Lijun Wu. Rest: Stress testing large reasoning models by asking multiple problems once, 2025. URL attts:://arxinn /rgg alss/ / 10541.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https: https://axxiv..rrg//////o/ http://ass//17.0333/
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang. Mingchuan Zhang. Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL hwtptt:ttt//iiiiee www.xiii.or.o.oooo orwoorrg/aaoooo abs/ 24C2. 03300.
Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and overthinking: An empirical study of reasoning length and correctness in Ilms, 2025. URL httbs//55550012.. https://arxiv: www./r/g//////oooo//oeoo/e.eeese
Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Webshaper: Agentically data synthesizing via information-seeking formalization, 2025. URL https:// wexiv.org/ae abs / 22000550770 15061.
5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng. Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, Yean Cheng, Yifan An, Yilin Niu, Yuanhao Wen, Yushi Bai, Zhengxiao Du, Zihan Wang, Zilin Zhu, Bohan Zhang, Bosi Wen, Bowen Wu, Bowen Xu, Can Huang, Casey Zhao, Changpeng Cai, Chao Yu, Chen Li, Chendi Ge. Chenghua Huang, Chenhui Zhang, Chenxi Xu, Chenzheng Zhu, Chuang Li, Congfeng Yin, Daoyan Lin, Dayong Yang. Dazhi Jiang, Ding Ai, Erle Zhu, Fei Wang. Gengzheng Pan, Guo Wang, Hailong Sun, Haitao Li, Haiyang Li. Haiyi Hu, Hanyu Zhang, Hao Peng. Hao Tai, Haoke Zhang, Haoran Wang, Haoyu Yang. He Liu, He Zhao, Hongwei Liu, Hongxi Yan, Huan Liu, Huilong Chen, Ji Li, Jiajing Zhao, Jiamin Ren, Jian Jiao, Jiani Zhao, Jianyang Yan, Jiaqi Wang, Jiayi Gui, Jiayue Zhao, Jie Liu, Jijie Li, Jing Li, Jing Lu, Jingsen Wang. Jingwei Yuan, Jingxuan Li, Jingzhao Du, Jinhua Du, Jinxin Liu, Junkai Zhi, Junli Gao, Ke Wang, Lekang Yang, Liang Xu, Lin Fan, Lindong Wu, Lintao Ding, Lu Wang, Man Zhang, Minghao Li, Minghuan Xu, Mingming Zhao, Mingshu Zhai, Pengfan Du, Qian Dong, Shangde Lei, Shangqing Tu, Shangtong Yang. Shaoyou Lu, Shijie Li, Shuang Li, Shuang-Li. Shuxun Yang. Sibo Yi. Tianshu Yu, Wei Tian. Weihan Wang, Wenbo Yu. Weng Lam Tam, Wenjie Liang. Wentao Liu, Xiao Wang, Xiaohan Jia, Xiaotao Gu, Xiaoying Ling, Xin Wang, Xing Fan, Xingru Pan, Xinyuan Zhang, Xinze Zhang, Xiuqing Fu, Xunkai Zhang, Yabo Xu, Yandong Wu, Yida Lu, Yidong Wang, Yilin Zhou, Yiming Pan, Ying Zhang, Yingli Wang, Yingru Li, Yinpei Su, Yipeng Geng, Yitong Zhu, Yongkun Yang. Yuhang Li, Yuhao Wu, Yujiang Li, Yunan Liu, Yunqing Wang, Yuntao Li, Yuxuan Zhang, Zezhen Liu, Zhen Yang, Zhengda Zhou, Zhongpei Qiao, Zhuoer Feng, Zhuorui Liu, Zichen Zhang, Zihan Wang, Zijum Yao, Zikang Wang, Ziqiang Liu, Ziwei Chai, Zixuan Li, Zuodong Zhao, Wenguang Chen, Jidong Zhai, Bin Xu, Minlie Huang, Hongning Wang. Juanzi Li, Yuxiao Dong, and Jie Tang. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025a. URL https: / rxiv. www.vorgg/aoo abs 2500 06471. 471.
14

---

Published as a conference paper at ICLR 2026
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Hongcheng Gao, Peizhong Gao, Tong Gao, Xinran Gu, Longyu Guan, Haiqing Guo, Jianhang Guo, Hao Hu, Xiaoru Hao, Tianhong He, Weiran He, Wenyang He, Chao Hong. Yangyang Hu, Zhenxing Hu, Weixiao Huang, Zhiqi Huang, Zihao Huang, Tao Jiang, Zhejun Jiang, Xinyi Jin, Yongsheng Kang, Guokun Lai, Cheng Li, Fang Li, Haoyang Li, Ming Li, Wentao Li, Yanhao Li, Yiwei Li. Zhaowei Li, Zheming Li, Hongzhan Lin, Xiaohan Lin, Zongyu Lin, Chengyin Liu, Chenyu Liu, Hongzhang Liu, Jingyuan Liu. Junqi Liu, Liang Liu, Shaowei Liu, T. Y. Liu, Tianwei Liu, Weizhou Liu, Yangyang Liu, Yibo Liu, Yiping Liu, Yue Liu, Zhengying Liu, Enzhe Lu, Lijun Lu, Shengling Ma, Xinyu Ma, Yingwei Ma, Shaoguang Mao, Jie Mei, Xin Men, Yibo Miao, Siyuan Pan, Yeho Peng. Ruoyu Qin, Bowen Qu, Zeyu Shang, Lidong Shi, Shengyuan Shi, Feifan Song, Jianlin Su, Zhengyuan Su, Xinjie Sun, Flood Sung, Heyi Tang, Jiawen Tao, Qifeng Teng, Chensi Wang, Dinglu Wang, Feng Wang. Haiming Wang, Jianzhou Wang, Jiaxing Wang, Jinhong Wang. Shengjie Wang, Shuyi Wang, Yao Wang. Yejie Wang, Yigin Wang, Yuxin Wang, Yuzhi Wang, Zhaoji Wang. Zhengtao Wang, Zhexu Wang, Chu Wei, Qiangian Wei, Wenhao Wu, Xingzhe Wu, Yuxin Wu, Chenjun Xiao, Xiaotong Xie. Weimin Xiong, Boyu Xu, Jing Xu, Jinjing Xu, L. H. Xu, Lin Xu, Suting Xu, Weixin Xu, Xinran Xu, Yangchuan Xu,, Ziyao Xu, Junjie Yan, Yuzi Yan, Xiaofei Yang. Ying Yang. Zhen Yang, Zhilin Yang. Zonghan Yang, Haotian Yao, Xingcheng Yao, Wenjie Ye, Zhuorui Ye, Bohong Yin, Longhui Yu, Enming Yuan. Hongbang Yuan, Mengjie Yuan, Haobing Zhan, Dehao Zhang, Hao Zhang. Wanlu Zhang, Xiaobin Zhang, Yangkun Zhang, Yizhi Zhang. Yongting Zhang, Yu Zhang, Yutao Zhang, Yutong Zhang, Zheng Zhang, Haotian Zhao, Yikai Zhao, Huabin Zheng, Shaojie Zheng, Jianren Zhou, Xinyu Zhou, Zaida Zhou, Zhen Zhu, Weiyu Zhuang, and Xinxing Zu. Kimi k2: Open agentic intelligence, 2025b. URL https: // /wwpvv..vvvv//.///////.../.a.av.
http//bs/25555 20534.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe Le, and Denny Zhou, Chain-of-thought prompting elicits reasoning in large language models, 2023, URL https::///// //wxxaxxivvooo/. hbsp://bb///200 11903.
Mingqi Wu, Zhihao Zhang, Qiaole Dong, Zhiheng Xi, Jun Zhao, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Huijie Lv, Ming Zhang, Yanwei Fu, Qin Liu, Songyang Zhang, and Qi Zhang. Reasoning or memorization? unreliable results of reinforcement learning due to data contamination, 2025a. URL /attps://azxiee
org/ 2507. 10532.
Yuyang Wu, Yifei Wang. Ziyu Ye, Tianqi Du, Stefanic Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in 1lms, 2025b. URL https: /ttp///axvvv//////////////..aa org. org.orrgg//o.... /bbs/ 2502. 07266.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujic Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025a. URL https: / arxivv org abs / 2500. 09388.
Wenkai Yang. Shuming Ma, Yankai Lin, and Furu Wei. Towards thinking scaling of test-time compute for 1lm reasoning, 2025b. URL https: http://awwww..oaa/////aaaaaa
18080.
Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 7-bench: tool-agent-user interaction real-world domains, 2024. URL https: /ttp:::///axvvv//// http:/awxxvvo..graraaaa /bsg//b///// 2406. 12045.
7-bnmhhhhhnbeehhhh A benchmark for
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong. Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source
15

---

Published as a conference paper at ICLR 2026
Ilm reinforcement learning system at scale, 2025. URL Attp::///axax 14476.
Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, Xiangyu Yu, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, Xin Liu, Mingxuan Wang, Yonghui Wu, and Lin Yan. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks, 2025. URL http::/ http://axxvvv.vvo. org/ 2504. 05118.
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- Z00: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. URL https::///// /aariiiiiivv////ia//aaaa///aitvv org/ abs/ 2200 18892.
Jiajie Zhang, Nianyi Lin, Lei Hou, Ling Feng, and Juanzi Li. Adaptthink: Reasoning models can learn when to think, 2025. URL https:tttt/... http://axvvv../........ wttp://gg/a....aaae/aeeee
Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, and Beidi Chen. Gsm-infinite: How do your iims behave over infinitely increasing context length and reasoning complexity?, 2025. URL https:: haaps://aaxiiie
0522.
16

---

Published as a conference paper at ICLR 2026
A R-HORIZON DATASETS CONSTRUCTION FOR CODE AND AGENTIC TASKS
Datasets Construction for Code Tasks For code tasks, we adopt a composition approach similar to mathematical tasks, using data points from existing datasets as seed questions for composition. We continue to employ the Expanded Problem Composition process described in Section 3.1. How- ever, unlike the sequential composition used for mathematical tasks, we apply a directly composed concatenation format for code tasks without adding explicit dependencies between problems. This design choice is motivated by the fact that code tasks require sandbox execution to obtaín answers, making it challenging to construct direct dependency relationships between problems and answers as in mathematical tasks.
For agentic tasks, we incorporate web search tasks for evaluation. We decompose questions based on the structured data from WebShaper (Tao et al., 2025), which consists of "Target" (target variable) and "Variable" (intermediate variables). Specifically, for each question, we follow this processing pipeline: We filter the original WebShaper dataset to obtain questions with varying complexity levels, ultimately selecting 50 questions. Each question's associated URLs are accessed using browsing tool, with browsing results stored for subsequent processing (URLs that cannot accessed are filtered out). We employ Saaude-Sonnet-aa to extract values for each variable V from the web pages (variables that cannot be extracted are excluded). The original questions and variables V are then assembled into a directed acyclic graph (DAG). Following topological sorting, we perform pruning to derive sub-questions and seed questions (questions with erroncous or duplicate decompositions are filtered out). This process yields a final dataset of 50 questions, with each question categorized into 5 levels based on the number of variables (ranging from to 5), resulting in a total of 250 seed problems.
Datasets Construction for Agentic Tasks
B How REINFORCEMENT LEARNING IMPROVES LONG-HORIZON REASONING
Despite reinforcement learning bringing long CoT thinking capabilities to models, we find that current mainstream LRMs still cannot achieve good performance on R-HORIZON evaluation. To further analyze the relationship between long-horizon reasoning capabilities and RL, we follow Skywork-ORI (He al., 2025a), an effective and scalable RL implementation for long CoT models, to observe the changes in long-horizon reasoning capabilities of long CoT models before and after RI.
Training Setup We follow the Skywork OR1 (He et al.. 2025a) multi-stage training approach, gradually increasing context length across different stages. Once the model's performance converged, we increased the context length in the subsequent stage. This approach led significant performance improvements on benchmarks while also enhancing training efficiency. We employ 3 stage training with max response lengths increasing from 8k (0-600 steps) to 16k (600- 1400 steps), and finally to 32k (1400-1 1680 steps). We train on the math subsets of the Skywork-RI. dataset. Additional training settings are provided in Appendix F.
ATME34
AIME25
Responsc Length
055
AIMIL (n=))
www MW M
AIMILI
1200
Kosponse Lejur
AIMEN te-2o
05
AIMES5 (1-2)
13009
0.4
18009
hym
Lergh
IC03
8000
7.09
8043
300
200
750
2008
1250
1300
1256
250
200
750
1006
1250
1506
1150
4:09
230
300
150
1800
1220
15%
1930
Tratnitn Steps
Triining Steps
Thaining Steps
Figure 11: The AIME24, AIME25 performance for single query and 2-query settings and response length evolution during multi-stage training progression across 8k, 16k, and 32k context lengths. Vertical dashed lines mark stage transitions.
17

---

Published as a conference paper at ICLR 2026
Observations During Training Process We find that RL training improve model performance on composed problems, but the improvement smaller than that on corresponding single problems (+36.6% on AIME24 and +9.1% on AIME24 on composed problems shows no clear correlation with the increase in response length. When training the 32k stage, although response length increases significantly. the model's performance both single and composed problems does not improve substantially.
2). Additionally,
observe that the improvement
C TRAINING DYNAMICS OF RL WITH R-HORIZON
We present the training dynamics of models trained with composed training data (n = 1, 2, 4) in Figure 12. The response length of models trained with composed data initially decreases and then increases as training progresses, ultimately reaching levels comparable those trained with original data, with similar training time per step. This indicates that models require fewer tokens to solve each problem, demonstrating that training with composed data promotes efficient reasoning. However, the entropy loss of models trained with composed data decreases more rapidly than those trained with original data, which may limit the model's s capacity for effective exploration.
Response Length
Time Per Step
Entropy Loss
1500
1409
1400
(100)
1200
1200
10000
1001
1000
200
3.6
409
100
200
3000
400
590
Tharsnn Stope
Transing
Thanning Stogo
Trakiing Deta (Pm
Thitiing (pNO)
Tranning Dara (w-1)
Figure 12: Training dynamics comparison across different training data compositions (n=1, n=2, n=4) showing response length, training time per step, and entropy loss evolution during the RL training process.
D ABLATION STUDY
D.I ABLATION DEPENDENCIES
We compare the difference between multiple dependent prob- lems and multiple independent problems. We remove the de- pendency construction step and directly concatenate multiple problems. We conduct experiments using R1 R1-Qwen-7B on Math500, with results shown Figure 13. We find that the accuracy of both problem composition methods falls below the theoretical accuracy, and the accuracy of multiple sequentially dependent problems is significantly lower than that of multiple independent problems. This indicates that current models still have substantial deficiencies when handling multiple correlated problems.
Sapoba
Zew
Qauy Necia
Figure 13: Comparison between multiple dependent and indepen- dent problems.
D.2 ABLATION ON EVALUATION METRIC
R- E-HORIZON adopts an all-or-nothing scoring criterion Accall to ensure models correctly answer all problems. An alternative evaluation metric AcCjast considers a response correct only the final problem is answered correctly. Theoretically, these two metrics should be identical for problems with sequential dependencies, as correctly answering the final problem requires sequentially solving all preceding problems. However, our ablation experiments reveal substantial differences between these metrics the number of
18

---

Published as a conference paper at ICLR 2026
problems increases, shown in Figure 14 (Left). The probability of correctly answering only the final problem far exceeds the probability of correctly answering all problems. We observe an anomalous phenomenon: models can correctly solve subsequent problems despite incorrect solutions to preceding ones, indicating that models can produce correct answers even when problems should be unsolvable. We provide statistics these anomalous cases in Figure 14 (right). We hypothesize that this phenomenon is related to data contamination in models (Wu et al., 2025a).
1.90
1B Ave,
7B Anuntlly Sumples Anomaaly Samples
1B Acc, Just
Aot,
AnomalySunmee Sample Count
200
Accurscy
150
Anomaly
0.20
50
0.00
2
8
12
16
20
2
16
20
Query Number
Number
Figure 14: R1- R1-Qwen models showing anomalous behavior in sequential reasoning. Left: Accau vs. Ast revealing increasing divergence. Right: Anomalous sample counts where models correctly answer final problems despite preceding errors.
D.3 IMPACT OF QUERY DIFFICULTY ORDERING
We conduct ablation study to examine whether the ordering of query difficulty affects model performance and thinking budget allocation. Using the pass rate of RI-Qwen-7B as the reference metric, define query easy if its pass rate excceds 0.5 and hard otherwise. We then compare the performance both 7B and 32B models under different orderings of casy and hard queries (i.c., easy-to-hard hard-to-easy). Figure 15 (b) and (c) show that DeepSeek-RI and R1 R1-Qwen-32B fail to allocate thinking budget reasonably according to problem difficulty, but can benefit from difficulty ordering in Figure 15 (a). We hypothesize that this because placing difficult problems the beginning leads models to allocate more token budget to difficult problems, thereby improving overall success rate, while smaller models (R1- [RI-Qwen-7Bi show significant benefit.
Perfomaance
Hary-Haad Token Distribution
Hard-eEay Token Distrilation
70k
Tokens por Preblam
Tokons per Prbbium
50L
Accanny
8.90
6.40
30k
.0k
20k
0.20
10k
5.00
R1 11-Qmmm7
R1-Qwa-- 120 Despook-A))
R1-Qwon-18
RI R1-Qwem-A3 A31 Despork-it
Rl-Qwa-71 7l
R1-Qjom- 128
DapuockRI
Figure 15: Ablation study the impact of query difficulty ordering for R1-Qwen-7B, R1-Qwen-322, and DeepsSeck-RI models. (a) Performance comparison between easy-to-hard and hard-to-casy query orderings. (b) Thinking budget allocation in the easy-to-hard scenario. (c) Thinking budget allocation in the hard-to-easy scenario.
19

---

Published as a conference paper at ICLR 2026
E EVALUATION IMPLEMENTATION DETAILS
E. MODELS AND DATASETS IN R-HORIZON BENCHMARK
Datasets Statistics and Evaluation Metric We present the statistics and evaluation metric of the R- E-HORIZON benchmark in Table 2, showing the number of problems in the original datasets, extracted seed questions, and final composed datasets.
Table 2: Dataset statistics and evaluation metric for R-HORIZON benchmark
Number of Problems
Dataset
Metric
Original
Seed
Composed
Mathematical Tasks
Math500
500
257
500
Accuracy
AIME24
30
28
Avg @32
AIME25
30
28
30
Avg 32
AMC23
40
37
40
Avg@8
Code Tasks
LiveCodeBench
279
279
279
Pass@1
Agentic Tasks
WebShaper
500
117
50
Avg@3
Model Details the R- R-HORIZON benchmark, we evaluate the following open-source models.
We present the model sources and their corresponding evaluation lengths (max new tokens for generation) follows: DeepSeek-R -0528 (64k), RI 11--wen-11.3 (64k), R1-Qwen-7B (64k), R1-
Qwen-32B (64k), R1-Llama8B (64k), R R1-I.lama70B (64k) (Guo et al., 2025), Qwen3-8B (40k), 101--222B-2500 2207 (64k), Qwen3-235B-A --2---22-------- TerB--hiininnnn5555 (64k), QwQ-32B (64k) (Yang et al., 2025a),
Qwen3-32B (40k), Qwcn3-235B-A-I 235--222-----------
Resttroneeeeerch--e Resssacn-- Seatinngg-wwnnn--eeee ween-1.5B (64k), Llama-3.1-
Nemotron- -aatroon--ao-----v (64k) (Nvidia et al., 2024), DeepScaleR- 1.5---1...--r-iiie
(64k) (Luo et al.,
2025b), Polaris-1.7B-B 1.7--Previe (64k), Milaris-B--Prvviee (64k) (An et al., 2025), Skywork-OR1-7B
(64k), Shywork--R1-3222 (64k) (He et al., 2025a), OpenThinker3-7B (32k) (Guha et al., 2025),
Efficient-R1-7B (a 0.2) (64k) (Arora & Zanette, 2025), Lase--DE-L4096-7B (64k) (Liu al.,
2025), DAPO-Qwen-32B (64k) (Yu et al., 2025).
Prompt Examples We present the prompt examples for math, code, and websearch tasks in Figure 16, Figure 17 and Figure 18.
Prompt 1: Prompt Example for Mathematical Tasks
Problem 1: (Question 1} Using the result ([answer 1]} from the previous calculation, ([variable 2]) (dependency function]. Problem 2: (Question 2/ Problem (Question n] Note: In this problem set: [variable k] represents the calculated variable needed to solve problem k. [answer k] represents the answer to problem k. Solve problems step by step and provide the answers for all problems the following format: #NB Final Answers Problem I: \boxed [[nswwe I]) Problem \boxed (answer 2]) ..... Problem n: \boxed ([ [[nswer n])
Figure 16: Prompt Example for Mathematical Tasks
E.2 EVALUATION METRICS CALCULATION
For mathematical and agent- WebShaper tasks, we utilize GPT-4.1 to extract answers from all problems and perform subsequent scoring. For code tasks, we first extract code blocks from the
20

---

Published as a conference paper at ICLR 2026
Prompt 2: Prompt Example for Code Tasks
### Question: [Questions] Solve all problems step by step and provide the code for all problems. For each problem, read the inputs from stdin solve the problem and write the answer to stdout (do not directly test on the sample inputs). Enclose your code within delimiters in the following format: "'python YOUR CODE HERE ### Answer: (use the provided format with backticks)
Figure 17: Prompt Example for Code Tasks
Prompt 3: Prompt Example for Web Search Tasks
You may call within <tools><<tools> XMI. tags: (Tools) Please answer the following question by using the available tools to search for information and visit relevant webpages. Before each action, conduct brief reasoning determine your next step. After reasoning, if you find you lack some knowledge, you can call the tools to get new information. you find no further external knowledge needed, you can provide the answer within <answer></answer> XML tags. For example, <answer> Beijing </answer>. {Question)
functions assist with the user query.In'n You are provided with function signatures
Figure 18: Prompt Example for Web Search Tasks
responses and assess their correctness via sandbox execution. The prompts used for scoring are presented in Figure 19 and Figure 20.
Prompt 4: Answer Extraction Prompt for Mathematical Tasks
You helpful assistant. Extract the question numbers and corresponding from the given results, and return them in JSON format, where the key is the question number and the value is the answer.
Figure 19: Answer Extraction Prompt for Mathematical Tasks
We also compare the consistency rate between using model-based answer extraction and rule- based "Aboxed()) )" pattern extraction in Table 3. We find that the number of problems increases, many models fail to accurately follow the output format, making model- based answer extraction more accurate for evaluation. Therefore, we uniformly adopt model-basee answer extraction for mathematical tasks.
E.3 INFERENCE HYPERPARAMETERS
We set the maximum generation length for inference to 64k tokens. For models with maximum lengths below 64k, set the max generation length to their max sequence length. For inference hyperparameters, we set temperature to 1.0, top-k to 10, and top-p to 0.95. For the Qwen series hybrid reasoning models that switch between thinking mode and non-thinking mode, we consistently test their thinking mode.
F TRAINING IMPLEMENTATION DETAILS
F.1 TRAINING SETUP
We show the training hyperparameters for training with R-HORIZON datasets Section 4.3. The maximum response length is set 40k tokens to prevent truncation. Training is conducted exclusively on the mathematical components of the Skywork-B RL dataset. All training progress are fine-tuned by optimizing the policy loss (see Eq. 5) with constant learning rate of 1 10-6 We set the batch size
21

---

Published as a conference paper at ICLR 2026
Prompt 5: Answer Extraction Prompt for WebShaper
You are an evaluation assistant. Please determine if the predicted answer is equivalent to the labeled answer. Question: [question]
Labeled Answer:
[correetaasswer
Predicted Answer: (response)
equivalent the labeled answer? Please respond with "Correct" if they equivalent, or "Incorrect" if they not equivalent. Do not include any other text.
Did the model give
Figure 20: Answer Extraction Prompt for WebShaper
Table 3: Consistency rate between model-based and rule-based extraction for R1-Qwen-7B on Math500
Composed Problem Num
2
4
Consistency Rate (%)
96.83
96.41
93.77
91.04
to 256, mini-batch size to 128, and group size to 16. We employ higher clip ratio of 0.265, target entropy of 0.2, sampling temperature of 1.0, and rejection sampling. Notably, we do not apply any KL loss in our training process.
We use the same training hyperparameters for standard RL training in Appendix B. We implement a three-stage training paradigm following Skywork ORI (He et al., 2025a). where context length incrementally expanded upon reaching performance convergence at each stage. This progressive approach, advancing from 8k to 16k and ultimately to 32k maximum response tokens, delivers both improved benchmark results and enhanced computational efficiency.
F.2 HORIZON TRAINING DATASETS
We initialize filtered data pool Diitered from the original Skywork-OR training data via the R-HORIZON Problem Filtering process (Section 3.1). To control the problem difficulty, we compose problems according their pass rates while maintaining AcCespected instances. We show the datasets' statistics Table
0.25 for all composed
Table 4: Dataset statistics for Skywork-ol-RL Data
Original
Seed
Composed ppass_rae>>0.255)
Skywork-o1-RD Data
48371
18015
18000
G THE USE OF LARGE LANGUAGE MODELS
Large language models employed exclusively writing aids to refine sentence clarity, format tables, and improve overall readability. They were not involved in the central research contributions, experimental design, or scientific content of this work. The authors bear full responsibility for all content presented in the paper.
H CASE STUDY
We provide a case study with an example prompt from AIME24 shown in Figure 21, and compare model outputs multi-horizon problems when trained with original data (n=l model in Section 4.3) versus R- R-HORIZON training data (n=4 model in Section 4.3), as illustrated in Figure 22 and Fig- 23. We observe that models trained with R-HORIZON training data consume fewer tokens per
22

---

Published as a conference paper at ICLR 2026
problem, avoid excessive thinking budget allocation individual problems, and successfully solve all problems.
Prompt 6: Example Prompt for Case Study
Instructions:
Problem 1: Alice and Bob play the following game, stack of n tokens lies before them. The players take turns with Alice going first. On each turn, the player removes either 1 token or 4 tokens from the stack. Whoever removes the last token wins. Find the number of positive integers n less than or equal to 2024 for which there exists strategy for Bob that guarantees that Bob will win the game regardless of Alice's play. Problem 2: Using result [answerl] from the previous calculation, [variable2] = [answerl] 805. Every morning Aya goes for 9-kilometer-tong walk and stops coffee shop afterwards. When she walks constant speed of kilometers per hour, the walk takes her [variable2] hours, including minutes spent in the coffee shop. When she walks s + 2 kilometers per hour, the walk takes her 2 hours and 24 minutes, including I minutes spent in the coffee shop. Suppose Aya walks at 2 kilometers per hour. Find the number minutes the walk takes her, including the t minutes spent in the coffee shop.
Problem 3: Using the result [answer2] from the previous calculation, [variable3] [answer2] 150. Let B be the set of rectangular boxes with surface [variable3] and volume 23. Let be the radius of the smallest sphere that can contain each of the rectangular boxes that are elements of B. The value of y² can be written y. where and are relatively prime positive integers. Find + q. Problem 4: Using the result [answer3] from the previous calculation, [variable4] = [answer3] 720. Let w * [pariabled] be a 13th root of unity. Find the remainder when
12
II 2 + 2B) k=
is divided by 1000.
Note: In this problem set: [variablek] represents the calculated variable needed to solve problem k.
[answerk] represents the answer to problem k.
Solve all problems step by step and provide the answers for all problems in the following format:
Final Answers
Problem 1:
[answer]
Problem 2:
[answer2]
Problem 3:
[answer3]
Problem 4:
[answer4]
Target Answers:
Problem I: 809
Problem 2:
204
Problem 3:
721
Problem 4:
321
Figure 21: Example Prompt for Case Study
23

---

Published as a conference paper at ICLR 2026
Prompt 7: Model Output for Model Trained with Original Data
<Problem Start>
Okay, have four problems here, and I need to solve each step step. Let start with Problem 1.
Problem 1: Game Theory
Alice and Bob play game with stack of n tokens. They take turns removing either or 4 tokens. Alice goes first, and the player who takes the last token wins. I need to find how many n I 2024 where Bob has a winning strategy, regardless Alice's moves.
Problem I thinking
Therefore, our answer for Problem is 809.
Final Answer
Problem 1: 809
<Problem END, Output Length=3154, Status=Correct>
<Problem Start>
Problem 2: Walking Speeds
Every morning, Aya walks km and stops at coffee shop. Walking speed km/h takes her t minutes in the shop. When she walks + 2 km/h, the walk takes hours and 24 minutes, including / minutes. Find the number of minutes the walk takes when she walks at + 1/2 km/h.
Wait, let me try to parse this. .... Problem 2 thinking
When walking
1. total time= time= #40.3
But without knowing and can't 't compute it.
think I've spent too much time without progress, so I might need to move to problem3 and return.
<Problem END, Output Length=14055, Status=Wronge:
<Problem Start>
Problem3:
Using the result [variable2] from probleml 1809 and [answer2] from problem2= let's say it's 100, find the remainder when [variable3] is divided by 1000.
Problem 3 thinking
But I'm not confident.
Wait, alternatively, perhaps problem2's answer number solutions 2024, which was 809, so problem3 that.
But without info, 1 can't compute. Given the time constraints, think I'll settle
Probleml: 809
Problem2: 100
Problem3: (809 + 100) = 909
<Problem END, Output 1en5th=16055, Status=Wronge
<Problem Start>
Status: NOT ATTEMPPEDD
<Problem 4 END, Output Length=16055, ATTTPPE
Figure 22: Case Study for Model Trained with Original Data
24

---

Published as a conference paper at ICLR 2026
Prompt 8: Model Output for Model Trained with Composed Data
<Problem Start>
Alright, have four problems solve. They're re all math problems, each building on the previous with some variables. Let me take them one by onc.
Problem 1: Alice and Bob play game with a stack of n tokens. They take turns removing or 4 tokens, Alice goes first. We need find number positive integers I 2024 where Bob has winning strategy regardless of Alice's play.
This is combinatorial game theory problem. Looking losing positions, I find they 0 or 2
(mod 5).
Problem thinking
So the answer for Problem is 809. So answerl=809.
<Problem END, Output Len5th=2015, Status=Correct>
<Problem Start>
Problem 2: Using the result [answerl] from the previous calculation, [variable2] = [answerl] 805. Every
morning Aya goes for 9-kilometer-long walk and stops a coffee shop afterwards.
So variable2 = 809 805
Problem 2 thinking
Solving, get 2.5 km/h and minutes.
When she walks So the answer is 204. So answer2=204.
km/h: Total time 204 minutes.
<Problem 2 END, Output Length=3081, Status=Correct>
<Problem Start>
Problem 3: Using the result [answer2] from previous calculation, [variable3] [answer2] 150. Let be the of rectangular boxes with surface [variable3] and volume 23.
So variable3 204 150 54.
Problem 3 thinking
Thus ,3
637 = 657, 64, p = 721.
64
So answer3=721.
<Problem END, Output Length=6819, Status=Corredt>
<Problem Start>
Problem Using the result [answer3] from the previous calculation, [variable4] [answer3] - 720. Let
[variable4] be 13th root of unity.
So variable4 = 721 720 = 1. Problem thinking
remainder when 8321 divided by 1000 321.
So answer4=321.
<Problem 4 END, Output Length=9336, Status=Correct>
Figure 23: Case Study for Model Trained with Composed Data
25


Reviews:
[{"review_id": "BKe6B6UxAW", "reviewer": "Reviewer_hTcg", "summary": "The authors propose a new dataset for long-horizon reasoning and then also train a model on their dataset with Reinforcement Learning (RL). The new dataset is based on six other datasets targeting math (MATH500, AMC23, AIME24 and AIME25), code generation (LiveCodeBench) and agentic tool usage (WebShaper). Long-horizon problems are generated by combining several independent problems into a single one, where the problems have to be solved (usually) sequentially since later problems rely on the results of earlier problems, thereby transforming them into a long-horizon task. 25 language models are then evaluated on the developed dataset and their results are analyzed. In addition the author also generate a training dataset based on the Skywork-RL dataset and use it to fine-tune an existing model with custom reward functions, which then is also evaluated.", "questions": "* Figure 2: What are the circles with a red outline?\n* 4.2, last paragraph: Why is AIME25 more challenging than AIME24?\n* Figure 6: How can the error position be above the token length? Because wrong answers tend to be longer for smaller query numbers?\n* How much can the six benchmarks (MATH500, AMC23, AIME24/25, LiveCodeBench and WebShaper) be considered out of distribution compared to the training dataset based on Skywork-RL?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* with the increase in context lengths of modern language models, long-horizon reasoning is a relevant topic and the dataset seems to be good contribution\n* the writing is mostly clear\n* good evaluation in general and large number of RLMs tested\n* good ablation study\n* detailed appendix with good overview over additional results as well as the evaluation setup", "weaknesses": "* I found the motivation a bit lacking: the new dataset is somewhat artificial in its dependencies - a proper motivational example would be beneficiary in my opinion\n* not a weakness per se, but I found the third chapter a bit too high-level without much details\n* writing primarily in the evaluation is at times a bit imprecise or not entirely accurate:\n  * 4.1:\n    * AMC23 is not discussed\n    * for MATH500 it seems that there is also data for 20 composed query numbers\n  * 4.2, first paragraph: \"R1-Qwen-7B drops from 93.6% (n = 1) to 0% (n = 16), which is 34.1% more than the 32B model\" - reference is unclear (probably MATH500); Where does the 34.1% come from?\n  * 5.1, Effective Reasoning Length of LRMs:\n    * \"gap between the actual accuracy and theoretical accuracy of models becomes increasingly larger\" - not necessarily true on AIME24\n    * \"7B model’s error range is (4-6k tokens) while the 32B model’s error range is (8-10k tokens)\" - context is missing, probably for MATH500\n  * Reflection is not defined or at least explained, which also makes it unclear why it matters.\n  * Sometimes \"expected accuracy\" and sometimes \"theorectical accuracy\" is used? I assume they are the same thing, so I suggest to use one term consistently.\n  * Appendix D.3: \"Figure 14 (b) and (c) show that all models fail to allocate thinking budget reasonably according to problem difficulty\" - seems to be somewhat not true for R1-Qwen-7B\n\nminor issues:\n* introduction: no reference for CoT\n* Figure 1: Theoretical Accuracy is not introduced at this point. (only on page 4)\n* 2.2:\n  * \"Su et al. (2025); Yang et al. (2025b); Wu et al. (2025b) investigates...\" - It should be \"investigate\", since it is a plural.\n* Algorithm 1: \"and Create\" - lowercase create?\n* 4.1: I believe, there is no whitespace before a footnote.\n* 4.3, Impact of Number of Composed Queries and Different Reward Schemes: n=1 is not really a composed problem\n* Figure 5: not very readable in terms of fontsize as well as chosen color for example for early stop and output truncation - I had to zoom in to 400% to find an instance for output truncation\n  * similar issue for Figure 6, especially the theoretical accuracy as well as Figures 7 to 9\n* Figure 6: missing whitespace before the brackets in the subfigure titles (also for Figure 9)\n* sometimes it is \"Math500\" and sometimes it is \"MATH500\" in the evaluation figures\n* Figure 8: x axis label might be misleading, maybe something like \"2 Queries\" or just \"2\" and so on would be better?\n* Appendix F.1: \"policy loss 5 with\" - should probably be something like \"policy loss (see Eq. 5) with\"\n* Appendix H: details on model, which generated the responses are missing, where the questions are coming from, etc.\n* references:\n  * consider the proper capitalization of the titles, at least for proper names and abbreviations to improve readability\n  * place of publication of arXiv references can only surmised from the URL\n  * consider adding the access date for web references\n  * [Guo et al. 2025] - cited differently than the other arXiv references\n  * [Luo et al. 2025a/b] - URL is not a proper link\n  * [MAA 2024/2025] - consider using something like the xurl package to fix the formatting\n  * [Muennighoff et al. 2025] - cited differently than the other arXiv references - maybe this could be the blueprint for the others", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yMoun17bnR", "reviewer": "Reviewer_JsjS", "summary": "The paper proposes R-HORIZON, a benchmark and data construction framework for evaluating and improving long-horizon reasoning in large reasoning models (LRMs). Using Expanded Problem Composition (EPC), it links independent problems into multi-step dependency chains. Evaluating 25 LRMs across math, code, and agentic tasks, the authors find significant performance degradation as reasoning length increases. They further integrate R-HORIZON data into RL from verifiable rewards (RLVR) using Group Relative Policy Optimization (GRPO), showing that such training enhances models’ sustained reasoning ability.", "questions": "**Questions for the Authors:**\n1. In **Figure 11**, the entropy of RL training decreases as *n* increases (e.g., *n=4* vs. *n=1*). Could the authors provide an explanation or hypothesis for why longer reasoning chains lead to lower entropy?  \n2. Is it possible that the poor performance of models on **R-HORIZON** arises because they have not been exposed to similar long-horizon or compositional data during pre-training or intermediate training? Have you attempted any supervised fine-tuning (SFT) or cold-start experiments on such data?  \n3. Could the improvement in original tasks after RL with R-HORIZON stem from the fact that the constructed datasets yield more balanced reward signals (i.e., less likely to produce all-correct or all-wrong rollouts), thereby improving rollout efficiency?  \n4. The paper reports an increase in reasoning efficiency (shorter thinking length) after RL. Could this be because higher *n* naturally leads to longer responses, which increases the truncation ratio during RL training, thus implicitly encouraging more concise reasoning?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Strengths:**\n\n1. The paper is well-written, clearly structured, and easy to follow.\n2. It introduces meaningful long-horizon benchmarks and corresponding datasets that are valuable for advancing research on large reasoning models (LRMs).\n3. The extended benchmarks effectively reveal the limitations of current LRMs in maintaining long-horizon reasoning.\n4. The authors conduct reinforcement learning (RL) experiments on the constructed datasets, demonstrating notable improvements in sustained reasoning performance.\n5. The paper provides comprehensive analyses and insights into the proposed data construction and evaluation methods.", "weaknesses": "**Weaknesses:**\n1. While the proposed methods enable verifiable long-horizon reasoning evaluation, the constructed benchmarks appear somewhat artificial and may not fully reflect realistic reasoning scenarios.  \n2. Similar efforts, such as *GSM-Infinity* [1], have explored benchmark construction for increasing reasoning complexity. The paper would benefit from a clearer discussion of how its approach differs from these prior works.\n\n[1] GSM-Infinite: *How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?* (ICML 2025) https://arxiv.org/abs/2502.05252", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0Ux1OEB0It", "reviewer": "Reviewer_sdPJ", "summary": "This paper introduces R-HORIZON, a benchmark and training framework designed to evaluate and enhance long-horizon reasoning in Large Reasoning Models (LRMs).\nExisting benchmarks mainly assess single-step reasoning, where each question is independent. R-HORIZON addresses this gap by composing interdependent multi-step queries from existing datasets (e.g., MATH500, AIME24/25, LiveCodeBench, WebShaper).\nThe authors evaluate 25 state-of-the-art reasoning models (e.g., DeepSeek-R1, Qwen-3-235B, Gemini-2.5, Claude-Sonnet-4) and show that all suffer severe performance degradation as reasoning horizons increase.\nFurthermore, they train models using R-HORIZON-based reinforcement learning (RLVR), demonstrating substantial gains on both composed and standard reasoning tasks (+17.4 on long-horizon tasks and +7.5 on AIME24).\n\nOverall, the paper provides a new perspective on reasoning length, reflection behavior, and token budget allocation, revealing fundamental limits in current LRMs and proposing a practical solution for improvement.", "questions": "None", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Novel and impactful evaluation paradigm.\nR-HORIZON systematically measures reasoning depth and breadth, exposing weaknesses that single-horizon benchmarks cannot reveal.\n\nComprehensive empirical analysis.\nThe evaluation across 25 strong LRMs and multiple domains (math, code, agentic tasks) convincingly demonstrates consistent long-horizon degradation patterns.\n\nActionable training insights.\nThe reinforcement learning experiments with R-HORIZON data provide a scalable and cost-effective way to enhance long-horizon reasoning, with clear empirical gains and behavioral analyses (reflection, budget allocation).", "weaknesses": "Multi-step reasoning is indeed necessary; however, in this paper, simply combining several basic math problems offers limited reflection of contextual consistency in true multi-step reasoning. Tasks such as “1 + 2 = ?” followed by “previous answer + 4 = ?” can essentially be summarized into a single query like “1 + 2 + 4 = ?”, which weakens the intended long-horizon reasoning challenge.\n\nMoreover, maintaining contextual consistency across multi-step reasoning is highly dependent on prompt design, yet the paper does not clearly explain how prompts are constructed or formatted for solving these composed problems.\n\nThe evaluation on closed-source models is insufficient, as only o4-mini was included.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eI4RClfXZw", "reviewer": "Reviewer_91MQ", "summary": "R-Horizon is motivated by existing LLM reasoning benchmarks' over-emphasis on _immediate, single-turn tasks_. It instead proposes a method to automatically generate _long-horizon, interdependent reasoning tasks_ via query composition, which enables it to curate the R-Horizon benchmark using existing (single-turn) datasets targetting math, coding, and agentic capabilities. The paper finds out that even the SOTA reasoning models suffer performance degradation as the number of composed queries (subproblems) increase. The paper then further suggests RLVR training over query-composed dataset, which demonstrates considerable improvement in both LRM's single- and multiple-problem performance. Extensive analysis and ablation studies provide important insights into the causes of performance degradation and why training with composed dataset works better.", "questions": "1. For the dataset construction for coding tasks, it's mentioned that _\"unlike the sequential composition used for mathematical tasks, we apply a directly composed concatenation format ... without adding explicit dependencies\"_. So (a) what is exactly this \"directly composed concatenation format\" (hard to tell simply from the given prompt in Appendix)? (b) isn't the point of R-Horizon benchmark to create interdependent, long-horizon reasoning tasks? I think the compromise made here undermines the overall validity (i.e. many of the analysis made in the paper, e.g. the \"dependency reasoning error, applies to math only).\n2. For the reward design, the distinction between the \"last\" and \"all\" rewards is a bit non-intuitive as I thought they should be very similar when the problems are inter-dependent. Is it even possible for a model to obtain positive \"last\" reward even when its \"all\" reward is 0 (the other direction is trivial)?\n3. For **Figure 6**, how can the error positions be even larger than the output lengths (even query num is low)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper is overall clearly motivated, well-written, and easy to follow.\n- The lack of long-horizon, multi-stage/problem evaluation of LRMs is indeed a practical concern, and the paper's approach of using query composition to automatically combine single-turn queries within one interdependent problem is (an example of) an elegant solution.\n- On top of the R-Horizon benchmark, the paper further conducts solid RLVR training on the composed dataset, which increases both single and multi-problem performance and further highlights the importance of equipping LRMs with multi-step compound reasoning skills.", "weaknesses": "- Even though the paper mentions that it constructs composed problems using both math, coding, and agentic datasets, most of the construction details and analysis are centering around the math criteria (see Q1 below). And the RL training is also solely done on math-related datasets.\n- For the analysis on the RL training, it would be great to see similar \"error type distribution\" plot (as in Figure 5) for the model before & after the training. This helps gain further insights on _why & how the performance on multi-query problems_ increase over the training.\n- While the writing is good overall, certain plots are hard to read due to ambiguous legends & captions. As an example, on Figure 8 the x-axis shows \"Query 1/2/etc.\", which I believe actually denotes the number of composed queries but can be misunderstood as query indices. Also certain plots have (unexplained) error bars/intervals while some do not.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)