LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts
Authors: Siyuan Wang
Gaokai Zhang
Li Lyna Zhang
Ning Shang
Fan Yang
Dongyao Chen
Mao Yang
Venue: ICLR 2026 Oral (2026)
Keywords: Long Context Reasoning
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: Reasoning over long contexts is essential for large language models. While reinforcement learning (RL) enhances short-context reasoning by inducing "Aha" moments in chain-of-thought, the advanced thinking patterns required for long-context reasoning  remain largely unexplored, and high-difficulty RL data are scarce. In this paper, we introduce LoongRL, a data-driven RL method for advanced long-context reasoning. Central to LoongRL is KeyChain,  a synthesis approach that transforms short multi-hop QA into high-difficulty long-context tasks by inserting UUID chains that hide the true question among large collections of distracting documents. Solving these tasks requires the model to trace the correct chain step-by-step, identify the true question,   retrieve relevant facts and reason over them to answer correctly. 
RL training on KeyChain data induces an emergent plan–retrieve–reason–recheck reasoning pattern that generalizes far beyond training length. Models trained at 16K effectively solve 128K tasks without prohibitive full-length RL rollout costs. On Qwen2.5-7B and 14B, LoongRL substantially improves long-context multi-hop QA accuracy by +23.5% and +21.1% absolute gains. The  resulting LoongRL-14B reaches a score of 74.2, rivaling much larger frontier models such as o3-mini (74.5) and DeepSeek-R1 (74.9). It  also improves long-context retrieval, passes all 128K needle-in-a-haystack stress tests, and preserves short-context reasoning capabilities. Code is available at https://loongrl.github.io.

Published as a conference paper at ICLR 2026
LOONGRL: REINFORCEMENT LEARNING FOR AD- VANCED REASONING OVER LONG CONTEXTS
Siyuan Wang
Gaokai Zhang
Li Lyna Zhang'!
Ning Shang'
Fan Yang'
Dongyao Chen
Mao Yang'
Microsoft Research Asia
Shanghai Jiao Tong University
Carneeie Mellon University
ABSTRACT
Reasoning over long contexts is essential for large language models. While re- inforcement learning (RI.) enhances short-context reasoning by inducing "Aha" moments in chain-of-thought, the advanced thinking patterns required for long- context reasoning remain largely unexplored, and high-difficulty RL data are scarce. In this paper, we introduce LoongRL, data-driven RL method for ad- vanced long-context context reasoning. Central to LoongRL KeyChain, synthesis ap- proach that transforms short multi-hop QA into high-difficlly long-context tasks by inserting UUID chains that hide the true question among large collections of distracting documents. Solving these tasks requires the model to trace the cor- rect chain step-by-step, identify the true question, retrieve relevant facts and rea- over them answer conrectly. RL training on KeyChain data induces emergent plan-retrieve-reasn--eeet beyond training length. Models trained 16K effectively solve 128K tasks with- out prohibitive full-length RL rollout costs. On Qwen2.5-7B and 14B, Loon- gRL substantially improves long-context multi-hop QA accuracy by +23.5% and +21.11 12.11% % absolute gains. The resulting LoongRL-14BB reaches a score of 74.2, rival- ing much larger frontier models such as o3-mini (74.5) and DeepSeek-FR R1 (74.9). It also improves cong-context retrieval, passes all 128K needle-in-a-haystter stress tests, and preserves short-context reasoning capabilities. Code is available https: http::://coogrrtr wtt...........i.................
reasoning pattern that generalizes far
INTRODUCTION
Reasoning over long input contexts is critical capability for large language models (LLMs), as many real-world tasks, from analyzing legal documents to debugging large codebases, require inte- grating information across thousands of tokens. Recent advances, such OpenAl o-series (Jaech et al., 2024) and DeepSeek-RI (Guo et al., 2025), show that reinforcement learning can improve reasoning by eliciting longer chain of thoughts (CoT) and emergent self-reflection. However, these methods mainly target short-context inputs and rely on model internal knowledge (c.g., math rea- soning). In contrast, long- cong-context reasoning requires both reasoning and the ability to retrieve and ground information from extensive external input contexts. Although modern models support longer context windows (Achiam et al., 2023; Ding et al., 2024), they excel mainly at retrieval, leaving rea- soning over long documents persistent challenge for real-world tasks (Ling et al., 2025).
This work aims to bridge this gap by enabling long-context models to move beyond retrieval and acquire advanced reasoning capabilities. Inspired by the short-context reasoning successes (Guo et al., 2025; Gandhi et al., 2025), we hypothesize that the key lies discovering and mastering thinking patterns specific to long-context reasoning. Since such patterns remain unclear, we adopt reinforcement learning approach to investigate whether high-quality reasoning patterns can emerge.
However, face significant challenges. First, effective RL training requires difficult long long-context problems that cannot be solved by retrieval alone. Questions must be sufficiently challenging to trigger reasonining and require retrieving relevant information from long input contexts. Moreover, since recent RL methods rely on outcome-only rewards to mitigate reward hacking (Liu et al., 2024;
"qqual contribution; work was done during the internship at Microsoft Research Asia 1 "ooresponndece Li I Lyna Zhang (li.zhang eechann@xingyaen @xingygzziiiii.ccommm

---

Published as a conference paper at ICLR 2026
(a) RL-Generated Trajectory with KeyChain Data (Plan: decompose problems multi-steps)
(b) RL-Generated Trajectory without KeyChain Data (Mixed reasoning with retrieva()
To determine when the last time Peter Tills sports team beat the winner of the 1894-86 86 FA
Firsl, we need to identily who won the 1894-95 FA Cup. From the passaige about the 1894-95 FA Cup, we can see that Villa
Cup, nead to follow these staps:
Albion 1-0 the compatiton by defeating Wast Bromwich winner
1. Identily the winner the 1894-95 FA Cup.
1694-95 Cup
2. Detormine toams Peter played
was Aston Villa.
3. Find the last time any of Petar Tillss teams beat Aston Villa.
Next, we need to find when Peter TIllss teom, which Evarton. last beat Aston Vila, Looking through the passage about Everton's history
(Retriove relevant document)
From Passage The 1894-88 96 FA Cup was won Aston Villa.
rivaltess we find the recent Aston
From Passage Peler Till played for several teams, need find the learss
the Premirr League montlonad: "Vila finally beat Blues in the Promiarship, thanks to Kavin Philips goal.
viclory of Everion
played profossion
(Step-laval reasoning)
However. we are asked about Everton specifically. Continuing to search the lest, we find "Everton's biggest rivalry with neighbours Liverpool, against whom the contests the Merseysida derby.
Now, need find the time any these toams beat Aston Villa from information
Passage
(Retrleve relevant document)
(Retrieve relevant document, recheck)
Therefore, the last time Peter Till's teem (Everton) beat the winner of the 1894-98 96- FA Cup (Aston Vilis) wes in the 2005- 2006 season. /000ko/ 2009 season)
Passage
recent relevant between Aston Villa and Birmingham Peter played Waisally mentioned...
City (one: of the
(boxed{1 Docember 2010)
Correct
X
Wrong
Figure 1: Model trajectories on long-conten multi-hop QA with and without KeyChain RL data. (a)
With KeyChain data, model exhibits an emergent plan retriveee retrieve-reason reason recchcck thinking pattern, improving reasoning reliability and can generalize to longer contexts. (b) Without KeyChain data, reasoning and retrieval are entangled, the model often lacks an explicit planning step and does not deeply reason over retrieved information, frequently leading to errors. Reasoning steps are marked in blue and retrieval steps in orange.
Li et al., 2025; Shang et al., 2025a), the answers must be verifiable. In practice, however, such high-diffiuulyy long-contex: questions are extremely scarce, and the ground-trute -tuth answers often take multiple valid forms, making reliable verification difficult. Second, strong long-contex, performance typically requires training at near-target lengths (Liiu et al., 2024; Li et al., 2025), but scaling RI. roll- outs from short inputs (i.e., current <IK tokens) to 128K contexts incurs prohibitive compute and memory costs, making direct training infeasible at standard compute scales. Third, even if feasi- ble, training exclusively long-context data risks degrading short-context and general reasoning abilities (Peng et al., 2023; Shang et al., 2025b), which remain critical in practice.
To this end, we introduce LoongRL, data-driven reinforcement learning method that incentivizes models to acquire effective thinking patterns for advanced long-contex: reasoning. At its core is KeyChain, data synthesis approach that transforms short multi-hop QA datasets into high- difficulty long-context problems by extending contexts with distracting documents and inserting random UUID "chains" that hide the true question across multiple hops. These non-semantic UUID) chains prevent the model from relying on lexical or semantic shortcuts during prediction, forcing it to trace the correct chains step-by-step, identify the actual question, retrieve relevant f'cts from the long context, and reason over them to generate the answer. Then, to enable effective RL training, we design a rule-based answer verifier, two-way substring exact match, which effectively evaluates free-form answers in general QA while mitigating reward hacking. Using KeyChain data, RL consis- tently elicits an emergent plan-retriere-reason-r Remarkably, this emergent patterns generalizes beyond the training length, enabling models trained at 16K to effectively handle 128K reasoning tasks without the prohibitive cost of full-length RL. Finally, we introduce balanced data data-mixing strategy enhance long-conter reasoning while pre- serving short-context general reasoning and long-context retrieval capabilities.
reasoning pattern, as shown in Fig. (a).
Extensive experiments across Diven2.5-BBIInsteen and Qwen2.5-14B- Instruct and diverse bench- marks demonstrate the superiority of LoongRL. Remarkably, LoongRL substantially boosts Qwen2.5-7B-Instrul and (wwen2.5-14B-Insti provements on long-context multi-hop QA tasks. The resulting LoongRI. 14B achieves a score of 74.2, significantly surpassing all baselines and closely approaching much larger models such as o3 mini at 74.5 and DeepSeek-RI at 74.9. Beyond the 16K training length, LoongRL generalizes ef- fectively inference lengths up to 128K tokens, substantially improving long-context retrieval and passing all soadle-in-a-haystar pressure tests. At the same time, it preserves short-context and general reasoning capabilities, setting a new state of the art for models at this scale, and shows that Lo can induce advanced reasoning patterns to substantially improve long-context reasoning.
by +23.5% and +21.1 % absolute accuracy im-
2 RELATED WORKS
Reasoning and by high-quality human-like chains of thought (CoT), typically obtained via teacher model distilla-
Reasoning. Recent advances in LLM reasoning are largely driven
2

---

Published as a conference paper at ICLR 2026
1tep1: Data Filtering
Step3: KeyChain Insertion
Qwen2. 5-328
7a0f9ecc
02 021553ad
bccc2507
835bdf8c
Key-Value Chain with real question
Key-Value Chain with distractor question
021553ad
15e85767
835bdf8c
9F1 8b2ffc
of
oq
Filter
15e85767
5e0db201
9f18b2fc
e400dd58
Step2: Long
Context Filling
5e0db201
original og
e400dd58
distractor
Document Insertion
C
Insert Key-Value Pairs Random Positions
Figure 2: Overview of our KeyChain data construction.
tion (Yang et al., 2025) or self-generation through reinforcement learning (Guo et al., 2025), Most existing studies focus short-context reasoning tasks, such mathematics (Wang al., 2025b; Shang et al., 2025a) and code (Liu et al., 2025; Ahmad et al., 2025), where emergent patterns like self-reflection and "aha" moments are crucial to the success (Gandhi et al., 2025). In contrast, ploration of advanced long-context reasoning patterns remains limited.
Existing efforts improve long-context reasoning largely fall into categories: prompting-ban based methods (Yen et al., 2024) and synthetic-data SFT (Li et al., 2024b;a;c), Prompting is limited by the base model's reasoning capacity, while synthetic-data SFT often introduces noise bias, con- straining advanced capabilities. QwenLong-Li (Wan et al., 2025) makes notable step by extending RI-distill
1-distill-Qwen-3222 Qwen-32B with RL on sequences up to 60K tokens, encouraging self exploratoo of long reasoning trajectories. However, it leaves open key questions about how to design high-quality RL training data. We address this gap by introducing KeyChain RI. data that fosters emergent reasoning patterns and generalizes from 16K to 128K contexts with significantly higher efficiency.
Long-Context Synthetic Data. Existing methods for long-context data synthesis primarily extend input contexts by padding questions with additional irrelevant documents. For example, Li et al. (2024c) augment MuSiQue (Trivedi et al., 2022) with extra unrelated passages; Li al. (2024a) use document-filll filllnn HotpotQA (Yang al., 2018) ()18)am)) and SQuAD (Rajpurkar et al., 2016); and Li al. (2024b) shuffle MuSiQue passages similarly. While these approaches increase context length, they limited in generating high-quality, challenging training data.
3 METHODOLOGY
3.1 KEYCHAIN DATA CONSTRUCTION
Overview. LoongRL is a data-driven reinforcement learning approach designed to train models with advanced reasoning over long contexts. It relies high-quality RL training dataset D {Li. qi, a;) constructed under three principles: (i) each question qi and answer are from real- world datasets datasess ensure reliability, synthetic data often suffer from hallucination (Liu et al., 2025); (ii) solving question requires reasoning over the full long input context L, not merely leveraging model internal knowledge direct retrieval. (iii) questions qi are sufficiently challenging to allow RL to incentivize advanced long-context reasoning capabilities.
Fig. 2 illustrates the KeyChain data construction. We begin with curated, high-quality short-context QA pairs {oL,, oq, oo,} from real-world tasks. The original input context oL; for each example is first expanded into long input C of 16K tokens by inserting distracting documents. KeyChain then transforms (C,, 00,; ) into {Li qt, a by randomly inserting multi-hop key-value chains that hide the original question oq; within Lis which significantly increases difficulty. Given the new question g:, model must first traces the chain to recover oq,, and then perform long-con long-context reasoning over C, to generate the correct answer aj, where a, = oa; This construction ensures that RL training focuses reasoning over long contexts rather than memorization or shallow retrieval.
Seed Dataset Curation and Context Extension. We curate a high-quality seed dataset from three real-world multi-hop QA datasets: HotpotQA (Yang et al., 2018), MuSiQue (Trivedi et al., 2022), and 2WikiMultiHoPQA (Ho et al., 2020). Each question oq, is paired with its ground-truth answer oa; and requires reasoning across multiple documents within short context oL,. This initial col-
3

---

Published as a conference paper at ICLR 2026
lection contains 277K QA instances. To ensure effective RL training, filter out tasks that overly easy or excessively hard questions. Specifically, we answer each question eight times using Qwen2.5-32B -nsstrutct (Qwen Team, 2024), and discard those with a pass rate of 0 or 1. This yields 72K examples of moderate difficulty.
We then extend each short context oL; into long context C by inserting additional real-world documents while keeping the original question oi; unchanged. The extra documents are sampled from the short-context documents of the 200K filtered-out QA tasks, excluding any overlap with oCi. Each extended context is approximately (<) 16,384 tokens, requiring the model to retrieve relevant information from large set of distractors. This construction closely simulates real-world long-context reasoning, where relevant information is often buried within extensive irrelevant text.
KeyChain Data Construction. To make reinforcement learning effective for long-context reason- ing, we build upon the above long-context multi-hop QA data to construct the KeyChain dataset. Fig. illustrates the process. For each long-context QA task {C', 0q, 00:). }. we insert linear key- value chains into the context, resulting in L. In each chain, key maps to a value that contains the next key, forming a step-by-step tracing path. We design two types chains: (i) multiple chains that instead resolve to distracting questions, and (ii) one chain that ultimately resolves to the original question oq.- Each key generated as a 32-character UUID, with character randomly sampled from 0-9 and A-F.
From the distracting chains, we sample questions from other QA instances the dataset, ensuring they are plausible but irrelevant. For target chain, we construct new question d-- This question requires the model to start from the initial key, trace the correct chain within L. recover the original question oq, and finally perform long-context reasoning over C; produce the correct answer a; = 00,, This design substantially increases task difficulty, as the model must first localize the hidden question among multiple distractors and only then reason over the extended context answer correctly, An example of an augmented KeyChain long-contex question is shown below:
Example Key SeyChain-ain augmenee long long-context question
Please reac the following text.
<Document
<original text> ("UUIDB-n"; "distracting question") <original text> <Document 1>
("UUIDA-1"; "UUIDA-2")
<Document
("UDIDB-1": "UUIDB-2")
("OUIDA-n": "correct question"]
...
the context above, there is one correct question answer. The correct question can only be found by following the correct consecution ive chain of key:value pairs encoded with JUID strings starting frem Find the correct question first, then answer it.
(e.g., -at.aaa-/aa-110000-------0-----
"starting UUIDA-1".
Importantly, our KeyChain method is generalizble and does not require specific UUID format; the essential property is that the identifiers are high-entropy and non-semantic. This design prevents the model from exploiting lexical shortcuts during token generation and enforces explicit key-value retrieval. In our experiments, performance remains nearly identical when UUIDs are replaced with randomly generated strings, shown in Appendix A.10.
Emergent Long-Context Reasoning Patterns.
We surprisingly find that RL training with Key- Chain data enables models to develop emergent, human-like long-context reasoning patterns. As shown in Fig. (a),, for each long-context QA task, the model first generates an explicit plan de- composing the problem into subproblems and substeps, retrieves relevant information for each step. and actively re-checks retrieved content when uncertain before proceeding. This structured hian-retrieev-rrason--eeeeee loop leads highly logical and reliable solutions. Furthermore, observe that this reasoning pattern also improves conventional long-context retrieval tasks, as illus- trated in Appendix A.5 with example trajectory the RULER benchmark, where the model
4

---

Published as a conference paper at ICLR 2026
performs step-by-step. human-readable retrieval, progressively locating the correct answer rather than directly jumping to it as in traditional retrieval approaches.
More importantly, the wew-rreieeeerssoo-eeeeee kens) generalizes to much longer contexts, up to 128K tokens (see Experiments). This allows training on 16K sequences while maintaining strong longer context performance, highlighting the robustness and scalability of the KeyChain RL approach.
behavior learned short contexts (16K to-
3.2
LONG-CONTEXT REINFORCEMENT LEARNING
This section introduces our long-contexs reinforcement learning methodology using KeyChain data, covering reward design, data mixing and multi-stage training recipe.
3.2.1
GROUP RELATIVE POLICY OPTIMIZATION FOR LONG-CONTER REASONING
Group Relative Policy Optimization (GRPO). For training, adopt the GRPO algorithm. Specifically, for each question 9, its long context L. and its ground-truth answer a from a dataset D, GRPO samples a group of rollout trajectories {or 02, , ... ,0) from the old policy moas and then optimizes the policy na by maximizing the following objective:
JGRPO (0) ()), 1 (-19)
lo:l
1
min [ (.000 BDL (Toot)
G
Oi
where (f)
Tew Te (l.. Hyper-parameters and / control the clipping range of impor- tance sampling ratio and the weight KI. penalty term, respectively. The estimated advantage A;, computed from group of rewards (r1,72,...rr for each rollout trajectory:
No xo
A, T: mean( (ri, 12. rc))
std((fi.72," "2, .ra.,a rra))
Here, r3 is the reward for trajectory o₁, evaluated using a rulc- bale-based verifier to mitigate reward hacking (Guo et al., 2025; Kimi Team et al., 2025).
To stabilize RL training, we follow best practices. A small KL penalty 3 = 0.001 prevents excessive policy deviation. Following prior works (Shang et al., 2025a), remove the entropy loss term, which while commonly used to encourage exploration, can cause uncontrolled entropy growth and destabilize training, so it is omitted in our experiments.
Rule-based Reward Design. In our long-context RL, most questions are general QA rather than math code problems with clear answers. The answers take many valid forms, making it difficult to determine whether a rollout trajectory truly reaches the correct solution. Prior works such QwenLong-Li (Wan al., 2025) address this by using LLM-as-a-judge, but this introduces additional complexity. In addition to the already expensive long-context RL training, it requires serving another model for answer judgment, while still leaving room for reward hacking.
We instead adopt rule-based reward, following the success of verifiable rewards in mathematical and code RL (Shang al., 2025a; Wang et al., 2025a; Guo et al., 2025). Our approach simple yet effective for long-context reasoning. First, we explicitly require the model to output its final answer within (boxed() in the training prompt (in Appendix. A.2), ensuring unambiguous answer extraction. Second, apply two-way substring exact match on the boxed answer. Each rollout trajectory o, receives a binary accuracy reward r; E {0, 1} depending on whether the extracted final answer Mans contains the ground truth answer Yans as a substring. Formally, the reward is computed as:
substring, or the ground truth answer a contains
{
if fa S Yans V Vans S a),
ri =
otherwise.
Compared to strict exact match, this design tolerates valid answer variations and avoids the rigidity that may otherwise exclude correct outputs. Experiments in Table 6 demonstrate its effectiveness.
5

---

Published as a conference paper at ICLR 2026
Table
Data recipe for long-context RL training.
Dataset
Description
Size
Length range
Difficulty
HoipotQQA-Key
HotpotQA
2,500
16,272-20,670
Hard
MuSiQuc-Key KhyCh
KeyChain-augmenne MuSiQue
2,500
16,495-20,623
Hard
KeyChain-auus aagain-d 2WikiMuliHHopQA
2,500
14,911 14,911-20,576
Hard
HotpotQA
Standard multi-hop QA
2,500
12.058-16.279
Medium
MuSiQue
Standard multi-hop QA
2,500
12,562-16,288
Medium
2WikiMultHHopOA
Standard multi-hop QA
2,500
10,727-16,274
Medium
Book RULER (Multi-key)
Long-context retrieval (20 keys, value)
512
12,038-17,387
Easy
Book RULER (Multi-value)
Long-context retrieval key, 20 values)
512
11,648-17,840
Hard
Math Choice
Multiple-choice math problems
2,500
40-425
Easy
DAPO Math
Mathematical reasoning
2,500
65-1,014
Hard
3.2.2 TRAINING RECIPE
We conduct LoongRL training on Qwenn..-77-IInttruet and Dwwen.55-14B-Instre both with 128K context window. The goals are (i) enhancing long-context reasoning through reinforcement learning, and (ii) preserving core abilities such as general short-context reasoning. To achieve this, we construct a mixed dataset (Table 1) and adopt a multi-stage RI. training strategy.
Training Length and Data Mix. As discussed in Sec. 3.1, KeyChain data effectively induces long- context reasoning patterns, enabling the model to generalize to longer contexts. To avoid the high cost of full 128K RL rollouts, train using 16K context length.
Table summarizes our training data sources, including their input context lengths and task dif- ficulty. Our dataset consists of four types. (i) High-difficulty KeyChain data is synthesized as described in Section 3.1, with 2,500 examples each from HotpotQA (Yang et al., 2018), MuSiQue (Trivedi et al., 2022), and 2WikiMuliHHoQAA (Ho et al., 2020), totaling 7,500 exam- ples. This set provides challenging examples that explicitly induce long-context reasoning. (ii) Medium-level multi-hop QA data consists of 2,500 examples from each of the same three datasets. These moderately difficult examples are especially important for smaller models (e.g., Qwen2.5- 7B-Instruct), enabling effective RL when the model initially struggles with harder KeyChain tasks. (iii) Long-context needle retrieval duta contains 1,024 synthetic examples designed to maintain the model's s ability to retrieve relevant information from long contexts. Each example uses a 16K-token book from PG19 as the base, into which multiple key-value "needles" are randomly inserted fol- lowing RULER(Hsieh et al., 2024), requiring the model to locate relevant values amid extensive distractors. (iv) Math data contains 5,000 short-context problems (<1K tokens) to preserve general short-context reasoning capabilities, including 2,500 hard problems from the DAPO training set (Yu al., 2025) and 2,500 easy multiple-choice questions from MATH (Hendrycks al., 2021).
Multi-Stage Training. Our reinforcement learning follows a three-stage curriculum. (i) Warm-up. We first train for one epoch the dataset excluding KeyChain data. Since KeyChain problems initially too difficult for small models, this stage allows the model to improve retrieval and general reasoning ability on easier data, ensuring stable optimization. (ii) Stage I (KeyChain augmentation), KeyChain data is then introduced to increase task difficulty, encouraging the model to plan effec- tively, retrieve precise information from distractor-heavy long contexts, and integrate evidence into coherent reasoning chains. (iii) Stage II (difficultyfocusee training). After Stage I, we generate eight rollouts per example using the best checkpoint. Examples solved correctly in all rollouts are discarded, leaving challenging subset (3-40%% of the data). RL continues on this subset, focusing updates on difficult cases to improve efficiency while avoiding overtraining on mastered problems.
EXPERIMENTS
4.1 SETUP
Training Setup. We run experiments two long-context instruction-tunnd models, Qwen2.5-7B- Instruct and 1-n-2.5--4B-Insrruue
Training uses GRPO using a group size G = 8 and a learning rate of le-6. Batch sizes are set to 512 for 7B model and 256 for 14B model. Rollouts are sampled with temperature 0.6 and top-p = 0.95, with a maximum output length of 4,096 tokens and long- context inputs of 16K. We adopt learning rate of 1 x 10-6 with cosine decay and gradient

---

Published as a conference paper at ICLR 2026
Table Results of LoongRL and frontier LLMs on long-context reasoning and general short tasks. LoongRL, delivers frontier-level long-context reasoning at much smaller scales (7B/14B), rivaling 03-mini and DeepSeek-RR while preserving general short-context abilities across all scales.
Long- Context Reasoning
General & Short Reasoning
Models
Avg. HotpoiQA 2WikMuuliHHoQAA MuSiQue NarrativeQA QASPER
Avg. MMLU
MATH
IFEval
03-mini (medium)
74.5
83.0
89.0
64.0
60.7
60.5
92.1
86.9
98.0
91.5
DeepSeck- RI
74.9
82.7
91.3
72.2
66.9
61.4
90.5
90.8
97.3
83.3
GPT-4o
64.7
82.5
78.0
54.0
60.5
48.5
82.5
88.7
74.6
84.3
QwQ-32B
69.6
78.5
87.4
62.7
61.1
58.5
85.9
75.7
98.0
83,9
R1 SIl-Distill-Lie 7OB
65.4
76.1
85.0
61.9
53.4
50.5
85.4
82.4
94.5
79.3
Qwen2.5-78 1002255-7--nnstuue
48.9
69.5
50.5
34,0
44,5
46.0
73.5
73.4
76.0
71.2
RI-Distill- 11-Distill-Qwen-77 Qwen-7B
31.2
40.2
53.3
11.1
8.9
42.5
69.9
62.3
92.8
54.7
LoougRL-7B
72.4
83.1
91.1
65.6
58.4
63.6
75.0
76.2
78.0
53.1
74,0
60.5
36.5
48.5
46.0
81.3
79.4
83,4
81.0
R1-Distill Saelll--wen--eee
14B
64.9
77.5
87.0
58.0
$1.0
$1.0
81.0
76.6
93.99
72.6
RI-Distill- Qwen-32B
65.5
76.3
87.6
59.8
52.7
50.9
82.4
80.5
94.3
72.5
QwenLong--L1332B
70.1
80.7
89.1
65.2
58.6
56.7
84.1
78.5
95.2
LoongRI-14B
74.2
82.2
93.3
67.5
63.4
64.5
80.7
80.5
83.2
78.4
Table 3: Results of LoongRL and frontier LLMs on the HELMET long-contedd generation bench- mark. LoongRL-7B and LoongRL-14B substantially outperform baseline models.
Model
Avg.
RAG
Generation with Citations
Summarization
NaturalQucstions
HotpotQA
PopQA
TrivisQA
ALCE ASQA
Qampari
BeBench Multi-LexSum
Qwvnn2.5-27 5-2-5--BBnnsccc
22.8
21.2
20.3
27.7
53.3
6.1
0.1
14,7
39.2
E1-Distill-ween-
LoongRL-7B
44,8
50.0
60.0
53.7
82.8
15.0
7,8
28.1
60.8
Qwen2 5-14t-Instte
40.5
45.3
81.7
24.1
27.7
50.4
RI-Disiil- Qwen- 14B
29.7
41.8
41.7
40.8
32.0
14.9
9.7
16.1
40.4
QwenLong-LL L1-g--1--5
LoongRL-144B 49.0
55.3
60.3
51.0
88.2
23.0
14.3
33.7
65.8
clipping at 1.0. For Qwen2.5-7B-Insttunl we apply the full three-stage RI. training: 42 steps in warm-up, 168 Stage and 118 in Stage For the larger Qwen2.5-144 1..5--1---nsstuc,, we skip warm-up stage since the model already possesses strong base abilities and can immediately handle Key ChyChain data. We train for 168 steps in Stage and 150 steps in Stage II. For the 7B model, we train on 16xA100 GPUs, while the 14B model is trained on 8xMI300X GPUs.
Evaluation Benchmarks. We evaluate LoongRL models across three dimensions. (i) Long- context reasoning: follow QwenLong-LI (Wan et al., 2025) and evaluate on multi-hov hop QA tasks in LongBench v1 (Bai et al., 2024), including HotpotQA (Yang et al., 2018), 2WikiMulti- HopQA (Ho al., 2020), MuSiQue (Trivedi et al., 2022)), NarrativeQA (Kočiský al., 2018) and QASPER (Dasigi et al., 2021), with input lengths from 4K to 64K tokens. We also evaluate evaluate on HELMET (Yen al., 2024), more challenging long-context generation benchmark. (ii) Gen- eral short-context reasoning: we use standard benchmarks including MMLU (Hendrycks et al., 2020), MATH-500 (Lightman et al., 2023), and the cnatuuctionfollowwiiii et al., 2023). (iii) Long-context retrieval: to measure the impact of long-contex, RL on retrieval abilities, we evaluate on Needle a Haystack (Kamradt, 2023) and RULER (Hsieh et al., 2024).
benchmark IFEval (Zhou
For inference, reasoning models and our models use temperature 0.6, with up to 128K input tokens and 10K output tokens. We sample eight solutions per problem and report average pass@ accuracy. Non-reasoning models (e.g., 1e-en..5-7B-Instrueee use temperature 0.
Baselines. We compare against three baselines: (i) leading frontier models, including o3-mini, GPT- 4o, DeepSeek-RI and QWQ-32B; (ii) state-of-the-ar models enhancing short-context reasoning on long-contex foundations, mainly RI -dsitille variants; and (iii) long-context reasoning models like the recent QwenLong- Matioonn----333 R1-32B, based on R R1-distill- diistill-Qwen-2221 Q2B and trained with 60K input context.
4.2 MAIN RESULTS
Competitive long-context reasoning at smaller scale. Table 2 summarizes the long-context rea- soning performance of LoongRL against state-of of-the-art models. We highlight two key observations: (i) LoongRL delivers frontier-level long-context reasoning at significantly smaller scales. Remark- ably, LoongRL-7B achieves an average of 72.4 on LongBench vl, surpassing all R RI-distilled mod- els and QwenIoong-LL1-322B At 14B, LoongRL. reaches 74,2, even rivaling the much larger, heavily
7

---

Published as a conference paper at ICLR 2026
Table 4: While being trained only on 16K, LoongRL generalizes impressively context up 128K.
NarrativeQA
RULER
Models
0-16K
16K-32K
32K-64K
16K
32K
64K
128K
Qwen2.5-7B-Instruuc
55.7
35.2
42.4
92.3
89.5
81.8
69.4
RI-Distill-Qwen-T1
55.7
35.2
42.4
18.9
4.4
1.4
0.9
LoongRI.-7B
69.8
47.4
57.2
93.4
91.4
86.2
76.8
Wwen2.5-14B-Instneee
55.7
40.7
93.4
92.5
82.3
73.6
RI-Distill- Soalll-wenn--4e1 14B
63.0
35.9
54.6
85.7
82.0
60.2
28.2
RI-Disiill-Qwen-322
Qwen-32B
57.4
44.4
58.9
90.3
88.9
71.5
40.9
OwenLong-L1-328
65.9
48.1
60.0
87.6
86.8
80.6
70.2
LoongRL-114B 69.5
55.2
64.3
95.4
95.1
87.1
79.9
trained 03-mini (74.5) and DeepSeck-RI (74.9). (ii) Our KeyChain-drivent RL proves far more effee- tive than existing methods. It improves Qwen2.5-7BBInsstru and Qwen2.5-14B 1i--tututnttt by +23.5% and +21.1 %, respectively. In contrast, RI -distille Qwen models, trained on long-CoT reasoning data, yield modest +1 1188% gain at 14B and even degrade 7B performance by -17.7%. Similarly, QwenLong-LL1-22
trained via conventional long-context RL on ww-diitill-Qwe--3222 improves by just +4.6% on average. Notably, LoongRL-7B even outperforms QwenLoon-LLI-321 by +2.3%, demonstrating that much smaller models can surpass larger baselines with our approach.
Table 3 compares the performance HELMET, challenging benchmark that includes out-domain long-context generation tasks such as RAG, citation-grounder generation, and long- Long-document sum- marization. Our models consistently outperform all baselines, further demonstrating that the rea- soning patterns induced by LoongRL generalize effectively to diverse long-context scenarios.
Training at short, generalize better long. The strong results in Table are largely driven by thinking pattern. Although trained on 16K input contexts, this patterns generalizes effectively to much longer con- texts. As shown in Table 4, both LoongRI--BB and 14B achieve substantial gains on longer-context reasoning and retrieval benchmarks. On NarrativeQA -26K), they achieve impressive absolute gains of +14.8% and +16.0% respectively, far exceeding RI Iddistilled models and QwenLong-Li 32B, which are trained with much longer contexts. On the RULER benchmark (up to 128K), while other baselines degrade sharply with increasing context length, our models maintain consistently strong performance, showing that the learned reasoning pattern transfers robustly to longer contexts.
KeyChain data, which enables our models to acquire plan- plan-retrieee-rrasonn
Near-lossless general short reasoning. Table 2 also reports LoongRL's performance short- context reasoning and general tasks, showing that effectively preserves the base models' capabil- ities. On MMLU, LoongRL. even yields gains of +2.8% (7B) and +1.1 % (14B). In contrast, both Rl-distilled models and QwenLong-LI-322 suffer performance drops. On instruction following (IFEval), RI I-distilled models degrade sharply (-16.5% 7B, -8.4% 14B), while LoongRL shows only minimal declines (-0.3% and -2.6%). For math reasoning, although R Rdddisileeed models benefit from heavy long-Cor CoT data distillation, our approach stably preserves the base models' math ability.
Improved long-context retrieval. We evaluate the impact of different approaches on retrieval using the Needle in Haystack benchmark, which measures model's ability find "needles" from long documents at varying depths. As shown in Fig. 3, the base Dwwen.5--7B-Instrus fails to fully pass this benchmark. In contrast, our LoongRL improves retrieval substantially, with LoongRL-7B achieving perfect accuracy across all depths. Other approaches remain limited, with RI RI-Distill-BB unable to retrieve beyond 20K and even the larger Swennong-LLl-33 failing to achieve a full pass.
4.3 ABLATION STUDY
Multi-stage RL training sustains improvements. understand how LoongRI. achieves strong performance, report step-by-step gains and average training lengths for 7B and 14B across the three RL stages. As shown in Fig. 4 (c.d), average response length steadily increases throughout training. Fig. (a.b) presents long-context reasoning accuracy, which grows consistently across each stage, demonstrating the effectiveness of the multi-stage RL. curriculum.
Ablation the KeyChain data. Our KeyChain training data effectively encourages models to acquire new long-context reasoning patterns during RL. To evaluate its effectiveness. we replace it with equal amount of regular long-context multi-hop QA data Qwen2.5-7BBInsttune while
8

---

Published as a conference paper at ICLR 2026
RL-
DeepSeelk Distill Qwen-78 Disttil--Qwe--BB
0-
Depth Percent (%g
22-
22
29-
23
44-
44
08
56-
89
-22
-02
2.00 1ork
200e
1200
201
210k
Context Lengtn
Contoxt Length
(won2.5-7B.nnstruu
Depth Percent (%)
22-
2.8
22
08
38-
44-
67-
-944
-04
76
78
89
101
125
Context Length
Length
Figure 3: Needle in Haystack retrieval across document depths. The base Qwen2.5-7BB-nnttmu.
does not fully pass the benchmark, whereas LoongRL-7B achieves perfect 100% retrieval accuracy.
(a) LoongRL-7B: LB1 Score
(b) LoongRL-148B LB1 Score
0.74
LongBengh) Avg. Score
0.72
0.70
Ammm
0.68
0.66
0.64
Warmup
0.62
Stagel
Stagel
Stage2
Sta ge2
0.60
50
100
150
200
250
300
350
100
150
200
250
són
350
Steps
Steps
(c) LoongRL-78: Response Length
(d) LoongRL-14B. Response Length
1400
Resporse Length
2000
800
600
Warmup Stagel Stage2
400
Sta geg
200
50
160
150
200
250
300
350
s0
1000
15.0
200
250
3án
350
Steps
Steps
Figure 4: Long-context reasoning accuracy and training response lengths throughout RL training.
keeping all other RL settings identical. Table shows the comparison results. RL with regular QA data yields moderate gains (66.2), whereas incorporating KeyChain data drives substantial leap to 72.4, reaching frontier-level performance.
Moreover, as shown in Fig. 1(b), models trained with regular long-context multi-hop data exhibit mixed reasoning-withh rett-vet pattern. They often lack an explicit planning step and do not perform careful reason over the retrieved information, making them more prone errors. This demonstrates that KeyChain not only significantly enhances long-context reasoning but also unlocks capabilities that cannot be achieved with conventional QA data, highlighting its unique and critical role.
Ablation on the answer verifier. To evaluate our two-way substring exact match for verifying answer correctness, compare it with three widely used baselines Qwen2.5-7B. I-strruct (i) Fl score between extracted answers and the ground truth (Shi et al., 2025; Chuang et al., 2025); (ii) LLLM-as-a-judee using DeepSeek-Vy V3 to assess consistency with the ground truth; and (iii) ex-

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study on the effectiveness of KeyChain data.
Models
HotpoiQA
2WikiMuhiHopQA
MuSiQue
Narrative
QASPER
Avg.
Qwen2.5-7B-mmg
69.5
50.5
34.0
44.5
46.0
48.9
LoongRL-7B (no KeyChain Data)
80.3
84.7
58.5
53.0
54.5
66.2
LoongRL-7B
83.1
91.1
65.6
58.4
63.6
72.4
Table Ablation study on the different answer verifiers on the 7B.
Reward Verifier
HotporQA 2WikiMultHHpQA MuSiQue NarrativeQA QASPER
Avg.
Fl score
79.5
86.4
58.0
46.6
55.0
65.1
LLM-as-a-judge
80.0
87.6
60.0
52.3
54.5
65.2
Exact match
82.7
91.3
66.3
51.0
54.9
69.2
Two-way Substring Exact Match (ours)
83.1
91.1
65.6
58.4
63.6
72.4
act match, requiring extract answer to match the ground truth exactly. As shown in Table 6, Fl and LLM-as-a-judge yield moderate gains, while exact match performs better but is overly strict, penalizing essentially correct answers with minor formatting differences. In contrast, our two-way substring exact match maintains high precision while allowing variations, boosting long-context reasoning scores to 72.4 and clearly demonstrating its practical reliability for RL training.
5 CONCLUSION
This work introduces LoongRI, data-driven reinforcement learning approach for advanced long- context reasoning. By creating a novel dataset, KeyChain, which transforms standard multi-hop questions into high-difficulty tasks, LoongRL trains models develop a "plan-retrieve-reason- recheck" " thinking pattern. A key finding is that this emergent reasoning ability generalizes re- markably well. Models trained on 16K token contexts can effectively solve tasks up 128K tokens. Our resulting LoongRL-144 14B model achieves 74.2 score on long-contex, QA benchmarks, rivaling much larger frontier models like 03-mini and DeepSeek-R1. These significant gains are achieved while successfully preserving the model's crucial short-context reasoning and retrieval capabilities.
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility of our work. Details the GRPO al- gorithm and hyperparameters are provided in Section 3.2.1 and Section 4.1. We provide training prompt template in Appendix A.2. The datasets used in our experiments are described in Table 1. To further facilitate reproducibility, the supplementary materials include (i) our RL training code, (ii) the code for synthesizing KeyChain data, and (iii) several representative samples of the synthe- sized KeyChain data. These resources, together with the descriptions in the main text and appendix, provide all necessary information for replicating our results.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXivv 08774, 2023.
Wasi Uddin Ahmad, Sean Narenthiran, Somshubra Majumdar, Aleksander Ficek, Siddhartha Jain, Jocelyn Huang, Vahid Noroozi, and Boris Ginsburg. Opencodereeasoning: Advancing data distil- lation for competitive coding, arXiv preprint arXiv:2504.01945. 2025.
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu. Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3119-3137, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 18653/v1l 2024.acl-long. 172. URL https: /ttp//clln/ttttt
org/ 2024 aal- 172.
Yung-Sung Chuang, Benjamin Johen-Wang, Shannon Zejiang Shen, Zhaofeng Wu, Hu Xu, Xi Vic- toria Lin, James Glass, Shang- Wen Li, and Wen-tau Yih. Selfcite: Self-supervised alignment for context attribution in large language models. arXiv preprint arXiv:2502.09604 2025.
Pradeep Dasigi, Kyle Lo. Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. A dataset of information-sccking questions and answers anchored in research papers. In Kristina Toutanova,
10

---

Published as a conference paper at ICLR 2026
Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Tur, Iz Beltagy, Steven Bethard, Ryan Cot- terell, Tanmoy Chakraborty, and Yichao Zhou, editors, Proceedings of the 2021 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, pages 4599-4610, Online, June 2021. Association for Computational Linguis- tics. doi: 10. (0.18653/v1/2000 maicl-m URL ht.ps: /ac////clllc. aaclllnhhhlggg org/2021. naacl menc--maing 365/
Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang. Jiahang Xu, Fan Yang, and Mao Yang. Longrope: Extending IIm context window beyond 2 million tokens, 2024.
URL http:::///// /ttpavaaiiiii.////a//ee.eaaaa. org/ abs
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cogni- tive behaviors that enable self- self-improving reasoners, or, four habits of highly effective stars. arXiv preprint ar arXiv:2503.0/300 2025.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint ar arXiv:2103.03877 2021.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi- hop QA dataset for comprehensive evaluation of reasoning steps. In Donia Scott, Nuria Bel, and Chengqing Zong, editors. Proceedings of the 28th International Conference on Compu- tational Linguistics, pages 6609-6625, Barcelona, Spain (Online). December 2020. Interna- tional Committee on Computational Linguistics. doi: 10.18653/ 580. URL https: //tcl/acccccc antho Molcoloogp ogy. - org/ wtt.0.00........eeo.e..e.....e cowlnngnaaaa wammmainn.. 580/
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. RULER: What's the real context size of your long-contex language models? In Proceedings of the 1st Conference on Language Modeling (COLM), Philadelphia, PA, 2024. URL https:ttp///..c...t.. / aaaaiiivivv..///vv./////////vi.m org/ abs/ 220../6644.....
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameey, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li. Charles de Bourcy, Chelsea Voss, Chen Shen. Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan. Dan Roberts, Daniel Kap- pler, Daniel Levy. Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras. Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Eliz- abeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas. Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclere, Hadi Salman, Haiming Bao, Hao Sheng, Hart An- drin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman. Hyung Won Chung, Ian Kivlichan, Ian O'Connell Ian Osband, Ignasi Clavera Gilaberte, and Ilge Akkaya. Openai ol system card. CoRR, abs/2412.16720 2024. URL https: / //oii 16720.
0.4855.//8rrii. 2412.
Greg Kamradt. Needle in a haystack pressure testing Ilms, 2023. URL https: httpt://ttthtiteeee //thhbb. com/ comm/gkaradt... /tt.aadddd.......ttt. wtW.Mest. Need1 e InAaaystacHa

---

Published as a conference paper at ICLR 2026
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang. Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang, Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang. Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haozhen Yu, Hongcheng Gao, Huabin Zheng. Huan Yuan, Jia Chen, Jianhang Guo, Jianlin Su, Jianzhou Wang, Jie Zhao, Jin Zhang, Jingyuan Liu, Junjie Yan, Junyan Wu, Lidong Shi, Ling Ye. Longhui Yu, Mengnan Dong, Neo Zhang, Ningchen Ma, Qiwei Pan, Qucheng Gong, Shaowei Liu, Shengling Ma, Shupeng Wei, Sihan Cao, Siying Huang, Tao Jiang, Weihao Gao, Weimin Xiong, Weiran He, Weixiao Huang, Weixin Xu, Wenhao Wu, Wenyang He, Xianghui Wei, Xianqing Jia, Xingzhe Wu, Xinran Xu, Xinxing Zu, Xinyu Zhou, Xuehai Pan, Y. Charles, Yang Li, Yangyang Hu, Yangyang Liu, Yanru Chen, Yejie Wang, Yibo Liu, Yidao Qin, Yifeng Liu, Ying Yang, Yiping Bao, Yulun Du, Yuxin Wu, Yuzhi Wang. Zaida Zhou, Zhaoji Wang, Zhaowei Li, Zhen Zhu, Zheng Zhang, Zhexu Wang. Zhilin Yang. Zhiqi Huang, Zihao Huang, Ziyao Xu, Zonghan Yang, and Zongyu Lin. Kimi k1.5; Scaling reinforcement learning with Ilms, 2025. URL https:: / aaxiv. or org/ abs / 2
Tomášs Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The NarrativeQA reading comprehension challenge. Transactions of the Association for Computational Linguisties, 6:317-328, 2018. doi: 10.1162/tacl a 00023.
URL hittps://act antool loy org/ /orrg//116 8- 1023/.
Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, al. Minimax-01: Scaling foundation models with lightning attention. arXiv preprint arXiv:2501.0831.3 2025.
Huayang Li, Pat Verga, Priyanka Sen, Bowen Yang. Vijay Viswanathan, Patrick Lewis, Taro Watan- abe, and Yixuan Su. ALR?: A retrieve-then-reason framework for long-context question answer- ing. arXiv preprint arXiv:2410.0322 2024a.
Siheng Li, Cheng Yang, Zesen Cheng, Lemao Liu, Mo Yu, Yujiu Yang, and Wai Lam. Large language models can self-improve in long-context reasoning. arXiv preprint a.X0422411.08147, 2024b.
Yanyang Li, Shuo Liang, Michael Lyu, and Liwei Wang. Making long-context language models better multi-hop reasoners. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2462-2475, Bangkok, Thailand, Au- gust 2024c. Association for Computational Linguistics. doi: 10.
135.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint wwXiv:2305.20050. 2023.
Zhan Ling, Kang Liu, Kai Yan, Yifan Yang, Weijian Lin, Ting Han Fan, Lingfeng Shen, Zhengyin Du, and Jiecao Chen. Longreason: A synthetic long-context reasoning benchmark via context expansion. arXiv preprini 2rXiv:2501.1508) 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 v3 technical report. arXiv preprint 14X44 2024.
Yifei Liu, Li Lyna Zhang, Yi Zhu, Bingeheng Dong, Xudong Zhou, Ning Shang, Fan Yang, and Mao Yang. rstar-coder: Scaling competitive code reasoning with a large-scale verified dataset. arXiv preprint MrXXi::5505.21999 2025.
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yam: Efficient context window extension of large language models. arXiv preprini arXiv:2309.0007/ 2023.
Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https: //weelmm. tttthu.../bbogg. qwen2. 5/.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383 2392,, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10. 10.18653/v//D1--11 1264.
12

---

Published as a conference paper at ICLR 2026
Ning Shang, Yifei Liu, Yi Zhu, Li Lyna Zhang, Weijiang Xu, Xinyu Guan, Buze Zhang, Bingcheng Dong, Xudong Zhou, Bowen Zhang, Ying Xin, Ziming Miao, Scarlett Li, Fan Yang, and Mao Yang. rstar2-agent: Agentic reasoning technical report, 2025a. URL https: /arxiv..rg//.///./........... abs/2508.0 20722.
22022
Ning Shang, Li Lyna Zhang, Siyuan Wang, Gaokai Zhang, Gilsinia Lopez, Fan Yang, Weizhu Chen, and Mao Yang. Longrope2: Near-lossless Ilm context window scaling, 2025b. URL https:
//aa/aaaaaa //aaxiv..rrg org. aww / 2502 20082.
Yaorui Shi, Sihang Li, Chang Wu, Zhiyuan Liu, Junfeng Fang, Hengxing Cai, An Zhang, and Xiang Wang. Search and refine during think: Autonomous retrieval-augmented reasoning of llms. arXiv preprint arXiv:2505. 1277, 2025.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multi- hop questions via single-hop question composition, 2022. URL https: ptttppp://axiiiii org/abs/
2108. 00573.
Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi. Chenliang Li, Ziyi Yang, Ji Zhang. Fei Huang, Jingren Zhou, and Ming Yan. Qwenlong-ll: Towards long-context large reasoning models with reinforcement learning, 2025. URL httpt::tttaaaaaee http://axxxvv....e./// /b550555 17667.
Shenzhi Wang, Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for Ilm reasoning. 2025a. URL https://axxvv.org/... 2506. 01939.
Yiping Wang. Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng. Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv:2504.2000
2025b. URL httpt::t////// ///x//axivvv//// org/ abs//b///554/ 20571.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 27X.v22505.09388, 2025.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun ichiciii Tsujii, editors, Pro- ceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium, October-November 2018. Association for Computational Lin- guistics. doi: 10.18653/v1/D1.. 1/55- URL https: /ttpaaaanannnttccccc//.a D08
Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, and Danqi Chen. Helmet: How to evaluate long-context language models effectively and thor- oughly. arXiv preprint wx.vv:2410.026444 2024.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guang- ming Sheng, Yuxuan Tong, Chi Zhang. Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen. Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingx- uan Wang. Dapo: An open-source Ilm reinforcement learning system at scale, 2025. URL https://arxiss wwwpii..rr//bbs///500/ 14477
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-followinn evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 USE OF LARGE LANGUAGE MODELS IN PAPER WRITING
In this work, we used large language models (LLMs) solely as general-purpose tools. Specifically, we employed LLMs to improve the clarity and readability of the paper. Additionally, during our ablation experiments, we evaluated the effectiveness of the answer verifier by using DeepSeek-V V3 as the baseline in an aLde setting.
14

---

Published as a conference paper at ICLR 2026
A.2 TRAINING PROMPT TEMPLATE
For reproducibility, we include the exact prompt format used during training (see Fig- ure 5). The model was trained to first generate intermediate reasoning enclosed in <think> </think>, and then provide the final answer enclosed \boxed() }
System Prompt
conversation between User and Assistant. The User asks question, and the Assistant solves it. The Assistant first thinks about the reasoning process in the mind and then provides the User with the answer. The reasoning process is enclosed within <think> </thi nk> and answer is enclosed within \boxed() tags, respectively, i.c., <thinks reasoning process here </think nh \boxed answer herel.
Figure 5: System prompt used during training
A.3 EXAMPLE OF KEYCHAIN-AUGMENT TRAINING DATA
Figure shows sample skeleton of training data, illustrating how carry out the Key-Chain augmentation for long-context reinforcement learning.
Please read the following text. Document 0:
Document 3:
Who's Who? is studio album by American jazz musician John Scofield. It features two different bands, one acoustic and one electric. The acoustic group, featuring Soofield's then-employer Dave Liebman on saxophones, Eddie Glu00f3mez on bass, and Billy Hart drums, recorded "The Beatles' and "How the West Was Won'
Bbodd--0rbb0066-4add 1a177-44--11-9----- 1-11-90-9111118 17fa3b 1799d": (eeeee9999444------- 46dd--beb1-3994466668653838333 (e0b1-330066882577
Document 10:
university is one smallest 23 CSU campuses California. Sonoma State offers Bachelor's degrees, 19 Master's degrees, Doctoral degree (Doctor Education), and 11 teaching credentials. (*972a8469- 164 44882-8b9d. 2434s465e 150": "Musiclan and satirist Allle Goertz wrote song about the "The Simpsons" character Milhouse, who Matt Groening named after who?")
Document 47:
Affleck
(e33d-11s7bc8960:ee 1972a8469-164 1-4f82-8b9d (1182-bb9d22334eee 243-2244444455500
Neil Affleck (born 1953) is a Canadian animator, director, and former actor. He has worked as an animator on "The Simpsons" "My Bloody Valentine", (" 1e644-88 5e-46d1-
'Family Guy', and actor appeared leading
bb8f-sa (oB8-sa113718Baa 837 "674d0/b 1od0fb1-33 1-32e7 22297-4629-8aal 3fad-c1a606cb00bb3y
context above, there one correct question answer. The correct question can only be found by following correct consecutive chain of keyivalue pairs encoded with UUID strings (e.g., dasffa---do--1100-- 0Da0c9 1eBb16), starting from 2dd640fb-066 (ttt--666-44a-----0 (add1-90800311//00 71a3311 799d".
Find the correct question first, then answer it.
Figure 6: A skeleton of KeyChain-augmenter training data used in LoongRL.
15

---

Published as a conference paper at ICLR 2026
A.4 COMPARISON BETWEEN TRACES OF KEYCHAIN VS. NON- MON-KEY MODELS
Here we contrast two models trained with same hyperparameters but different data. The KeyChain- augmented model shows structured. step-by-step reasoning (colored as green) and clear re-checking when ambiguity arises, while the non-Key KeyChain model often shorteuts retrieval and proceeds to reasoning prematurely, leading to emrrs (colored red). The following traces illustrate these dif- ferences on the same question:
Question: Gary L. Bennett was part of the space missions that have primary destination of what celestial body?
KeyChain-trained Model
To determine the primary destination of the space missions that Gary L. Bennett was part of, we need to he worked on from the given passages and then find the primary destination of those
identify the space missions.
From Passage 1, we can see that Gary L. Bennett worked on the following space missions: "His professional career included work on Voyager, Galileo, and Ulysses space missions." Now, we need to find the primary destination of these missions. Let's check Passage for information about the Ulysses mission:
"Ulysses yoo-LLSS-ecz. UK also YOO-liss-ecz) was robotic space probe whose primary mission was orbit the Sun and study it at all latitudes."
Therefore, the primary destination the Ulysses space mission is the Sun.
Although we should check the other missions mentioned (Voyager and Galileo) for completeness, we from Passage 3:
transit Earth visible from Mars on 11 May 1984 and the Viking missions had been terminated year previously. Consequently, the next opportunity
"No missions were planned to coincide
observe such an alignment will be in 2084." "On 21 December 2012. the Cassini-Huygen Huygens probe, in orbit around Saturn, observed the planet Venus transiting the Sun." " "These Schiaparelli EDM lander, the Mars Polar Lander, and Deep Space 2. These were all sterilized for surface missions but not for special regions (Viking pre-steriizzatio only). Mars Polar Lander, and Deep Space 2 crashed into the polar regions which are now treated as special regions because of the possibility forming liquid brines."
And from Passage 4: "Ulysses (spacecraft)" "The spacecraft was originally named Odysseus, because of its lengthy and indirect trajectory to study the solar poles. It was renamed Ulysses, the Latin translation of "Odysscus". ESA's request honor not only of Homer's mythological hero but also of Dante's 's character in the Inferno. Ulysses was originally scheduled for launch in May 1986 ahoard the Space Shuttle Challenger on STS-61-F. Due the 28 January 1986 loss of Challenger, the launch of Ulysses was delayed until October 1990 aboard Discovery (mission STS-41)."
The Voyager mission is known to have a primary destination of the outer planets, specifically Jupiter, Saturn, primary destination of Jupiter.
Uranus, and Neptune. The Galileo mission However, the question specifically asks about the space missions that Gary L.. Bennett
part of, and from the information provided, the Ulysses mission is clearly stated to have the Sun as its primary destination.
Therefore, the primary destination the space missions that Gary L.. Bennett was part of, specifically the Ulysses mission. is the Sun.
the Sun
No KeyChain Model
(Lack of systematic decomposition of the question)
First, we need to identify the space missions mentioned in the passage that Gary L. Bennett was involved with. From the biography provided, we see that Bennett was involved several space missions, including the Voyager. Galileo, and Ulysses space missions. The question asks for the primary destination of these missions. For the Voyager mission, the passage states: "He was the flight safety manager for the radioisotope power sources currently in use Neptune and beyond)." For the Galilen mission, it mentions: "Prior to coming to NASA, Bennett held key positions in DoE's space radioisotope power program, including serving Director of Safety and Nuclear Operations for the radioisolope power sources that were used on the Galilco mission to Jupiter and that are heing used on the Ulysses mission to explore the polar regions the Sun."
Voyager and Voyager 2 spacecruft (which went Jupiter, Saturn, Uranus,
For the Ulysses mission, the passage states: "Ulysses -eo-LLSS-ezz. UK also YOO-liss-eez)
robotic
space probe whose primary mission was to orbit the Sun and study it at all latitudes."
From these statements, we can deduce that Bennett was involved missions to Jupiter, Saturn, Uranus, Neptune, and the Sun. However, the primary destinations mentioned directly in relation to Bennett's roles are Jupiter for the Galileo and Ulysses missions, and multiple planets including Jupiter, Saturn, Uranus, and Neptune for the Voyager missions.
Given the specific mention of Bennett's 's role in the Galileo mission to Jupiter, and the broader statement about the Ulysses mission studying the Sun, Jupiter stands out as a primary destination that Bennett was directly involved with
Jupiter
16

---

Published as a conference paper at ICLR 2026
A.5 TRAJECTORY EXAMPLES ON LONG-CONTEXT RETRIEVAL
We show trajectories to highlight the impact of KeyChain data on general long-context retrieval. In the Ruler VT examples in A.S, the KeyChain-traiie trained model demonstrates clearer, stepwise retrieval, while the non-Key thain-trrinned model conflates intermediate variable assignments and propagates errors, showing KeyChain data's role in eliciting structured and correct retrieval behaviors.
RL Trajectory Example with KeyChain Data: Long-Coon Context Retrieval on Ruler VT
<think>> However, the specific value 92018 is only directly assigned Let's break it down step by step:
variable in the text provided.
1. The first assignment in the text is: "VAR QPE = 64886 The grass is green. The sky blue. The sun yellow. Here we go. There and back again." " This assigns the value 64886 the variable QPE.
2. The next assignments form chain: "VAR SEJ VAR QPE The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again." This assigns the value of QPE (which is 64886) to the variable SEJ.
"VAR ZQO VAR SEJ The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again." . This assigns the value of SEJ (which is 64886) to the variable 2Q0.
4. "VAR RVU = VAR ZQO The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again." This assigns the value of ZQO (which is 64886) to the variable RVU
is yellow. Here go. There and back again." This assigns the value of RVU (which is 64886) to the variable FAI.
"VAR FAI VAR RVU The grass is green. The sky is blue. The
6. The value 92018 is assigned to the variable SGMLJ: "VAR SGMLJ = 92018 The grass is green. The sky blue. The sun yellow. Here we go. There and back again."
7. Theen forms new chain: "VAR PBDME = VAR SGMLJ The grass is green. The sky is blue. The is yellow. Here we go. There and back again." This assigns the value of SGMLJ (which is 92018) to the variable PBDME.
"VAR EANSM VAR PBDME The grass green. The sky is blue. The sun yellow, Here we go. There and back again." " This assigns the value of PBDME (which is 92018) the variable FANSM.
9. "VAR QPKBX = VAR EANSM The grass is green. The sky is blue. The sun is yellow. Here we go. There and back again." This assigns the value of EANSM (which 92018) to the variable QPKBX.
10. "VAR YYZJM VAR FANSM The grass green. The sky blue. The sun is yellow. Here go.
There and back again." " This assigns the value of EÄNSM (which is 92018) to the variable YYZJM.
So, the variables that are assigned the value 92018 are: SGMLJ, PBDME, EANSM, QPKBX, and YYZJM. </hink>>
<think>> The variables that are assigned the value 92018
1. VAR SGMLJ = 92018 2. VAR PBDME = VAR SGMLJ 3. VAR EANSM = VAR PBDME 4. VAR QP-
KBX = VAR EANSMS. VAR YYZJM = VAR EANSM Thus, the variables are SGMLJ, PBDME, EANSM, QPKBX, and YYZIM. </thin>>
(boxed(SGMLJ, PBDME, EANSM, QPKBX. YYZIM)
RL. Trajectory Example WITHOUT KeyChain Data: Long-Conter Retrieval on Ruler VT
The reasoning process involves identifying the lines where the value 92018 is first assigned to a variable, which is "VAR PBDME = 92018". Then, tracking this value through all subsequent assignments where this variable is reassigned other variables. The variables that inherit this value through the assignments are EANSM, YYZJM, SGMLJ, and FAI. Since the question asks for variables assigned the value 92018, and FAI is assigned 92018 through the chain VAR PBDME = VAR SGMLJ, VAR SGMLJ = 92018, and so on, the final list of variables PBDME, EANSM, YYZJM, SGMLJ, and FAI. </think> (BDMME/PBDME, EANSM, YYZIM, SGMLJ, FAI)
<think>
A.6 LOONGRL TRAINING DYNAMICS
Fig. 7 and Fig. 8 show the training dynamics for the 7B and 14B models, respectively. Unlike the 7B setting, the 14B model is trained using only two curriculum stages, distractor and hard-mined, skipping the initial warm-up due to its strong base capabilities. As shown in two figures, a similar cyclical trend emerges: counts of consistently correct predictions and average rewards rise within each stage, reset when transitioning
harder task pool, and increase again. This multi-staee training continues to provide informative learning signals and prevents saturation, while the steady growth in response length indicates that the model persistently extends its retrieval and reasoning chains in long-context multi-hop reasoning.
A.7 VON
We evaluate the LoongRL series and baseline models on the LongBench-ve v2 benchmark. For models without long cfain-of-thought (Long Co'T) reasoning capability, we follow the original LongBench-
17

---

Published as a conference paper at ICLR 2026
Warmup (1 epoch 42 steps)
Stagel (4 epoch 168 steps)
Stage2 (9 epoch 117 steps)
Consistent Pre ctions per Batch
Average Reward
Average Response Length
400
MMMM
1400
Stage2
350
Stage2
Augrage Reward
0.6
Stage2
Resporse length
1200
Stagel
Stagel
Stage1
1.000
g300
800
@250
Warmup
600
200
Wammup
Warmup
400
150
200
Step
Step5
200
Steps
200
Figure 7: Training metrics for the three-stage schedule. Vertical dashed lines mark the transitions Warmup Stage Il diistractor-augmeenii and Stage - Stage III (hard-mined); stage lengths correspond to our setup (about 42, 168, and 117 steps).
Stage1 (2 epochs: 168 steps)
Stage2 (3 epochs; 150 steps)
Consistent Predictions per Batch
Average Reward
Average Response Length
1200
200
Stage2
Stage2
Stage2
80.5
E1000
160
g0.4
Salve
140
Stagel
80.3
Stage1
Ressponses
800
90.2
600
120
Stage1
0.0
400
Steps
Steps
Steps
Figure 8: Training metrics for the LoongRL-14B's two-stage schedule. Vertical dashed lines mark the transitions Stage I (distractor-aute ddistacctor-augmenneed) and - Stage III (hard-mined); stage lengths corre- spond our setup (about 168, and 150 steps).
Table 7: Comparison LoongRL models with other baselines on the LongBench-ve benchmark, grouped by Difficulty, Length, and Task Type.
Model
Overall
Diffccllty
Length
Task Type
Easy
Hard
Short
Medium
Long
Long ICL
Long SDU
Code SingleDoc QA Long Dialogue MultiDoc QA
56.1
40.6
GPT-40
48.4
44.4
50.0
QwQ-31B
RI-Destil-LLaMa-70B
46.2
O.een_5.7B-nnstrnn
RI-DistillQOwen-TB
27.0
29.2
25.7
30.6
23.7
27.8
21.0
18.2
32.0
25.1
33.3
32,0
LoongRL-7B
36.2
41.1
33.1
40.6
34.4
32.4
35.8
39.4
44.0
38,9
59,0
21.6
Q.ee Instructt
35.3
34.9
33.5
43.3
32.6
27.1
33.8
33.3
32.0
38.3
35.9
33.6
R1 RI-Distil-Qwenn Qaan---wnnBBB 148
36.2
40.2
33.8
44.1
31.4
32.6
36.8
36.4
28.4
38.4
44,1
33.4
RI-Distill- Distill-Qwae Owen 32B
38.6
40.1
37.6
48.9
33.5
31.5
29.6
39.4
38.0
39.4
51.3
39.2
OwennLong-L1-320
40.8
46.4
37.1
52.4
35.7
31.5
37.0
32.7
43.2
10.0
55.1
41.0
L4ongR1-14B 14B
42.3
46.4
39.9
44.4
43.3
37.0
39.5
45.8
38.0
44,0
59.0
37.6
v2 CoT setting, using a temperature of 0.1, sampling five responses per query, and reporting the average score across them ()vg@5). For models with Long CoT reasoning ability, we instead adopt a temperature of 0.6, again sampling five responses and reporting Avg@5 5. addition, the Qwen family and our LoongRL models, apply the YaRN method to extend the context length to 128k tokens. The overall comparison results are summarized Table
A.8 RULER
We evaluated the retrieval capabilities of models on long-text tasks using the RULER benchmark. For models without long-context reasoning abilities, we followed the original RULER setting by appending prompt suffix designed to guide the model to produce completion-style answers, e.g. "What is the special magic number for wandering- age mentioned in the provided text? The special magic number for wandering-age mentioned the provided text is". . In contrast, for models capa- ble of long-context reasoning, removed this completion-style suffix, preliminary experiments
18

---

Published as a conference paper at ICLR 2026
Table 8: RULER benchmark results across different context lengths. Qwen2.5 model series, we report their YaRN variants for 64k and 128k.
For QwQ QwenLong.
Model
4k
8k
16k
32k
64k
128k
Avg.
03-mini
96.58
96.85
94.69
90.85
74.81
65.40
86.53
DeepSeek-RI
98.46
97.98
97.18
96.06
94.92
85.10
94.95
GPT-40
97.69
96.73
96.73
96.02
94.46
89.10
95.12
QwQ-32B (YaRN@644/128k)
89.10
86.46
83.84
78.42
64.72
59.68
77.37
RI-Distil-LLaaMa-006
94.89
95.60
93.75
89.60
79.65
0.00
75.58
1lama3.1-70B-mm
96.78
96.64
95.82
94.87
89.21
64.53
89.64
R
RI-Disiil-LLaMM--88
83.89
79.80
73.77
64.46
51.06
1.28
59.04
-ama3.1-8B-Instme
96.10
93.81
90.91
86.73
84.77
74.15
87.75
Owen2.5-7B-Instruee
(YaRN@64k/128k)
95.16
93.73
92.31
89.46
81.79
69.41
86.31
RI-Disiill-Qwen-7B
65.70
48.29
18.86
4.38
0.88
23.25
LoongRL-7B (YaRN@644/1288)
95.06
94.34
93.37
91,36
86.18
76.84
89.53
Owen2.5-14B-Instmuu 2Yt.a/innssmc\aoaNReeG6/6///2200 (YaRN@644/128k)
96.27
95.11
93.38
92.53
82.33
73.57
88.86
RI-Distill-Qwnn-14B
91.44
86.29
85.73
82.00
60.24
28.23
72.32
Ri-Distill-Qwen-32B
93.61
91.64
90.27
88.90
71.51
40.88
79.47
QwenLong-LI-32B (YaRN (14K/128)))
91.71
88.51
87.55
86.81
80.64
70.19
84.24
LoongRL-14B (YaRN 64K/128K)
97.56
96.14
95.36
95.11
87.14
79.92
91.87
indicated that these models tend not to provide direct completions but instead perform explicit rea- soning before answering. After removing the suffix, we allowed the reasoning-capate models to generate up to 8192 tokens and subsequently extracted the model's answer from the text following the "</think>" token in its output.
A.9 NEEDLE-IN-A HAYSTACK
We further evaluated the trieval ability of models in extremely long-text settings. Figure 9 reports the performance of our LoongRL-1444 14B model. Results demonstrate that LoongRL-14B maintains strong retrieval accuracy across extended context lengths, showcasing its robustness for long-context information retrieval.
(NIAH) task, which specifically measures the
Loong
0
1.0
11
Depth Percent
33
44
0.6
56
-0.4
78
89
0.0
10x
19k-
20c-
37x"
4GK
55x
8.3
92K
10
11ok
119K
128k
Context Length
Figure 9: Needle-in-a-Haystecl performance of LoongRL-14BB.
A.10 UUID ABLATION
Table 9: Analysis of the design choice of KeyChain data. The RL experiments are conducted on Qwen2.5-7B-Insttee
KeyChain data
Avg.
HotpoiQA
2WikiMuliHopQA
MuSiQue
NarrativeQA
QASPER
UUID (Reported)
72.4
83.1
91.1
65.6
58.4
63.6
Random String
72.2
81.5
92.3
65.5
58.5
63.3
As described in Section 3.1, we use UUIDs as unique identifiers for variables in the KeyChain data. To ensure that the observed performance gains not tied the specific UUID format, conduct an ablation study in which UUIDs are replaced with random strings of the same length. As shown in Table 9, this modification results in nearly identical performance, indicating that the keys do not
19

---

pattern that integrates structured Published as a conference paper at ICLR 2026
need to follow specific UUID format; they only need to be high-entropy and non-semantic to prevent shorteut learning.
A.11
EMERGING SKILLS ANALYSIS DURING RL
Table 10: Example responses of LoongRL-14B different RL training steps, showing how retrieval, reasoning, rechecking, and planning behaviors evolve over training.
Stage
Ckpt at different training steps
Response Length
Skills
Accuracy
Step
2,228
Retrieve Reason
Step 20 Step
2.963
Retrieve + Reason Recheck
Stage
5,685
Retrieve Reason Recheck
Step 118
6,101
Retricve Reason Recheck
Step 199
1,480
Plan Retrieve Reason
Stage
Step 229
2,596
Plan + Retrieve + Reason + Recheck
Step 319
4,143
Plan Retrieve Reason Recheck
To understand the content of model responses and how LoongRL progressively acquires reasoning skills, we manually inspected LoongRL-14B outputs across RL training steps and observed clear evolution reasoning behavior.
As shown in Table 10, early in training (Step 10), the model attempts retrieval and reasoning but often fails. Over time, develops retiivve-rcsoo--rchhekk pattern, producing longer and more accurate responses (e.g., 2K - 6K tokens). By Step 118, responses are even longer due to repeated retrieval, reasoning, and rechecking. though the model still struggles to answer correctly. After difficulty-fouused Stage 2 training (Step 199), explicit planning emerges before retrieval and rea- soning, yielding shorter responses that correctly solve the tasks. Continued training consolidates these abilities, producing the full plan-retreeve-resso--eeehh planning, accurate retrieval, reasoning, and self-verification.
In summary, longer responses primarily reflect the emergence of rechecking and iterative reasoning. while Stage 2 introduces explicit planning, completing the full reasoning pattern.
20


Reviews:
[{"review_id": "EIIg5MGt7o", "reviewer": "Reviewer_6KJK", "summary": "This work proposes a synthetic training data construction method to improve long context reasoning for large language models. Basic idea is to insert irrelevant documents in addition to the relevant one as a challenging dataset together with key-value chains or arbitrary key-strings, e.g., UUIDs, so that the model has to traverse key-value pairs until reaching the correct question. Experiments are carried out on standard long-context reasoning benchmarks, e.g., HotpotQA, and general benchmarks, e.g., MMLU, showing gains when compared with other large language models with long reasoning capacity.", "questions": "- The performance is evaluated on the standard dataset without distractor contexts or questions, but I'm curious of the performance when such distractors are inserted for LoongRL and other models.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- This work proposes an interesting approach of synthetic data construction for training a language model for long-context reasoning. Basic idea is to insert distractors both for contexts and questions so that a model has to pay attention not only the correct context, but correct question at the same time. It is interesting that a model trained only on \"shorter\" context, i.e., 16K, can scale to 128K contexts.\n- The idea of inserting key-value pairs is quite interesting so that a model has to traverse key and value pairs until reaching the correct question. This additional complexity might enforce a model to pay more attention to the question itself during the GRPO training.\n- Experiments are carried out systematically by comparing with diverse models and settings.", "weaknesses": "- The motivation is not clear why UUIDs are used as keys. There exist alternatives, e.g., entity names, or other random strings, could be possible.\n- The detail settings are missing, e.g., the number of distractor contexts and questions, inserted to construct the synthetic dataset. It is also not clear whether the distractor questions are related to the irrelevant contexts already inserted in the long context filling step.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kV34AVzyhc", "reviewer": "Reviewer_WxW6", "summary": "The paper's core contribution is an RL training task designed to elicit reasoning behaviours that are adapted to long-context problems. The authors augment existing multi-hop QA tasks by adding additional documents to the problem context and constructing key-value chains that need to be traversed for the appropriate question to be retrieved and then answered based on the documents' content. Experiments show that this task composition incentivises models to learn a structured plan, retrieve, reason, re-check reasoning pattern for long-context tasks. Two-way substring matching is used for verifiable rewards. A mixture of KeyChain, standard QA, retrieval and math reasoning tasks are used for training to improve long-context reasoning and maintain short-context reasoning performance. Results show that short-context reasoning is indeed preserved and 16k training contexts lead to long-context reasoning abilities that generalise to 128k evaluation contexts. Needle-in-a-Haystack evaluations are passed with 100% accuracy, demonstrating strong retrieval performance.", "questions": "How did you detect or measure the plan, retrieve, reason, recheck reasoning pattern?\n\nCould you show how much each curriculum stage contributes independently?\n\nHave you tested LoongRL on non-QA long-context tasks (e.g., summarisation, code reasoning, document comparison)?\n\nHow do the training costs for LoongRL compare to training QwenLong-L1?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper addresses an important open problem: reasoning over long contexts beyond basic retrieval. By enabling RL to target nontrivial but verifiable long-context reasoning problems, the KeyChain dataset overcomes a key bottleneck of long-context RL finetuning. The results indicate strong empirical performance gains on long-context benchmarks without regressing on the short-context reasoning benchmarks considered, however the latter is expected given the inclusion of short-context reasoning tasks in the training mixture.", "weaknesses": "The KeyChain data construction is highly task-specific: synthetic multi-hop QA with UUID breadcrumbs. It is unclear from the current results if the learned reasoning behaviour generalises to other domains, such as open-ended dialogue, summarisation, or multi-document synthesis. There is no evaluation on long-context generation tasks, which has been artificially decoupled from reasoning over long, static input contexts. A major claim is the emergence of a general reasoning pattern for long-context problems, but it is unclear whether the structured reasoning pattern that is qualitatively demonstrated arises from task scaffolding (i.e., explicit chain following) rather than self-organised reasoning behaviour. Finally, the ablations are currently incomplete, particularly with regards to the multi-stage curriculum. Whilst accuracy is showing to rise across stages, there is no direct comparison between the full curriculum and: skipping the warm-up, doing a single RL stage on all data, or removing the Stage II difficulty filtering. The wam-up stage is even omitted for the 14B model, but the impact of this on results is never shown.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "goZfBet2q1", "reviewer": "Reviewer_kHZc", "summary": "- Problem: Current RL-for-reasoning mostly targets short contexts; long-context scenarios require both retrieval from large inputs and multi-step reasoning. Direct long-context RL rollouts are costly; high-quality, verifiable, and challenging long-context data are scarce.\n- Key idea: A data-driven RL pipeline (LoongRL) centered on KeyChain, a synthesis method that transforms standard multi-hop QA into high-difficulty long-context tasks by inserting UUID key→value chains that hide the true question among distractors. Solving requires: follow chain → recover the real question → retrieve relevant evidence → reason → answer.\n- RL method: Train with GRPO on ~16K-token inputs to induce advanced patterns that purportedly generalize to 128K without full-length RL. Reward is outcome-only, via a rule-based “two-way substring exact match” applied to a boxed final answer to reduce reward hacking and tolerate formatting variants. A balanced data mix and multi-stage schedule preserve short-context capabilities.\n- Emergent pattern: Empirically, RL over KeyChain elicits a plan–retrieve–reason–recheck behavior that improves reliability and long-context generalization beyond training lengths.\n- Results: On Qwen2.5-7B/14B, LoongRL delivers large gains on long-context multi-hop QA (e.g., +23.5% and +21.1% absolute on LongBench v1), matches or approaches larger frontier models (o3-mini, DeepSeek-R1) at much smaller scales, improves Needle-in-a-Haystack and RULER (up to 128K), and preserves/mostly maintains general short-context abilities (MMLU, MATH, IFEval).\n- Ablations: (i) Replacing KeyChain with regular long-context QA yields notably lower gains; (ii) two-way substring match outperforms F1, LLM-as-judge, and strict EM as a training reward; (iii) multi-stage curriculum helps.", "questions": "- Data and leakage\n  - Please detail the exact splitting and de-duplication strategy to ensure that augmented KeyChain instances do not overlap (content-wise) with evaluation items in LongBench v1/v2. How do you prevent direct reuse of the same question/evidence pairs?\n  - When filling long contexts with distractors (including reusing documents from filtered tasks), how do you ensure no evaluation leakage?\n- Reward design\n  - Provide analyses of reward false positives/negatives: e.g., on a held-out set with human-verified correctness, report precision/recall for two-way substring vs EM/F1/LLM-as-judge. Include examples of where two-way substring fails and mitigation (e.g., normalization, entity linking).\n  - Clarify the reference policy used for KL (π_ref): is it the base instruct model frozen? Any mixing with SFT references?\n- Fairness and settings\n  - Add a “strictly matched inference” comparison where all models use the same temperature, pass@1, and no YaRN (or apply YaRN uniformly where possible) on a subset of tasks, to isolate the contribution of LoongRL vs. differences in decoding/context-extensions.\n  - Report sensitivity to decoding hyperparameters (temperature/top-p) on the main long-context benchmarks.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- Clear, focused objective and thoughtful problem decomposition:\n  - Tackles a real gap: moving beyond retrieval to robust long-context reasoning.\n  - Designs data and training to be verifiable and compute-aware (shorter rollouts, longer generalization).\n- Novel and pragmatic data construction:\n  - KeyChain is a neat way to force chain tracing and disambiguation under heavy distractors, requiring both retrieval and reasoning.\n  - Uses real QA seeds (HotpotQA, MuSiQue, 2Wiki) to ground tasks in natural language rather than entirely synthetic QA.\n- RL methodology matched to constraints:\n  - Outcome-only rule-based reward (boxed answer + two-way substring) is a strong, reproducible alternative to LLM-as-judge, with an ablation showing its advantage for training.\n  - GRPO and small KL with a staged curriculum is standard yet well-justified to stabilize training.\n- Strong empirical results at favorable scale/cost:\n  - Large absolute gains vs. strong baselines on LongBench v1, competitive with much larger models.\n  - Convincing length generalization from 16K training to 128K evaluation across NarrativeQA, RULER, and Needle-in-a-Haystack.\n  - Maintains short-context and instruction-following capabilities, often outperforming R1-distilled counterparts that degrade on these.\n- Diagnostic evidence:\n  - Ablations disentangle the contribution of KeyChain vs. conventional long-context QA.\n  - Qualitative trajectories demonstrate the claimed plan–retrieve–reason–recheck behavior.\n- Reproducibility:\n  - Clear training details, datasets, prompts, and claim of releasing code and sample data.", "weaknesses": "- Synthetic structure risk:\n  - The KeyChain format (UUID chains with a designated starting key) has a highly regular, explicit structure. There is a risk the model learns to exploit these patterns rather than developing general long-context reasoning skills. Although downstream improvements suggest transfer, additional tests against structurally varied chains would better establish robustness.\n- Reward and evaluation concerns:\n  - Two-way substring match is pragmatic but may still admit false positives (e.g., partial overlap on ambiguous entities) or false negatives (paraphrases). The ablation is helpful, but more analysis on reward precision/recall would increase confidence against reward hacking.\n  - Inference differences across baselines (temperature, sampling, pass@1 vs pass@k) and the use of YaRN to extend contexts for only some models can complicate fairness. While partially addressed, I recommend additional matched settings or a dedicated “strictly matched” comparison table.\n- Overlap and contamination:\n  - Training uses multi-hop QA seeds overlapping with evaluation domains (e.g., HotpotQA). The paper does not explicitly state safeguards for train/test separation or whether exact question/passages overlap with benchmark test items after augmentation. Without a rigorous de-duplication policy, there is a risk of leakage.\n- Limited statistical characterization:\n  - No variance across seeds or CI reported; improvements are large but stability under re-runs is not quantified.\n- Claims of emergent patterns:\n  - The plan–retrieve–reason–recheck behavior is shown qualitatively. A quantitative metric (e.g., automated tagging of plan/recheck segments; retrieval step correctness rates) would make the claim stronger and trackable across training stages and datasets.\n- Scope and breadth:\n  - Most evaluations are reading comprehension/multi-hop QA and retrieval. It would be informative to include tasks where the query is obscured in different ways (tables, code bases, non-UUID chains), or tool-augmented settings (e.g., retrieval tools outside the context), to demonstrate broader generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7zypHLvPQT", "reviewer": "Reviewer_ZCVK", "summary": "This paper introduces LoongRL, a data-driven reinforcement learning (RL) method to improve long-context reasoning in LLMs, tackling the dual challenges of scarce high-difficulty data and high computational costs. The core contribution is KeyChain, a novel data synthesis technique that transforms short multi-hop QA into difficult long-context tasks by hiding the true question behind a multi-step UUID chain within a distractor-filled document. The authors find that training with RL on KeyChain data induces an emergent \"plan-retrieve-reason-recheck\" pattern that, critically, generalizes from 16K training contexts to 128K test contexts. This method yields substantial accuracy gains on Qwen2.5 models (+21-23%), allowing them to rival much larger frontier models while preserving short-context reasoning and retrieval abilities.", "questions": "1. Since you have included RULER QA, multi-key, multi-value NIAH in the training set, then can we still evaluate on RULER in Table 3? I think model may already learn the test data distribution in your RL training.\n2. Can we try training on HotPotQA keychain data but evaluate on other dataset like MuSiQue to see whether the training data can help to generalize in unseen dataset? Also, I am interested if your data can generalize your model to unseen domain more than Wikipedia. \n3. In Figure 4, you showed model learned to increase response length. Can you explain what includes in the response? Do model first learn to plan, then retrieve, reason and recheck? What skills do model learn progressively during training?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The KeyChain synthetic data is a clever and novel approach to enable long-context RL training. This work shows adding tracing and distractors in the multi-hop QA data can improve the reasoning of long-context tasks.\n2. The authors show models trained on 16K contexts can generalize to 128K, directly addressing the prohibitive computational cost of RL rollouts at full context length.\n3.  The authors designed their training data mix to prevent catastrophic forgetting of short-context and retrieval skills. The results in Table 2 (showing stable MMLU, MATH, and IFEval scores) and Figure 3 (perfect NIAH scores) are crucial for demonstrating that this specialization does not come at an unacceptable cost.", "weaknesses": "1. The testing benchmark (LongBench and RULER) and training dataset are almost in the same domain (e.g., Wikipedia). We can see large improvement on these two benchmarks but relatively small improvement on LongBenchv2. It would be better to show more long-context benchmark improvements like HELMET, AA-LCR, MRCR, etc.\n2. The underlying reasoning data is derived entirely from multi-hop QA datasets (HotpotQA, MuSiQue, etc.). It is unclear if the emergent reasoning pattern would generalize to other long-context tasks, such as summarizing a novel or analyzing a large codebase.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)