SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models
Authors: Ziyi Yang
Weizhou Shen
Chenliang Li
Ruijun Chen
Fanqi Wan
Ming Yan
Xiaojun Quan
Fei Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Self-Play
Reinforcement Learning
Long-Context Reasoning
Large Language Models
Primary area: reinforcement learning
TL;DR: A label-free RL framework that drives the autonomous evolution of LLMs in long-context reasoning

Abstract: Progress in long-context reasoning for large language models (LLMs) has lagged behind other recent advances. This gap arises not only from the intrinsic difficulty of processing long texts, but also from the scarcity of reliable human annotations and programmatically verifiable reward signals. In this paper, we propose SPELL, a multi-role self-play reinforcement learning framework that enables scalable, label-free optimization for long-context reasoning. SPELL integrates three cyclical roles—questioner, responder, and verifier—within a single model to enable continual self-improvement. The questioner generates questions from raw documents paired with reference answers; the responder learns to solve these questions based on the documents; and the verifier evaluates semantic equivalence between the responder’s output
and the questioner's reference answer, producing reward signals to guide continual training. To stabilize training, we introduce an automated curriculum that gradually increases document length and a reward function that adapts question difficulty to the model’s evolving capabilities. Extensive experiments on six long-context benchmarks show that SPELL consistently improves performance across diverse LLMs and outperforms equally sized models fine-tuned on large-scale annotated data. Notably, SPELL achieves an average 7.6-point gain in pass@8 on the strong reasoning model Qwen3-30B-A3B-Thinking, raising its performance ceiling and showing promise for scaling to even more capable models. Our code is available at https://github.com/Tongyi-Zhiwen/Qwen-Doc.

Published as a conference paper at ICLR 2026
SPELL: SELF-PLAY REINFORCEMENT LEARNING FOR EVOLVING LONG-CONTEXT LANGUAGE MODELS
Ziyi Yang 1.2 Weizhou Shen?
Chenliang Li
Ruijun Chen'
Fanqi Wan'
Ming Yan 21 Xiaojun Quan 1,3 Fei Huang²
Sun Yat-sen University Tongyi Lab, Alibaba Group Shenzhen Loop Area Institute
yangzy39 (engzyyy@mail2 sysu. edu. en, mm119608@alibabee iiniiini... com.
xiaojunquaneelaat edu.cn cn
ABSTRACT
Progress in long-context reasoning for large language models (LLMs) has lagged behind other recent advances. This gap arises not only from the intrinsic difficulty of processing long texts, but also from the scarcity of reliable human annotations and programmatically verifiable reward signals. In this paper, we propose SPELL, multi-role self-play reinforcement learning framework that enables scalable, label-free optimization for long-context reasoning. SPELL integrates three cyclical roles coles-questioner, responder, and verifier- within single model to enable tinual self-improvement. The questioner generates questions from raw documents paired with reference answers; the responder learns solve these questions based on the documents; and the verifier evaluates semantic equivalence between the responder's s output and the questioner's reference answer, producing reward signals to guide continual training. To stabilize training, we introduce an automated cur- riculum that gradually increases document length and a reward function that adapts question difficulty to the model's 'ss evolving capabilities. Extensive experiments on six long-context benchmarks show that SPELL consistently improves performance across diverse LLMs and outperforms equally sized models fine-tuned on large- scale annotated data. Notably, SPELL achieves an average 7.6-point gain in pass @ 8 the strong reasoning model Qwen3-30B-AS ---B---A-BThhhiiiiii
raising its performance ceiling and showing promise for scaling to even more capable models. Our code is available httph:t///// /ttp://gtttttbbbb www. Tongyi Zhiwent /Cwnn -oo Doc
Questioner
Responder
Question
Boseline
Pass Rato (%)
66
+SPELL
56.4
Memory
$1.7
50.8
50-
Single Model
37.3
Resperse
Qwen2.5 5-18B
Owen2 5-1 48 Ii--nsttnct
RI-AMaull Qwee-11R
Question
Reward
Reward
75
Qwen1-30n-A Boen--300-A3B-T-ieee -3----in
+RIVR
Pasa Rate (%g)
72-
SPELL
69-
gemini-2..mm 2.5--2225pp---- 70.4
Forward
Verifier
63
Backward
8
Reward
Number Samples (loog Scale)
Figure 1: (Left) An overview of the SPELL framework, where single LLM self-evolves by dynamically adopting the roles of questioner, responder, and verifier. (Right) SPELL consistently boosts performance across various models (top) and exhibits superior test-time scaling over traditional RLVR (bottom).
Work done during internship at Tongyi Lab, Alibaba Group.
Corresponding authors.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
In recent years, reinforcement learning (RL) has emerged as promising approach for enhancing
the reasoning capabilities of large language models (LLMs) (Guo et al. 2025 Yang et al. 2025
Jaech et al. 2024 Team et al. (025). Among these methods, reinforcement learning with verifiable
rewards (RLVR) has shown particular promise in domains where correctness can be programmatically
verified, such as mathematics, logical reasoning, and software engineering (Lambert et al. 2024] Hu
ctt al. 2025 Liu et al., [025c] Wei et al., 2025). RLVR methods employ rule-based or programmatic verifiers to generate reward signals, which then guide policy optimization through algorithms such as Proximal Policy Optimization (PPO) (Schulman et al. 2017b), Group Relative Policy Optimization
(GRPO) (Shao et al. 2024), and related variants (Shao et al. 2024 Yue et al. 2025
Liu et al. 2025d).
Despite these advances, most RLVR research has been restricted to short-context settings (e.g.. <1024 tokens), where models primarily rely on their parametric knowledge for reasoning (Wan et al. 2025). In contrast, reasoning over long documents like long-context question answering requires not only locating relevant evidence scattered across extended contexts but also executing multi-step reasoning. Extending RLVR to long-context reasoning presents significant challenges, which stem from the inherent difficulty of processing long texts, as well as two critical bottlenecks: the prohibitive cost and unreliability of human annotations, and the absence of programmatically verifiable rewards.
Empirical evidence highlights the severity of these issues. On benchmarks such LongBench-V2 human accuracy for extra-long multppe-choice reasoning tasks drops to 25.1% % effectively ap- proaching random chance (Bai et al. 2025). This not only limits the performance achievable under human supervision but also imposes a scalability ceiling, particularly as LLMs approach superhuman reasoning capabilities (Zhao et al. 2025). Specifically, as context length grows, producing reliable an- notations becomes increasingly costly and unstable, and supervision diversity diminishes. Moreover, the lack of verifiable reward mechanisms in long-context settings further constrains the applicability of RLVR, posing a fundamental challenge to advancing reasoning capabilities at scale.
To address these limitations, we turn to self-play RL, where single model learns to self-evolve by generating and solving its own tasks without human labels (Zhou et al., 2025: Chen et al.) 2025b] Huang et al. 2025). However, applying self- play to long-context reasoning poses a unique challenge: answers may be semantically correct yet differ substantially in expression, rendering string matching or naive majority voting unreliable reward signals. Thus, the model should not only generate questions and answers, but also verify its own solutions reliably. This observation motivates our framework, in which one LLM assumes three complementary roles: questioning, responding, and verifying.
In this paper, we introduce SPELL (Self-Play Reinforcement Learning for Evolving Long-Context Language Models), a self-play RL framework for long- contect reasoning. In this setup, unified policy alternates among three roles: the questioner, which formulates questions with reference answers from raw documents; the responder, which attempts to solve them; and the verifier, which compares the responder's ss output with the reference answer to produce reward signals for joint optimization. To steer this process, SPELL incorporates three key design elements. First, verifier trained for self-consistency on verifiable tasks produces stable rewards, even for outputs that cannot be verified by strict rules, thereby overcoming the brittleness of string matching. Second, an automated curriculum uses history memory of question-answe pairs and documents to progressively increase task difficulty. A Gaussian-shaped reward further calibrates difficulty around the responder's competence frontier, ensuring questions are neither too easy nor impossibly difficult. Third, a role- specific dynamic sampling strategy balances contributions across roles to stabilize training of the shared policy. Together, these components form a self-sufficient, closed-loop system that enables LLMs autonomously evolve long-context reasoning without numan-labele data
We evaluate SPELL across 12 open-source LLMs ranging from 4B to 32B parameters, including both dense and Mixture-off -xperrtss (MoE) architectures. On six long-context QA benchmarks. SPELL de- livers consistent performance gains. Remarkably, training base model with enElLe it to surpass its instruction-tun -unedd counterpart that relies on extensive aunan-annotated data, highlighting the data efficiency of our label-free self-play approach. Against a strong RLVR baseline trained on static dataset synthesized by SeeppSek-R1--e R10228 (Guo et al. 2025). SPELL achieves larger and more reliable gains. For capable models such as Swen3-30B-AAe A3BB -.h--iin... SPELL's dynamic curriculum continually elevates performance and enables it to outperform the leading gemini-2.5- pro (Comanici et al., (025) in pass@4, @ 4. These findings firmly establish our self-play approach as a scalable and effective path toward advanced long-context reasoning without human supervision.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES
Long-Context Reinforcement Learning We formulate the long-context generation task as reinforcement learning (RL) problem. Given of documents {cJ]== and question q. the goal of long-context RL is to optimize policy model to generate a response that maximizes reward function rg (c. q. The standard objective to maximize the KL-regularized expected reward (Schulman et al. 2017a; Wan et al. 2025):
max P. [ro q, v)] kL [x 7ty
c, Terly c,
where e c= Concat(cr (ancat(c1.com C...... Cn), D is the training dataset, denotes reference policy, and B controls the strength of the KL regularization to prevent large deviations from the reference policy.
Group Relative Policy Optimization (GRPO) For long-context inputs, the quadratic complexity of the attention mechanism renders PPO (Schulman al. 2017b), which relies generalized advantage estimation (GAB) (Schulman et al. 2015) via value network, computationally prohibitive. Therefore, we employ GRPO (Shao al. 2024) optimize the objective in Eq. (T). For each input (c, 9). GRPO first samples a group of G candidate responses {ys / from the old policy Tew It then estimates the advantage through group-wise reward z-score normalization, thereby obviating the need for separate value network. Formally, the objective is:
M6
(O) e. (..) mon (1eg))
min (pi.(0)A
G
51D (Te (Tel| ]
clip (1,)()),)-,,1+++)))))))))))
where (0) = mo(uee (c..9.v.........)
Fewe (.-...-......))) is the importance sampling ratio for token in sequence i. The group-relative advantage A, is shared across tokens of the i-th sequence and computed by normalizing the sequence-level rewards {r,),
mean( ([rx)
A;
std ad((r=)(=))
THE SPELL FRAMEWORK
In this section, detail the core design of SPELL, self- reinforcement learning framework that enables LLMs to improve their long-context reasoning capabilities without external supervision. The key principle of SPELL that single policy model मe dynamically assumes three complementary roles: questioner T que autonomously generates and solves questions while producing reliable reward signals. This closed- loop interaction creates an evolving curriculum in which the model progressively adapts to longer contexts and more complex reasoning (Section 3.1). Role-specific reward designs (Section 3.2 and unified optimization procedure (Section 3.3 jointly drive this co-evolution.
responder Tb and a verifier x Through their interaction, the model
3.1 THE SELF-PLAY EVOLUTIONARY LOOP
As illustrated in Figure 2 and Algorithm T. SPELL proceeds iteratively: given a cluster of n documents C = {c, ) and task type r, the policy лa first generates new questions ? then attempts to solve them, and finally verifies the solutions before performing a unified policy update.
drestion-answer pairs in an iterative curriculum. In the very first iteration, it is conditioned only on randomly sampled subset of m documents (m < n) and produces pair (q. a). After each solvable pair is created, we append it to history memory 7. that stores the I. most recent solvable question-answer pairs and their associated source documents: Hc = {(Ci, a, / In subsequent iterations, the questioner is conditioned on both newly sampled
Questioning The questioner nT generates
Details of dataset construction task definition are provided in Appendix E. TTo direct the policy in enacting three distinct roles, we adopt zero-shot prompting using tailored templates for each role and task type. Details of these templates are provided in Appendix G
3

---

Published as a conference paper at ICLR 2026
Role-Specific Rollout
History Memory
Retrieve
Cnew
que
Questioning
prite 0.?
3C
Grounding Filter
Responding
Verifying
Piu
m
Full Context
ver x
Yi
Model
Data
Unified Policy Update
Pat
Psu
Role Specific
GRPO
Dynamic Sampling
NLX
Role Rele-Speclfic
MGa
Training
Reward
Advantage
Estimation
Figure 2: Overview of our proposed SPELL for self-evolution longecontext reasoning. The process operates a continuous loop that alternates between two stages: (1) Role-Specific Rollout, where a single policy model enacts three distinct roles a questioner (ngg), ) responder (x²") ). and verifier (r1²")-t generate training data. (2) Unified Policy Update, where the unified policy is refined using the collected data, and the enhanced model serves the starting point for the next rollout eyele.
subset Cnew and the stored memory. The resulting context is X = =UUL, Ci) Ul(a, at) o) As the memory fills, the context for the questioner expands include both previously seen and newly sampled documents, which allows the questioner to generate questions that integrate information across more documents. The history memory also raises difficulty by including past {(g, a)}} these exemplars discourage redundancy and, via prompting, push T00h0 to generate harder questions than those already solved. Consequently, the questioner's s difficulty increases for two complementary reasons: (I) the context que expands over iterations more documents are brought into scope, and (2) explicit conditioning on historical { }t)} encourages the model to escalate question complexity.
Responding The responder gØ attempts to solve the generated question based on documents. To mitigate the generation of non-grounded or hallucinated questions, we employ a grounding filter process to discard questions that can be answered without documents. For valid questions, the responder is presented with the complete set documents, where the remaining documents unseen by the questioner serve as distractors to increase grounding and reasoning difficulty. This design enforces reliance on the provided document context rather than parametric memory. encourage exploration of diverse reasoning trajectories, the responder generates G' independent rollouts {34 )
Verifyying The verifier n evaluates the semantic equivalence between the responder's output y: and the questioner's reference answer a. For each y:. it produces G' independent binary judgments {vs.g 15.11 Ui.j E {0, 1}, which are then aggregated through majority voting:
G 2
yer Σ
Ui
where T(-) is the indicator function. This ensemble-based verification reduces variance and produces a stable, semantically aware reward signal, which is essential for sustaining a self-play system.
3.2 ROLE-SPECIFIC REWARD DESIGN
The three roles co-evolve under specialized rewards that align their objectives while remaining compatible within a single shared policy. In what follows, we detail these rewards.
Verifier The verifier is trained to improve its judgment reliability through self-consistency (Wang et al. 2022 Zuo et al. 2025). For candidate output yı, the verifier produces G rollouts with judgments U.,. Each rollout is then assigned a reward:
b.f [[Ve,j = )
(5)
where o is the majority vote over G rollouts.
4

---

Published as a conference paper at ICLR 2026
Responder The responder's reward for the i-th solution is the maximum of deterministic, rule- based check and the verifier's consensus score, denoted as:
r max (2a a), )
(6)
The rule-based function, Rull. provides a binary reward based on cover exact match (CEM) crite- ria (Wan et al., 2025 Song al., 2025 it returns 1 if the ground-truti answer appears in the generated response yi and otherwise. The maximum reward plays a crucial role: when yi is a correct paraphrase that CEM fails to capture, majority vote of ot = 1 prevents the policy from being misled by nelse-ne noise, which stabilizes learning and encourages continual improvement.
Questioner The questioner is incentivized to generate questions of intermediate difficulty, as learning is most efficient at the frontier of the LLM's capabilities (Bae et al. [2025] Huang et al.) 2025). For binary-rewart tasks, this frontier corresponds reward variance and provides the richest learning signal. We therefore define the questioner's 's reward as a Gaussian function centered at this optimal point. Given the responder's average success rate, res, the reward is:
success probability of 0.5, which maximizes
prres = t
(1TH (""
exp
if0 < prs <1 puS = 0 p
2
yiqu
0
005
if the question is not grounded in documents if the question-answe pair has formatting errors
We set the mean 0.5 to target the point of maximum learning efficiency and the standard deviation 0.5/3 to concentrate the reward around this level. Additionally, the questioner penalized for producing ill-formatted (e.g., non- non-parsable) question-ande answer pairs questions that can be solved without context, thereby enforcing both correct formatting and strong grounding the provided text.
3.3 UNIFIED POLICY OPTIMIZATION
A central feature of SPELL is that samples generated under different roles supervise single policy Ty. The optimization must control both sample efficiency and gradient balance across roles.
Role-Specific Dynamic Sampling The raw samples collected for each document instance are highly imbalanced: one questioner sample, G responder samples, and G verifier judgments. To prevent the verifier's ' samples from dominating updates and to prioritize improvements in the responder's s introduce role-specific sampling strategy that leverages the statistical structure of each role's signals. For the responder, we retain all groups with non-zero reward labeled as positives for the questioner,
document- -rruuddd reasoning,
variance (std( (r ] 0). The associated questions and an equal number of negatives are drawn from questions with non-positive reward, as defined in Eq. (7). For the verifier, we preserve instances where the majority vote agrees with the rule-based check and subsample groups with conflicting verifications to match the number of questions. This role-specific sampling strategy reduces the training set to roughly 1/G of all samples, accelerates optimization, and prevents the responder's 's gradients from being overwhelmed by verifier samples. Importantly, although most verifier samples are omitted, their collection cost low, see Appendix F.T
Advantage Estimation For the responder and verifier, which generate G outputs per prompt, we use group-level advantage estimation defined in Eq. (3):
rol mean (1 G Jk=1)
A
role E (res, ver).
(8)
std
(
role G
The questioner generates only a single output per instance and thus lacks a group-level baseline. Therefore, adapt the normalization method from REINFORCE+++basclne (Hu 2025 and
normalize its reward against other questioner rewards within the training batch Bque.
y.que
mean(
que
(300))
que
std(rqqe
yeque
Bque)
Unified Policy Update After collecting and sampling batch of samples, the policy parameters are updated by jointly optimizing the GRPO objective across all three roles:
TG)Po Ture (0) + (A) (#)
(10)
The updated na is reused to execute all roles in the next iteration. This closes the self-evolutionary cycle and keeps one unified policy for questioning, responding, and verifying.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 EXPERIMENTAL. SETUP
Training Details Our SPELL RL framework is implemented using VeRL. (Sheng et al. 2025). During generation, we employ a sampling temperature of 0.7 and a top-p value of 0.95. The maximum input length is 16K tokens, while the maximum output length is set to 4K for non- reasoning models and extended 20K tokens for reasoning models. To balance rollout diversity and computational efficiency, we utilize a group size of G = 8. The maximum number of recent solvable question-answer question answer pairs cached in history memory is set L
and the number of candidate documents drawn when proposing a new question is set to m 5. We conduct a purely on- on-policy RL training with a batch size of 128 and a constant learning rate of 2 x 10-6, At the beginning of each rollout, we randomly sample one of three predefined task formats document general QA, financial math QA, or multiple-choice choicc-llonng with a relevant document list from the corpus. Prompt templates for each task and each role provided in Appendix G For the RLVR bascline, synthesize a dataset using DeepSeek-R -1--52 (Guo et al. 2025) over the same document corpus and maintain identical hyperparameters ensure fair comparison. For comprehensive details data construction, RL algorithm, and baselines, please refer to Appendix E E.3 and E.5
Evaluation Benchmarks We evaluate multiple-choice QA on LongBench- V2 (Bai et al. 2025) and multi-hop QA across Frames (Krishna al. 2025).
models on six long-context benchmarks, spanning
AotporQA (Yang al. 2018), 3 2Wiki MWikiiuutihopAAAA (Ho et al. 2020), MuSiQue (Trivedi et al. 2022 and the DocMath (Zhao et al. 2024) for financial report reasoning task. We evaluate all
models with maximum input lengths of 16K and 100K tokens, and report the average accuracy over eight runs Further details on the benchmarks and evaluation protocol are available in Appendix E.2
E.2
4.2
MAIN RESULTS
Table summarizes the results of SPELL across 12 open-source LLMs on six long-context QA benchmarks under maximum input lengths of 16K and 100K tokens. These results offer valuable insights into SPELL's effectiveness and generalization, as elaborated below.
SPELL consistently enhances performance across diverse models. Our self-play framework exhibits strong universality, and delivers substantial improvements across different architectures, sizes, and families. This versatility is evident across the following dimensions. (1) Model types and sizes: SPELL cultivates complex reasoning skills from scratch. For unaligned base models, the average improvement at 16K is large and robust, with Qwen2.5-7B Qwen2.5- and Qwen2.5-32B improving by 13.9, 14.4, and 9.1 points, respectively. Remarkably, these trained models consistently counterparts of the same size, which are trained with extensive
outperform their instruction-turet human-annotated data. This result highlights that SPELL data-efficient and practically valuable in scenarios where labeled data is scarce. SPELL also benefits instruction-tune tuned models, e.g.. Qwen2.5-7B Qwen2.5-7B-Inntrucc improves by 9.0 points. For highly specialized reasoning models such as R1-Distill-Qwen- 14B, the performance still increases by 3.4 points. (2) Architecture: Beyond dense models, the framework is also applicable to Mixture-ou of- of-Experts (MoE) models, where it improves Owen3-30B-ARE -3n-stuuutttt and Qwen3-30B- 30---30-------- A3B-A3nnT by 4.4 and 2.0 points, respectively. (3) Improvements extend across families. For example, Llama-3. I-8B-Instruct and RI-Distill-LJama-St increase by 4.4 and 3.4 points, respectively. Collectively, these results establish SPELL as a broadly effective paradigm for advancing LLMs in long-context tasks.
Model families:
SPELL is superior to traditional RL with static data. We compare SPELL against the RLVR baseline trained on a fixed dataset synthesized by DeepSeek-RI-On 1-0228. Although such static data offers high-quality supervision for RL training, it cannot adapt the policy's s evolving capabilities. In contrast, SPELL constructs a self-play curriculum that tracks the model's current ability: the questioner focuses on instances near the responder's 's competence boundary. maintaining alignment between the training signal and the policy throughout optimization. The advantage becomes increas- ingly evident as the policy model's 's capabilities grow. For Qwen2.5-7B, RLVR achieves performance comparable to SPELL,, indicating that a static corpus appears sufficient for weaker policies. However, for Qwen3-30B- -3--33------B-- 1.3B-Thinking SPELL improves average scores by 2.0, whereas RLVR yields no gain. On the more challenging benchmarks for the same model, RLVR decreases accuracy
We use the subsets of HotpotQA, 2WikiMulthhoOAA, and MuSiQue from LongBench
Bai al. 2024b

---

Published as a conference paper at ICLR 2026
Table Overall results of proposed SPELL method with maximum input lengths of 16K and 100K long- context benchmarks. "LB-MQU LLBMMQA represents the average performance across 2WikiMulihopQA, HotpotQA, and MuSiQue. "LB-V2 -2" refers to LongBench-v2. For the average score (Avg.), indicates the relative improvement over the hase model within each group. The best score in each model group is highlighted in bold.
Models
16k
100K
LB-MQA
Avg.
DucMuth
LB-MQA
Avg.
Buse Models
Qwen2.5-7B + RI
10.9
27.9
23.6
41.8
41.0
50.0
30.2
40.8
42.7
40.3
49.2
26.0
39.5
SPELL
40.0
39.2
50.9
32,3
40.0
39,9
40.1
50.8
28.2
39.8
Qwen2.5-1448
38.0
37.2
41.9
32.1
37.3
36.2
37.5
43.3
27.5
36.1
+ RLVR
52.2
51.0
63.3
32.9
49. 49.9-12a
53.2
52.1
64.2
30.5
50,0115
SPELL
$7.6
52.6
63.0
33.5
$1.7
56.8
53.0
63.2
31.2
S1.1
Qwen2.5.32B
46.8
42.6
49.0
33.7
43.0
40.7
42.2
50.1
28.7
40.4
+ RLVR
58.3
50.0
59.5
32.8
50.2
57.5
49.9
60.1
32.7
50.1
SPELL
52.4
Instruct Models
Qwen2.55-BB
38.4
40.3
39.4
44.5
28.4
38.4
+ RLVR
43.0
48.7
59.6
30.1
45. 91
44.1
48.6
57.4
28.2
44.8
SPELL
45.8
46.7
63.1
33.2
47.2
44.5
48.2
60.7
32.4
46.5
Qwen2.5-14B. 1-105-5--1--nnttecc
56.7
36.6
52.5
+ RLVR
56.1
59.6
71.0
36.4
56.0
56.7
59.9
73.4
38.5
67. 1
SPELL
89.6
62.1
72.8
36.8
57.8
60.1
63.9
74.8
40.1
59.7
Swve225-328-Insttuu
60.0
19.9
61.4
36.0
51.8
63.0
49.1
61.5
36.2
52.5
RLVR
59.9
60.5
70.4
36.3
56.8
59.7
62.3
69.6
36.9
57.1
SPELL
62.3
61.2
40.1
59.5
63.3
74.1
40.8
60.1
Qwen 3. I0--a3--ns
62.3
55.3
70.5
36.9
56.3
63.0
57.8
70.3
44.1
58.8
RLVR
62.5
59,9
71.8
39.8
58 58.5
64.0
62.0
72.4
47.4
61.5
60.7
Llama3. 1-8t-Instruct
33.2
45.6
52.5
29.1
40.1
34,9
47.3
53.5
27.1
40.7
REVR + SPELL
37.9
45.0
58.8
27.5
42.3
36.9
47.6
57.2
26.1
42.0-13
39.2
48.9
61.6
28.4
44.5
39.7
50.8
60.9
26.2
44.4
Reasoning Models
KI-Disnil-Llama-B8
42.0
50.3
41.5
26.4
47.5
+ RLVR
43.4
51.4
67.8
30.0
48. 48.2*14
45.4
$4.0
68.0
28.3
18. 18.9+14
SPELL
48.9
534
68.4
30.2
$0.2
49.2
54.3
70.0
29.3
50,7
R. Distill-Qeen-ll
57.7
59.2
72.4
36.2
56.4
59.5
60.6
73.3
33.3
56.7
RIIR
59.6
61.7
74.6
37.2
58.3
61.0
63,8
76.0
35.9
50.2-25
50.4
Qwan3-4B-Thhnking
58.6
56.7
69.9
32.9
$4.5
61.4
59.2
70.9
40.7
58.1
RINR
60.5
56.6
33.8
M5.
63.3
71.1
43.4
10.1
+ SPELL
61.9
56.6
56.7
64.8
72.4
43.0
60.2
Qwen MMB-AABBThhnkiae
62.9
64.5
75.7
39.7
60.7
63.8
65.8
77.9
46.7
63.6
RLVR
60.74M
77.2
64.5-
+ SPELL.
64.1
66.5
78.0
42.3
62.7
66.7
68.1
78.4
50.5
63.9
+RLVR
SPEL2
gemini-2.5
Overall
DocMath
Frames
LonglRocc-MQQA
1 LongBeenh-VV V/2
84-
Pass Rate
60
83
66
72.1
80-
55-
52.4
78
Number Samples Scalo)
Test-time scaling performance @pass@k) across benchmarks. Qwen3-30B-AN -00B--3-Thiiniin model trained with SPELL shows significantly steeper improvement as the number of samples (K) increases compared to the base model and the RLVR baseline. Notably, its pass @ass@4 performance surpasses gemini-2.8, 2.5--ro..
Figure 3:
DocMath (-0.2) and LongBench- V2 (-1.2), whereas SPELL delivers consistent gains of 1 1.2 and 2.6 points, respectively. These results validate that when models approach or surpass the quality static training data, a self-play curriculum proves more effective for sustaining performance gains.
SPELL generalizes to longer contexts. All models are trained with a 16K input limit and evaluated at 100K without additional tuning. The results remain consistent under this out-of-distribution input length, demonstrating that the benefits of SPELL extend beyond the training window. For Qwen2.5- 14B, the average improvement is 14.4 at 16K and increases to 15.0 100K. This consistency suggests that the framework strengthens document- document-grounded reasoning in a way that remains effective input lengths grow substantially, rather than producing gains limited to specific context length.
7

---

Published as a conference paper at ICLR 2026
Ovarlll
Genraa
Fimuncial Math QA
Mulliplechhicee
SPELL
Update
wio History Memory
0.8
0,8
I-Passie
0.6
-Paas
0.6
-Paox
0.6-
0.2
0.2
0.2
50
10
Training Steps
Training Steps
Training Stops
Figure 4: Analysis question difficulty - pass@1) 1) on three tasks over training steps. (Left): The full SPELL framework shows a clear upward trend in difficulty. (Middle): Without questioner updates, difficulty stagnates. (Right): Without the history memory, difficulty becomes erratic and unstable.
SPELL boosts exploration and raises the performance ceiling. We assess test-time exploration with the pass@k @k metric at a 100K input limit. As shown in Figure Qwen3-30B-AS 1-3--Aiin-TTiin trained with SPELL exhibits markedly steeper improvement curve increases compared both the base model and the RLVR baseline. Its pass @8 score reaches 74.5, significantly outperforming the RLVR baseline (68.1) and the original base model (66.9). This enhanced exploratory ability further allows the SPELL-trriiee model to surpass the performance of the leading gemini-2.5-pro (Comanici) al., 2025) at a pass @4 rate. These results indicate that SPELL effectively broadens the model's test-time search space and raises its attainable performance ceiling, highlighting a promising path toward elevating the capabilities of even more powerful foundation models.
4.3 ABLATION STUDIES
To validate the key design choices within the SPELL framework, we conduct ablation studies on Qwen2.5-7B- I-stuutnst We individually remove each core component of the questioner and verifier roles to quantify their individual contributions to the overall performance.
Questioner As shown in T the removal of the format penalty and the grounding filter degrades the average score by 1.1 and 1.0 points, respectively. The format penalty keeps the question well-formed, and the grounding filter prevents the generation of hallucinated questions. The largest drops come from disabling the update mechanism and the history memory: freezing the questioner lowers the average score by 4.6, and re- moving history memory lowers it by 2.9. The declines appear across DocMath, Frames, LB-MQA, and LB-V2, which indicates that these components have broad impact rather than task-specific effects.
Table 2: Ablation study of SPELL Qwen2.5-7B I-Istruttctt with 16K maximum input length. and + indicate relative decreases and increases, respectively, compared to the full SPELL model.
Method
DocMath
LB-MQA
Average
SPELL
45.8
46.7
63.1
33.2
47.2
Questioner
Format Penalty
46.0
48.2-15
59.3
31.0
$8.1
who Grounding Filter
47.0
46.491
60.1
31.3
46.2
pddte History Memory
45.0
gas
60.9
30.3
42.64
45.6
46.6
51.2
30.8
11.3
Verifier
Verifier
44.0"
w/o U pdate
45.1
48 2-14
61.447
32 32.69
46.8
Majority Voting
45.5
48. 18.1
61.9
30.743
46.6
Update Consistency
46.6
31.3
We further examine how these components affect generated question difficulty over Qwen2.5-7B-
Instruct training steps, measured by 1-pass@) with an external responder (Qwen3-300) 30B-A3BBBBBBB--- Instruct) and an external verifier (gpt-oss- The full SPELL model (Figure 4 left) shows clear upward trend in overall question difficulty, which ensures the questioner proposes questions that are challenging enough for the responder's evolving capabilities. In contrast, freezing the questioner causes difficulty to stagnate (Figure 4 middle), while removing the history memory makes it erratic (Figure 4 right). The evidence supports the conclusion that continual updates and access to recent history are necessary to form a stable and progressively more challenging curriculum for the responder, which is essential for sustained improvement in self-play system. This dynamic prevents one role from exploiting the static weaknesses of another, as observed in Liu et al. (2025a).
Verifier Removing the verifier and relying solely rule-based rewards decreases average score by 3.2 points, with a 6.4-point drop on DocMath. The CEM-based reward function brittle and can penalize semantically correct but lexically different answers; the verifier provides complementary
8

---

Published as a conference paper at ICLR 2026
SPELL
AZR
R-Zero
0.25
638
0.2
Respore Leaghh
Pus Rati (4.)
faue
-0.00
Emioop
Jarann AM
-2
Triiiing
Training
(n) Reward Mapping
(b) Entropy
(c) Response Length
(d) Performance
Figure Comparison of different reward mapping strategies. (a) Visualization of the reward functions for SPELI,, A7R, and R-Zero. (b) SPELI. exhibits more stable entropy dynamics during training. (c) SPELL maintains a more moderate and controlled growth in response length. (d) These factors contrihute to consistent performance improvement, ultimately leading our method to achieve the highest final pass rate.
signal in such cases. Interestingly, disabling verifier updates or switching to single-pass decisions leads moderate declines, which indicates that Qwen2.5-7B-Inm In-trrcctt is already competent the simpler verification task. However, removing the consistency update mechanism still causes a 1.5- point performance drop. This result shows that the verifier's updates are susceptible to noise from its own erroneous majority votes, which degrades its reliability. On rele-verifiable tasks, the verifier learns to filter this noise by aligning its majority vote with the ground-trute rule-based outcome. This process provides the verifier with reliable learning signals, which in turn enhance its ability to generate stable rewards for uuve-ulverifiable outputs. This illustrates how verifiable rewards can guide the calibration of non-verifinn verifialle rewards, finding that aligns with the self-judging methodology in Kimi-K2 (Team et al., 2025).
4.4 ANALYSIS OF QUESTIONER REWARD MAPPING
We compare our Gaussian- mapped reward function for questioner Eq. with the reward mapping used in AZR (Zhao et al. 2025) and R-Zero (Huang et al. (025). Figure 5 5(a) visualizes these distinct mapping functions. While the AZR reward function also penalizes high-accuracy questions, is susceptible noise from spurious correctness, which can destabilize the training process. In contrast, our Gaussian function, which peaks when the average responder accuracy ydr 0.5, selectively encourages questions the frontier of the responder's competence. Additionally, this mechanism mitigates the impact of data noise. Questions with wrong reference answers typically result in success rates zero or one, corresponding scenarios of random guessing on unsolvable questions or consistent matching with the incorrect reference, respectively. Both extremes naturally fall into the low-reward tails of the Gaussian function, effectively suppressing incorrect questions during policy optimization. While R-Zero also centers its peak reward at 0.5, our Gaussian mapping provides more targeted reward by offering stronger incentives for questions of moderate difficulty and imposing steeper penalty on those that are either too easy or too hard. This creates focused and smooth reward distribution that guides the questioner away from generating both trivial and overly difficult questions. The training dynamics corroborate these design differences. As shown in Figures (]) and (c), our method maintains a more stable training entropy and exhibits more controlled growth in response length under the Gaussian mapping than under AZR or R-Zero. These advantages in training stability lead to superior overall performance. As Figure d) demonstrates, our method not only achieves more consistent performance growth but also reaches the highest final pass rate among all compared approaches. This evidence supports the view that concentrating the questioner's 's reward the responder's competence frontier stabilizes the optimization process while preserving headroom for their mutual co-evolution.
4.5 HYPERPARAMETERS ANALYSIS
Selection of standard deviation o The choice of a in Eq. (7) is derived from the statistical properties of the Gaussian distribution, where approximately 99.7% of data points fall within three standard deviations (3o) of the mean. In SPELL, we aim to concentrate the questioner's reward at the point of the responder's maximal learning efficiency, where fees 0.5. Accordingly, the mean is set to 0.5. Given this mean, the distance to either boundary of the valid average responder reward range [o, 1] is 0.5. By setting 30 0.5, we ensure the effective range of the questioner reward

---

Published as a conference paper at ICLR 2026
covers the responder reward space, yielding o = 3. To further vali- date this theoretical choice, conduct an ablation study on using Qwen2.5- 7B-Instruct. As shown in Table 3 nar- rowing the curve (a = 0.5/6) has a minimal negative impact, as the reward remains well-focused. However, widen- ing the curve (or 0.5/2) significantly degrades performance, likely because it assigns higher rewards to overly easy or hard questions, providing less tar- geted training signal. This confirms that
Table 3: Ablation analysis of SPELL varying the standard devia-
tion o and the rollout group size G using Qwen2.5-78 105---Innsttuee The default configuration is a = 0.5/3 3 and G = 8.
Settings SPELL
DocMath
Frames
LB-MQA
LB-V2
Average
45.8
46.7
63.1
33.2
47.2
Standard Deviation (d)
0.5/6
45.3
47.0
62.3
32.8
46.9
0.5/2
45.5
46.0
59.4
31.8
45.7
Group Size (G)
G 4
44.3
47.1
62.5
31.8
46.4
16
46.0
47.4
62.7
31.9
47.0
0.5/3 / is both theoretically sound and empirically effecttiee
Sensitivity of group size (G) We examine the impact of the rollout group size G model performance using Qwen2.5-7B-Instruuc As shown in Table while G = yields the best overall results, SPELL remains robust across different group sizes. We select G 8 as the default setting strike a balance between performance gains and computational efficiency during training.
4.6 ROLE OF EXTERNAL JUDGES IN VERIFICATION
We investigate whether replacing the rule-based judge (CEM-based reward function) with stronger external model sgpt-oss-120b) benefits the self-play process. As shown in Table introducing stronger external judge does not yield significant overall improvement. This suggests that Qwenn.5-7B-Innsruuct is already capable of learning semantic verification through self-play without external supervision. Notably, when an external judge is introduced, the internal verifier becomes less important; removing it results in only a minor 0.5 0.S-point drop, compared to the significant 3.2-point drop observed when using the rule-based judge. This highlights the critical role of the internal verifier in complementing the brittle CEM- reward function when an external judge is not available.
Table 4: Comparison of SPELL trained with rule-based judge versus an external judge (gpt-oss- 120b). The verifier crucial when using rule-based judge, but becomes less critical when including an external judge.
Method
DocMath
Frames
LB-MQA
LB-V2
Average
Qwen2.5-7B Twen2.5-7B-Instruett
38.4
40.3
45.1
29.0
38.2
+ SPELL (Rule-based Judge)
45.8
46.7
63.1
33.2
47.2
+ SPELL (Gpt-oss-120) Judge)
47.1
48.0
61.6
32.1
47.2
SPELL (Rule-based Judge) w/o Verifier
39.4
46.6
60.4
29.4
44.0
+ SPELL (Gpt-oss-120) Judge) Verifier
47.0
47.2
61.1
31.3
46.7
5 CONCLUSION
This work introduces SPELL, a multi-role self-play reinforcement learning framework for evolving the long-context reasoning capabilities of LLMs without human supervision. A single policy model alternates among the roles of questioner, responder, and verifier to generate questions, solve them, and assess the solutions, which reduces reliance on costly and unreliable human annotation while enabling stable self-evolution. Extensive experiments across 12 models of diverse architectures and sizes show that SPEL.. delivers consistent and substantial improvements long-context reasoning.
This study concludes with three notable findings. First, signals from verifiable tasks can calibrate and strengthen the verifier's assessment on non-verifiable tasks, thereby ensuring reliable self-rewarding mechanism. Second, within multi-role self-play framework, sustaining a dynamic equilibrium among the capabilities of different roles is critical for the stable evolution the shared policy. Finally, surpassing human performance, where
results demonstrate that for models approaching external supervision emerges as a fundamental bottleneck, autonomous self-evolution transitions from promising alternative indispensable strategy for sustained adyancement.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This research focuses the development of long-context LLMs through self-play that requires no human supervision. While we believe our methodology does not inherently raise significant ethical issues, acknowledge the potential for misuse of this technology. We also recognize that an unsupervised learning approach may perpetuate or amplify societal biases in the model. Our research conducted using only publicly available datasets, in compliance with their licenses, and involves no personally identifiable information. We have adhered to all relevant ethical and legal standards and declare no conflicts of interest that could have influenced the outcomes of this study.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, provide full experimental details in Section 4. and Appendix E These include our methods for dataset construction, training configurations, and the evaluation setup. The implemented code, the data used, and a comprehensive guide reproduce our method are available in the supplementary materials.
ACKNOWLEDGEMENTS
This work was supported by Alibaba Group through Alibaba Research Intern Program and National Natural Science Foundation of China (No. 62576368).
REFERENCES
Sanghwan Bae, Jiwoo Hong. Min Young Lee, Hanbyul Kim, Yeon Nam, and Donghyun Kwak. Online difficulty filtering for reasoning oriented reinforcement learning. arXiv preprint 033300 2025.
Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, and Juanzi Li. LongAlign: recipe for long context alignment of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024a.
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024b.
Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang. Xin Lv, Shulin Cao. Jiazheng Xu, Lei Hou, Yuxiao Dong. Jie Tang, and Juanzi Li. LongBench v2: Towards deeper understanding and reasoning on realistic long-context multitasks. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), 2025.
Guanzheng Chen, Xin Li, Michael Shieh, and Lidong Bing. Longpo: Long context self-evolution of large language models through short-to-long preference optimization. In International Conference on Learning Representations, 2025a.
Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Self-questioning language models. arXiv preprint arXiv: 03682, 2025b.
Zhi Chen, Qiguang Chen, Libo Qin, Qipeng Guo, Haijun Lv, Yicheng Zou, Hang Yan, Kai Chen, and Dahua Lin. What are the essential factors in crafting effective long context multi-hop instruction datasets? insights and best practices. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), 2025c.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.141683 2021.
11

---

Published as a conference paper at ICLR 2026
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.0626/ 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, et al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi. Yiran Wu, al. A survey of self-evolving agents: On path to artificial super intelligence. arXiv preprint ar arxi::2507.21046 2025.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in Ilms through reinforce- 2025.
ment learning. Nature,
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representationss 2021a.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Thirly-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021b.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop qa dataset for comprehensive evaluation of reasoning steps. Proceedings of the 28th International Conference on Computational Linguistics, 2020.
Jian Hu, Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262, 2025.
Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung- Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503 242990, 2025.
Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. R-zero: Self-evolving reasoning 11m from zero data. arXiv preprint arXiv:2508.0500 05004, 2025.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint 1rX1n:2412.16720, 2024.
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarez, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: unified evaluation of retrieval- augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2025.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:241..15124 2024.
Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, and Ivan Oseledets. Confidence is all you need: Few-shot rl fine-tuning of language models. arXiv preprint arXiv:2506.060 06395, 2025.
Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi, Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, et al. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning. arXiv preprint arXiv:2506. 24/ 2025a.
12

---

Published as a conference paper at ICLR 2026
Jiaheng Liu, Dawei Zhu, Zhiqi Bai, Yancheng He, Huanxuan Liao, Haoran Que, Zekun Wang, Chenchen Zhang, Ge Zhang, Jiebin Zhang, et al. A comprehensive survey on long context language modeling. arXiv preprint arXiv 2025b.
Junteng Liu, Yuanxiang Fan, Zhuo Jiang, Han Ding, Yongyi Hu, Chi Zhang, Yiqi Shi, Shitong Weng. Aili Chen, Shiqi Chen, et al. Synlogic: Synthesizing verifiable reasoning data at scale for learning logical reasoning and beyond. arXiv preprint urXiv:2505. 19641, 2025c.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi. Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025d.
MAA. American invitational mathematics examination AIME, 2025. URL https: ht///maa..... /aap::s/mmaa.rggg
maa maa--Lovttatioonaa--cperrrrrr
OpenAl. gpt-oss-120b & gpt-oss-20b model card, 2025. URL
pdf
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 2023.
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02444 0203802438. 2015.
John Schulman, Xi Chen, and Pieter Abbeel. Equivalence between policy gradients and soft q-learning. arXiv preprint arXiv: mete::1044444444 2017a.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint (0d.:v:777.66444 2017b.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.0338 03.00000003 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin WU, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, 2025.
Tianyuan Shi, Canbin Huang, Fanqi Wan, Longguang Zhong, Ziyi Yang. Weizhou Shen, Xiaojun Quan, and Ming Yan. Mutual-taught for co-adapting policy and reward models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025.
Anshumali Shrivastava and Ping Li. In defense of minhash over simhash. In Artificial intelligence and statistics, pp. 886-894. PMLR, 2014.
Toby Simonds, Kevin Lopez, Akira Yoshiyama, and Dominique Garmier. Self rewarding self improving. arXiv preprint 20Xi:2550.0088777 2025.
Huatong Song, Jinhao Jiang, Wenqing Tian, Zhipeng Chen, Yuhuan Wu, Jiahao Zhao, Yingqian Min, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. RI R1-searcher+4: Incentivizing the dynamic knowledge acquisition of Ilms via reinforcement learning. arXiv preprint arXiv:2505. 2025.
Huashan Sun, Shengyi Liao, Yansen Han, Yu Bai, Yang Gao, Cheng Fu, Weizhou Shen, Fanqi Wan, Ming Yan, Ji Zhang, et al. Solopo: Unlocking cong-context capabilities in 1lms via short-to-long preference optimization. arXiv preprint arXiv:2505.11 11666 2025.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.200 204.20444 2025.
13

---

Published as a conference paper at ICLR 2026
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop hop question composition. Transactions of the Association for Computational Linguistics, 10:539-554, 2022.
Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni, Kelvin Xu, Sanil Jain, Rakesh Shivanna, Jeffrey Hui, Nishanth Dikkala, Mehran Kazemi, Bahare Fatemi, al. Michelangelo: Long context evaluations beyond haystacks via latent structure queries, arXiv preprint arXiv:2409.126400 2024.
Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi Yang, Ji Zhang, Fei Huang, Jingren Zhou, and Ming Yan. Qwenlong-ll: Towards long-contexs large reasoning models with reinforcement learning. arXiv preprint arXiv:2505. 17667, 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery. and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint 2xXv:22233.1111, 2022.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
Yudong Wang, Zixuan Fu, Jie Cai, Peijun Tang, Hongya Lyu, Yewei Fang. Zhi Zheng, Jie Zhou, Guoyang Zeng, Chaojun Xiao, al. Ultra-fnewwb: Efficient data filtering and verification for high-quality Ilm training data. arXiv preprint a5X5v22505.05427, 2025.
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing Ilm reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502. 18449, 2025.
Tian-Shi Xu, Hsiao-Dong Chiang, Guang- Yi Liu, and Chin-Woo Tan. Hierarchical k-means method for clustering large-scale advanced metering infrastructure data. JEEE Transactions on Power Delivery, 32(2):609-616, 2015.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, et al. Qwen2 technical report. ArXiv, abs/2407. 10671, 2024a.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2.5-math technical report: Toward mathematical expert model via self- celf-improvement. arXiv preprint arXiv:2409.1 12122, 2024b.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09300 2025.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D) Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018
Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, and Danqi Chen. HELMET: How evaluate long-context models effectively and thoroughly. In International Conference on Learning Representations, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 11.50..14477 /4476, 2025.
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models. In International Conference on Machine Learning, 2024.
14

---

Published as a conference paper at ICLR 2026
Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du et al. Vapo: Efficient and reliable reinforeement learning for advanced reasoning tasks. arXiv preprint arX4v:250..05118 2025.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025.
Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang. Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint urXiv:2505.033: 503355....3,..... 2025.
Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang. and Arman Cohan. Doemath-eval: Evaluating math reasoning capabilities of Ilms in understanding long and specialized documents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging lim-as-a -andde-- with mt-bench and chatbot arena. Advances in neural information processing systems, 2023.
Yifei Zhou, Sergey Levine, Jason Weston, Xian Li, and Sainbayar Sukhbaatar. Self-challenging language model agents. arXiv preprint arXiv:2506.01716, 2025.
Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, et al. Ttrl: Test-time reinforcement learning. arXiv preprint arXiv:2504. 16084, 2025.
15

---

Published as a conference paper at ICLR 2026
A STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
In preparing this manuscript, we employ large language models (LLMs) purely as an assistive writing tool, without influencing the intellectual contributions of this work. Their use limited to checking grammar, correcting spelling, and improving the clarity and precision of the text. The proposal of the research question, the development of the methodology, and the experimental design are the original outputs are subject to critical review, editing, and
contributions of the authors. All final verification by the authors to ensure the authenticity of the content.
B LIMITATIONS AND FUTURE WORK
First, while our study provides strong empirical evidence, a theoretical framework explaining the co-evolutionary dynamics of the three roles within single model has yet be explored. Second, due to framework and efficiency constraints, our experiments are limited to a 16K input context. Although the acquired skills generalize well to longer contexts, it critical next step develop more efficient frameworks for self-play RL on ultra-long contexts, such as 128K tokens and beyond. Finally, our self-play framework still relies on a degree of human intervention, such as pre- pre-processing the document corpus and crafting prompt templates for each role. Future work could explore pathways toward greater autonomy, such system where an LLM interacts with a real-world environment to generate and evolve its own tasks, templates, and reward functions.
C RELATED WORK
C.I LONG-CONTEXT ALIGNMENT
Developing long-context language models (LCLMs) has become a central research area, as many real-world applications require reasoning paradigm in this field is to enhance models through various post-training alignment techniques on well-curated, synthesized datasets. One prominent approach is supervised fine-tuning (SFT), For instance, Algg (Bai et al. 2024a) utilizes a self-instruct pipeline to construct a large-scale, long-instructton dataset for SFT, while MIMG (Chen al. 2025c) employs a multi-agent system to generate more complex, multi-hop reasoning data. Another line of work focuses on preference optimization. LongPO (Chen et al., 2025a) and SoL.PO (Sun et al. 2025) generate preference pairs by contrasting outputs from compressed versus full contexts and leverage direct preference optimization (DPO) (Rafailov et al. 2023) to transfer short-context capabilities to longer inputs. More recently, QwenLong-Li (Wan et al. 2025) introduces the concept of long-context reasoning RL and employs a two-stage progressive context scaling training to develop long-context reasoning models. While proven effective, all these approaches exhibit some form of reliance external supervision.
extended inputs (Liu et al., (025b). A dominant
C.2 SELF-PIAY LANGUAGE MODELS
To mitigate the reliance on external supervision, such as human annotation or labeled datasets, researchers are developing self-play language models (Gao et al.) 2025). These models achieve autonomous improvement by generating their own training data, reward signals, or both. These approaches be categorized into two paradigms: multi-model optimization, which co-evolves several distinct models, and single-model optimization, where one model assumes multiple roles.
Multi-Model Optimization This paradigm orchestrates the co-evolution of multiple specialized models. In -aught (Shi et al. 2025), a policy model and a reward model are reciprocally and iteratively refined: the policy model generates data enhance the reward model, which in turn provides more accurate feedback to improve the policy model. Similarly, in R- R-Zero (Huang et al. 2025), challenger and a solver LLM are independently optimized and co-evolve: the challenger generates new, challenging math problems for which it is rewarded based on the solver's uncertainty, and the solver is fine-tuned on these questions, rewarded for correctly solving them. While effective, this paradigm substantially increases systemic complexity, and performance gains often plateau after a finite number of iterations.
Single-Model Optimization In contrast to the multi-model approach, single-model optimiza- tion reduces systemic complexity by leveraging a single model to assume multiple roles for self- improvement. For instance, Absolute Zero Reasoner (AZR) (Zhao et al. 2025) employs one model as
16

---

Published as a conference paper at ICLR 2026
both proposer that generates complementary coding tasks (induction, abduction, and deduction) and a solver that addresses them, with code execution feedback serving as the reward signal. Similarly, the Sclf-Chas Self-Challenging Agent (SCA) (Zhou et al. 2023) employs model to formulate novel "Code-as cill--s--ite "Code-as-Task"" questions and subsequently solve them, using self-generated code functions to provide the verification signal. A significant limitation of such approaches, however, is their dependence external code ecutors, which confines their applicability to domains with programmatically verifiable outcomes, To overcome this limitation and enhance autonomy, other approaches further incorporate self-rewarding mechanisms that leverage self- cefnissssst (Zuo et al. 2025), internal confidence scores (Li et al. 2025), or self-generated evaluations (Yuan et 2024). For example, Self Questioning Language Models (SQLM) (Chen et al. 2025b) utilize a model to propose and then answer questions, adapting its reward mechanism between self-consisteney for arithmetic and proposer-generatee unit tests for coding. Similarly, the Self Self-Rewarding Self-Improvin framework (Simonds et al. (025) also generates its own questions and solutions but uses self-nudging mechanism for reward computation.
Our work extends the single-policy self-play paradigm to long-context understanding and reasoning. Unlike existing methods that focus on short-context tasks like coding or math, SPELL is designed for reasoning over long documents. In framework, single LLM learns by playing three roles: questioner, a responder, and a verifier. These roles interact to create a self-sufficient learning loop for long-context comprehension, thereby addressing key gap in current self-play learning research.
D SPELL ALGORITHM
In this section, we outline the step-by-step algorithm for SPELL in Algorithm T
Algorithm 1 The SPELL Algorithm
1: Require: Initial policy model Te; Dataset D ()); Subset size History memory size
L; Batch size N; Group size
2: for (C,+)
do
3: Ho + Queue(0, L)
Initialize history memory for each document cluster
4: for each training step f = ........................ do
5: Ique Bros Bros, Br <- 0,0,0
Initialize empty sample batches for three roles
6:
while |B < do
7:
(C, D
Sample a document cluster and task type Sample subset of m documents for questioner
8:
Cnew f- SampleDocs Xque +- GerQuestionerContexl (Cnew; Hc) D Prepare questioner input; see $3.1 (g, a) ~ T que (-IXque)
9:
10:
Questioning: Generate a question with reference answer > Attempt to answer the question without document context > Grounding filter que q(X .q. a, quee 0.5)] Penalize and store ungrounded question
11:
context ~ T Tes (-1g)
12:
Rruk (yno _hoo_context: a) = then
Eque
14:
continue
b Discard question and proceed to the next iteration Responding:
15:
[ ~ u (-IC,)) for to do
Generate group of G responses
17:
G ~ m. you (-1g, Vi, a) Verifying: Generate G verifications for each response
que (17h)) h. ([rgg ComputeReward(gr a, {ys) , ( ((o2)))))))
A Compute role-specific rewards; see Eq. 44-Eo 7
20:
if que then PushTootistryyMemor(tHc
> Append solvable questions to the history memory
21:
q.a)
22:
B + T U ((Xque q. a, rTT))
ges - ges {(C, q, (s (re))))
Bver + Bver [(g. a, (ucho (Vighess (199)()))
25:
B f- SampleBatch(B#4 Bres BBer)
Apply role-specific dynamic sampling; see $33
26:
TO UpdatePolicy(Fe.. B)
Unified Policy Update; see $3.3
27: Ensure: Updated policy model T
17

---

Published as a conference paper at ICLR 2026
method techni
que
Impact
app
onn
LongBenck V2
strategy
ed
Tesear ch
a
data
on
nerwork
DucMaah
Franes
HopprizA
activity
500
ZwikiMuhihopOt
chemical
Count
MuSiQue
ana
VSS
600
eacti ion
coff
writing
st
sructture equation
concept
Fealculatinn
nanageni
Maated
n
ig
100
reac
Popeent
fonnt educationa
0k-4k
4k-1dk
Token Length Interval
16k-52k
32k-64k
64k-128k
128k+
theory word
th
Figure (Left) The word cloud training document lahels, with font size indicating frequency. The prominence of terms like education, analysis, and learming underscores the dataset's focus on knowledge-intensse content. (Right) The token length distribution, calculated by the Qwen2 tokenizer. shows that our evaluation benchmarks cover wide spectrum of context lengths.
E IMPLEMENTATION DETAILS
E. TRAINING DATA CONSTRUCTION
Our training data supports three distinct tasks: document general QA, financial math QA, and multiple-choice QA. We construct the dataset from two complementary sources. The first is the DocMath dataset (Zhao et al., 2024). which provides specialized data comprising long, complex financial reports that necessitate numerical reasoning. We used only the raw documents, discarding the original unlabeled questions. From this dataset, we select 2,150 instances with a total token length below 16K, which are designated for the financial math QA task.
The second component is a general-domain corpus designed to enhance the model's fundamental document understanding. This dataset originates from the 1.16 billion English documents in the Ultra-Fineweb corpus (Wang et al. 2025) and is curated through the following multi-stage pipeline. First, an initial filtering stage retains high-quality texts of appropriate length by selecting documents with perfect quality score of 1.0 and character lengths between 100 and 32,768. This step downsam- ples the corpus to a high-quality subset of 4 million documents. Second, a cleaning procedure ensures data diversity and prevents test set contamination. We employ the MinHashl algorithm (Shri? vastava & Li, 2014) with a threshold of 0.8 and 128 permutation functions for deduplication and to decontaminate the data against all documents in our evaluation benchmarks. This cleaning phase refines the corpus to 1.1 million unique and decontaminated texts. Third, a hierarchical clustering approach structures the dataset into different topics. We generate -196-dimensional embeddings for each text using the Ereenddn 8B model (Zhang et al.) 2025), which are then grouped into 50,000 distinct clusters via hierarchical k-means algorithm (Xu al. 2015). Each cluster contains an average of 20 semantically related documents, and the resulting domain distribution, visualized in confirms the dataset's broad topical diversity. This curated corpus from Ultra- Finte- is used to generate training instances for the document general QA and multiple-choice tasks. To ensure comparable scale across tasks, the number of clusters selected for these two tasks matches the data size of the DocMath portion. Ultimately, our combined training data consists of 6,450 unique document sets, with 2,150 designated for each of the three tasks.
Figure (left), (leff),
E.2 EVALUATION DETAILS
We evaluate our models using suite of well-established benchmarks designed to assess long-context comprehension and reasoning. These benchmarks fall into two primary categories: multiple-choice and multi-hop question answering (QA). For the multiple- choice task, we use LongBench-V2 (Bai et al. 2025). benchmark of 503 questions that assesses deep comprehension across six areas: single-document QA, multi-document QA, long in-context learning, long-dialogue history understanding, code repository understanding, and long structured data understanding. For multi-hop QA, our evaluation incorporates several benchmarks: Frames (Krishna et al. 2025), containing 824 questions diverse Wikipedia topics such history, sports, science, animals, and health; three subsets from LongBench (Bai et al. 2024b), each with 200 questions, namely HotpotQA (Yang al. 2018) (2-hop), 2WikiMultihopOA (Ho et al. 2020) (requiring up
Evaluation Benchmarks
18

---

Published as a conference paper at ICLR 2026
to five hops), and MuSiQue (Trivedi et al. 2022)
(requiring up four hops); and DocMath (Zhao
eet al.] 2024), which focuses on numerical reasoning within financial reports. For DocMath, we use the testmini subset of 800 queries, which is orthogonal to training data. As shown in Figure (right), the test instances across these benchmarks cover a wide range of context lengths.
Evaluation Configurations We evaluate all models at two maximum input lengths: 16K tokens, which aligns with our training configuration, and 100K tokens to test for generalization to longer contexts. The maximum generation length is 4K tokens for non-reasoning models and is extended to 20K tokens for reasoning models. For prompts exceeding the maximum context window, we employ the middle truncation strategy from [Bii et al.) (2024b) to preserve the front and tail portions of the context. All experiments are conducted using a sampling temperature of 0.7 and a top-p value of 0.95. For each query, generate main experiments and the pass @ k metric for the test-time scaling analysis. The pass@k metric is an unbiased estimator for the probability that at least one of & candidate solutions is correct, given candidates per problem, of which are correct. It is calculated as:
8 candidate responses, reporting the average score (pass @ 1) for
pass @k =
(11)
Our scoring is tailored to each benchmark's s format. For multiple-choice tasks, we report standard accuracy. For multi-hop QA tasks, simple string matching is often insufficient to assess the semantic correctness of free-form text answers. Thus, we supplement the cover exact match (CEM) score with LLM-as- a--uud (Zheng al. (023), which uses gpt-oss- 120b (OpenAl 2025 evaluate semantic equivalence between a model's 's prediction and the ground-truth answer. T'ee prompt for this evaluation is detailed in Table The final score for these tasks is the maximum of the two, providing a more comprehensive and robust assessment of model performance.
Table Prompt template LLM-as-a-judge to compare predicted answer the ground truth given question. Modified from Frames Krishna et al. 2025)
LLM Judge Prompt for Multi-Hop QA Tasks.
## TASK
I need your help in evaluating answer provided by LLM against ground truth answer. Your task is to determine if the ground truth answer is present in the LLM's response. Please analyze the provided data and make decision. ## Instruction
1. Carefully compare the "Predicted Answer" with the "Ground Truth Answer".
2. Consider the substance of the answers look for equivalent information or correct answers.
Do not focus on exact wording unless the exact wording crucial to the meaning.
3. Your final decision should be based on whether the meaning and the vital facts of the
"Ground Truth Answer" are present in the "Predicted Answer".
4. Your decision must be one of the "[[YES]]] or "[[NO]]".
## Input Data
Question: {question)
Predicted Answer: |predicted answer)
Ground Truth Answer: (ground truth)
## Output Format
Provide your final evaluation in the following format: "Explanation:" "How you made the decision"
"Decision:" " "[[YES]]" "[[N]]""
Please proceed with the evaluation.
E.3 RL ALGORITHM DETAILS
To enhance stability and practical performance, we integrate two key techniques inspired by Decou- pled Clip and Dynamic Sampling Policy Optimization (DAPO) (Yu al.) (2025) for the baselines and our method. First, we adopt a token-level policy gradient loss, which normalizes each token's contribution by the total number of tokens in the group. This approach ensures every token in the
19

---

Published as a conference paper at ICLR 2026
same group contributes equally to the final objective, which prevents the learning signal from valuable tokens in high-quality, long responses from being diluted while ensuring that undesirable patterns in low-quality. lengthy outputs are effectively penalized. Second, employ dynamic sampling strategy: any group of generations where the rewards [ri) G exhibit zero variance discarded from the training batch. This ensures the advantage in Eq. is always well-defined.
Consistent with recent findings suggesting that removing KL regularization can improve exploration and accelerate convergence (Hu al. (025 Yu et al. 2025 Wan al.) (025). we set O. Besides, we operate in a strictly on-policy setting, performing only a single gradient update per batch of samples. This design choice implies that the policy being updated, Te, remains identical to the policy that generated the data, Tew.. Since the importance sampling ratio (d) is strictly equal to I, the clipping function becomes inactive, and we can remove it from the objective. Note that the advantage A; is independent of I, the training objective in Eq. simplifies to:
(o) = E (Dg)) MER
A
(0)
(12)
2 VJ
E.4 DETAILS OF OPEN-SOURCE MODELS AND THE DATASET
In T we provide the Huggingface repository names of all policy models, the embedding model, the judge model, and datasets used in experiments.
Table 6: Details of open-source models and datasets in experiments.
Model/Dataset
Huggingface ID
Qwen2.5-775 5-7B
Qwen/ Swccn//wcn2.557BB
Qwen2.5-14
Owen/ Srren/Qwen2.5-111
Qwen2.5-32B
Qwen/ Stoen/OwenZ..-322
Qwen2.5-7B-Insse
Owen/OwenZ..
Qwen/(Qeen.5-114h)
Swen2.5-32B-Insttuu
Qwen3-30B-AA3B-nnsti
incaa-lanaa/Meaas-laaaa11111111 Instruct
R1 R1-Distill-Llama
Deepseelaii/Deppsck-----Diiilll 8B
R1-Distill-Qwen
14B
deepseek
DpppssccaaaDDeppeecKK-iSSiiiuu 14B
Qwen3-4B-Thinkiee
Qween/(wwenS-BBBTTTnn
Qwen3-30B-
Qwen/((wen3-300-AA11 1hw.---g--hnnnngg2221115
Qwen3-Fmbeedinngg3
gpt-oss-120h
openal/gnt-oss-120g
DocMath
yale-nlp/DocMath-tHi
Ultra-Fineweb
openbmb/ CatubmbbUltta-PPe Ultra Web
E.5 DETAILS OF BASELINES
We compare SPELL with two categories of baselines. The first category comprises the base models on which SPELL is built, organized into three groups: (1) base models, including Qwen-2.5-7B, Qwen- 2.5-14B, and Qwen-2.5-32B: (2) instruction-tuned models, including Qwven-2.57BB-inntruee
Qwen-
2.5- In-sstet-,sttruct, 2wwen22.5-32B-Innmre
and Llama-3. 1.taa-..118----- -88...111--s--trcct and
(3) reasoning models, including R1-Distill-LLann Liastill-Llaaa---- 8B. RI-Distill Swailll--we----- 14B, Owen3-4B-Thie Thinnking
and 1-3-33---A3BThhine
(Yang et al. 2025: 2024a Guo et al. 2025 Dubey et al. 2024).
The second baseline is traditional RLVR. For data construction, we use SeepSeek-R1-0552 (Guo) et al. 2025) as both the questioner and responder to synthesize dataset from the same document clusters used for SPELL. The synthesized dataset is then filtered by a verifier model (gpt-oss-I -spt-oss-120b), 120b), and retains only the instances where the answers from the questioner and responder are identical. This process yields a dataset of approximately 3,000 verifiable samples that covers multiple-choice document general QA. and financial math QA tasks. For RLVR training, we employ the cover exact match (CEM) as the rule-based reward function. We generate eight trajectories per question and maintain all other hyperparameters identical to those of SPELL to ensure fair comparison.
20

---

Published as a conference paper at ICLR 2026
F ADDITIONAL ANALYSIS
This section provides further insights into the properties of SPELL. We begin by analyzing the training cost in Section F.I and exploring generalization short-context tasks in Section F.2 We
then extend our evaluation to additional long-context benchmarks in Section F.3 and compare our
approach with state-of-tee of-the-art long-context alignment baselines in Section F.4 The section concludes
with an analysis of the training dynamics of role evolution in Section F.3 and a discussion on reward hacking risks and mitigations Section
F.I TRAINING COST ANALYSIS
We analyze the computational cost of SPELL using the Qwen2.5-7B-Be Inn-ttuccttttt model single node with 8 x 80GB NVIDIA A100 GPUs. Figure 7 Iillutraates the time breakdown for the two primary stages of our framework: role-specific rollout (questioning, re- sponding, and verifying) and unified updating. The total time per training step averages approximately seven minutes, and the total training cost is about eight hours. Although the number of verifier rollouts times greater than that of the responder, as out- lined in Section 3.3 its computational cost is lower. This efficiency stems from two factors: the verifier processes shorter inputs, it does not require the long documents provided to the responder, and its task of generating brief verifications is less demand- ing than the responder's task of reasoning over long contexts. During the later stages of training, the time for verifying constitutes only about half of the time required for responding. Consequently, generating G' verification judgments for each response to create a reliable reward signal does not introduce significant computational bottleneck. As shown in Section 4.3 and Table the verifier provides a significant 3.2 3.2-point performance gain (from 44.0 to 47.2). The significant performance gain by introducing the verifier far outweighs the minor increase training cost.
Questioning Responding
Verifying Updasing
200
Time
100
10
20
40 Training Steps
30
50
Figure 7: Time breakdown per training step for the four main stages of SPELL. Verifying constitutes small amount the total cost.
F.2 SHORT-CONTEX REASONING RESULTS
We further investigate the generalization of our proposed SPELL method from long-context reasoning short-context reasoning tasks. Our evaluation suite includes five mathematical benchmarks and the MMLU Pro general knowledge benchmark as follows:
AIME 24/25 (MAA 2025) is the American Invitational Mathematics Examination, prestigious high-school mathematics competition administered by the Mathematical Association of America (MAA). The AIME consists of two exams (I and II) annually, each containing 15 problems, for total of 30 problems per year. The answer to each question is an integer from 0 to 999, These problems demand deep mathematical knowledge and creative mroblem-solving strategies, making them challenging benchmark for advanced mathematical reasoning.
AMC 2 Yang et al., 2024b) refers to the 2023 American Mathematics Competition. This competi- tion features 25 questions designed to test advanced high school mathematics, covering topics such as trigonometry, advanced algebra, and advanced geometry.
MATH (Hendrycks al. 2021b) is dataset of math problems ranging in difficulty from middle school to high school competition level. It is designed to test a wide range of mathematical skills, including algebra, geometry, number theory, and counting & probability. For our evaluation, we utilize a subset of 500 problems, referred to as MATH-500.
GSM8K (Cobbe al. 2021) is dataset of 1.3 grade school math word problems. These problems are designed to be solvable by capable middle school student and require two to eight steps of reasoning using basic arithmetic operations.
httta:://uggingaaacc//aaassssss 1o-aaaa0aaaaaaaaama idat ion- amc
21

---

Published as a conference paper at ICLR 2026
MMLU-Pro (Wang et al. 2024 is enhanced version of the MMLU (Hendryeks al. 202 la dataset, designed to address issues such as noisy data and reduced difficulty resulting from ad advances in model capabilities and increased data contamination. MMLU-PPu raises the difficulty by expanding the multiple-choice options from four to as many as ten and incorporating expert-reviewed annotations for improved quality and reduced noise.
Table 7: Evaluation results base models on short-context reasoning tasks.
Model
AIME24
AIME25
AMC23
MATH500
GSM8K
MMLU Pro
Average
Qwen2.5-7B
5.42
3.33
33.44
53.60
76.57
40.24
35.43
+ SPELL
9.17
5.00
40.31
63
86,28
49.78
42.36
Qwen2.5-14B + SPELL
6.67
5.83
37.81
61.68
84.31
46.67
40.50
12.08 1541
10.42:43
50.31 n12:
72.40'm022
91.36'
58.86 12219
49.24 8.74
Qwen2.5-32B
9.17
5.83
45.31
66.25
87.34
48.
43.80
+ SSEELL
15.83
8.33
55.62
76.00
90.25
60.22
51.01-724
Consistent with our main experiments, all models are evaluated with a maximum output of 4K tokens, a sampling temperature of 0.7, and a top-p value of 0.95. We report the accuracy averaged over 8 independent runs for each task. As shown in Table 7 SPELL, trained solely on the long-context data, improves performance on short-context reasoning benchmarks across all base models. The average scores of Qwen2.5-7B, Qwen2.5-144 and Qwen2.5-32B increase by 6.93, 8.74, and 7.24 points, respectively. The consistent gains indicate that reasoning competencies acquired through long-context self-play transfer effectively to short-context settings.
F.3 ADDITIONAL LONG-CONTER BENCHMARKS
We further evaluate SPELL on two challenging long-context benchmarks: MRCR (Vodrahalli et al. 2024 a multi-needle "Needle in Haystack" " benchmark, and three subsets HELMET (enn et al. 2025). which covers Setrivval-Augmenter Generation (RAG), In-Context Learning (ICL), and Summarization (Summ). We evaluate the Qwen2.5 base models against the RLVR baseline and SPELL with maximum input length of 16K and maximum output length of 4K. The results in Table show that SPELL consistently and significantly outperforms both the base models and the RLVR baseline across these diverse long- long-context tasks, demonstrating strong generalization capabilities beyond the standard QA tasks.
Table 8: Evaluation results for base models on MRCR and HELMET subsets. The best score in each model group is highlighted in bold.
Model
MRCR.2needle
MRCR-Sneedle
Helmet-RAG
Helmet-ICLL
Average
Qwen2.5-7B 6.9
2.0
2.2
50.0
3,5
4.1
11.5
RLVR
22.0
10.5
2.6
18.5
SPELLL
34,5
16.5
16.0
54.2
10.4
13.7
24.2
Qwen22.5114B
25.1
10.5
10.0
42.4
1.5
15.2
RL.V
20.9
9,5
9.1
46.7
42.0
24.7
25,5
17.8
Qwen2.5-32B
27.0
11.5
12.0
59.0
42.8
16.2
28.1
RLYR
36.7
14.6
13.0
52.7
16.7
21.0
25.8
SPELL
38,0
18.5
14.7
61.4
56.4
21.2
35,0
F.4 COMPARISON WITH LONG-CONTERS ALIGNMENT BASELINES
We compare SPELL against three recent long-context alignment baselines baselines-SoLowl SoLoPO (Sun et al. 2025), LongPO (Chen et al. 2025a), and QwenLong-Li (Wan et al. 2025) Qwen2.5-7B- Instruct the base model. To ensure fair comparison, reimplement these methods using the same document corpus employed in SPELL. For SoLoPO and LongPO, the core step is to con- struct preference pairs from short contexts containing key information and long contexts containing distractors given the same question. Specifically, for each data instance comprising n documents, we first randomly sample documents as the short text and employ DeepSeek-RI-Os as
s http:://tuggiggfsssccooossssssap
22

---

Published as a conference paper at ICLR 2026
the questioner to generate QA pairs. Then, swer the proposed questions using the full set of n documents. We retain only those QA pairs where the answers from the questioner and responder are consistent. Next, uments from the questioning stage as short texts, corresponding to Fishort in SoloPO and Is in LongPO, and take all documents long texts, corresponding Flong in SoLoPO and 2t. in LongPO. Finally, we apply their respective preference pair construction strategies and training configurations on Qwen2.5-7B-Instruuc to reproduce these methods. For QwenLong-LI, we use the same synthesized data as our RLVR baseline and follow their official GRPO training setup. We evaluate both our reimplemented models and the official I LongPO checkpoint using a maximum input length of 16K and a maximum
DeepSeck-R1-0529 the responder to
take the 5 doc-
Table 9: Comparison of SPELL against different long-context alignment baselines. The best score is highlighted in bold
Model
DocMath
Frames
LB-MQA
LB-V2
Average
Dowan..5-B-innsee
38.4
40.3
45.1
29.0
38.2
RLLRR
+ -LongPO (Reimpl.)
41,4
44.2
53.7
32,0
42.8
output length of 4K. The results in Ta-
+ +LongPO (Official)
42.3
41.1
15.7
30,9
10.1
SoLoPO (Reimpl.)
45.3
43.9
56.0
31,6
44.2
ble 9 demonstrate that SPELL consis-
-Quent.ong-L11 ong-1.1 (Reimpl.)
tently outperforms these long-context alignment baselines.
SPELL.(Ours)
33.2
F.5 EVOLUTIONARY DYNAMICS OF QUESTIONER AND VERIFIER
To understand the self-evolutionary process and identify potential failure modes, analyze the behavior of the questioner and verifier roles across different training steps.
Questioner dynamics We track the dis- tribution of valid questions generated by the questioner throughout the training pro- cess. As shown in Figure a)), the task distribution notably imbalanced during the first 10 steps, with Financial Math QA accounting for over 70% of solvable tasks. This imbalance likely occurs because the model transfers its strong mathematical rea- soning capabilities the Financial Math QA task, which necessitates substantial nu- merical calculation. As training progresses and the policy evolves, the distribution be- comes increasingly balanced. This indi- cates that SPELL effectively drives the questioner explore diverse range of task types.
Gronal GA
Fonvii Mirk
Marplo-taie
Desgrement Rac PN
16
Trakking Stepe
(a) Questioner Task Distibutionn
(b) Verifier Disagreement Rate
Figure 8: (a) Evolution of the task distribution generated by the questioner. (b) The disagreement rate between the learned verifier and the rule-based judge.
Verifier calibration We analyze the disagreement rate between the verifier's 's majority vote and the rule-based judge (CEM). Figure b) plots this metric for General QA and Financial Math QA, both of which contain questions that are partially non-verifiable by strict rules. Initially, the verifier struggles with General QA, which often involves open-ended semantic equivalence that rule-based checks fail to capture. In contrast, Financial Math QA exhibits a lower initial disagreement rate, attributed to the model's 's relatively strong numeric reasoning ability. The disagreement rate consistently decreases for both tasks, indicating that the verifier progressively aligns with the rule-based judge. This trend further suggests that the verifier's 's updates guide the questioner toward generating questions that are more verifiable by the rule-based judge.
F.6 ANALYSIS OF REWARD HACKING
Reward hacking a significant concern in self-play systems. Throughout our exploratory experiments, we identified several potential failure modes and implemented specific mitigations.
Questioner stagnation Without the automated curriculum and history memory, the questioner tends to repeatedly propose similar, trivial questions about the same document to maximize the responder's ' success rate. The history memory module conditions the questioner on recently solvable QA pairs and newly introduced documents. The prompt in Appendix G explicitly instructs the questioner to produce novel and more difficult questions.
23

---

Published as a conference paper at ICLR 2026
Responder mode collapse When the responder receives rewards for outputs that are merely substrings of the ground-truth answer, it can hack the metric by generating short phrases like "The answer is". We address this by enforcing stricter cover exact match (CEM) criteria, which requires the responder to include the complete ground-truth answer to receive a positive reward.
Verifier self-delusion Updating the verifier solely based on its own majority vote as a pseudo-label can lead to verifier hacking. Without external supervision, verification errors accumulate, eventually causing the verifier to label all responders' outputs as correct. To mitigate this, we introduce the consistency check mechanism, which aligns the verifier's judgments with rule-based rewards verifiable questions, thereby preventing this self-delusion.
These observations confirm that the architectural components of SPELL memory, prompt templates, consistency checks, and CEM-based reward function- are essential to mitigate reward hacking and ensure stable self-evolving.
the history
G PROMPT TEMPLATE
In this section, we detail the prompt templates for the questioner, responder, and verifier across all tasks. For the questioner, apply different prompting strategies for document clusters with and without history memory; these prompts are modified from the guidelines for human annotators in LongBench-V V2 (Bai al. 2025). The responder prompt for financial math QA is modified from DocMath (Zhao et al. 2024). and the prompts for document general QA and multiple-choice modified from LongBench-V2 V2, The verifier prompt for document QA tasks modified from Frames (Krishna et al. 2025) and requires the model to evaluate the semantic equivalence of generated answer against ground-trui trrtt reference.
Questioner Prompt for Document General QA Task without History Memory
You are an expert in document analysis. We are building a benchmark to evaluate the capabilities of large language models (LLMs) fact retrieval, reasoning across multiple constraints, and accurate synthesis information into coherent responses. Your primary task is propose challenging question based the provided document context enclosed between <text> and </text>, The question must require both document comprehension and multi-hop reasoning. You must also provide the correct answer and detailed step-by-step derivation showing how the answer is obtained from the document.
## Principles for Question Design
Adhere strictly to the following principles when crafting your question, answer, and derivation Questions, answers, and derivations must be English.
1. Language Requirement:
2. Standalone & Context--ndeepeddent Questions should not contain any references to
"Article 1", "Article 2", etc. They should be understandable without any additional context.
3. Unambiguous Answer: Each question should have single, clear, and factual answer.
4. Multi-hop Reasoning: Answering each question should require combining information
from ALL provided documents. The final answer cannot be found in any single document.
Guideline for Question Phrasing: Strive for a natural and seamless integration of
information from each document. A good question often:
Starts with a clear question word (What/How/W http///ow/hhee/WWeeee
Links constraints from different documents using logical connectors.
Example connectors: 'in relation to', 'given the condition of", 'as result of", 'which also affects'. 'in addition to'.
6. Answer & Step-by-Step Derivation:
The answer must be concise phrase or sentence. An answer with more than 20 tokens forbidden.
The derivation must be a clear, step-by-step logical chain. Each step must explicitly cite the specific data point phrase and its source from the context (e.g., "From Table 3, Row "Revenue', Year 2023..." or "As stated in paragraph 2..."). ## Output Format
Your response must conclude with a JSON object containing the following keys: "question" and "answer", placed after your reasoning.
24

---

Published as a conference paper at ICLR 2026
"question":
"<A well-structured English question that adheres to all design principles>",
"answer": "<A concise answer, under 20 tokens>".
## Document Context
<text>
[context]
</text>
Questioner Prompt for Financial Math QA Task without History Memory
You are an expert in document analysis and numeric reasoning. We building a benchmark to evaluate the numerical reasoning capabilities of large language model's (LLMs) when analyzing specialized documents containing both text and tables. Your primary task is propose a challenging question based on the provided document context enclosed between <text> and </text>. The question must require both document comprehension and multi-step mathematical reasoning to arrive at single, non-zero numerical answer. You must also provide the correct numerical answer and detailed step-by-step derivation showing how the answer is obtained from the document.
## Principles for Question Design
Adhere strictly to the following principles when crafting your question, answer, and derivation:
1. Language Requirement: Questions, answers, and derivations must be English.
2. Complexity and Reasoning Depth:
The question must be challenging, requiring the LLM to go beyond simple retrieval. should not be solvable trivially or in few inference steps.
It must involve multi-step mathematical reasoning (e.g., requiring two or more distinct calculation steps).
It should necessitate integration of information from different parts of the document (e.g., combining data from a table with information from a text paragraph, or using multiple rows/columns from table).
Aspects like summarization or complex information extraction can be part of process.
3. Avoided Question Types:
Simple Counting: Avoid questions like "How many X are there?" X is easily countable or directly stated. If counting is involved intermediate step for larger calculation and the count is small (<=10), it's acceptable.
Direct Retrieval: Avoid questions answerable by looking up single, isolated piece of information.
Questions should primarily rely on the provided document. Only common sense or minimal domain -senaiii knowledge (e.g., basic financial concepts like 'profit revenue cost' if contextually appropriate and derivable) inferable from the document is allowed.
Excessive External Knowledge:
4. Information Obscurity:
Start with clear question word Sethaa/How/Whhre//hhhaee
Do not explicitly mention or paraphrase key numerical values from the document within the question itself. The LLM should identify and extract these values.
Phrase questions to require inference and understanding relationships between data points rather than just locating them.
5. Factual Grounding:
All information required answer the question must be present in directly derivable from the provided document.
Do not introduce hypothetical scenarios, fictional data, or assumptions supported by the document.
Questions should not contain any references "Article 1", "Article 2", etc. They should be understandable without any additional context.
6. Numerical Answer:
The final answer must be single non-zero numerical value.
An answer with more than two numerical values is unacceptable.
25

---

Published as a conference paper at ICLR 2026
If the document implies units (e.g., millions of dollars, percentages), the question should be phrased such that the numerical answer alone sufficient (e.g., "What is the value in millions of dollars?" rather than expecting the answer to include "million dollars").
7. Step-by-Step Derivation:
Provide a clear, hep-by-step derivation for your answer.
This derivation must explicitly reference specific data points or phrases from the document (e.g.. "From Table 3, Row 'Revenue', Year 2023..." "As stated in paragraph 2..."). Detail all mathemattial operations performed in each step. This helps verify the question's solvability and reasoning path. ## Output Format
Your response must conclude with JSON object containing the following keys: "question" and "answer", placed after your reasoning.
"question": "<A well-structured English question that adheres to all design principles>", "answer": "<A single, non-zero numerical answer>"
## Document Context
<text>
[context)
</text>
Questioner Prompt for Document Multiple-Choice Task without History Memory
You are an expert in document analysis. We are building a benchmark to evaluate the capabilities of large language models (LLMs) on fact retrieval, reasoning across multiple constraints, and accurate synthesis of information into coherent responses. Your task to generate multiple choice question based on the provided document context enclosed between <text> and </text>. The question must require document comprehension and multi-hop reasoning. You must provide one correct answer and three plausible, distinct distractors. Crucially, you must also provide detailed explanation for why correct answer is correct (including derivation steps) and why each distractor is incorrect. ## Principles for Question and Option Design
Adhere strictly to the following principles when crafting your question, answer, options, and derivation:
1. General Requirements:
All questions, options, and explanations must be in English.
Questions should be challenging, requiring more than simple retrieval or a few inference steps.
2. Cognitive Complexity Requirements for the Question:
Must necessitate multi-step reasoning (e.g., involving three more distinct logical or calculation steps).
Should require the integration of at least three distinct data points from different parts of the document (e.g., combining data from a table with text, or using multiple rows/columns/cells, Should demand the synthesis of quantitative data with qualitative information found in the text.
The problem setup should have the potential for common misinterpretations, which will inform distractor design.
3. Content Validity Criteria:
The question and all options must be exclusively answerable using information from the provided document. No external knowledge beyond common sense or very basic, universally understood concepts (e.g., profit revenue cost, if directly applicable and data is provided) should be required.
If applicable to the document type (e.g., financial reports), prioritize questions with regula- tory/compliance implications or those highlighting significant financial outcomes. Ensure numerical values involved in the question or options require contextual interpretation within the document, not just direct look-up.
26

---

Published as a conference paper at ICLR 2026
Avoid trivia; focus on questions that address material information key insights derivable
from the document.
4. Distractor Development Guidelines:
Each of the three distractors must be plausible yet clearly incorrect upon careful analysis.
Distractors should represent distinct error paths or common misinterpretations.
At least one distractor should represent a common conceptual misunderstandinng related to the document's 's content how information is presented.
5. Forbidden Question/Option Patterns:
Simple Counting: Avoid questions like "How many X there?" if X is easily countable directly stated. Small counts (<=5) as part of larger calculation are acceptable.
Direct Retrieval: Avoid questions where the answer (or its direct components) can be found in single, obvious location without further processing.
Excessive External Knowledge: Questions must not require significant domain-speciin knowledge not provided or clearly inferable from the document.
No Fabricated Information: Strictly adhere to document content. Do not introduce hypothetical scenarios, data, assumptions not explicitly stated or directly inferable.
Ambiguous Scenarios: The question must have one unambiguously correct answer based solely on the provided document.
Vague Options: All options, including distractors, must be precise and unambiguous,
6. Answer and Explanation Requirements:
The correct answer must be
A detailed derivation for the correct answer must be provided, showing step-by-step calcu- lations and referencing specific parts of the document (e.g., "From Table X, Row Y...", "As stated in paragraph Z...")
For each distractor, provide brief explanation of why it is incorrect, ideally linking it to the type of error it represents (e.g., "Option incorrect because it omits the X deduction mentioned in...", "Option B results from incorrectly summing X and Y instead of finding their difference...").
## Output Format
Your response must conclude with a JSON object containing the following keys: "question", "options", and "answer", placed after your reasoning.
"question": "<A well-structured multiple choice English question, exclude choices and an- swer>"
"options":
"A": "<Text for choice A="
"B": "<Text for choice B>"
"C": "<Text for choice C>"
"D" "<Text for choice D>"
}.
"answer"; "ccorrect
## Document Context
<text>
[context)
</text>
Questioner Prompt for Document General QA Task with History Memory
You are an expert in document analysis. We are building a benchmark to evaluate the capabili- ties of large language models (LLMs) on fact retrieval, reasoning across multiple constraints, and accurate synthesis of information into coherent responses. Your primary task is to propose ONE new, significantly more difficult question based on the provided document context and set of existing, simpler questions. The new question must be fundamentally different and more complex than the provided examples, requiring both document comprehension and multi-hop reasoning. You must also provide the correct answer and a detailed step- stepp derivation showing how the answer is obtained from the document.
27

---

Published as a conference paper at ICLR 2026
## Principles for Question Design
Adhere strictly to the following principles when crafting your question, answer, and derivation Questions, answers, and derivations must be in English.
1. Language Requirement:
Standalone & Context-Independent: Questions should not contain any references to
"Article 1", "Article 2", etc. They should be understandable without any additional context.
3. Unambiguous Answer: Each question should have a single, clear, and factual answer.
4. Multi-hop Reasoning: Answering each question should require combining information
from ALL provided documents. The final answer cannot be found in any single document.
5. Guideline for Question Phrasing: Strive for natural and seamless integration of
information from each document. A good question often:
Starts with a clear question word That/Hoo/WWher/WWW
Links constraints from different documents using logical connectors.
Example connectors: 'in relation to', "given the condition of", 'as result of", 'which also affects', 'in addition to'.
Escalate Question Difficulty: The new question must demonstrate higher order of reasoning than the Previous Examples. First, analyze the examples to identify their simple reasoning patterns (e.g., fact retrieval, single-step comparison). Then, create a new question that incorporates one more of the following advanced reasoning types:
Quantitative Reasoning & Calculation: Requires performing mathematical operations (e.g..
addition, subtraction, percentage change, averaging) data from multiple sources.
Comparative & Superlative Analysis: Requires comparing multiple entities based on synthe- sized criteria find the one that highest, lowest, best, etc.
Conditional or Causal Reasoning: Structured as an "if-then" scenario or asks for the cause/effect of situation by linking different documents (e.g.. "What would be the total cost if the discount from Document A were applied to the price listed in Document B?").
Synthesis Across Data Types: Forces connection between qualitative information (e.g., a policy description) and quantitative data (e.g., number in table) to reach conclusion.
7. Answer & Step-by-Step Derivation:
The answer must be concise phrase sentence. An answer with more than 20 tokens is forbidden.
The derivation must be clear, step-by-step logical chain. Each step must explicitly cite the specific data point or phrase and its source from the context (e.g., "From Table 3, Row 'Revenue', Year 2023..." or "As stated in paragraph 2..."). ## Output Format
Your response must conclude with a JSON object containing the following keys: "question" and "answer", placed after your reasoning.
"question": "<A well-structured English question that adheres to all design principles>", "answer": "<A concise answer, under 20 tokens>".
## Document Context
<text>
(context)
</text>
## Previous Examples
### Example
Question: (question 1|
Answer: (answer 1}
### Example K:
Question: (question k) Answer: (answer k)
Questioner Prompt for Financial Math QA Task with History Memory
You are an expert in document analysis and numeric reasoning. We are building a benchmark to evaluate the numerical reasoning capabilities of large language models (LLMs) when
28

---

Published as a conference paper at ICLR 2026
analyzing specialized documents containing both text and tables. Your primary task is to pro- pose ONE new, significantly more difficult question based on the provided document context and set of existing, simpler questions. The new question must be fundamentally different and more complex than the provided examples, requiring both document comprehension and multi-step mathematical reasoning arrive at single, non-zero numerical answer. You must also provide the correct numerical answer and a detailed step-by-step derivation showing how the answer is obtained from the document.
## Principles for Question Design
Adhere strictly the following principles when crafting your question, answer, and derivation:
1. Language Requirement: Questions, answers, and derivations must be in English.
2. Complexity and Reasoning Depth:
'The question must be challenging, requiring the LLM to go beyond simple retrieval. It should not be solvable trivially or few inference steps.
It must involve multi-step mathematical reasoning (e.g., requiring two or more distinet calculation steps).
It should necessitate integration of information from different parts of the document (e.g., combining data from a table with information from a text paragraph, or using multiple rows/columns from table).
Aspects like summarization or complex information extraction can be part of the process.
3. Avoided Question Types:
Simple Counting: Avoid questions like "How many X are there?" X is easily countable or directly stated. If counting is involved as intermediate step for larger calculation and the count is small (<=10), it's acceptable.
Direct Retrieval: Avoid questions answerable by looking up single, isolated piece of information.
Excessive External Knowledge: Questions should primarily rely on the provided document.
Only common sense or minimal domain-specifie knowledge (e.g.. basic financial concepts like 'profit = revenue cost" if contextually appropriate and derivable) inferable from the document is allowed.
4. Escalate Question Difficulty: The new question must demonstrate higher order of reasoning than the Previous Examples. First, analyze the examples to identify their simple reasoning patterns (e.g., direct lookups, single calculations). Then, create a new question that incorporates one more of the following advanced reasoning types:
Period-over-Period Calculation: Requires calculating growth, decline, or change between different time periods.
Ratio or Metric Derivation: Requires calculating financial metric or ratio not explicitly stated in the document.
Aggregation and Filtering: Requires aggregating data across multiple
rows/colummss/sections after filtering based on a text-based condition.
Projection or Implication: Requires using data from the document answer "what if" forward-looking question based only on the provided numbers.
5. Information Obscurity:
Start with a clear question word Baap/Hoo/Whhee//eeeee
Do not explicitly mention paraphrase key numerical values from the document within the question itself. The L.MM should identify and extract these values.
Phrase questions require inference and understanding relationships between data points rather than just locating them.
6. Factual Grounding:
All information required to answer the question must be present in or directly derivable from the provided document.
Do not introduce hypothetical scenarios, fictional data, assumptions not supported by the document.
Questions should not contain any references to "Article 1", "Article 2", etc. They should be understandable without any additional context.
7. Numerical Answer:
The final answer must be a single non-zero numerical value.
An answer with more than two numerical values is unacceptable.
29

---

Published as a conference paper at ICLR 2026
If the document implies units (e.g., millions of dollars, percentages), the question should be phrased such that the numerical answer alone sufficient (e.g., "What is the value in millions of dollars?" rather than expecting the answer to include "million dollars").
8. Step-by-Step Derivation:
Provide a clear, step-by-stey step derivation for your answer.
This derivation must explicitly reference specific data points or phrases from the document (e.g.. "From Table 3, Row Revenue; Year 2023..." " "As stated paragraph 2...". Detail all mathematical operations performed in each step. This helps verify the question's solvability and reasoning path. ## Output Format
Your response must conclude with a JSON object containing the following keys: "question" and "answer", placed after your reasoning.
"question": "<A well-structured English question that adheres to all design principles>", "answer": "<A single, non-zero numerical answer>"
## Document Context
<text>
[context]
</text>
## Previous Examples
### Example 1:
Question: (question
Answer: (answer }
### Example K:
Question: [question k}
Answer: (answer k}
Questioner Prompt for Document Multiple-Choice Task with History Memory
You expert in document analysis. We are building a benchmark to evaluate the capabilities large language models (LLMs) on fact retrieval, reasoning across multiple constraints, and accurate synthesis of information into coherent responses. You will be provided with document context and a set of simpler, existing questions. Your primary task is generate ONE new, highly challenging multiple-choice question with one correct answer and three plausible, distinct distractors. The new question must be fundamentally different and more complex than the provided examples, requiring both document comprehension and multi-hop reasoning. You must provide one correct answer and three plausible, distinct distractors. Crucially, you must also provide detailed explanation for why the correct answer is correct (including derivation steps) and why each distractor is incorrect.
## Principles for Question and Option Design
Adhere strictly to the following principles when crafting your question, answer, options, and derivation:
1. General Requirements:
All questions, options, and explanations must be in English.
Questions should be challenging, requiring more than simple retrieval or few inference steps.
2. Cognitive Complexity Requirements for the Question:
Must necessitate multi-step reasoning (c.g., involving three or more distinct logical or calculation steps).
Should require the integration of at least three distinct data points from different parts of the document (e.g., combining data from a table with text, using multiple rows/columns/cells)) Should demand the synthesis of quantitative data with qualitative information found in the text.
The problem setup should have the potential for common misinterpretations, which will inform distractor design.
30

---

Published as a conference paper at ICLR 2026
3. Content Validity Criteria:
'he question and all options must be exclusively answerable using information from the provided document. No external knowledge beyond common very basic, universally understood concepts (c.g.. profit revenue cost, directly applicable and data provided) should be required.
If applicable to the document type (e.g., financial reports), prioritize questions with regula-
tory/compliance implications or those highlighting significant financial outcomes. Ensure numerical values involved in the question or options require contextual interpretation within the document, not just direct look-up.
Avoid trivia; focus on questions that address material information or key insights derivable from the document.
4. Distractor Development Guidelines:
Each of the three distractors must be plausible yet clearly incorrect upon careful analysis.
Distractors should represent distinct error paths or common misinterpretations.
At least one distractor should represent a common conceptual misunderstandinng related to the document's 's content how information is presented.
5. Forbidden Question/Option Patterns:
Simple Counting: Avoid questions like "How many X are there?" if X is easily countable directly stated. Small counts (<=5) as part of larger calculation are acceptable.
Direct Retrieval: Avoid questions where the answer (or its direct components) be found in a single, obvious location without further processing.
Excessive External Knowledge: Questions must not require significant domain-specific knowledge not provided or clearly inferable from the document.
No Fabricated Information: Strictly adhere to document content. Do not introduce hypothetical scenarios, data, or assumptions not explicitly stated or directly inferable.
Ambiguous Scenarios: The question must have one unambiguously correct answer based solely on the provided document.
Vague Options: All options, including distractors, must be precise and unambiguous. 6. Escalate Question Difficulty: The question must demonstrate higher order of reasoning than the Previous Examples. First, analyze the examples to identify their simple reasoning patterns (e.g., fact retrieval, single-step comparison). Then, create new question that incorporates one or more of the following advanced reasoning types:
Quantitative Reasoning & Calculation: Requires performing mathematical operations (e.g..
addition, subtraction, percentage change, averaging) on data from multiple sources.
Comparative & Superlative Analysis: Requires comparing multiple entities based on synthe- sized criteria to find the one that is highest, lowest, best, etc.
Conditional or Causal Reasoning: Structured as an "if-then" scenario or asks for the cause/effect of situation by linking different documents (e.g.. "What would be the total cost if the discount from Document A were applied to the price listed Document B?"). Synthesis Across Data Types: Forces connection between qualitative information (e.g.. policy description) and quantitative data (e.g., a number in a table) to reach a conclusion.
7. Answer and Explanation Requirements:
The correct answer must be Ecorrect_answedt
A detailed derivation for the correct answer must be provided, showing step-by-step caleu- lations and referencing specific parts of the document (e.g.. "From Table X, Row Y...", "As stated in paragraph Z...").
For each distractor, provide brief explanation of why it is incorrect, ideally linking it to the type of error it represents (e.g., "Option is incorrect because it omits the X deduction mentioned in...", "Option B results from incorrectly summing X and Y instead of finding their difference....".
## Output Format
Your response must conclude with a JSON object containing the following keys: "question", "options", and "answer", placed after your reasoning.
"question": "<A well-structured multiple choice English question, exclude choices and an- swer>".
"options":
31

---

Published as a conference paper at ICLR 2026
"A": "<Text for choice A>".
"B": *<Text for choice B>*
"C": *<Text for choice C>",
"D". <<ext for choice
1.
"answer": "<correct_ane "anweert
## Document Context
<text>
[context]
</text>
## Previous Examples
### Example
Question: (question 1}
Answer: (answer 1
### Example K:
Question: (question k}
Answer: (answer k}
Responder Prompt for Document General QA Task
Please read the following text and answer the question below.
<text>
(content)
</text>
Question: (question)
Format your answer as follows: "The correct answer is (insert answer here)."
Responder Prompt for Financial Math QA Task
You are an expert in document analysis and numeric reasoning, you supposed answer the given question based on the provided context. You need to first think through the problem step by step, documenting each necessary step. Then you are required conclude your response with the final answer in your last sentence "Therefore, the answer is (insert answer here)". The final answer should be a numeric value. <text>
[content]
</text>
Question: (question)
Please reason step by step, and format your answer follows: "Therefore, the answer is (insert answer here)."
Responder Prompt for Document Multiple-Choice Task
Please read the following text and answer the question below.
<text>
(content)
</text>
Question: What the correct answer to this question: {question]
Choices:
(A) [choice_At A
(B) [choice_Bi B)
(C) (choice_ C)
(D) |choice_D) __
Format your answer as follows: "The correct answer is (insert answer here)."
32

---

Published as a conference paper at ICLR 2026
Verifier Prompt for Document QA Task
## TASK
You are an expert in verifying answers are the same. Your input problem and two answers, Answer and Answer 2. You need to check if they are equivalent. Your task is to determine two answers are equivalent, without attempting solve the original problem.
## Instruction
1. Carefully compare the Answer and Answer
2. Compare the answers to verify they represent identical values or meaning, even when written in different forms or notations.
3. For numerical answers, you should allow #0.15% tolerance.
4. Your decision must be one the "[[YES]]" or "[[NO]]".
## Input Data
Problem: |problem|
Answer 1: {answer_] 1
Answer 2: [answer_2)
## Output Format
Provide your final evaluation in the following format:
"Explanation:" Provide an explanation for why the answers are equivalent or not. "Decision:" "[[YE]]]" or "[[N]]]" Please proceed with the evaluation.
33


Reviews:
[{"review_id": "pTZiM2nFLQ", "reviewer": "Reviewer_wF6e", "summary": "The paper propose SPELL, a multi-role self-play reinforcement learning framework that enables scalable, label-free optimization for long-context reasoning. SPELL integrates three cyclical roles—questioner, responder, and verifier—within a single model to enable continual self-improvement. Extensive experiments on six long-context benchmarks show that SPELL consistently improves performance across diverse LLMs and outperforms equally sized models fine-tuned on large-scale annotated data.", "questions": "Given that the entire multi-agent RL pipeline is largely hand-engineered, the paper should discuss whether reward hacking could theoretically occur. For example, the response generator might collapse to a short, low-effort policy while the questioner learns to ask only trivial questions, yet the pair still receives a high combined reward.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The motivation of this paper is well-grounded.\n2. The multi-agent RL system designed is both novel and sound. The reward designed is intuitively justified and appears effective for long-context reasoning.\n3. The experiments is extensive and carefully designed.", "weaknesses": "Given the unreliability of human annotations in long-context reasoning, how do you ensure that the questions and their corresponding answers provided by the questioner are correct?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bdiXIyH47M", "reviewer": "Reviewer_HDua", "summary": "This paper proposes SPELL (Self-Play Reinforcement Learning for Evolving Long-Context Language Models), a framework designed to address the scarcity of human annotations and verifiable rewards in long-context reasoning. The core idea is a multi-role self-play loop where a single LLM policy alternates between three roles:\n\nQuestioner ($\\pi^{que}_\\theta$): Generates question-answer pairs from raw documents, conditioned on a growing history memory to increase difficulty.\n\nResponder ($\\pi^{res}_\\theta$): Answers the generated questions based on the full document context.\n\nVerifier ($\\pi^{ver}_\\theta$): Assesses the semantic equivalence between the responder's output and the reference answer via majority voting over multiple rollouts.", "questions": "1. Computational Overhead: Could the authors provide an estimate of the computational cost (e.g., FLOPs or GPU hours) of SPELL compared to the RLVR baseline? Given the $O(G^2)$ verifier rollouts, how do you justify the cost-benefit trade-off, and are there strategies to reduce this overhead?\n\n2. Verifier Degeneration: The verifier's training signal is based on self-consistency and alignment with a rule-based check. How does the authors ensure the verifier does not degenerate, for example, by developing a biased \"style preference\" aligned with the responder instead of true semantic equivalence, especially on non-verifiable tasks?\n\n3. Scaling to Larger Models: The results are compelling on models up to 32B. What is your hypothesis regarding how SPELL would perform on much larger foundation models (e.g., 70B or 140B parameters)? Would the adaptive curriculum provide even greater relative gains, or would the need for high-quality data diminish its advantage?\n\n4. Comparison to Simpler Baselines: Did you consider comparing against a simpler \"self-training\" baseline, where you generate synthetic QA pairs with a teacher model (like DeepSeek-R1) once and then fine-tune the student model on this static dataset? This would help isolate the benefit of the online, adaptive self-play loop from the mere use of synthetic data.\n\n5. Failure Mode Analysis: Beyond aggregate metrics, can you provide a qualitative analysis of common failure cases? For instance, what types of questions does the evolved questioner struggle to generate, and what mistakes does the verifier consistently make? This would greatly inform future iterations of the framework.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Well-Motivated Problem: Tackles the critical and underexplored challenge of applying RL to long-context reasoning without reliable human annotations or programmatic verifiers.\n\nComprehensive Evaluation: The scale of the experimentation is impressive, spanning 12 models of various sizes and architectures across 6 benchmarks. This strongly supports the claim of generalizability.\n\nStrong Ablations: The ablation studies are a key strength, providing convincing evidence for the necessity of each core component (history memory, verifier updates, Gaussian reward).\n\nEffective Methodological Design: The three-role framework is elegant. The Gaussian reward for targeting the \"competence frontier\" and the hybrid responder reward are intelligent designs that directly address stability and learning efficiency in self-play.\n\nCompelling Results vs. Baselines: Demonstrating superiority over a strong RLVR baseline and instruction-tuned models effectively highlights the value of the adaptive, self-play curriculum.", "weaknesses": "Limited Analysis of Scaling Laws: The paper shows results up to 32B models. A key question for a \"scalable\" method is its behavior on larger, state-of-the-art models (e.g., 70B+). Does the performance gap over baselines widen or narrow?\n\nSuperficial Treatment of Computational Cost: The framework requires $G + G^2$ generations per data point, which is computationally expensive. A discussion of training cost (e.g., GPU hours compared to the RLVR baseline) and its trade-off with performance is missing but crucial for a complete assessment.\n\nPotential for \"Covariate Shift\" in Verification: The verifier is trained and evaluated on data generated by the same, evolving policy. There is a risk of the verifier overfitting to the specific style or errors of its companion roles, leading to a degenerate equilibrium. The paper does not investigate this potential pathology.\n\nNarrow Benchmarking: While the benchmarks are diverse, they are all QA-style tasks. It is unclear if the improvements translate to other long-context tasks like summarization, retrieval-augmented generation, or long-form dialogue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eUo1wfilRG", "reviewer": "Reviewer_czEk", "summary": "This paper introduces a SPELL (Self-Play Reinforcement Learning for Evolving Long-Context Language Models), a self-play RL framework for long-context reasoning, in which a unified policy is trained using reward signals derived from three roles: the questioner, the responder, and the verifier. The reward functions involve no human supervision and drive self-improvement via an easy-to-hard curriculum. The questioner generates ground-truth answers for the responder and is incentivized to produce questions of intermediate difficulty. The verifier is trained via self-consistency (e.g., agreement across multiple samples). The responder is optimized to match the ground-truth answer and to satisfy the verifier’s majority vote.\n\nSPELL yields consistent gains over a strong RLVR (reinforcement learning with verifiable rewards) baseline across 12 open-source models on six long-context QA benchmarks, evaluated at maximum input lengths of 16K and 100K tokens. SPELL also generalizes to longer contexts and improves test-time exploration and performance.", "questions": "1. How often do the verifier and the rule-based judge disagree? The paper mentions that “the CEM-based reward function is brittle and can penalize semantically correct but lexically different answers”. What happens if we replace CEM with a stronger external judge (e.g., an independent 120B model)? In that setting, what is the relative importance of the verifier versus the rule-based judge?\n\n2. How sensitive is performance to G (number of responder and verifier rollouts)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper’s originality lies in extending the single-policy self-play paradigm to long-context understanding and reasoning through a unified tri-role self-play formulation: \n\n(1) A single policy is supervised under three distinct roles, which form a self-improvement loop.\n\n(2) All reward signals are generated entirely through the model’s own self-evaluation, without any human knowledge or supervision. This design fully leverages the base model’s inherent potential to improve and evolve autonomously.\n\nThe experimental comparison with RLVR is solid, and the core components of the questioner and verifier are thoroughly examined through ablation studies.\n\nThe paper is clearly presented and easy to follow.", "weaknesses": "**Baseline breadth.** A major weakness is the absence of comparisons against recent long-context SOTA methods. The main RL baseline is RLVR on static data synthesized by DeepSeek-R1-0528. Adding head-to-head comparisons with long-context SOTA methods would better position SPELL among competitive approaches. In particular, SoLoPO (short-to-long preference optimization) and LongPO (long-context self-evolution via preference optimization) report substantial gains and improved length/domain generalization on long-context benchmarks. \n\n**Risk of self-reinforcing errors.** Because the self-consistency and mutual dependencies among the questioner, responder, and verifier drive the co-evolution, one potential weakness of the SPELL framework is its susceptibility to self-reinforcing errors over time. It would strengthen the work to discuss or incorporate mechanisms that mitigate it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6ugn3hHec0", "reviewer": "Reviewer_Zsjp", "summary": "The paper proposes a self-play RL framework for training LLMs to do long-context reasoning. Since there is little human annotations for long-context reasoning questions, they design a framework which includes a questioner, a responder and a verifier. Each module is trained via RLVR with a specially designed reward function. Overall the experiment results demonstrate that their proposed SPELL method is able to improve the long-context ability of LLMs and surpass the vanilla RLVR.", "questions": "- For the questioner reward, why do you use a Gaussian-shaped reward instead of the linear piecewise reward as in R-zero? Is there a specific reason that the Gaussian reward is better? Why do you set σ = 0.5/3 and are there any ablation studies on this?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem of long-context reasoning is very relevant to the field. The self-play framework which includes three well-specified roles for generating questions, generating responses and verifying responses is carefully designed with reasonable rewards.\n- An interesting design is the iterative curriculum which keeps generating more and more difficult questions. They demonstrate in experiment results that the increasing difficult questions are better than static questions in RLVR.  \n- Their experiments show that SPELL has consistent advantages over RLVR on most datasets and both long and short context settings. In addition, the verifier and the consistency rewards are useful for the overall model training.", "weaknesses": "- The paper aims to improve long-context reasoning ability. The method is quite effective but seems more like a general RLVR training data generation method, and less relevant to the problem of long-context. It would be better to compare their SPELL method with at least one long-context alignment method other than only RLVR. For example, authors mentioned LongPO, SoLoPO and QwenLong-L1, which seems to be good baselines (and can be compared with SPELL given same documents as the starting point).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)