IterResearch: Rethinking Long-Horizon Agents with Interaction Scaling
Authors: Guoxin Chen
Zile Qiao
Xuanzhong Chen
Donglei Yu
Haotian Xu
Xin Zhao
Ruihua Song
Wenbiao Yin
Huifeng Yin
Liwen Zhang
Kuan Li
Minpeng Liao
Yong Jiang
Pengjun Xie
Fei Huang
Jingren Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Long-Horizon Agents
Deep Research
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Recent advances in deep-research agents have shown promise for autonomous knowledge construction through dynamic reasoning over external sources. However, existing approaches rely on a mono-contextual paradigm that accumulates all information in a single, expanding context window, leading to context suffocation and noise contamination that limit their effectiveness on long-horizon tasks. We introduce \textbf{IterResearch}, a novel iterative deep-research paradigm that revisits long-horizon research through the lens of Interaction Scaling. Instead of relying on linear context accumulation, we adopt an MDP-inspired architecture with strategic workspace reconstruction. By maintaining an evolving report as memory and periodically synthesizing insights, our approach preserves consistent reasoning capacity across arbitrary exploration depths. To effectively train this paradigm, we employ Efficiency-Aware Policy Optimization (EAPO), a training strategy that adapts geometric reward discounting to incentivize efficient exploration and utilizes adaptive downsampling for stable distributed training. Extensive experiments demonstrate that IterResearch achieves substantial improvements over existing open-source agents with average +14.5pp across six benchmarks and narrows the gap with frontier proprietary systems. Remarkably, our paradigm exhibits unprecedented interaction scaling, extending to 2048 interactions with dramatic performance gains (from 3.5\% to 42.5\%), and serves as an effective prompting strategy, improving frontier models by up to 19.2pp over ReAct on long-horizon tasks. These findings position IterResearch as a versatile solution for long-horizon reasoning, effective both as a trained agent and as a prompting paradigm for frontier models.

Published as a conference paper at ICLR 2026
ITERRESEARCH: RETHINKING LONG-HORIZON AGENTS WITH INTERACTION SCALING
Guoxin Chen!:4, Zile Qiao"*, Xuanzhong Chen", Donglei Yu", Haotian Xu Wayne Xin Zhao Ruihua Song..", Wenbiao Yin Huifeng Yin², Liwen Zhang? Kuan Li,, Minpeng Liao², Yong Jiang', Pengjun Xie Fei Huang', Jingren Zhou !Gaoling School of Artificial Intelligence, Renmin University of China
TTongyi Lab, Alibaba Group, 'OpenRLHF
"Beijing Key Laboratory Research on Large Models and Intelligent Governance (gx. chen.chry chn, batmanf fy) gma 1. com, songru hua_bl conêout com griaozile.qzl, yongjiang jy 8 Balibaaaa-aaaaaaaaaaa inc. com
ABSTRACT
Recent advances deep-research agents have shown promise for autonomous knowledge construction through dynamic reasoning over external sources. How- ever, existing approaches rely on a mono-contextual paradigm that accumulates all information in single, expanding context window, leading to context suf- focation and noise contamination that limit their effectiveness on long-horizon tasks. We introduce IterResearch, novel iterative deep-research paradigm that revisits long-horizen research through the lens of Interaction Scaling. Instead relying on linear context accumulation, we adopt MDP-inspired architecture with strategic workspace reconstruction. By maintaining an evolving report as memory and periodically synthesizing insights, our approach preserves consis- tent reasoning capacity across arbitrary exploration depths. To effectively train this paradigm, we employ Efficiency-Awan Policy Optimization (EAPO), train- ing strategy that adapts geometric reward discounting to incentivize efficient ex- ploration and utilizes adaptive downsampling for stable distributed training. Ex- tensive experiments demonstrate that IterResearch achieves substantial improve- ments over existing open-source agents with average +14.5pp across six bench- marks and narrows the gap with frontier proprietary systems. Remarkably, paradigm exhibits unprecedented interaction scaling. extending to 2048 interac- tions with dramatic performance gains (from 3.5% to 42.5%), and serves an ef- fective prompting strategy, improving frontier models by up to 19.2pp over ReAct long-horizon tasks. These findings position IterResearch for long-horizon reasoning, effective both as a trained agent and as prompting paradigm for frontier models.
versatile solution
RerResearch-330-ABB MiroThinker- 328,
WebDancer-QWQ WebThinker QwQ
80
Asearcher
72.8
60
55.4
Performance (%)
52.8
40
29.430.11
28.8
18:0
15.1
15.6
9.8
6.8
5.2
3.5
BrowseComp-zh Figure Performance of IterResearch against state-of-the-ard open-source long-horizon agents.
BrowseComp
Humanity's Last Exam
GAIA
"Corresponding Authors.
Code

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent advances in deep-research agents represent a transformative shift for Large Language Models (LLMs), moving beyond passive knowledge acquisition from the model itself towards autonomous agents that construct knowledge through dynamic reasoning over external sources (OpenAl 2025a
Google
2025 XAI 2025 Perplexity 2025; Anthropic 2025 Team et al. 2025 These frontier
proprietary systems have demonstrated remarkable performance on long- tang-orizonn tasks that require sustained reasoning and information--seeking capabilities over extended interactions.
When tackling long-horizon tasks, recent works (Chen et al. 2025a, Song et al., 2025; Zheng et al, 2025h Jin et al. 2025 Li et al.] 2025c Tao et al.] (025) typically append all retrieved information and intermediate reasoning steps to a single, continuously expanding context window, which we term the monto-contettull paradigm. While straightforward to implement, this paradigm fundamen- tally undermines the sustained reasoning capabilities required for long-horizon tasks: (1) context suffocation: the context window fills with all prior interactions, the available space for model reasoning progressively shrinks, forcing increasingly constrained responses that ultimately degrade into premature or superficial conclusions. (2) noise contamination: irrelevant information from web searches and early exploration errors become permanently embedded in the context, creating cascading interference that dilutes signal quality throughout the entire reasoning process.
To address these limitations, we introduce IterResearch, a novel Iterative Deep-Researde Paradigm that fundamentally reimagines how autonomous agents maintain sustained reasoning capacity in long-horizon scenarios. Our key insight is that effective long- morihhoizoon research requires periodic syn- thesis and strategic forgetting lorgeting-capabiliiiss absent in current mono-contextual approaches. Specif- ically, we adopt an MDP-inspired architecture to structure deep research with a distinctive state design: rather than maintaining an ever-expanding history, each state a strategically reconstructed workspace containing only essential elements: the question, an evolving report serving the agent's memory, and the immediate context needed for current reasoning. This iterative structure, where future exploration depends only on the current reconstructed state rather than the entire history, en- ables the agent to maintain consistent reasoning capacity across arbitrary exploration depths while naturally circumventing the degradation that plagues mono-contextual approaches.
To effectively train this iterative paradigm, we employ Efficiency-Aware Policy Optimization (EAPO), training strategy specifically designed for IterResearch. EAPO addresses two critical challenges unique to our iterative paradigm: First, recognizing that not all successful trajectories are equally valuable, we adapt standard geometric discounting to create efficiency-awaan that geometrically discount based on trajectory cength-agente reaching correct conclusions through concise, focused exploration receive higher rewards than those requiring extensive iterations. Sec- ond, since our paradigm naturally decomposes trajectories into independent training samples at each round, we employ adaptive downsampling to handle the variable sample counts based on data- parallel size, ensuring stable distributed training while preserving over 99% of training data.
rewards
Extensive experiments demonstrate that IterResearch significantly outperforms existing open-source agents, achieving an average improvement of 14.5 percentage points (pp) across six challenging benchmarks. More remarkably, IterResearch narrows the performance gap with frontier propri- etary systems, even surpassing some on these benchmarks. Furthermore, our work reveals three fundamental insights about deep-research agents. First, our iterative paradigm unlocks extreme in- teraction scaling- a capability theoretically extensible to infinite depths yet structurally infeasible for current mono-contextual approaches. To our knowledge, we are the first to successfully extend agents to 2048 interactions with only 40K context length, exhibiting dramatic performance im- provements (3.5% + 42.5%) as maximum interactions increase from 2 to 2048, suggesting that the perceived difficulty of Long-horizon tasks may stem from insufficient exploration capacity. Second, we observe cross-paradigg knowledge transfer: trajectories generated by IterResearch signifi- cantly enhance mono-contextual agents, demonstrating that our paradigm induces superior explo- ration behaviors that create high-quality training signals transferable even across paradigmaticall different approaches. Third, our iterative paradigm serves as an effective prompting strategy: without any training, simply applying it to frontier models yields substantial improvements over the standard cono-contextual approach. ReAct (Yao et al. 2023), particularly on long-ho morizooi tasks (+ (+12.7-19.2pp on BrowseComp), revealing that IterResearch offers model-agnostic solution to long- horg-horizon reasoning. These results confirm the effectiveness of our iterative paradigm in en- abling both deeper exploration and higher-quality reasoning in long-horizon scenarios.
2

---

Published as a conference paper at ICLR 2026
In summary, main contributions be summarized follows:
We propose IterResearch, novel iterative deep-research paradigm that revisits long-horizon research through an MDP-based formalism, maintaining sustained reasoning capacity through periodic synthesis and an evolving report memory- eliminating the context suffocation and noise contamination that plague mono-contextual approaches.
We employ Efficiency-Aware Policy Optimization (EAPO) which adapts geometric discounted rewards that incentivize efficient exploration and utilizes adaptive downsampling for stable dis- tributed training, enabling effective learning from our paradigm's unique trajectory structure.
We demonstrate IterResearch's 's exceptional capabilities and broader impact: (1) achieving an av- erage 14.5 pp improvement across six challenging benchmarks; (2) exhibiting interaction scaling 2048 interactions with dramatic performance gains; (3) enabling cross-paradigm knowledge transfer to enhance mono-contextual agents; (4) providing model-agnostic prompting strategy that significantly improves frontier models on long-horizan tasks without training.
2 RELATED WORK
Retrieval-Augmente Generation (RAG). RAG is crucial approach to overcome knowledge lim- itations of large language models (LLMs) by integrating external information sources (Nakano et al.)
2021 Yu et 2024 Asai et al. 2024 Wei et al. 2025b; Chen al. 2025a Jin et al. [225 Song et al. 2025 Zheng et al. 2025b). However, traditional RAG methods are typically confined to static retrieval environments, such Wikipedia, with limited exploration spaces, making them inadequate for complex, long-horizon reasoning tasks that require dynamic information gathering.
Deep Research. Recent advances in deep research (OpenAT 2025a Google 2025 XAI 2025
MoonshotAl 2025) have transcended RAG's limitations by deploying autonomous agents in real- world environments, demonstrating remarkable capabilities in navigating complex web environ- ments and synthesizing information from diverse sources. However, existing open-source meth-
ods (Li et al. 2025b/e Tao al.] 2025 et al.) (0255) predominantly adopt a mono-contextual paradigm, continuously appending all retrieved information and reasoning steps to a single expand- ing context. This linear accumulation leads to progressive workspace suffocation and irreversible noise contamination, limiting their effectiveness in long-horizon tasks. In contrast, our eterResearch reimagines deep research by adopting an MDP-based architecture with a iterative workspace recon- struction mechanism, eliminating accumulation-induce degradation and enabling sustained reason- ing capacity at arbitrary research depths critical advantage absent in existing approaches.
3 METHODOLOGY
In this section, detail IterResearch, which extends the Markov Decision Process framework to deep research through iterative workspace reconstruction (33.1) as illustrated in Figure Then, we further introduce Efficiency-Awar Policy Optimization for training ($3.2).
3.1 ITERATIVE REEE-REE PARADIGM
3.1.1 MDP-INSPIRED FORMULATION
To address the limitations of mono-contextual paradigms, adopt an MDP-inspired formalism to structure long-horizon research, defined by the tuple (S, E, separates internal thought updates from external environmental interactions enforce state inde- pendence.
T. Rt) This formulation explicitly
State Space S: Each state s; = Me, {ap-1, TR-11)) represents the agent's explicit workspace. 8, serves the constant question q: (2) the evolving report M,, which acts as a compressed memory of prior validated findings; (3) the immediate context {a--1, TR, } from the previous step.
synthesized representation of the research history, comprising: (1)
3

---

Published as a conference paper at ICLR 2026
Mono-contextual Paradigm:
linear accumulation of information
response budget
Question
Think Tool Call
Tool Response
Noise
Think
Tool Call
Tool Response
Noise
Context Suffocation
Noise Contamination
length
Iterative Deep-Research
Decision Space
Action Answer
Paradigm:
Final
Answer
t t=t+1
Think;
Report,
Action,
If Action Tool Call
Workspace S
Environment &
Report;
G
Question
Action,
Tool Response:
Python
IterResearch
Update Workspace
T
Tool Response;
Report:
Action,
Tool Response:
Figure 2: (Top) The mono-contextual approach linearly accumulates all information into single, ever-expanding context, leading to context suffocation and noise contamination. (Bottom) IterRe- search models deep research with iterative workspace reconstruction. Each round begins with reconstructed workspace 8, containing the question, an evolving report M₁, and immediate con- text. The agent generates structured decisions de (Think, Report, Action) and interacts with environment E. The transition function reconstructs the workspace, maintaining an iterative state structure while preventing context bloat and enabling sustained reasoning and ceermaation-seeiing
Decision Space D: At each state 8t, the policy æ generates composite decision de consisting of distinct internal thought and external actions:
de Think,,Mes
|~ (-(ss)
(1)
Internal Thought External Action
Here, M,+1 represents active memory update generated by the agent, while a, is the actual interaction with the environment or the final answer.
Environment & and Reward R: The environment executes action a and returns a stochastic response TR, ~ E()la)). The reward function R[ss, ag) equals if and only if the action a is terminal answer evaluated as correct by the oracle, and O otherwise.
Transition Function T: The state transition is deterministic given the decision and environmen- tal response. explicitly reconstructs the workspace rather than appending to history:
8++1 = T(s,, de, TR,) = (q. M TR.))
This reconstruction mechanism ensures that 8 depends solely on the current state st (via de) and the immediate environment feedback TR,, strictly adhering to the MDP intuition.
The complete research process IterResearch can be formalized as a sequence of state transitions driven by the agent policy n:
Policy Step:
(Think,, Met1, (-1st) TR, E(-la:) 8t41 + M141, TR.))
Environment Step:
State Update:
where TR, E(a)), initial state 50 Mo. 0) with empty report Mo. The iterative process generates a trajectory - = {(S0, do. TRo), (5), d1, (RI),...,(1)) dr)} terminating when ar answer. Unlike mono-contextual approaches where the state dimension grows linearly (|sel x t), our formulation ensures |sy| ~ O(1), enabling theoretically unbounded exploration steps.
3.1.2 ITERATIVE WORKSPACE RECONSTRUCTION
The cornerstone of our paradigm is workspace reconstruction, which fundamentally departs from traditional linear accumulation approaches (Li et al. 2025cla Tao et al. 2025). While existing
4

---

Published as a conference paper at ICLR 2026
methods suffer from (tt) context growth leading to inevitable performance degradation, intro- duce a principled reconstruction mechanism that maintains bounded workspace complexity while preserving complete task-relevant information through selective compression.
At round f, the workspace 8; contains only three essential components: (1) the question g, providing the constant objective; (2) the evolving report MM. serving as compressed memory of all critical findings; and (3) the immediate context {a--1: TR,-1 1 from the last interaction. The key insight is that the report Me41 is naturally generated by the LLM as part of its structured decision output (Think,, Me+1 Oz). This natural flow leverages the LLM's inherent capabilities for information compression and relevance filtering, without requiring explicit algorithmic intervention.
As shown in Eq. 3 the transition function T implements strategic forgetting by reconstructing the workspace each round. The historical trajectory (sg. do. TRg. .... $t-1. d IR is deliber- ately discarded, with only the synthesized knowledge preserved in M This design ensures a constant workspace regardless of trajectory length, in stark contrast to mono-contextual approaches:
mono
(q. do, TRo, ..., a1-1: TR.-1
VS.
(4,, Mr: {a-- TR,-1)
(4)
Mono-cootextull:
O(t) growth
leetResearhh (Ours): (11) constant
Through the iterative workspace. the agent maintains consistent reasoning capacity throughout the research process, avoiding the performance degradation that inevitably occurs when context win- dows approach their limits. Furthermore, through end-to-end training ($3.2), the agent progressively learns to synthesize reports that effectively filter noise and preserve essential information. Thus, ir- relevant information or errors from early rounds cannot directly propagate to future decisions- they must first pass through the agent's ' synthesis to be incorporated into the report. This selective tention aligns with the MDP formalism: the current state $141 contains all decision-relevan in- formation, making the full history unnecessary for optimal decision-make making. The transformative impact of this design manifests in interaction scaling. While mono-contextual approaches typically fail or degrade severely beyond dozens of interactions due context limitations, our IterResearch enables theoretically unbounded exploration, sustaining consistent reasoning quality at arbitrary depths. This scaling capability, empirically validated through experiments with up to 2048 interac- tions ($ 4.4. fundamentally expands the scope of problems that deep-research agents can tackle.
3.2 EFFICIENCY-AWARE POLICY OPTIMIZATION
3.2.1 DISCOUNTED REWARD SHAPING FOR EFFICIENCY
While the iterative workspace reconstruction ensures scalable exploration, a critical question re- mains: how
train agents not just explore deeply, but do so efficiently? We now address this challenge by employing an efficiency-aware training strategy.
In deep research tasks, the agent receives a binary reward signal Ry {0. 1} only upon termination, where Rt = 1 if the final answer is correct and 0 otherwise. This terminal-only reward stems from the inherent difficulty of evaluating intermediate research steps-it is challenging determine the value of any particular search query or exploratory action (Chen et al. 2025a).
However, this sparse signal alone insufficient for guiding efficient learning, as it treats all success- ful trajectories equally regardless of their computational cost. An agent that arrives at the correct answer in 5 well-chosen steps should be preferred over one that requires 20 steps of meandering exploration, even if both ultimately succeed. This efficiency consideration is not merely about com- putational resources: real-world deployment, each interaction incurs API costs, and unnecessary exploration can lead to increased latency. To address these issues, we adapt the standard episodic reward formulation from Sutton & Barto (2018). By utilizing geometric discounting, we create an implicit efficiency pressure:
Ft = T- Ry. 7 E 10,
(5)
where is the terminal step, is the current step, and is the discount factor. This exponential decay creates an implicit efficiency pressure: actions contributing to earlier task completion receive proportionally higher rewards, naturally incentivizing more direct exploration strategies while main- taining the simplicity of terminal-only evaluation.
5

---

Published as a conference paper at ICLR 2026
3.2.2 POLICY OPTIMIZATION WITH MULTI-ROUNE TRAJECTORIES
A distinctive feature of our iterative paradigm is that each trajectory naturally decomposes into multiple independent training samples (one per round), whereas one trajectory typically yields single training sample in mono-contextual approaches. Specifically, for each question q. perform G rollouts generating G independent trajectories. Each trajectory Tr unfolds over T; rounds, where round produces state-decision pair (si,s, ds,e) following our MDP formulation (Eq. 3)
This yields rich training corpus C {( Tz.)) i [L. Gl.t E [1. with Σ Ti sam- ples, far exceeding the G' trajectory-level samples from traditional approaches. While this paradigm significantly enriches training data, the variable sample count across questions requires careful han- dling for distributed training. We address this through adaptive downsampling that reduces the training corpus to the largest multiple of data parallel (DP) size:
C
|rrainl
x DP
(6)
DP
This approach ensures minimal data loss (typically 1% of samples) while maintaining uniform sampling across trajectories. To optimize IterResearch, we combine these efficiency-awann rewards and adaptive downsampling into unified training strategy (EAPO), implemented on top of the Group Sequence Policy Optimization (GSPO) algorithm (Zheng et al.. 2025a), enabling stable train- ing on variable-length trajectories:
T
(0) = Egg Ciin (11)
min(p::(0)A... clip(p,.,)p (iin +,) -,
(7)
Crrain
i=1 1=1
where all E Ti rounds from the G trajectories for question q form one group, with normalized
advantages computed across all samples within this group A = (t,)(0) is the importance ratio based on sequence likelihood (Zheng et al. 2023
Ite Q is the training set, and
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. To rigorously assess the effectiveness our IterResearch, we evaluate on six chal- lenging benchmarks including Humanity's Last Exam (HLE) (Phan et al. (2025). BrowseC-
omp (Wei et al., 2025a), BrowseComp-zh (Zhou et al., 2025a) GAIA (Mialon et al. 2023), Xbench-DeepSearce (Xbench-Team 2025), SEAL-0 (Pham at... 2025). These benchmarks com-
prehensively assess the essential capabilities for effective deep research in multi-step tool use, web navigation, complex reasoning, long-horizon information-seeking and cross-lingual synthesis.
Baselines. We comprehensively compare our IterResearch against state-of-the-ar methods includ- Direct Inference: We evaluate frontier LLMs including GPT-4o and GPT-4.1 (Hurst
ing:
et al. 2024), 04-mini (OpenAl) 2025b), and DeepSeek-R1-0575 (Guo et al., 2025) (2) Propri-
etary Deep-Research System: We compare with commercial deep-research systems including
OpenAl's Deep Research (OpenAl 2025a), Perplexity Research (Perplexity (025), Gemini Deep
Research (Google, 2025), Grok3-Researchhearce (xAl (025). and Kimi-Researchhr (MoonshotAll
2025 (3) Open-source Agents: Recent open-source deep-research agents including Search-ol (Li et al. 2025b), WebThinker (Li et al. 2025c), WehDancer (Wu et al. (025), WebSailor (Li et al.
2025a), Ascarcher (Gao et al. 2025). and MiroThinker (MiroMindAl 2025).
Implementation Details. We implement our IterResearch using Qwen3-30B-A3B (Yang et al.) 2025) as the backbone model, considering both model performance and computational effi- ciency. Our training follows a two-stage process: we first employ rejection sampling fine-tuning (RFT) (Yuan et al. 2023) to equip the model with our iterative deep-research paradigm capabilities, then apply reinforcement learning to further enhance its search strategy and reasoning abilities. For brevity, provide comprehensive training details and hyperparameters in Appendix C.3

---

Published as a conference paper at ICLR 2026
Table 1: Main results across six deep-research benchmarks. We report accuracy (%) for all metrics.
The best results are in bold, and the second best among open-source agents are underlined. Model
Tools
HLE
BC
BC-zh
GAIA
Xbench-DS
SEAL-0
Direct Inference
GPT-4o
X
2.3
0.6
GPT-4,1
X
4.9
1.5
14.4
22.3
o4-mini
X
18.9
6.1
15.2
33.3
60.0
4.5
DeepSeek-RI 11aaccck-R--0555 R10528
X
17.7
2.0
26.3
16.5
5.4
Proprietary Deep-Research System
OpenAl DeepRescarch
/
26.6
51.5
42.9
67.4
Perplexity Research
/
21.1
22.6
Gemini DeepResearch
/
26.9
50.0
Trok3-ResearchSeare
12.9
50.0
Kimi-Rescarcher
26.9
69.0
36.0
Open-source Agents
Search-ol- -loh
/
5.4
2.8
17.9
39.8
40.3
WebThinker-QwQ
/
6.8
32.8
WebDancer-QwQ
/
7.6
3.8
18.0
51.5
40,0
20.7
Asearcher-Wai Web-QwQ
12.5
5.2
15.6
52.8
42.1
WebSailor-32B
9.6
10.5
25.5
53.2
53.3
16.2
WehSailor-72B
9.8
12.0
30.1
55.4
55.0
19.8
Thinke- 14B,0.2
20.0
26.6
62.1
47.0
MiroThinker-3255 32B,o.2
19.1
17.2
29.4
64.1
56.0
Iter R
37.3 u.7
45.2 +
72.8
71.0 +5
39.6 =
+ Improvement
8.8
1 20.1
15.8
8.7.7
15.0
1 18.9
4.2
MAIN RESULTS
Table presents the comprehensive evaluation results across six challenging benchmarks. First, IteRResearch outperforms all existing open-source agents, with an average margin of 14.5 percent- age points across the six benchmarks. More remarkably, demonstrates competitive or superior per- formance compared to proprietary deep-research systems-Surpassing -suraassing OpenAl's DeepResearch on HLE and BrowseComp-zh, while achieving comparable results BrowseComp and GAIA. These results confirm that our iterative paradigm successfully bridges the gap between open-source and commercial systems. Second, the consistent improvements across benchmarks with distinct charac- teristics validate our core design principles. On information-seeking benchmarks requiring extensive web navigation (BrowseComp. BrowseComp-zh, SEAL-0), our method demonstrates substantial advantages mono-contextual baselines. These tasks particularly suffer from context suffoca- tion traditional approaches, as agents must navigate through numerous web pages while synthe- sizing vast amounts of information. Our workspace reconstruction mechanism maintains consistent reasoning capacity by strategically compressing findings into the evolving report, preventing the inevitable degradation that plagues mono-contextual methods. On complex reasoning benchmarks demanding deep analytical capabilities (HLE, GAIA, Xbench-DS), the advantage stems from our ability to mitigate noise contamination. While mono-contextual approaches irreversibly accumu- late errors and irrelevant information throughout their trajectories, our iterative paradigm provides natural breakpoints for filtering noise through periodic synthesis. The evolving report preserves only validated findings while discarding exploratory dead-ends, enabling more focused reasoning in subsequent rounds. These consistent improvements across diverse task types demonstrate that paradigm provides principled solution to the fundamental limitations of linear information accumulation
the iterative
4.3 ABLATION STUDY
To thoroughly understand the contributions of approach, conduct comprehensive ablation studies examining both the effectiveness of our Efficiency-Aweer Policy Optimization (EAPO) and the fundamental advantages of our iterative paradigm traditional mono-contextual approaches.

---

Published as a conference paper at ICLR 2026
Table 2: Ablation studies training methodology and paradigm design. The paradigm ablation uses identical training data and external environment to ensure fair comparison.
HLE
BC
BC-zh GAIA
Xbench-DS
SEAL.-0
Avg
Ablation on Methodology
IterResearch-EAl --APO
28.8
37.3
45.2
72.8
71.0
39.6
49.1
(eereecaacchGGPP
28.2
38.3
45.6
70.9
67.0
39.6
48.3
IterResearch-SPP
25.3
34.9
40.8
68.9
65.0
37.8
45.5
Ablation Paradigm
Knowledge Transfer)
Mono-Agent
18.7
25.4
34.6
62.1
55.0
23.4
36.5
Mono-Agent Iter
25.4
30.1
40.4
63.1
62.0
30.6
41.9
+ Improvement
1 6.7
1 4.7
1 5.8
t 1.0
7.0
1 7.2
t 5.4
(1) Effectiveness Efficiency-Aware Policy Optimization. The upper section of the impact of our EAPO compared to standard GSPO and SFT. Analysis of average interactions re- veals that EAPO requires 18.04 turns, compared to GSPO's 19.13 turns and SFT's 16.45 turns. While EAPO and GSPO achieve comparable accuracy across benchmarks, the critical distinction emerges in interaction efficiency: EAPO reduces average interactions by 5.7% while maintain- ing or improving accuracy. This validates our core hypothesis that geometric discounted rewards successfully incentivize the discovery of more efficient research strategies- agentss learn to reach correct conclusions through more focused, deliberate exploration rather than exhaustive searching.
(2) Superiority of the lterative Paradigm. To rigorously validate our paradigm's 's advantages, we con- duct controlled comparison using identical training data across different paradigms. The middle section of Table reveals striking performance gaps: our iterative paradigm outperforms the mono- contextual baseline (Mono-Agent) by an average of 12.6 percentage points across all benchmarks, with particularly dramatic improvements on long-horing Long-horizon information-seekin BC-zh: +10.6%). Notably, to ensure the mono-contextual agent operates at its optimal capacity and mitigate the inevitable context accumulation issues inherent to its design, we deliberately equipped it with a substantially larger context window (64K vs. our 40K tokens). This substantial perfor- mance gap persists despite providing the mono-contextual approach with more context length, which confirms our theoretical analysis: workspace suffocation fundamentally limits mono-contextual approaches-simphy expanding the context window cannot resolve this limitation. In contrast, our workspace reconstruction mechanism maintains consistent reasoning quality at arbitrary depths through strategic information compression and filtering, enabling effective handling of Long-horizon tasks that overwhelm traditional approaches regardless of their context size.
tasks (BC: +11.8% hetterraatiiiiiiii
(3) Cross-Paradiey Knowledge Transfer. An unexpected yet significant finding emerges: trajecto- ries generated by our iterative paradigm can enhance mono-contextual agents when incorpo- rated into their training data. As shown in the bottom rows of Table 2 augmenting Mono-Agent with iterative-pon iterative-paradigm data while maintaining total data volume (Mono-Agent + Iter) yields con- sistent improvements across most benchmarks, with average gain of 5.4 percentage points. The fact that trajectories generated through our iterative paradigm can enhance mono-contextual agents indicates that our paradigm induces superior research behaviors that create higher-quality training signals, partially transferable even across paradigmatically different approaches.
4.4 SCALING ON INTERACTION
A fundamental advantage of our iterative paradigm is its ability maintain consistent performance at arbitrary interaction depths- property critical for tackling genuinely complex long- horizoo tasks that may require extensive exploration. To empirically validate this capability, we conduct scaling experiments on BrowseComp (200 subset), the most interaction-intensive benchmark in our evalu- ation suites. Figure presents our scaling analysis as we exponentially increase the maximum al- lowed turns from 2 to 2048, a range that would be computationally prohibitive for contexall approaches due to context window limitations. Two key insights emerge from these results:
8

---

Published as a conference paper at ICLR 2026
First, performance scales grace- fully with interaction budget.
Accuracy Avg. Turns
41.0
425
160
40
Accuracy improves from 5.5%
37.0
125,6
129.5
with only 2 turns to 50. 1% at 2048
120
Average Turns
Accuracy (%
turns, with the steepest gains curring between 2 and 2" turns. This demonstrates that complex information-seekee seeking tasks genuinely benefit from extended exploration- capability that mono-contextual approaches cannot provide due to inevitable context overflow. Notably, 2048 turns represents an extreme challenge that is currently infeasible for mono-contextual agents due to catastrophic context accumulation, yet our approach operates smoothly within its constant 40K token workspace through iterative state reconstruction. Second, the agent learns intelligent resource allocation. Despite having access to 2048 turns, the agent uses only 80.1 turns on average, indicating adaptive termination once sufficient information is gathered rather than exhaustively consuming the budget. Notably, the growth pattern of average turns mirrors the accuracy surve-both increase rapidly in the 24-27 range before plateauing Soinnnggnuggggnnn that exploration depth naturally aligns with task complexity. This sublinear growth in average turns (compared to exponentially increasing budget) demonstrates that the agent develops increasingly efficient search strategies as more interactions become available, rather than simply extending existing patterns.
30
23.5
73.9
20
18.0
14.5
37.6
10
40
6.0
23.6
3.5
4.0
71
2
22
23
2
2
26
27
2
29
_0
2
Maximum Turns
Figure Interaction Scaling.
4.5 RESERESE AS A EFFECTIVE PROMPTING STRATEGY IN LONG-HORIZOO TASKS
03
Comparison
DeepSeek.Va.1 Comparison
03-ReAct (mono)
70.5
DeepSeek-v3.1- (mona)
70.9
a3. carch
DeepSeek wt..eeekk....t.kkeeeee...e Itereseearc
62.4
58.1
55.7
Performance (19
49.2
49.2
33.1
29.8
30.0
24.6
HLE
Browsadcompprooscccopp.al
GAIA
HLE
BrowsaCoomBBrowaCCoppp.
GAIA
Figure 4: Performance comparison between IterResearch and ReAct as Prompting Strategies.
Having demonstrated IterResearch's 's effectiveness trained agent, investigate whether our iterative paradigm can serve an effective prompting strategy for long-horizon tasks without any training. We compare with ReAct (Yao et al. 2023). the prevailing momo-contextual prompting paradigm, using frontier models o3 (OpenAl 2025b) and DeepSeek-V3.1 (DeepSeek) 2025).
Figure 4 reveals that IterResearch consistently outperforms ReAct across benchmarks, with particularly dramatic improvements on the most challenging horghorizon task BrowseComp (o3: +12.7pp. DeepSeek: +19.2pp). These gains validate two key insights: (1) The iterative paradigm with workspace reconstruction provides more effective cognitive structure for long-horizon rea- soning, enabling models to maintain focus through periodic synthesis rather than drowning in ac- cumulated context. (2) The paradigm's benefits model-aggostte both o3 and DeepSeek model architectures exhibit substantial improvements, suggesting that our approach addresses fundamen- tal limitations in how current models handle extended reasoning chains rather than model-specific

---

Published as a conference paper at ICLR 2026
weaknesses. The improvements peak BrowseComp the most inlboratioonintensivv bench- mark confirming that our paradigm's advantages scale with task horizon length, making it par- ticularly valuable for complex real-world problems.
CONCLUSION
In this work, we presented IeeRRssearch, novel iterative deep-research paradigm that addresses the context suffocation and noise contamination plaguing mono-contextual approaches long-horizon tasks. By iterative workspace reconstruction and developing Efficiency-Aware Policy Optimization for effective training. we achieved substantial improvements over existing agents (average + +14.5pp across six benchmarks). Furthermore, experiments reveal three transformative insights: this iterative paradigm enables unprecedented interaction scaling to 2048 interactions with dramatic per- formance gains (3.5% 42.5%), serves models by up to 19.2pp. and induces superior exploration behaviors transferable across different paradigms. These findings establish that iteration with strategic synthesis, rather than accumulation, is fundamental to conquering long- horizon reasoning challenges, providing both a powerful agent architecture and versatile framework applicable across different models and paradigms.
effective prompting strategy that improves frontier
ACKNOWLEDGMENTS
This work was supported by Alibaba Research Intern Program. This paper partially supported by the National Natural Science Foundation of China No. 92470205 and Beijing Major Science and Technology Project under Contract No. Z251 100008425002.
REFERENCES
Anthropic. Claude takes research to new places. research April 2025.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannanch Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
URL httttt::///ppereeeee
net Forum? dd=hhy55goSy
Guoxin Chen, Minpeng Liao, Peiying Yu, Dingmin Wang. Zile Qiao, Chao Yang. Xin Zhao, and Kai Fan. C-3PO: Compact plug-and-play proxy optimization to achieve human-like retrieval- augmented generation. In Forty-second International Conference Machine Learning, 2025a. URL htl.htttss//opeeeeviieee net /atatfforee forum? id=h1pwAm@4w2
Guoxin Chen, Jing Wu, Xinjie Chen, Wayne Xin Zhao, Ruihua Song, Chengxi Li, Kai Fan, Day- iheng Liu, and Minpeng Liao. Reform: Reflective autoformalization with prospective bounded sequence optimization. arXiv preprint 1rXiv:2510.24599 2025b.
Xuanzhong Chen, Zile Qiao, Guoxin Chen, Liangcai Su, Zhen Zhang, Xinyu Wang, Pengjun Xie, Fei Huang, Jingren Zhou, and Yong Jiang. Agentfrontier: Expanding the capability frontier of Ilm agents with zpd-guided data synthesis. arXiv preprint arXivv2510.24695, 2025c.
DeepSeek. 2025.
Yiming Du, Wenyu Huang. Danna Zheng. Zhaowei Wang. Sebastien Montella, Mirella Lapata, Kam-Fai Wong, and Jeff Z Pan. Rethinking memory in ai: Taxonomy, operations, topics, and future directions. arXiv preprint arXiv:2505.00675. 2025.
Jiaxuan Gao, Wei Fu, Minyang Xie. Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu. Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl. arXiv preprint 2508 07976, 2025.
Google. Deep research
available deep reep-research-oo ni ni wwwrroxxxpeiimmeta1....
gemini 2.5 pro
experi-
mental.. 2025.
URL
10

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 2025.
Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in Ilm agents via incremental multi-turn interactions. arXiv preprint arXiv:2507.05257, 2025.
Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcome Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Ka- mali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codis- poti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Baek, An- gela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang, Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll L. Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Con- ger. Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang. Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czamecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, and Dane Sherbur.. Gpt-4o system card. CoRR, abs/2410.21276, 2024. doi: 10448500/ARXIV22410220
URL atttss://dot wtt..l/00//.///0......... http:://.ool..oo/....55//////222 2410.
21276
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint 1rXiv:2503.09510 2025.
Kuan Li, Zhongwang Zhang. Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baix- uan Li, Zhengwei Tao, Xinyu Wang. et al. Websailor: Navigating super-human reasoning for web agent. arXiv preprint wxXi:2507.00252. 2025a.
Xiaoxi Li, Guanting Dong. Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. Search-ol: Agentic search-enhanced large reasoning models. arXiv preprint vrXiv:2501.05366. 2025b.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability. CoRR, 000//2504.21776,66 2025c. doi: 1044550/RXXV2550222777
CRL/httss://di. /
10. 00.44850/arrX55 wtt.:ri.................. 21776
Zhiyu Li, Shichao Song, Chenyang Xi, Hanyu Wang, Chen Tang, Simin Niu, Ding Chen, Jiawei Yang, Chunyu Li, Qingchen Yu, al. Memos: memory os for ai system. arXiv preprint
arXiv:2507.03724. 2025d.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference Learning Representations, 2023.
MiroMindAI. Mirothinker, 2025.
URL
httpes///githubb.cm///MMMwaaaa
MiroThinker
MoonshotAL. Kimi-researcher, 2025. Rese-eReeer
URL
http:://ronshotai.ggtthb.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji. Jeff Wu, Long Ouyang, Christina Kim. Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:21/2.09332, 2021.
11

---

Published as a conference paper at ICLR 2026
Alexander Novikov, Ngån Vü, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov. Borislav Kozlovskii, Francisco JR Ruiz. Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506. 13131, 2025.
OpenAl. Deep research system card, 2025a.
URL https//cch.cpenaa..
deep deep-resea resea rcch wystrrchsssste--cccc card. pd.
OpenAl. Introducing openai o3 and 04-mini. April 2025b.
https://openai.comm/ns
int -awpuiiiinn--aa---------
Perplexity. Introducing perplexity deep research, 2025. URL httttpp://w.ppppppeeeelle// a http:////mmmooooooooooeerrrepppe
research
Thinh Pham, Nguyen Nguyen, Pratibha Zunjare, Weiyuan Chen, Yu-Min Tseng, and Tu Vu. Sealqa: Raising the bar for reasoning in search-augmented language models. arXiv preprint arXiv:2506.0m 0/062, 2025.
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity's last exam. arXiv preprint arXiv:2501.14249, 2025.
Zile Qiao, Guoxin Chen, Xuanzhong Chen, Donglei Yu, Wenbiao Yin, Xinyu Wang. Zhen Zhang, Baixuan Li. Huifeng Yin, Kuan Li, Rui Min, Minpeng Liao, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. WebRescarcher: Unleashing unbounded reasoning capability in Long-Horizon Agents. arXiv preprint arXiv:2509. 1333v, 2025.
Huatong Song, Jinhao Jiang, Yinggian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. R1-searcher: Incentivizing the search capability in ilms via reinforcement learning. arXiv preprint arXiv:2503.05552, 2025.
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, Cambridge, Massachusetts, second edition, 2018
Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al. Webshaper: Agentically data synthesizing via information-seeking formalization. arXiv preprint arXiv:2507.1500 2025.
Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingehu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, et al. Tongyi deepresearch technical report. arXiv preprini arXiv:2510.27011, 2025.
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: simple yet challenging benchmark for browsing agents. arXiv preprint S.Kiv:2200.125111 2025a.
Zhepei Wei, Wei-Lin Chen, and Yu Meng. InstructRAG: Instructing retrieval-augmentted gener- ation via self-syythesized rationales. In The Thirteenth International Conference on Learning Representations, 2025b. URL
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekum Xi, Gang Fu, Yong Jiang, et al. Webdancer: Towards autonomous information seeking agency. arXiv preprint arXiv:2505.22644, 2025.
xAI. Grok beta - the age of reasoning agents, 2025. URL httpsttpp///.aaa/eess
Xbench-Tee Team. Xbench-deepsearch 2025. URL httpp:ss//xooncc.o///////eeeeeee
Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Hinrich Schutze, Volker Tresp, and Yunpu Ma. Memory-rl: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint ar0iv:2508. 19828, 2025.
12

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint M0XY::25005003888, 2025.
Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, et al. Memory3: Language modeling with explicit memory. arXiv preprint arXiv:240.00117. 2024.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference Learning Representations (ICLR), 2023.
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, al. Memagent: Reshaping long-context Ilm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259 2025.
Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. Rankrag: Unifying context ranking with retrieval- -ugmeettd generation in llms. Advances in Neural Information Processing Systems, 37:121 111111111111181881- 121184, 2024.
Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuangi Tan, Chang Zhou, and Jingren Zhou. Scaling Relationship on Learning Mathematical Reasoning with Large Lan- guage Models. arXiv preprint 2rXiv:2308..0888 2023.
Zeyu Zhang, Quanyu Dai, Rui Li, Xiaohe Bo, Xu Chen, and Zhenhua Dong. Learn to memorize: Optimizing Ilm-based agents with adaptive memory framework. arXiv preprint arXiv:2508. 16629, 2025.
Chujie Zheng, Pei Ke, Zheng Zhang, and Minlie Huang. Click: Controllable text generation with sequence likelihood contrastive learning. arXiv preprint 23Xiv:2306.033500, 2023.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint ar0iv:2507. 2507718777 2025a.
Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environ- ments. arXiv preprint Stt1:22044031600, 2025b.
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zhh Benchmarking web browsing ability of large language models in chinese. arXiv preprint arXiv:2504. 19314, 2025a.
Zijian Zhou, Ao Qu Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Meml: Learning synergize memory and reasoning for efficient long-horizon agents. arXiv preprint arXiv:25066 15841, 2025b.
13

---

Published as a conference paper at ICLR 2026
A ADDTIONAL RELATED WORK
Memory Mechanisms in LLMs. Memory mechanisms have emerged as a critical component for extending LLM capabilities beyond single-turn interactions (Du et al. 2025 Zhang et al., 2025 Chen et al., (025b). While early works explored explicit memory architectures with separate storage and retrieval modules (Yang et al. 2024). recent approaches have focused on memory management for LLM agents. MemoryLLM (Hu et al. 2025) and MEMI (Zhou et al., (025b) investigate how agents can learn to synthesize and utilize memory across multi-turn interactions, while Memory- R1 (Yan et al., 2025) employs reinforcement learning to train agents for adaptive memory manage- ment. MenAgent ( Yu et al., 2025) and MemOS (Li et al. 2025d) further advance this direction by introducing memory operating systems that unify representation, scheduling, and evolution of memories as manageable system resources. However, these memory-centric approaches primarily focus on explicit memory module design or retrieval optimization within fixed context windows, fundamentally differing from our approach. IterResearch naturally integrates memory through the evolving report M2 within our Markovian workspace reconstruction rather than maintaining sep- arate memory modules or databases, our report serves as a compressed, task-focused memory that seamlessly updated through the agent's structured decisions. This design eliminates the overhead of explicit memory management while ensuring that memory evolution is intrinsically aligned with the research trajectory, enabling more efficient and coherent long-horizon exploration.
Iterative Reasoning and Convergent Paradigms. Concurrently, similar iterative paradigms have emerged in different domains, most notably in algorithmic discovery with systems like AlphaE- volve (Novikov et al., 2025). AlphaExolve, designed for coding tasks, shares a strong conceptual architecture with Iteresearch: maintains main objective (the task), an "evolving report" of high- level ideas (analogous to our Me), and an immediate context (the code). Despite this architectural parallel, the domains are fundamentally different. AlphaEvolve operates on structured code and re- ceives verification from unit tests, whereas IterResearch is designed for the unstructured, noisy, and dynamic environment of web research. We believe this connection strengthens our core thesis. The "convergent evolution" of two distinct, long-horizon domains (algorithmic discovery and deep web research) independently arriving at an iterative synthesis paradigm with an evolving report strongly suggests this is a fundamental and highly effective solution to the limitations of mono-contextual reasoning. It indicates that our IterResearch paradigm is not a niche method, but a general and advanced architecture for complex, long-horizon AI.
B
MORE ANALYSIS
B.1
THEORETICAL MOTIVATION: EFFICIENCY THROUGH DISCOUNTING
The discounted reward formulation in Eq. 5 elegantly encodes preference for efficiency that emerges naturally from the MDP framework. To illustrate this, consider two successful research trajectories for the same question: trajectory тA reaching the correct answer in TΛ 5 steps, and trajectory TB requiring TB- = 20 steps.
Under our discounting scheme with 0.995, each step in the trajectories receives different re- wards based on its temporal distance from the terminal state. For any intermediate step f. the rewards are:
= 7 TA-t t. RT = =75- = TTT Rr 70
(8)
(9)
This creates fundamental learning signal: earlier steps in shorter trajectories receive substan- tially higher rewards than corresponding steps in longer trajectories. To illustrate the magnitude of this difference, consider the reward at step 3:
5-3 72 0.99
(10)
B -0-3 20 = Rd 0.918
(11)
The 7.8% reward difference for the same step position creates a strong gradient that guides the policy toward more efficient research strategies. This consistent multiplicative advantage across all shared steps systematically guides the policy toward discovering more efficient research strategies.
14

---

Published as a conference paper at ICLR 2026
Redundant exploration: Searching for similar information multiple times delays progress, with each redundant step reducing future rewards by factor 7
Circular reasoning: Revisiting previously explored hypotheses without new insights wastes steps, exponentially diminishing the trajectory's 's total return
Unfocused browsing: Following tangential information that doesn't contribute to the final an- accumulates geometric penalties
Importantly, this efficiency incentive emerges without any explicit length penalty or auxiliary ob- jectives-it is an inherent property of geometric discounting applied to our MDP formulation. The discount factor y serves as a single hyperparameter that controls the trade-off between exploration thoroughness and efficiency: values closer to 1 allow more exploratory behavior, while smaller val- ues create stronger pressure for direct problem-solving. Our empirical choice of = 0.995 strikes a balance that permits necessary exploration while maintaining sufficient efficiency pressure, vali- dated by the 5.7% reduction in average trajectory length observed in our ablation studies (Table
B.2 COMPUTATIONAL COMPLEXITY ANALYSIS.
(TR)) with t rounds and average response size [TR], our algorithm maintains constant workspace size of O(IM) |TRR), where .M is the report size bounded by design through the agent's ' learned synthesis behavior. This ensures consistent computational efficiency regardless of the research depth. Table 3 provides a detailed complexity comparison. the Table, / is the number of rounds, |TR] is the average tool response size, [M] is the bounded report size, and L the model's context limit. The key distinctions are:
Unlike mono-contextual approaches where context size grows as O(t
Table Computational complexity comparison between paradigms.
Metric
Mono-contextual
IterRescarch (Ours)
Used Context Size
O(t (R))
O [R]) M(I + )R)
Attention Computation
O((t (TR)
Effective Reasoning Window (maxx L (R)))
OLL M (TR))
Maximum Rounds
(TL/[TR)
(xo) (theoretically unbounded)
Used Context Size: Mono-contextual approaches accumulate all past responses, growing lin- early with rounds until reaching the context limit. Our approach maintains constant size through workspace reconstruction, with the report M serving compressed memory that synthesizes all essential findings.
The quadratic attention cost becomes prohibitive for mono-contextual approaches as t increases, with complexity scaling as O( (TR)))). Our bounded workspace ensures consistent computational cost of (([M) [TR))*) per round, independent of trajectory length.
Attention Computation:
Effective Reasoning Window: In mono-contextual approaches, the available context for new reasoning diminishes as max(0, L ---tt t [TR)), eventually reaching zero when accumulated history exhausts the context limit. Our approach maintains a consistent reasoning window of L L-IM| [TR] across all rounds, ensuring sustainable reasoning capacity throughout the research process.
Maximum Rounds:
Mono- contextual approaches face a hard limit approximately L/ rounds before context overflow. In contrast, our iterative paradigm is theoretically un- bounded-at as long as [M| + [RR] < T. (which is maintained through report synthesis), the agent can continue exploration indefinitely.
These complexity advantages become critical in long-horizon tasks: proaches face inevitable failure when t. [TR] L (context overflow), our approach can theoretically extend to arbitrary depths. This theoretical advantage translates to practical benefits, as empirically demonstrated in scaling experiments (Figure B), where successfully extend agents to 2048 interactions using only 40K context length- feat structurally impossible for mono-contextual ap- proaches.
while mono-contextual ap-
15

---

Published as a conference paper at ICLR 2026
The constant complexity also ensures predictable resource consumption: each round requires ap- proximately the same computational resources regardless of position in the trajectory, enabling bet- ter resource planning and allocation in deployment scenarios. This predictability, combined with the unbounded exploration capability, makes our iterative paradigm particularly suitable for genuinely complex research tasks that may require extensive investigation.
B.3 EXTRAPOLATION BEYOND TRAINING HORIZON
A remarkable property of iterative paradigm is its ability extrapolate far beyond the training horizon. While we train with max = 32 to promote efficient research strategies, the learned agent can seamlessly operate with maxx 2048 even higher during inference-a 64x extrapolation factor that would be structurally impossible for mono-contextual approaches.
This extrapolation capability is enabled by fundamental design choices:
Workspace: Each round's decision depends only on the current reconstructed state (q, TR, not on absolute position or the full trajectory history. This position- agnostic design ensures that the agent's decision-making process remains consistent whether round 10 or round 1000.
Report-based Memory: The evolving report M, provides a scale-invariant representation of
research progress. Unlike trajectory accumulation, the report's bounded complexity ensures the state distribution remains stable regardless trajectory length, allowing coherent reasoning any depth.
We deliberately constrain training to Tmax 32 for strategic reasons: (1) it provides sufficient signal for learning effective research strategies while keeping computational costs manageable, and (2) it creates pressure for the agent to develop concise exploration patterns rather than relying exhaustive search. This constrained training paradoxically enhances extrapolation- learning to maximize information gain within limited rounds, the agent develops robust strategies that scale gracefully when given additional capacity.
Our experiments (Figure 3 empirically validate this extrapolation capability: agents trained with Tmax = 32, achieve 42.5% accuracy on BrowseComp when extended to Tmax 2048 during inference, compared to only 15.2% with Tmax = 32. This dramatic improvement demonstrates that the agent effectively utilizes the additional exploration capacity without any degradation in decision quality or coherence.
Contrast with Mono-contextual Limitations. barriers to such extreme extrapolation:
Mono-contextual approaches face fundamental
Absolute position encodings trained on sequences of length 32 often produce undefined or degraded representations beyond the training range
Position Embedding Overflow:
Attention Pattern Collapse: Attention distributions learned on short sequences fail to generalize to dramatically longer contexts, leading to degenerate focus patterns
Context Saturation: The accumulated context from 2048 rounds would exceed most models' context limits, causing hard failures rather than graceful degradation
Theoretical Foundation. The extrapolation capability stems directly from our MDP formulation where the optimal policy defined over states, not trajectory positions. Since our state space S and decision space 'D remain constant regardless of horizon length, a policy learned shorter trajectories naturally generalizes to longer ones, provided the state distribution remains similar. The report synthesis mechanism ensures this distributional stability by maintaining bounded complexity O(MM) regardless of trajectory length, preventing the distribution shift that would otherwise oceur with unbounded context accumulation.
This extrapolation capability fundamentally expands the applicability of our approach: agents can be efficiently trained on moderate-lent length trajectories yet deployed quiring extensive exploration, providing a practical path to handling real-wor world research challenges of unknown complexity.
on arbitrarily complex tasks
16

---

Published as a conference paper at ICLR 2026
B.4
TRAINING DYNAMICS EFFICIENCY-AWARE POLICY OPTIMIZATION
0.85
BC-2h
0.75
Accuracy (%)
Reward
0.65
40
0.55
35
Original Smoothed
0.45
50
100
150
30
50
100
150
Step
Step
Figure 5: Training dynamics of our RL. (Left) Training Rewards Curve. (Right) Accuracy Curve.
Figure iillustrates the training dynamics our EAPO framework across 150 optimization steps.
Reward Convergence. The left panel demonstrates stable convergence with training rewards in- creasing from 0.55 approximately 0.72, representing 30.9% improvement. The smooth EMA curve exhibits only minor oscillations, confirming that our adaptive downsampling successfully han- dles variable sample counts from our iterative paradigm while maintaining stable gradient signals. The consistent upward trend without plateauing suggests the geometric discounting continues to provide meaningful learning signals throughout training.
Performance Evolution. The right panel reveals distinct learning patterns that reflect fundamental differences in task characteristics:
BrowseComp (English): The sharp performance jump from 32% to 39% at step 50 followed by stabilization suggests the agent discovers critical search strategies likely effective query formulation or result filtering patterns specific to English web content. The subsequent plateau indicates these strategies generalize robustly.
Comp-eComp-zh (Chinese): The monotonic improvement from 40% 45% reflects a smoother optimization landscape, possibly due to more structured Chinese web content or different infor- mation organization patterns that allow incremental strategy refinement.
The correlation between reward growth and performance improvement validates our core hypothe- sis: geometric discounted rewards successfully guide the agent toward more efficient exploration. Notably, the reward improvement (30.9%) exceeds the accuracy gains (BC: 18.8%, BC-zh: 12.5%), indicating the agent learns not just to solve tasks but solve them efficiently. This is empirically confirmed in our ablation studies (Table 2 where EAPO achieves 5.7% shorter trajectories than standard GSPO while maintaining comparable accuracy, demonstrating that our reward design cessfully shapes more focused exploration behaviors without compromising task performance.
C MORE IMPLEMENTATION DETAILS
In this section, we provide comprehensive implementation details of our proposed method. For additional insights and more intricate details, we refer the reader our supplementary materials.
C. ALGORITIMIC FRAMEWORK
Algorithm
D
presents the complete procedure of our iterative deep-research paradigm.
The algorithm proceeds through discrete research rounds, where each round t follows a structured sequence:
17

---

Published as a conference paper at ICLR 2026
Algorithm Iterative Deep-Research (IteRResearch)
Require: Question g, Agent model π, Environment &, Max rounds Tax
Ensure: Final answer to
Initialize: Ma + 0, so (q, Mo, ++)
b Empty report and context
2: while t < max do
3: de < ((+))
> Generate structured decision
4: Parse: (Think,, M
(4+1,a)) at) t de
5: if a, = answer then
6:
break
> Agent decides to terminate
7:
end
8: TR, E(a))
Execute tool and get response D Reconstruct workspace
9:
Sc+1 + Me, fa,, TR,))
1+1+1
11:
end while
return final answer
Decision Generation (Lines 3-4): The agent æ processes the current state 8, to produce struc- tured decision d, which is then parsed into three components: reasoning (Think,), updated report (Me+1), and next action (at).
Termination Check (Lines 5-7): If the agent outputs a₂ = answer, the algorithm terminates with the agent's final answer. This allows autonomous determination of information sufficiency.
3. Tool Execution (Line 8): For non-terminal actions, the environment & executes the requested tool (search, browse, compute) and returns the response TR,
Workspace Reconstruction (Line 9): The crucial step distinguishing our paradigm erratigm-iinteadd instead of appending to ever- werr-roowing context, reconstruct bounded workspace containing only the question q. updated report Me+1, and latest interaction (at, TR,).
Report Evolution Mechanism. The report M serves as the agent's evolving memory, dy- namically synthesizing information across rounds. At each step, the agent updates the report by incorporating new findings from TR, while preserving essential insights from MM. This selee- tive retention ensures critical findings persist while redundant information is filtered, maintaining bounded complexity regardless of trajectory length.
Termination Conditions. The algorithm terminates under two conditions: (1) Natural Termina- tion: The agent determines sufficient information has been gathered and outputs answer (Line 6). (2) Forced Termination: The round counter reaches Tmax (Line 11), preventing infinite loops.
C.2 TOOL ENVIRONMENT
Our environment provides four complementary tools that enable comprehensive research capa- bilities. Each tool is designed handle specific aspects of the research process, from information gathering to computational analysis. We provide the detailed tool schema in Appendix E. We implement the tool environment using production-grade APIs and services:
Google Search: Returns top-10 10 search results with snippets for general web queries. The tool accepts multiple queries single call, enabling efficient batch searching. Each result includes title, URL, and brief snippet, providing the agent with sufficient context determine relevance before deeper exploration.
Google Scholar: Returns top-10 search results with snippets for academic papers, citations, and scholarly metadata. Similar web search, it supports batch queries and returns structured bibliographic information including authors, publication venues, citation counts, and abstract snippets. The tool also includes fallback general web search for comprehensive coverage. Both Google Search and Google Scholar are accessed via SerpAP[" providing reliable and rate- limited access search results.
https://serpapl
18

---

Published as a conference paper at ICLR 2026
Visit (Web Browser): Enables detailed content extraction from specific URLs with goal-oriented summarization. The agent specifies both the target URLs and a specific goal (e.g., "find the methodology section" "extract statistical results"), allowing focused information extraction. The tool handles both HTML webpages and PDF documents, automatically detecting and parsing the appropriate format. Our summarization model (Qwen3-30B-A.) processes the raw content with the agent's 's goal to produce concise, relevant summaries. We employ Jina Reader for robust web content extraction.
Executes arbitrary Python code in a secure, sandboxed environment for computational tasks and data analysis. The interpreter comes with standard libraries (NumPy, Pandas, Matplotlib, etc.) pre-installed and can handle complex calculations, data manipulations. and logical operations. All outputs must be explicitly printed, ensuring clear communication of results back to the agent. We use Code Sandbox ensuring secure and isolated computation.
Python Interpreter:
C.3 IMPLEMENTATION DETAILS
This section provides comprehensive implementation details of our IterResearch. We also provide all of the code and training data for easy reproduction.
Table 4: Key hyperparameters in the supervised warm-up phase.
Hyperparameter
Value
Learning Rate
1e-5
Batch size
512
#Epochs
Chat template
Owen Yang et al. 40960
2025
Maximum Context Length (Prompt Response)
Warmup ratio
0.03
LR scheduler type
Cosine
Supervised Fine-tuning Phase. Since existing LLMs lack inherent capabilities for our itera- tive deep-research paradigm. we conduct a two-stage data preparation process: Stage 1: High- quality QA Collection. We curate 30K high-quality question-answer pairs from recent web re- search datasets (Li et al. 2025a Tao et al.) 2025;; Chen et al., 2025c Qiao et al.] 2025). These pairs are filtered based on answer quality, factual accuracy, and research complexity ensure they re- quire genuine multi-step investigation. Stage 2: Trajectory Synthesis. To bridge the gap between standard QA pairs and our iterative paradigm, we employ Qwen3-235B-A22B (Yang et al. (025) to synthesize research trajectories following our framework. This process yields 110K training tra- jectories with an average of 3.7 rounds per trajectory, providing rich supervision for learning the iterative research pattern. We utilize S iime" as our training framework for the initial supervised fine-tuning phase. The detailed apper-parameters for this phase are presented in Table
Reinforcement Learning Phase. We employ a strategic data selection process to identify ques- tions with optimal learning potential: (1) Difficulty Calibration: Using the best checkpoint from SFT, we evaluate each of the 30K questions with 5 independent trials, recording success rates. (2) Learning Zone Selection: We retain questions with success rates between 20%-60% (1-3 cor- rect out of 5 attempts), identifying 4,096 questions that fall within the model's "zone of proximal ceyelopmmen""-calleenngini enough to provide learning signal but achievable enough to generate successful trajectories. Questions that are too easy (> 60% success) provide weak learning signals, while overly difficult questions (< 20% success) lead sparse rewards and unstable training. Ta- ble summarizes the key hyperparameters used during the reinforcement learning phase. We also 51 im RI. frameowrk due to its efficient and easy use.
htttps://iia..at
https://github. TT..m/HHJJM///// me
19

---

Published as a conference paper at ICLR 2026
Table 5: Key hyperparameters in the RI. phase.
Hyperparameter
Value
Learning Rate
1c-6
Base model
Qwen3-30B-A3h Yang et al.
2025
Batch size
16
Group size per Question (G)
16
temperature
1.0
Top P
0.95
KL loss coefficient (A)
0.
entropy coefficient
0.
Maximum Context Length (Prompt Response)
40960
Maximum interaction rounds (7
32
Table 6: Maximum round settings across different stages and benchmarks.
Stage/Benchmark
T Tmax
Training
32
Inference Phase:
GAIA Mialon et al. 2023
32
HLE (Phan el al. 2025
64
BrowseComp zh Zhou et 2025a)
64
BrowseComp (Wei et al. 2025a)
256
Maximum Round Settings. We adopt task-adaptive Tima values to balance training efficiency with inference flexibility. Table I summarizes our configuration across different stages and bench- marks. We constrain Tmax 32 during both SFT and RL phases to instill efficiency-oriented behaviors. This limit, combined with our geometric reward discounting (Equation 5). creates strong incentives for the agent to develop concise research strategies rather than exhaustive exploration pat- terns. During inference, we adjust Tmax based on benchmark characteristics. This adaptive configu- ration ensures that simple tasks remain efficient while complex questions have sufficient exploration budget, all while maintaining the efficiency patterns learned during training.
Reward Design. We employ LLM-as-judge evaluation following established practices (Chen al. 2025a). Specifically, we use 3wen3-28 -235m----3----- A2BBB to assess answer correctness:
1.0 if answer is correct
Rr
(12)
0.0
otherwise
Baselines. The baseline performance figures for prompting-based comparisons in Table 1 (e.g., ReAct on BrowseComp) were sourced directly from the State-of-the-Art (SOTA) results reported in their original papers. We adopted this methodology for two critical reasons: (1) Many these prior SOTA works do not publicly release their exact optimized prompts. (2) Therefore, directly reporting their published score the most rigorous and fair method for comparison. This ensures our results are benchmarked against the true SOTA, not potentially weaker or un-optimized re- implementation.
C.4 AVERAGE TOKEN BUDGETS
To provide further insight into the reproducibility and computational cost of our method, detail the average generation tokens required for the IterResearch-309 30B---3BBBBBBB-- agent to complete tasks across the six primary benchmarks.
is important to note that this data **xcludee*** tokens returned by tools (e.g., the content of web pages search results). It represents only the tokens consumed by the agent's own "Think" and "Report" steps. This figure thus reflects the actual computational cost of the agent's internal reasoning.
20

---

Published as a conference paper at ICLR 2026
Table 7: Average generation tokens (excluding tool responses) for IterResearch-30B-AN. across
benchmarks.
Benchmark
HLE
BC
BC-zh
GAIA
Xbench-DS
Seal-0
Avg. Tokens
31,426
376,349
81,136
33,198
31,600
28,186
As shown in Table benchmarks.
7 the required reasoning cost (i.e., generation tokens) varies dramatically across
BrowseComp (BC) consumes a token budget (approx. 376k) that is an order of magnitude larger than all other benchmarks. This provides strong empirical evidence of its extreme complexity long-horizon task, requiring the agent to perform extensive iteration, thinking, and report synthesis to succeed in the complex web environment. This aligns with our scaling experiments (Figure 3), where performance continually improves as more interaction steps (and thus more generation tokens) are permitted.
BrowseComp-zh (BC-zh), approx. 81k tokens, is the second most complex task.
The other four benchmarks (GAIA, HLE, Seal-0, Xhench-DS) are similar magnitude (ap- prox. 28k-33k tokens), suggesting that while they are complex multi-step reasoning tasks, they require significantly less iterative synthesis compared to the long-horizon 'BC' benchmarks.
This analysis demonstrates that our method's performance gains (e.g., in Table 2) do not stem from uncontrolled token expenditure. Instead, they are achieved by superior paradigm (IerRRsearrh) operating within a manageable, task-dependent reasoning budget.
D CASE STUDY OF ITERRESEARCH
We present representative example demonstrating how IterResearch solves complex biology question through iterative research. This case highlights three key capabilities: (1) evolving report synthesis, (2) efficient information gathering, and (3) autonomous termination decision.
Examples of our IterResearch in Humanity's 's Last Exam (HLE)
data source: Humanity's 's Last Exam
split: test
question: A researcher is interested in studying insects in the family Micromalthidae. They develop a lab colony where larvae are supplied with decaying wood to feed on. Suppose an adult male is found in the colony. Upon its death, what will be the only thing that this individual will have fed on?
Answer Choices: A. Its mother B. Fungus C. Decaying wood D. Cellulose E. Nothing
Iteration
In the initial iteration, the evolving report and intermediate context are empty. <think>
First, the question is about a researcher studying insects in the family Micromalthidae. They have lab colony with larvae fed decaying wood. An adult male is found, and upon its death, we need to determine what it fed on based on the answer choices: A. Its mother, B. Fungus, C. Decaying wood, D. Cellulose, E. Nothing.
I need to understand the biology of Micromalthidae. I recall that Micromalthidae is family of beetles, and they might have unique feeding habits or life cycles. Specifically, I think Micromalthus debilis is a species this family, and it's known for having a complex life eycle with paedogenesis, where larvae can produce offspring. But the question is about an adult male and what it fed on. Adult males might not feed at all they have a short lifespan if their role is solely reproduction. The answer choices include "Its mother," which sounds bizarre, but it might be related to matriphagy or something similar. Also, fungus and decaying wood common food sources for wood-feeding insects, but cellulose
21

---

Published as a conference paper at ICLR 2026
component of wood. 1 should verify the feeding habits of Micromalthidae adults. Since this is specific entomological question, I need to search for reliable information. Available tools include google search, google scholar, Visit, and Pythonlnterpreter. For this, I should use google scholar or google search to find academic sources on Micromalthidae feeding behavior.
scoglerscholar
Let me plan my approach:
1. Search for information on Micromalthidae, specifically about adult feeding habits.
2. Focus on the family Micromalthidae and any unique aspects like matriphagy or non- feeding adults.
I remember that in some insects, adult males don't feed, but need confirmation for this family. Also, the question mentions "upon its death." which might imply that it didn't feed much or at all, leading evidence. I'll start by searching using google scholar for Micromalthidae feeding habits or life cycle. Since the question is in English, my output should be English. Now, for the report section, need to provide:
choice E. Nothing. But let's not jump conclusions. need
Problem Essence Analysis: Deeply analyze core elements, complexity levels, and potential pitfalls.
Problem-Solviine Planning: Based understanding, clarify what needs to be confirmed, information sufficiency, and cognitive blind spots.
Then, must make one tool call.
Potential pitfalls: Misinterpreting the life cycle or feeding habits. Micromalthidae might have unusual behaviors.
Cognitive blind spots: Assuming adult beetles always feed, but some might not.
Information sufficiency: currently don't know enough, so need to search. should search for "Micromalthidac adult feeding" "Micromalthus debilis life cycle". Let me formulate the search query.
Query: "Micromalthidae adult male feeding" Or more broadly: "Micromalthidae life cycle"
But the question specifies adult male, so perhaps focus on that. Also, "matriphagy in Mieromalthidae" because choice A is "Its mother," which might relate that. I recall that in Micromalthus debilis, there is matriphagy, where larvae eat the mother. But the question about an adult male, not larva. The adult male might feed, or perhaps has a different diet. Upon second thought, in some sources, adult Micromalthus might not feed all. But let's verify. I'll use google_scholar to search. For the tool call, need to specify the query. Let me decide on the query. Since google_scholar returns academic papers, should use precise terms.
Query: "Micromalthidae adult feeding habits" Or "Micromalthus debilis adult nutrition" But to be safe, I'll use multiple queries to cover bases. The tool allows multiple queries, so I can do an array. Let me think. First, need to confirm the feeding behavior. Also, the Visit tool might be useful if find specific URLs, but first, need search. Pythonlnterpreter probably isn't 't needed for this, as it's about information retrieval, not calculation. So. I'll use google scholar. Now, for the report section. Problem Essence Analysis: The core is to determine the diet of an adult male Micromalthidae insect based on its life cycle and biological characteristics. Complexity arises from potential unique behaviors like matriphagy non-feeding adults. Pitfalls include misattributing larval behaviors to adults or generalizing from other beetles. Problem-Solving Planning: I need to confirm the feeding
22

---

Published as a conference paper at ICLR 2026
habits of adult males in Micromalthidae. Current knowledge is insufficient, I must search academic sources. Blind spots might be assuming all adults feed or not, without specific data. Then, tool call: Use google_scholar scholar with queries about Micromalthidae adult feeding. Let me write the queries. Queries: ( I"Micromalthidae adult male feeding", "Micromalthus debilis life cycle". "matriphagy Mieromalthidae"] included matriphagy because it might be relevant, even though it's about larvae eating mother, but perhaps adults have connections. But the question about adult male feeding, so focus on that. Perhaps "do Micromalthidae adults feed?" But let's go with the queries.
Now, output format: First <report>, then <tool call>. Language: English. So, draft the report.
</think>
<report>
### Status Report and Deep Analysis
*Problem Essence *nalysis** The core of this problem lies in understanding the unique biological characteristies and feeding habits of insects in the family Micromalthidae, specifically focusing adult males. Micromalthidae is small family of beetles, with Mieromalthus debilis being the most studied species, known for complex life cycle involving paedogenesis (reproduction by larvae) and potential cannibalistic behaviors. The complexity here stems from the possibility of unusual feeding patterns, such as matriphagy (consumption of the mother by offispring) non-feeding adults, which could influence the answer. Potential pitfalls include conflating larval behaviors with adult ones, misinterpreting ecological roles, or generalizing from other beetle families without specific evidence. The answer choices range from logical options like decaying wood or fungus to more extreme possibilities like matriphagy complete lack of feeding, requiring precise taxonomic knowledge.
**Problem Sooiving Planning Based on the current understanding, the key information needed is the confirmed feeding behavior of adult male Micromalthidae insects. While larvae are supplied decaying wood, adults may have different diets or may not feed at all due to short lifespans or specialized roles. Information sufficiency is low without verified sources, personal recall suggests possible non-feeding matriphagy-relace behaviors. this must validated. Cognitive blind spots include assuming adult beetles feed or overlooking species- species-specific specific anomalies. The next step is to search academic literature using Google Scholar to retrieve authoritative sources on Micromalthidae life cycle and feeding ecology, focusing adult males. </report>
<tooL_call>
"name": "google scholar". "arguments": "query": ["Micromalthidae adult male feeding", "Micromalthus debilis life cycle", "matriphagy Micromalthidae", "do Micromalthidae adults feed"]
</tool call:>
We will get the tool resposne from Environment here.
tool.ree response>>
Google scholar for 'Micromalthidae adult male feeding' found 10 results: ## Scholar Results
1. [5.3 Micromalthidae Barber, 1913](no available link) publicationInfo: T Hörnschemeyer Coleoptera, Beetles. Morphology and 2016 books. google.commmmmmoooo
Date published: 2016
citedBy: 3
larva cerambycoid type, which legless and feeds on wood and probably the fungi therein. It... From this pupa finally an adult male hatches. Males are haploid with ten chromosomes...
23

---

was its Published as a conference paper at ICLR 2026
2. [The remarkable life-history of
As the content is too long, we will omit it here. </tool response>
Iteration 2
Workspace Reconstruction: Following our transition function, each iteration maintains only: (1) the evolving report between <report>... </report> tags, (2) the current ac- tion between <tool_call>, between croo.rrepoosee compressed into the report, not retained in full.
</tool_call tags, and (3) the previous tool response wttool.responae tags. Historical interactions are
<think>>
Hmm, let's start by understanding the question. The user is asking about what an adult male Micromalthidae insect would have fed before death in lab colony with decaying wood. The answer choices include its mother, fungus, decaying wood, cellulose, or nothing.
Looking at the previous tool call, used Google Scholar to search for information on Micromalthidae feeding habits, life cycle, and matriphagy. The results from the tool response are quite detailed. Several sources mention that male larvae practice matriphagy, cating their mothers. For example, the Current Biology article by Normark (2013) states that it takes male about a week to devour his entire mother before pupating into adult male. Other papers like Pollock & Normark (2002) and Perotti et al. (2016) also confirm this behavior.
The Wikipedia entry and other web results reinforce this, mentioning that adult males rarely develop and consume their mothers. The key point here that adult males themselves don't 't feed; it's the male larvae that engage in matriphagy. Once they become adults, they might not feed at all, as indicated by references to vestigial reproductive organs and non-functional adults.
So, putting together: the adult male hasn't fed anything after becoming an adult because its feeding happened during the larval stage part of matriphagy. Therefore, the only thing fed on was its mother. The correct answer should be mother.
I need to make sure there's no conflicting info. Some sources mention larvae feeding on decaying wood fungus, but specifically for males, the matriphagy is highlighted. Also, the question specifies "upon its death," referring to the adult male, so we' re looking back at what it consumed during its life. Since the adult doesn't feed, the last thing mother during the larval stage.
The status report should document all findings, emphasizing the matriphagy behavior and the fact that adult males don't feed. The next steps aren't needed since the information sufficient from the sources. So, the final answer is A.
</think>
<report>
### Status Report and Deep Analysis
Progress *chieved:**
Based on comprehensive research through Google Scholar and web searches, I have gathered substantial information about Micromalthidae biology. specifically focusing on Micromalthus debilis, the only extant species in this family. The key findings are:
1. Unique Life Cycle Characteristicc**: Micromalthus debilis exhibits one of the most complex life cycles among beetles, involving multiple reproductive strategies including
24

---

Published as a conference paper at ICLR 2026
paedogenesis (reproduction by larvae), haplodiploidy, thelytoky, and arrhenotoky. The species is primarily parthenogenetic with female larvac reproducing asexually.
2. **Male Development and Feeding Behavior**; Male development occurs through specialized pathway where male eggs hatch into active triungulin larvae that immediately begin feeding. Multiple authoritative sources (Normark 2013 in Current Biology, Pollock Normark 2002, Perotti et al. 2016) confirm that male larvae exhibit obligate matriphagy they consume their mothers from within. According to Normark (2013): "It takes male about a week to devour his entire mother, after which he pupates and molts into a winged adult male."
3. ** Adult Feeding Habits**: Adult males of Micromalthus debilis appear be feeding or have vestigial mouthparts. Research by Beutel & Hörnschemeyer (2002) and Yavorskaya al. (2018) indicates that adults have reduced morphological features related to feeding, suggesting they do not feed as adults. The Wikipedia entry telephone-pos pole beetles confirms that adult beetles have "vestigial reproductive organs" and likely do not feed.
4. **Contrast with Larval Feeding**; While larvae (particularly female larvae) do feed on decaying wood and associated fungi, this feeding behavior is specific to the larval stage and different between sexes. Male larvae have a specialized feeding requirement focused on matriphagy rather than wood consumption.
**Ecological C The laboratory colony described in the question provides decaying wood, but this substrate is primarily utilized by female larvae rather than male individuals. The matriphagy behavior represents an evolutionary adaptation to nutrient acquisition nutrient-poor environments.
**Source Verification Status**:
High confidence Normark (2013) Current Biology article and Pollock & Normark (2002) journal paper
Supporting evidence from Perotti et al. (2016) Scientific Reports
Morphological studies (Beutel & Hörnschemeyer 2002, Yavorskaya al. 2018) provide additional confirmation
Wikipedia and other secondary sources align with academic literature
*Uncertainties *
Confirmed that adult males do not feed as adults
Established that male larvae exclusively consume their mothers
Clarified that decaying wood feeding is primarily associated with female larvae
**Next Steps *
No further research is required sufficient authoritative information has been obtained to answer the question definitively. The collected evidence clearly establishes the feeding history of adult male Micromalthidae insects. </report>
Having synthesized sufficient evidence, the agent autonomously decides to terminate with a comprehensive answer. The evolving report now contains all critical findings, enabling
25

---

Published as a conference paper at ICLR 2026
confident conclusion without further exploration.
<answer>>
Based on comprehensive research into the biology of Micromalthus debilis, the sole extant species in the family Micromalthidae, the correct answer A. Its mother.
**Detailed Explanation:
The telephone-pole beetle (Micromalthus debilis) possesses one the most extraordinary and complex life cycles known in the insect world, particularly regarding the feeding behavior of male individuals. When an adult male found laboratory colony where larvae supplied with decaying wood, this individual will have exclusively fed its mother during its developmental history.
**Matriphagy as Obligatory Feeding Strategy:**
Male Micromalthus debilis develop through a unique pathway characterized by obligate matriphagy the consumption of the mother by her offspring. This process begins when male eggs are laid by paedogenetic (reproducing while still in larval form) females. These eggs hatch into active, legged triungulin larvae that immediately commence feeding on their living mother from within. According to research published in Current Biology (Normark, 2013), this feeding process requires approximately one week to completely devour the maternal body. Only after consuming sufficient nutrients from their mother do the male larvae pupate and eventually emerge as winged adult males.
*#Non-Fedding Adult Stage:**
Critical research by Beutel and Hörnshhmmeer (2002) in Systematic Entomology and Yavorskaya al. (2018) in Arthropod Structure & Development reveals that adult Micro- malthus debilis exhibit significant morphological reduction in feeding structures. Their mouthparts are vestigial, and they possess non-functional digestive systems, confirming that adult males do not feed at all during their brief adult stage. The adults represent the terminal phase of the life cycle, having completed all necessary feeding during their larval development.
**Contrast with Female Feeding :ehavior:
While the laboratory colony provides decaying wood as food for the larvae, this substrate is primarily consumed by female larvae rather than males. Female Micromalthus debilis feed on rotting wood, often infested with fungi, which provides them with the energy required for their parthenogenetic reproduction. The male-specific matriphagy strategy represents an evolutionary adaptation to obtain concentrated nutrients in environments where wood decomposition products may be nutritionally limited.
**Evolutionary Context:**
This extreme feeding specialization reflects the unique selective pressures facing Micro- malthus debilis. The matriphagy behavior ensures that male offspring receive high-quality nutrients directly from their genetically identical mother (given thelytokious parthenogen- esis), while allowing the population to maintain its asexual reproductive strategy. The vestigial nature of adult males suggests they serve primarily as dispersal units rather than feeding organisms.
Therefore, upon the death of an adult male Micromalthidae in the described laboratory colony, the only thing this individual would have consumed throughout its entire life cycle its mother, making option A the correct answer.
26

---

Published as a conference paper at ICLR 2026
</answer>
E INSTRUCTION TEMPLATES
E. TOOL SCHEMA SPECIFICATION
The agent interacts with tools through a structured schema that defines available functions and their parameters. Below we present the complete tool specifications used in our system.
Listing 1: Google Search Tool Schema
"type": "function", "function"; { "name": "google_search", "description": "Perform Google web searches then returns string of the top search results. Accepts multiple queries.",
"parmeterss"
"type": "object", "properties"; {
"query": {
"type": "array",
"items": ("type": "string"), "minItems"; 1, "description": "The 1ist of search queries." "
b, requ"ree"; ["query"]
Listing 2: Google Scholar Tool Schema
"type": "function".
"function";
"name": "google scholar", "description"; "Leverage Google Scholar retrieve relevant information from academic publications. This tool also returns results from Google search.",
"parameters";
"type": "object", "properties":
"query": {
"type": "array",
"items": ("type": "string"),
"minItems"; 1, "description": "The list of search queries."
requireed; ["query"]
Listing 3: Visit (Web Browser) Tool Schema
"type": "function".
27

---

Published as a conference paper at ICLR 2026
"function":
"name": "Visit",
"description"; "Visit webpage (s) or paper (s) and return the summary of the content.",
"parameters":
"type": "object", "properties":
"uri":
"type": "array", "items": 1"type*: "string"), "minitems"; 1, "description": "The (Rb(S) to visit." b
"goal": "type": "string",
"description": "The geal of the visit."
"parse_type";
"type": "string",
"enum": ["html", "pdf"),
"default"; "html", "description", "Speci fy /html' or 'pdf' format."
"required;; ["uri", "goal"|
Listing 4: Python Interpreter Tool Schema
"type": "function",
" "function":
"name": "BythonInterpreter",,
"description": "Executes Pythen code in secure sandbox. Designed for calculations, data manipulations, and general programming tasks.",
"parameters";
"type": "object", "properties":
"code": {
"type": "string", "description": "The Pythen code to execute. Output must use print () functions."
"required;; ["code"]
E.2 INSTRUCTION OF OUR ITERRESEARCH
Prompt of IterResearch
You professional problem-solving agent with rigorous information verification capabilities and deep analytical thinking.
## CRITICAL OUTPUT FORMAT REQUIREMENTS
You MUST follow this exact format. Every response must contain:
28

---

Published as a conference paper at ICLR 2026
1. <report>..<<hepport (always required)
2. Either <answer>.. </answerr> OR stool.llll (never both)
## Input Format
** Currenn Date** Current Date
**Question** The problem posed by the user that needs to be solved
** Last Status Report and Deep Analysis**; A summary overview current work progress
Last Tool Call**; The specific action taken in the previous round
Last Observation** * The results and feedback obtained after the previous action
## Output Format
<report>>
### Status Report and Deep Analysis **Progress *chieved:**
Based on the Last Status Report and Deep Analysis and Last Tool Response provided in the input, compile a comprehensive and complete documentation of all currently collected information, conclusions, data, and findings. This section must capture ALL important information without any omissions, presented in plain text format with corresponding sources clearly annotated. You must directly record the actual information content rather than using referential markers summaries. This includes:
1. All factual data and evidence collected
2. All analytical conclusions and insights derived
3. All source materials and their verification status
4. All uncertainties, limitations, gaps identified
5. Complete integration of previous progress with new findings The documentation must be sufficiently detailed and complete that someone can fully inherit and understand all achieved progress to seamlessly continue the research without losing any critical information or context.
Next Steps Plan:*
Based on the comprehensive progress achieved above, formulate detailed and actionable plan for next phase of research or investigation. </report> You MUST output this section enclosed with <report> </report> tags!
<report><<hepor>>>>>>>>>>>>>>>>>
** Decision *oin Are you certain that further verification or information gathering is needed to provide the final answer?
**1f YES Information is sufficient:**
<answer>>
Answer Format:
1. ** =aLguuage*** Your answer should be in the same language the question. If the question uses English, answer in English. If the question uses Chinese, answer Chinese. 2. The answer should include much relevant content possible. Organize the content into separate paragraphs to avoid overly long sections. Avoid content duplication in the answer.
3. Do not include any non-text elements such as URLs, images, or tables that appeared in the reasoning.
4. Output only the answer text. Do not use any additional symbols or start with phrases like
'Here my answer'.
5. First, output a direct answer to the question.
Do not just output the answer the question; provide rich and lengthy response by synthesizing all relevant information, and format it using markdown.
For statistical data with least items, use a markdown table to present the results, ensuring the table description is clear. For less than 3 items, describe them directly in text. For research-type questions, try to generate a report over 1000 words, using subhead-
29

---

Published as a conference paper at ICLR 2026
ings and other elements to improve readability and logic.
</answer>>
You MUST output this section enclosed with "aanswr>></answeee tags!
"*If NO Further action needed:** **
<tool_call>
"name": "tool name here", "arguments": "parameter name here": parameter value here,
"another parameter name here": another parameter value here.
</tooLcall>>
You MUST output this section enclosed with <tool call></tool call>> tags!
## Working Principles
1. Riggorous Verification Critically evaluate all information sources
** *Deepp Thinking**; Pursue essential understanding, not satisfied with surface phenom- ena
*#Evidence-Driven*** Make reasoning decisions based on reliable evidence through deep thinking
You are required to maintain detailed documentation in all your reports and actions, providing sufficient information for others to fully grasp your progress and effectively continue or modify the research trajectory based on your contributions." **
## Special Requirements
All tools in the tool list real and functional as long you make correct tool calls, you will receive their returned results.
Clearly distinguish between "confirmed facts," "highly credible inferences," and "hy- potheses to be verified"
Clearly indicate uncertainty when information is insufficient
Always focus on the original question
When outputting [Status Report and Deep Analysis], never omit key actions and results, even if these actions or results do not meet expectations, these conclusions must still be documented.
**When further action is needed, you must select an appropriate tool from your available tool list and carefully configure the tool call parameters based on the tool's specific characteristics and requirements**
** When the current status sufficient to answer the question, must provide the final answer enclosed with <annwer>></anwwee tags rather than continue with actions**
## FORMAT REMINDER
Start with <report>... </reporr> section
Then choose: <answer>...<answerr if sufficient info, OR ctool.call..../oor call>
need more action
Never output both answer and tool call tags same response
## Input
Current Date: (date_to_use}
Question: {question]
Available Tools [tools}
-Last Status Report and Deep Analysis:
<report>
(report)
</report>>
Last Tool Call:
<tool
(action)
</tool
Last Tool Response:
30

---

Published as a conference paper at ICLR 2026
resoo_sespons>>>
{observation]
</tool response>>
Now please begin your deep analytical work. The language of your output must be consistent with the language of the question. If the question is in Chinese, output in Chinese; if the question English, output in English.
31


Reviews:
[{"review_id": "cBQcawjAdm", "reviewer": "Reviewer_reAn", "summary": "This paper introduces IterResearch, an iterative deep-research paradigm that overcomes the limitations of mono-contextual approaches in long-horizon reasoning. It maintains an evolving report as memory and periodically synthesizes insights. The system was trained on an aggregated QA dataset, using both supervised fine-tuning and a reinforcement learning phase. Experiments demonstrate substantial improvements over existing open-source agents and notable gains when used as a prompting strategy for frontier models.", "questions": "Q1: Could the authors clarify how the baselines were adopted for the studied benchmarks? Were the open-source agents also fine-tuned on the same data as IterResearch?\n\nQ2: Which datasets were used during the RL phase? Were there tasks in the training set similar to those on which the approach was evaluated, or was it still only the QA Collection? How does the test set intersect with the QA Collection?\n\nQ3: What is the token budget and inference cost for the baseline LLMs? Which budget is needed to reproduce the results.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The problem is well-motivated: mono-contextual approaches accumulate information in a single context window, causing noise and limiting effectiveness on long-horizon tasks.\n- The presented approach significantly outperforms current state-of-the-art models in solution quality on selected benchmarks.", "weaknesses": "W1: The novelty of the approach is somewhat limited, as many of its core ideas, such as using discounted rewards and iterative refinement of trajectories, are already widely used in classical reinforcement learning.\n\nW2: The MDP formulation appears incomplete. It omits a reward function (e.g., a verification signal), and the transition function is likely not deterministic, since tool outputs can vary over time (e.g., search results). The decision space is unconventional: including Think and Report as part of the action reflects internal computation rather than true environmental actions. Additionally, the “reconstructed workspace” manages context by discarding some history, but this alters the state definition across timesteps and may violate the Markov property.\n\nW3: While I understand the intuition behind why discounted reward shaping works, the analogy with RL and the discount factor appears somewhat misleading. First, if we treat this reward as in a typical RL environment, it becomes biased: two similar trajectory prefixes may receive different rewards depending on the trajectory length (the length of the suffixes). Consequently, this mechanism not only amplifies the signal at the end of successful trajectories but also implicitly encourages shorter trajectories. I think this modification is more akin to how value is computed in RL, where there exist well-established alternatives, such as GAE. \n\nW4: The approach was trained exclusively with Qwen-3-30B-A3B, which may limit its generalization to other models. Experiments with models used without fine-tuning only partially address this concern.\n\nW5: Provide a stronger comparison with AlphaEvolve [1] (and similar systems), as the three components used, main objective question (task), evolving report (previous high-level ideas), and immediate context (code), closely mirror its design (in the case of AlphaEvolve, for coding tasks). The use of a reward function applied at the end of generation similarly provides the necessary learning signal for optimization. Although Alpha-Evolve itself is closed-source, several open-source implementations exist (e.g., OpenEvolve). I recommend that the authors at least position their work in relation to these approaches, and ideally include a direct empirical comparison.\n\nW6: The paper assumes that the LLM can reliably summarize, filter noise, and preserve crucial information in each round. In practice, this compression step could lead to the loss of essential context, especially under noisy tool outputs, and may worsen compounding summarization errors.\n\n[1] Novikov A, Vũ N, Eisenberger M, Dupont E, Huang PS, Wagner AZ, Shirobokov S, Kozlovskii B, Ruiz FJ, Mehrabian A, Kumar MP. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131. 2025 Jun 16.\n\n**Minor suggestions**:\n- I recommend that the authors provide confidence intervals in the table, and include the token budget used for each model.\n- It is recommended to include in the description of Table 1 an explanation of why proprietary models were not tested on some benchmarks, to make the table self-contained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YYbSsK0Ev8", "reviewer": "Reviewer_wSFE", "summary": "The paper proposes IterResearch, a long-horizon reasoning framework where an agent maintains a Markovian workspace instead of accumulating the full dialogue history. The evolving report acts as compressed memory, keeping context size nominally constant. Training uses Efficiency-Aware Policy Optimization (EAPO) with geometrically discounted rewards to promote concise, successful trajectories.\n\nIterResearch achieves notable gains (+14.5 pp on six benchmarks) and reportedly scales from 32 to 2048 reasoning steps, outperforming prior open-source systems and even rivaling proprietary ones. However, the \"constant-context\" claim is partly theoretical - since the report must still grow or overwrite information, true unbounded reasoning is unproven. The paper also lacks experiments confirming recovery from early false summaries or testing smaller training horizons. Comparisons rely on older baselines (GPT-4.1, o4-mini), and it remains unclear whether gains stem from the paradigm itself or its prompting format.\n\nOverall, IterResearch is an elegant compression-based reformulation of long-horizon reasoning, but its claimed scalability and memory efficiency need stronger empirical validation.", "questions": "1. What happens when $T_{max}$ during training is reduced to, say, $8$ or $16$? Does extrapolation still hold, or does generalization degrade proportionally?\n\n2. How can we confirm that high-turn (>100) trajectories generate genuinely new insights rather than re-searching similar content? Have the authors analyzed overlap of retrieved URLs or paraphrased content between early and late rounds?\n\n3. How do accuracy and average step count vary with $\\gamma$ (e.g., $0.99$/$0.997$/$0.999$)? Is the $5.7%$ length reduction robust, and where does performance begin to trade off?\n\n4. If the evolving report contains an early false hypothesis, can the agent recover? Have the authors measured recovery rate with stress tests where earlier synthesized content is wrong/misleading?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear, principled formulation. The Markovian workspace reconstruction is simple and well-motivated, giving constant context size and avoiding \"context suffocation\"/\"noise contamination\". The formal transition s_{t+1}=T(s_t,d_t,E(a_t)) is explicit.  \n\n2. Strong empirical coverage. Six benchmarks with diverse characteristics; competitive vs. proprietary systems on several, e.g., surpassing OpenAI DeepResearch on HLE and BrowseComp-zh. \n\n3. General usefulness as a prompting recipe. IterResearch prompting outperforms ReAct with frontier models (o3, DeepSeek-V3.1), suggesting paradigm-level value beyond one implementation.", "weaknesses": "1. Markovian constant context is a conceptual simplification rather than a true breakthrough.\nThe claimed $O(1)$ workspace complexity is only formal when the evolving report $|M|$ is treated as fixed. In practice, $|M|$ grows with the number of synthesized summaries, and each tool response $|TR|$ can vary widely across steps. Therefore, the overall process still depends on in memory and computational complexity, merely folded into a different structure. This makes the \"constant\" context claim more of a heuristic compression scheme than a provably unbounded process. Once the report becomes saturated, the system risks discarding fundamental early-stage information, effectively re-introducing the limitations it sought to remove.\n\n2. Potential inefficiency from report initialization.\nIf the evolving report $|M|$ is kept constant, the first few interactions still include a large workspace prompt even when the agent has accumulated little information. This may yield unnecessary compute overhead at the start of every episode, contradicting the claimed \"efficiency-aware\" principle.\n\n3. Questionable extrapolation to long horizons.\nThe claim of scaling from $T_{max}=32$ (training) to $2048$ interactions (inference)rests on the assumption that the agent can reconstruct forgotten information through new searches. Yet, no evidence is given that this process retrieves previously lost or overwritten insights, as opposed to repeatedly circling around similar queries or rediscovering the same partial results. Without qualitative trajectory inspection, it is unclear whether high-turn runs reflect genuine continued reasoning or redundant loops.\n\n4. No measurements for smaller training horizons.\nWhile Section 4.4 evaluates scaling at inference, there are no experiments showing performance under smaller $T_{max}$ during training. Thus, it is impossible to assess whether longer training horizons produce better generalization or if the model’s extrapolation is accidental.\n\n5. Outdated or inconsistent baseline comparisons.\nThe benchmark comparison includes GPT-4.1 and o4-mini but omits GPT-5-series or contemporary baselines. This limits the credibility of claims about competitiveness \"with frontier proprietary systems\".\n\n6. Lack of ablation on report-based prompting alone.\nThe paper does not include a variant where the same prompt structure is given to a baseline model without the workspace-reconstruction mechanism (i.e., the same input text but no internal state machine). It remains unclear how much of the improvement arises from the report-prompt structure rather than the Markovian update itself.\n\n7. Novelty of EAPO may feel incremental. Geometric discounting for terminal rewards is standard MDP practice; while appropriate here, the RL contribution beyond applying discounting + GSPO + downsampling reads as engineering rather than fundamentally new learning theory. Ablations isolate length reduction but not a deeper analysis of policy changes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5Qkvn6CH6t", "reviewer": "Reviewer_XKsG", "summary": "The paper proposes IterResearch, a new method for deep research. Current approaches keep expanding the context in the think->act phases which can limit the space available for a response. IterResearch is inspired from MDPs and compresses the current context by summarizing it into a report. Thus, the new context consists of the original question, the report, and any actions to take.\n\nThe authors also introduce EAPO - Efficiency Aware Policy Optimization by using discounted rewards for binary, sparse deep research settings where the reward is only given on the final step.\n\nFinally, IterResearch can also be used as a prompting paradigm directly so that no training etc is needed and can work with current models.\n\nThe authors then conduct an empirical evaluation with different baselines.", "questions": "Please see weaknesses", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clear and well-written\n\n2. The overall idea is intuitive\n\n3. Results demonstrate significant improvement (although there are some open questions here)", "weaknesses": "Overall, the results clearly seem to advance the SOTA so Im hoping for more clarifications to my identified weaknesses. Im happy to engage in discussion and revise my score if my concerns are adequately addressed.\n\n1. I think one of the major claims of the paper is the Markovian State Reconstruction. There is not really any guarantee that this report is Markovian in any sense. The Markovian State suggests that the future state is independent of the history given the current state. \n\nIf that were the case, then definitely the transition function with the input history or with the report is not going to change the outcome. So the paper title feels a bit bold. It is more apt to call it summarization but calling it Markovian State Reconstruction is a bit too much. I think experiments would be needed to back your claims.\n\nThe idea is intuitive to improve the performance due to limitations on attention windows/distributions but to claim that summarization of the past context and making it a report is making it Markovian seems a bit overblown IMO.\n\n2. I tihnk one problem with the results is that the prompts are not provided for the baselines in the prompt-based approach. The prompt used in appendix F.2 is quite exhaustive and Im not sure if some instructions that are agnostic to the approach were used in the baselines.  For example, the entire section of ## Working Principles. This might be an issue with clarity in the paper.\n\n3. I think the claim of unbounded interactions is overstated. Even summarization has limits. Are the summaries never going to grow? How does the LLM know what information is important. There could be some tool response that is only going to be needed several steps later. This is often the case in partial-order planning. One issue here is that once that information is not included in the summary, it is lost forever. At least in the mono-context case it is possible to be \"re-\"discovered by a later step based on the generated output and attended to. I am a bit confused on how the LLM knows to generate a good report so that such issues do not occur.\n\n3. I dont think EAPO is really novel. RL algorithms use discounting by definition and this does not seem like a new contribution. Please justify why you think EAPO is novel?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)