Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window

## Abstract

Abstract While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a novel framework that elicits such abilities by introducing high-difficulty training tasks and dynamic context window. DeepMiner presents a reverse construction method to generate complex but verifiable question-answer pairs from authentic web sources, which ensures the challenge and reliability of training data while injecting cognitive capabilities into multi-turn reasoning scenarios. We further design an elegant yet effective dynamic context management strategy for both training and inference, utilizing sliding window mechanisms while eliminating the dependency on external summarization models, thereby efficiently empowering the model to handle continuously expanding long-horizon contexts. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent by almost 20 percentage points, and demonstrates consistent improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within standard 32k context length , effectively addressing the context limitations that constrain existing multi-turn interaction systems.

## 1 Introduction

Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have enabled significant breakthroughs in mathematical reasoning and code generation (DeepSeek-AI, 2025; Yang et al., 2025; Liu et al., 2025b). These methods empower large language models to exhibit sophisticated cognitive behaviors such as self-verification, backtracking, and subgoal decomposition in single-turn settings (Gandhi et al., 2025).
The release of OpenAI’s DeepResearch (OpenAI, 2025b) demonstrates that such cognitive capabilities can be successfully extended to multi-turn and long-horizon tasks, yet how to achieve this extension remains unknown.
While existing open-source efforts have explored this direction through high-quality data generation (Li et al., 2025a; Tao et al., 2025; Gao et al., 2025; Wu et al., 2025) and specialized reinforcement learning algorithms (Song et al., 2025; Jin et al., 2025; Zheng et al., 2025), they have not achieved the same level of capability enhancement observed in single-turn settings. Substantial performance gaps persist compared to proprietary systems that maintain stable performance across dozens of interactions.

In this paper, we focus on search agents and identify two fundamental challenges that prevent existing approaches from effectively leveraging long-horizon interactions:

- •
Insufficient Task Complexity. Current datasets such as TriviaQA (Joshi et al., 2017), 2WikiMultihopQA (Ho et al., 2020), and HotpotQA (Yang et al., 2018), while involving multi-step reasoning, rely on structured Wikipedia data and permit success through shallow information retrieval. These tasks fail to elicit sophisticated cognitive behaviors such as verification, backtracking, and strategic planning that characterize expert-level reasoning.
- •
Context Management Limitations. Long-horizon interactions rapidly consume available context through accumulated tool responses, with a typical 32k context length supporting only approximately 10-15 effective interaction turns. Current solutions primarily rely on summarization compression of tool outputs (Li et al., 2025a; Gao et al., 2025), but this approach introduces several limitations. First, summarization inevitably loses fine-grained information critical for precise reasoning. Second, additional summarization models increase system complexity and computational overhead. Third, summarization components cannot be integrated into end-to-end RLVR optimization, creating optimization blind spots.

To address these challenges, we present DeepMiner, a training framework for long-horizon multi-turn agents via constructing high-difficulty training tasks and managing continuously growing model contexts. For task complexity, we develop a reverse construction method that generates challenging QA pairs by synthesizing questions across multiple authentic web sources, with rigorous filtering to ensure genuine reasoning demands. Our primary contribution lies in addressing context management limitations through a dynamic sliding window strategy that selectively compresses distant tool responses while preserving assistant reasoning traces. Additionally, our strategy maintains access to original webpage content rather than compressed summaries, avoiding the information loss and optimization blind spots inherent in summarization-based approaches.

We implement DeepMiner on Qwen3-32B and evaluate the resulting model across multiple deep research benchmarks. DeepMiner-32B achieves 33.5% accuracy on BrowseComp-en, surpassing the previous best open-source agent nearly 20 percentage points. Consistent improvements are observed across BrowseComp-zh, XBench-DeepSearch, and GAIA. These advantages suggest that high-quality training data and effective context management can jointly facilitate the development of deep reasoning capabilities in long-horizon, multi-turn interactions. Notably, our dynamic context management enables sustained interactions 100 tool calls within standard 32k context length, demonstrating the effectiveness of our approach for extending the operational horizon of long-horizon reasoning agents.

## 2 Complex Question Construction

Training agents with deep reasoning capabilities in long-horizon interactions requires tasks that inherently demand extended exploration and sophisticated multi-step reasoning. Existing datasets predominantly enable success through simple retrieval or shallow reasoning patterns, failing to elicit the sophisticated cognitive strategies required in complex information-seeking tasks, such as verification, backtracking, and subgoal setting. We tackle this challenge through a reverse construction approach that produces complex QA pairs demanding extended reasoning across multiple authentic information sources.

Our approach grounds question generation in real-world entities and authentic web information, creating deliberately complex tasks that challenge expert-level reasoning abilities. Figure [1](https://arxiv.org/html/2510.08276v1#S2.F1) illustrates our three-stage pipeline: entity-driven information collection, multi-source question generation, and strict quality filtering.

#### Entity-driven Information Collection.

We begin with entity selection from Wikipedia, targeting individuals with moderate visibility (^1^111,000$\sim$10,000 page views over the last six months.) to strike a balance between sufficient information availability and excessive familiarity. This range excludes both overly obscure figures lacking sufficient web presence and highly popular individuals whose information is already included in most models’ parametric knowledge. For each selected entity, we conduct comprehensive information gathering through two primary search strategies: direct name queries for biographical information and news searches for recent developments, typically yielding dozens of potentially relevant web pages per entity.
The collected sources undergo three-stage filtering to ensure source quality: entity correspondence verification, information complementarity assessment, and credibility validation. Entity correspondence verification compares each webpage against Wikipedia to eliminate entity confusion and confirm that pages refer to the right individual. Information complementarity assessment excludes sources that provide no substantial and unique knowledge. Credibility validation filters unreliable websites, retaining only trustworthy sources.

Figure: Figure 1: Overall pipeline for complex question construction.
Refer to caption: x1.png

#### Question Generation.

The question generation process takes multiple selected sources as input while deliberately excluding Wikipedia pages to force synthesis across distributed sources. We provide the LLMs with detailed demonstrations of complex reasoning patterns and explicitly constrain each question to synthesize information from at least four distinct sources, requiring cross-document inference rather than simple fact extraction. To further increase complexity, generated questions undergo secondary obfuscation processing. This obfuscation process increases reasoning demands by replacing specific information with more generic descriptions, transforming concrete details into broader categorical terms while maintaining answer uniqueness. The obfuscated questions require the model to resolve generic descriptions through cross-document synthesis, while ensuring that questions remain solvable with sufficient exploration.

#### Multi-stage Filtering.

Generated QA pairs undergo difficulty filtering and quality filtering to ensure both difficulty and reliability. Difficulty filtering ensures questions require extensive reasoning beyond current model capabilities through two mechanisms: direct search engine queries and zero-shot prompting of reasoning models. We then eliminate questions that find the entity or correct answer through either path, ensuring retained questions genuinely demand tool-assisted multi-step exploration. Quality filtering controls multiple aspects that could compromise reliability. We eliminate questions with the following attributes: 1) contains elements or expressions that may introduce interpretation ambiguity, 2) has evasive or ambiguous answers, and 3) the answer is not logically derivable from available evidence in the given reference documents. By eliminating ambiguity, filtering ambiguous answers, and verifying question validity, we ensure each QA pair provides a reliable training signal.

This reverse construction approach generates training tasks that demand extended multi-step reasoning and strategic planning across long-horizon scenarios. The requirement for multi-source information integration and deliberate obfuscation creates an environment where models must engage in genuine cross-document reasoning across extended interaction sequences, providing the challenging training substrate necessary for effective reinforcement learning optimization. Question examples can be found in Appendix [E](https://arxiv.org/html/2510.08276v1#A5).

## 3 Reinforcement Learning with Dynamic Context Window

### 3.1 Dynamic Context Management Strategy

Figure: Figure 2: Preliminary experiments with gpt-oss-120b on the BrowseComp dataset. Left: Length distribution of incorrect trajectories. Right: Context length changes across rounds with/without sliding window. Without sliding window, tool responses grow exponentially and squeeze assistant context. With sliding window, tool length stays constant while assistant content grows normally.
Refer to caption: x2.png

#### Empirical Analysis of Context Challenges.

Complex agent tasks typically require language models to engage in long-horizon interactions. However, in real-world scenarios, the extent of these interactions is fundamentally constrained by the model’s maximum context length. To illustrate this challenge, we analyze error patterns of open-source models on the complex information-seeking task BrowseComp. As shown in Figure [2](https://arxiv.org/html/2510.08276v1#S3.F2), even when setting the max context length to 128k, the majority of model failures occur when the maximum context length is reached. Case analysis reveals that models often recognize they have not found the answer and attempt to continue searching, but are forced to terminate due to context constraints rather than task completion.

We further examine model output patterns under context limitations, with results presented in Figure [2](https://arxiv.org/html/2510.08276v1#S3.F2). Under most used 32k length constraints, models typically reach context limits after only 10-15 turns, which proves insufficient for complex deep research tasks. Detailed analysis of context composition reveals that tool responses are typically 5-10 times longer than assistant responses, creating rapid context consumption that severely limits interaction depth. Further investigation into tool response shows that in most cases, current tool responses primarily influence the model’s immediate next decision, with minimal impact on outputs at distant interaction positions. This observation suggests that while tool responses are essential for local decision-making, their long-term retention may not be necessary for maintaining coherent reasoning strategies across extended interactions.

#### Sliding Window Mechanism.

Motivated by these findings, we design a context management strategy that selectively retains tool responses while preserving assistant reasoning traces.
Let $\tau=\{q,a_{1},t_{1},a_{2},t_{2},\ldots,a_{T-1},t_{T-1},a_{T}\}$ represent a complete interaction trajectory, where $q$ denotes the user query, $a_{i}$ represents the $i$-th assistant response, and $t_{i}$ represents the corresponding tool response. We implement sliding window management with parameters window size $\mathcal{W}$ and slide step $\mathcal{S}$ to control tool response visibility.

As shown in Figure [3](https://arxiv.org/html/2510.08276v1#S3.F3), the sliding operation triggers when the number of accumulated tools $|R_{t}|$ reaches $\mathcal{W}$. At this point, we identify the sliding boundary $b=\max(1,t-\mathcal{W}+\mathcal{S})$, then replacing tool responses $\{t_{1},t_{2},\ldots,t_{b-1}\}$ with placeholder token $\phi=$ “[Previous tool output skipped. Re-run tool if needed.]”, while maintaining visibility of recent tool responses $\{t_{b},t_{b+1},\ldots,t_{k}\}$. This operation creates a modified trajectory where early tool responses are compressed while all assistant reasoning outputs remain intact, preserving the essential strategic reasoning and decision-making logic that enables coherent long-term planning. As shown in Figure [2](https://arxiv.org/html/2510.08276v1#S3.F2), within the tool response sliding window mechanism, the search agent can even reach 100 turns within only 32k context.

#### Training-Testing Consistency.

The sliding window mechanism introduces a new challenge: training models to operate effectively under dynamic context conditions. Direct training on complete trajectories would create a mismatch with inference behavior, where models encounter varying context states as sliding operations occur. To address this, we decompose each trajectory $\tau$ into multiple training sequences that reflect the different context states experienced during inference.

For a trajectory with $T$ tool calls, we generate $K=\lfloor\frac{T-\mathcal{W}}{\mathcal{S}}\rfloor+1$ training sequences. The first sequence $\tau^{(1)}$ contains complete initial context with all assistant responses trained. For subsequent sequences $k>1$, we construct training instances where early tool responses are replaced with placeholders according to the sliding boundary, while recent context remains within the active window.

To prevent optimization conflicts, we ensure each assistant response is trained exactly once across all sequences through careful masking:

$$ $M^{(k)}_{i}=\begin{cases}0&\text{if }i<\mathcal{W}+(k-2)\cdot\mathcal{S}+2\\ 1&\text{otherwise}\end{cases}$ $$

This training strategy ensures models learn to reason effectively under the same dynamic context conditions they encounter during inference, maintaining consistency between training and deployment scenarios while enabling scalable optimization on arbitrarily long interaction sequences.

Figure: Figure 3: Sliding window mechanism for dynamic context management. Upper: inference trajectory with sliding window (size=3, slide=2) where older tool responses are replaced by omission tokens while assistant outputs are preserved. Lower: corresponding training sequences where only newly generated responses receive gradient updates while previous outputs serve as fixed context.
Refer to caption: x4.png

### 3.2 Cold Start

To establish foundational tool-calling and long-horizon reasoning capabilities, we employ supervised fine-tuning as a cold-start phase. We design a pipeline to generate action trajectories through powerful language models, where the sliding window mechanism is also applied during trajectory generation to eliminate turn limitations imposed by context constraints. We filter trajectories with incorrect answers or excessive length, retaining high-quality examples that demonstrate effective tool usage and multi-step reasoning.
During training, we employ the multi-sequence construction method described in Section [3.1](https://arxiv.org/html/2510.08276v1#S3.SS1.SSS0.Px3) to ensure training-testing consistency.
This cold-start phase provides models with essential capabilities for structured tool interaction and coherent reasoning under dynamic context conditions, establishing the foundation for subsequent reinforcement learning optimization.

### 3.3 Reinforcement Learning Training

Trajectory-level Advantage with Sequence-level Training. We employ Group Relative Policy Optimization (Shao et al., 2024) as our reinforcement learning algorithm, and adapt the advantage computation to work with our sliding window context management. The key challenge lies in reconciling trajectory-level reward signals with sequence-level training requirements imposed by sliding window processing.

For each question $q$, we generate $G$ complete trajectory rollouts using the sliding window mechanism during inference. Rewards are computed at the trajectory level based on the final answer, and advantages are calculated through group-relative comparison among these $G$ trajectories:

$$ $\hat{A}_{i}=\frac{R_{i}-\text{mean}(\{R_{j}\}_{j=1}^{G})}{\text{std}(\{R_{j}\}_{j=1}^{G})}$ $$

What we modified lies in advantage propagation: each trajectory $\tau_{i}$ is decomposed into multiple training sequences $\{\tau_{i}^{(1)},\tau_{i}^{(2)},\ldots,\tau_{i}^{(K_{i})}\}$ following the sliding window protocol. The trajectory-level advantage $\hat{A}_{i}$ is then propagated to all training sequences derived from that trajectory, ensuring each sequence receives the same advantage signal regardless of its specific context state. This approach maintains the group-relative advantage computation that drives effective policy learning while enabling training under the dynamic context conditions that models encounter during inference.

Reward Design. We employ a straightforward reward design where an LLM judge evaluates whether the final answer matches the ground truth, assigning a binary reward of 1 for correct answers and 0 for incorrect ones. This binary signal provides clear optimization targets while avoiding complex reward engineering that might introduce training biases or instabilities in the long-horizon setting.

## 4 Experiments

### 4.1 Experimental Setups

Tool Configuration. Our agent operates with three tools: web_search returns top-10 web search results with titles, URLs, and snippets; fetch retrieves webpage content in Markdown format with pagination, enabling efficient navigation of long documents; and find performs in-page search to locate specific information within webpages. This tool suite supports comprehensive information retrieval while maintaining computational efficiency.

Training Details. We adopt Qwen3-32B Yang et al. (2025) as our base model, with the thinking mode enabled. We employ a two-stage training strategy. For SFT, we train on approximately 3,000 high-quality trajectories using a batch size of 256 and learning rate of 1e-5. The RL phase uses around 4,000 questions with a batch size of 32 and learning rate of 2e-6. During on-policy RL training, we generate 8 rollouts per question for policy optimization, with a maximum trajectory length of 40k tokens and turn limit of 60. To manage context length, we set the tool window size to 5 with sliding size of 3. All training is implemented using the VERL framework.

Evaluation. We evaluate DeepMiner on four challenging deep research benchmarks: BrowseComp (Wei et al., 2025), BrowseComp-zh (Zhou et al., 2025), XBench-DeepSearch (Chen et al., 2025), and GAIA (Mialon et al., 2023). We use complete test sets for main experiments and a BrowseComp subset for detailed analysis.
Our evaluation employs the following configuration: temperature of 0.6, top-p of 0.9, maximum of 100 interaction turns. For context management, we set the sliding window size to 5 with a slide step of 3. All results are evaluated using ChatGPT-4o-Latest as the judge model, with evaluation prompts provided in Appendix [D](https://arxiv.org/html/2510.08276v1#A4).

Baselines. We compare DeepMiner against three categories of systems for comprehensive performance evaluation. First are commercial systems, primarily including leading products such as OpenAI DeepResearch (OpenAI, 2025b), Metaso DeepResearch (Metaso, 2025), and Kimi Researcher (Team et al., 2025). Second are advanced general models, including OpenAI-o3 (OpenAI, 2025a), Claude-4-Sonnet (anthropic, 2025), DeepSeek-R1 (DeepSeek-AI, 2025), DeepSeek-V3.1 (DeepSeek Team, 2025), Kimi-K2 (Team et al., 2025), GLM-4.5 (Zeng et al., 2025). Finally, recent open-source work includes Webshaper (Tao et al., 2025), ASearcher (Gao et al., 2025), WebDancer (Wu et al., 2025), WebSailor (Li et al., 2025a), WebExplorer (Liu et al., 2025a), and DeepDive (Lu et al., 2025b). We adopt the official results reported in their papers.

**Table 1: Performance comparison on deep research benchmarks.**
| Model | BrowseComp | BrowseComp-zh | Xbench-DS | GAIA |
| --- | --- | --- | --- | --- |
| Commercial Agents |  |  |  |  |
| OpenAI DeepResearch | 51.5 | 42.9 | - | 67.4 |
| Metaso DeepResearch | 12.0 | 45.4 | 64.0 | - |
| Kimi Researcher | - | - | - | 69.0 |
| General LLMs with tools |  |  |  |  |
| OpenAI-o3 | 49.7 | 58.1 | 66.7 | 70.5 |
| Claude-4-Sonnet | 12.2 | 29.1 | 64.6 | 68.3 |
| DeepSeek-R1 | 8.9 | 35.7 | 55.0 | - |
| Kimi-K2-Instruct-1T | 14.1 | 28.8 | 50.0 | 57.7 |
| GLM-4.5-355B | 26.4 | 37.5 | 70.0 | 66.0 |
| DeepSeek-V3.1-671B | 30.0 | 49.2 | 71.2 | 63.1 |
| Open-source Agents |  |  |  |  |
| WebShaper-72B | - | - | - | 60.0 |
| ASearcher-32B | - | - | 42.1 | 52.8 |
| WebDancer-32B | 3.8 | 18.0 | 39.0 | 51.5 |
| WebSailor-72B | 12.0 | 30.1 | 55.0 | 55.4 |
| DeepDive-32B | 14.8 | 25.6 | 50.5 | - |
| WebExplorer-8B | 15.6 | 32.0 | 53.7 | 50.0 |
| Our Agents |  |  |  |  |
| DeepMiner-32B-SFT | 21.2 | 28.0 | 53.0 | 54.4 |
| DeepMiner-32B-RL | 33.5 | 40.1 | 62.0 | 58.7 |

### 4.2 Main Results.

We evaluate DeepMiner across four challenging deep research benchmarks. As shown in Table [1](https://arxiv.org/html/2510.08276v1#S4.T1), DeepMiner achieves substantial improvements over existing open-source agents while approaching the performance levels of leading proprietary systems.

DeepMiner achieves substantial improvements over existing open-source agents, with particularly impressive results on the challenging BrowseComp benchmarks. On BrowseComp-en, DeepMiner-32B reaches 33.5% accuracy, substantially outperforming all the previous open-source agents and notably exceeding DeepSeek-V3.1-671B, a model over 20 times larger.
Notably, even our SFT model demonstrates strong performance, significantly outperforming most existing open-source agents and highlighting the effectiveness of our reverse construction method for generating high-quality training data.
The performance gains extend consistently across BrowseComp-zh, XBench-DeepSearch, and GAIA. This consistent performance across benchmarks validates both the generalizability of our approach and the diversity of our automatically constructed training data.

The progression from supervised fine-tuning to reinforcement learning demonstrates substantial and consistent improvements across all evaluated benchmarks, confirming the effectiveness of our training methodology. RL optimization provides significant gains over SFT model: +12.3 percentage points on BrowseComp-en, +12.1 points on BrowseComp-zh, +9.0 points on XBench-DeepSearch, and +4.3 points on GAIA. The magnitude of improvement is especially pronounced on the most challenging benchmarks—BrowseComp-en and BrowseComp-zh—where tasks demand sophisticated reasoning strategies. This pattern suggests that reinforcement learning with our generated dataset effectively enhances the model’s ability to develop multi-step reasoning strategies and maintain coherent search policies across extended interactions.

### 4.3 Context Management Efficiency Analysis.

**Table 2: Comparison of different context management strategies across multiple dimensions. All evaluations are conducted on GPT-OSS-120B. DeepMiner’s performance remains stable, as it nearly reaches the maximum of 100 rounds within 32k context length.**
| Strategy | Detail<br>Availablity | End to End<br>Optimization | 32k | 64k | 128k |
| --- | --- | --- | --- | --- | --- |
| Vanilla | ✓ | ✓ | 9.0 | 23.7 | 30.3 |
| Summary | ✗ | ✗ | 10.0 | 25.3 | 31.6 |
| DeepMiner | ✓ | ✓ | 33.3 | 33.3 | 33.3 |

To compare the efficiency of different context management strategies, we evaluate three approaches across varying context length limits: vanilla approach (no context management), external summarization, and our sliding window method. We conduct this evaluation using GPT-OSS-120B. Table [2](https://arxiv.org/html/2510.08276v1#S4.T2) shows that our sliding window method achieves superior context utilization efficiency while maintaining critical technical advantages.

Our approach demonstrates remarkable context efficiency: DeepMiner achieves 33.3% accuracy on BrowseComp using only 32k context, exceeding the performance of alternative methods that require 128k context length.
These results confirm that dynamic sliding window management enables more efficient context utilization while providing fundamental technical advantages. The superior performance achieved in this training-free evaluation demonstrates the inherent efficiency of our approach for enhancing long-horizon reasoning capabilities.

### 4.4 Detailed Analysis

Figure: Figure 4: Training Dynamics of DeepMiner Reinforcement Learning.
Refer to caption: x5.png

#### Training Dynamics.

Figure [4](https://arxiv.org/html/2510.08276v1#S4.F4) illustrates the training dynamics of our reinforcement learning process. As shown in the figure, our trajectory length steadily increases with training steps while remaining within the 40k trajectory length limit. As analyzed in Section [3.1](https://arxiv.org/html/2510.08276v1#S3.SS1.SSS0.Px1), this indicates continuous growth of assistant context, reflecting the model’s progressively enhanced capabilities in multi-step reasoning and complex search strategies. This improvement is reflected in the training rewards, which exhibit a sustained growth trend throughout the process, gradually increasing from 0.45 to 0.60, indicating that our tasks are of sufficient difficulty for effective policy learning. The in-domain rewards translate to advantages on evaluation benchmarks, where BrowseComp performance improves from 22% to 33.5%, confirming that our reverse-constructed tasks can stimulate the model to perform deep cognitive behaviors in general long-horizon multi-turn scenarios.

Figure: Figure 5: Scaling on tool call budget and context length.
Refer to caption: x8.png

#### Context Scaling.

We analyze how DeepMiner scales with the tool-call budget and the maximum context length on BrowseComp. As shown in the left subplot of Figure [5](https://arxiv.org/html/2510.08276v1#S4.F5), DeepMiner’s performance increases steadily with larger tool-call budgets, surpassing DeepSeek-V3.1-671B at around 60 calls. At 100 calls, performance reaches 33.5, clearly exceeding the majority of open-source agents and approaching the level of leading proprietary systems, which indicates that tool-call capacity is a central determinant of agent capability. The right subplot reports DeepMiner’s performance under varying context lengths. With a commonly used 32k context length, performance approaches 33.0 – nearly double that of other open-source agents under the same settings. This pattern highlights not only the efficiency of DeepMiner’s context management strategy but also its ability to sustain close to 100 rounds of tool interactions within a constrained context space, demonstrating that careful management of context may be more effective than simply expanding capacity.

#### Data Efficiency.

**Table 3: Comparison between HotpotQA-SFT and DeepMiner-SFT.**
| Data | BC | BCZH | Xbench | GAIA |
| --- | --- | --- | --- | --- |
| HotpotQA | 15.6 | 21.8 | 47.0 | 51.4 |
| DeepMiner | 21.2 | 28.0 | 53.0 | 54.4 |

To validate the efficiency and effectiveness of our constructed QA pairs, we compare the DeepMiner dataset with HotpotQA (Yang et al., 2018), a widely used dataset in search agent training (Jin et al., 2025). We conduct the same cold-start process as in our main experiments to construct an SFT model based on HotpotQA training data.
As shown in Table [3](https://arxiv.org/html/2510.08276v1#S4.T3), model trained on HotpotQA demonstrate inferior performance compared to DeepMiner SFT model. The HotpotQA-trained model achieves only 15.6% on BrowseComp, while the DeepMiner-trained model reaches 21.2%. This performance gap demonstrates that conventional multi-hop datasets are insufficient to elicit the cognitive behaviors required for complex web agent tasks, validating the necessity of our deliberately challenging data construction approach.

## 5 Related Work

#### Reinforcement Learning with Verifiable Rewards.

Currently, Reinforcement Learning with Verifiable Rewards (RLVR) has become the standard method for training LLMs (DeepSeek-AI, 2025; Team et al., 2025; Yang et al., 2025). Various RL algorithms, such as GRPO (Shao et al., 2024), have been proven to demonstrate superior performance across multiple complex tasks. Recently, several works have attempted to introduce RLVR into web agent optimization, employing cold-start instruction tuning followed by RL (Lu et al., 2025b; Wu et al., 2025; Tao et al., 2025; Jin et al., 2025; Wang et al., 2025). However, the unique application scenarios of Web Agents introduce new challenges, such as sparse rewards and context limitations, making it difficult for RLVR methods to achieve their expected benefits. To address this, rather than focusing on constructing new training mechanisms or rewards for reinforcement learning (Yu et al., 2025; Liu et al., 2025b; Lu et al., 2025a), we leverage a dynamic context window mechanism to substitute the context management framework of the ReAct Yao et al. (2023) used in most multi-turn RL. This extends the application scenarios of RLVR without significantly increasing computational costs, achieving efficient long-horizon RL training.

#### Deep Research Agents.

Deep research agents are LLM-based systems that autonomously leverage search engines, web browsing, and various tools to accomplish complex research tasks through multi-step deep reasoning processes (Xi et al., 2025). Proprietary systems like OpenAI’s DeepResearch (OpenAI, 2025b) and others (Google, 2025; Claude Team, 2025) have exhibited unprecedented ability in complex tasks, matching or exceeding human experts. However, their closed architectures and inaccessible training data hinder reproducibility and collaborative research.
Concurrently, the open-source community has achieved significant progress in fundamental tasks (Anonymous, 2025; Li et al., 2025b; OpenPangu Team, 2025). Recent works have shifted toward deep research tasks that demand complex nonlinear reasoning capabilities. Researchers have developed various data synthesis approaches for agent optimization (Liu et al., 2025a; Lu et al., 2025b; Tao et al., 2025). Some other works design context management strategies, such as summarization (Li et al., 2025a; Gao et al., 2025), for deeper interactions.
Despite these advantages, a huge gap persists - even state-of-the-art open-source models achieve less than 20% accuracy on the BrowseComp, far below proprietary systems. Meanwhile, DeepMiner significantly narrows this gap through reverse-constructed complex training tasks and dynamic context window.

## 6 Conclusion

In this work, we present DeepMiner to build deep research agents with cognitive behaviors that can effectively handle sustained long-horizon interactions. DeepMiner addresses two fundamental limitations for web agent training: insufficient training task complexity through reverse construction of verifiable QA pairs from authentic web sources, and context explosion through dynamic context management without external summarization models. DeepMiner preserves complete assistant reasoning traces while compressing distant tool responses, enabling interactions that exceed 100 turns within 40k contexts. DeepMiner-32B achieves 33.5% on BrowseComp, nearly doubling the performance of prior state-of-the-art open-source agents. DeepMiner represents a fundamental shift from context-limited, shallow reasoning to unbounded, deep exploration.

## References

- Anonymous (2025)
Anonymous.
Evolvesearch: An iterative self-evolving search agent.
In *Submitted to ACL Rolling Review - May 2025*, 2025.
URL [https://openreview.net/forum?id=DNS0NlnT63](https://openreview.net/forum?id=DNS0NlnT63).
under review.
- anthropic (2025)
anthropic.
Introducing claude 4, 2025.
URL [https://www.anthropic.com/news/claude-4](https://www.anthropic.com/news/claude-4).
- Chen et al. (2025)
Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, et al.
xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations.
*arXiv preprint arXiv:2506.13651*, 2025.
- Claude Team (2025)
Claude Team.
Claude research, 2025.
URL [https://www.anthropic.com/news/research](https://www.anthropic.com/news/research).
- DeepSeek-AI (2025)
DeepSeek-AI.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
URL [https://arxiv.org/abs/2501.12948](https://arxiv.org/abs/2501.12948).
- DeepSeek Team (2025)
DeepSeek Team.
Introducing deepseek-v3.1: our first step toward the agent era!, 2025.
URL [https://api-docs.deepseek.com/news/news250821](https://api-docs.deepseek.com/news/news250821).
- Gandhi et al. (2025)
Kanishk Gandhi, Ayush K Chakravarthy, Anikait Singh, Nathan Lile, and Noah Goodman.
Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective STars.
In *Second Conference on Language Modeling*, 2025.
URL [https://openreview.net/forum?id=QGJ9ttXLTy](https://openreview.net/forum?id=QGJ9ttXLTy).
- Gao et al. (2025)
Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu.
Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl.
2025.
URL [https://arxiv.org/abs/2508.07976](https://arxiv.org/abs/2508.07976).
- Google (2025)
Google.
Gemini Deep Research: your personal research assistant, May 2025.
URL [https://gemini.google/overview/deep-research/](https://gemini.google/overview/deep-research/).
Accessed: 2025‑08‑05.
- Ho et al. (2020)
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa.
Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps.
In Donia Scott, Nuria Bel, and Chengqing Zong (eds.), *Proceedings of the 28th International Conference on Computational Linguistics*, pp. 6609–6625, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.
doi: 10.18653/v1/2020.coling-main.580.
URL [https://aclanthology.org/2020.coling-main.580/](https://aclanthology.org/2020.coling-main.580/).
- Jin et al. (2025)
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han.
Search-r1: Training llms to reason and leverage search engines with reinforcement learning.
*arXiv preprint arXiv:2503.09516*, 2025.
- Joshi et al. (2017)
Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer.
Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.
*arXiv preprint arXiv:1705.03551*, 2017.
- Li et al. (2025a)
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou.
Websailor: Navigating super-human reasoning for web agent, 2025a.
URL [https://arxiv.org/abs/2507.02592](https://arxiv.org/abs/2507.02592).
- Li et al. (2025b)
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou.
Webthinker: Empowering large reasoning models with deep research capability, 2025b.
URL [https://arxiv.org/abs/2504.21776](https://arxiv.org/abs/2504.21776).
- Liu et al. (2025a)
Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, et al.
Webexplorer: Explore and evolve for training long-horizon web agents.
*arXiv preprint arXiv:2509.06501*, 2025a.
- Liu et al. (2025b)
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin.
Understanding r1-zero-like training: A critical perspective, 2025b.
URL [https://arxiv.org/abs/2503.20783](https://arxiv.org/abs/2503.20783).
- Lu et al. (2025a)
Fanbin Lu, Zhisheng Zhong, Shu Liu, Chi-Wing Fu, and Jiaya Jia.
Arpo:end-to-end policy optimization for gui agents with experience replay, 2025a.
URL [https://arxiv.org/abs/2505.16282](https://arxiv.org/abs/2505.16282).
- Lu et al. (2025b)
Rui Lu, Zhenyu Hou, Zihan Wang, Hanchen Zhang, Xiao Liu, Yujiang Li, Shi Feng, Jie Tang, and Yuxiao Dong.
Deepdive: Advancing deep search agents with knowledge graphs and multi-turn rl, 2025b.
URL [https://arxiv.org/abs/2509.10446](https://arxiv.org/abs/2509.10446).
- Metaso (2025)
Metaso.
Metaso, 2025.
URL [https://metaso.cn/](https://metaso.cn/).
Accessed: 2025-08-05.
- Mialon et al. (2023)
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom.
Gaia: a benchmark for general ai assistants.
In *The Twelfth International Conference on Learning Representations*, 2023.
- OpenAI (2025a)
OpenAI.
Introducing openai o3 and o4-mini.
[https://openai.com/index/introducing-o3-and-o4-mini/](https://openai.com/index/introducing-o3-and-o4-mini/), 2025a.
Accessed: 2025-06-24.
- OpenAI (2025b)
OpenAI.
Introducing deep research.
[https://openai.com/index/introducing-deep-research/](https://openai.com/index/introducing-deep-research/), 2025b.
Accessed: 2025-06-26.
- OpenPangu Team (2025)
OpenPangu Team.
Openpangu deepdiver-v2: Multi-agent learning for deep information seeking, 2025.
URL [https://ai.gitcode.com/ascend-tribe/openPangu-Embedded-7B-DeepDiver](https://ai.gitcode.com/ascend-tribe/openPangu-Embedded-7B-DeepDiver).
- Shao et al. (2024)
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024.
URL [https://arxiv.org/abs/2402.03300](https://arxiv.org/abs/2402.03300).
- Song et al. (2025)
Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen.
R1-searcher: Incentivizing the search capability in llms via reinforcement learning.
*arXiv preprint arXiv:2503.05592*, 2025.
- Tao et al. (2025)
Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al.
Webshaper: Agentically data synthesizing via information-seeking formalization.
*arXiv preprint arXiv:2507.15061*, 2025.
- Team et al. (2025)
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al.
Kimi k2: Open agentic intelligence.
*arXiv preprint arXiv:2507.20534*, 2025.
- Wang et al. (2025)
Liang Wang, Haonan Chen, Nan Yang, Xiaolong Huang, Zhicheng Dou, and Furu Wei.
Chain-of-retrieval augmented generation, 2025.
URL [https://arxiv.org/abs/2501.14342](https://arxiv.org/abs/2501.14342).
- Wei et al. (2025)
Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese.
Browsecomp: A simple yet challenging benchmark for browsing agents, 2025.
URL [https://arxiv.org/abs/2504.12516](https://arxiv.org/abs/2504.12516).
- Wu et al. (2025)
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Yong Jiang, Pengjun Xie, et al.
Webdancer: Towards autonomous information seeking agency.
*arXiv preprint arXiv:2505.22648*, 2025.
- Xi et al. (2025)
Yunjia Xi, Jianghao Lin, Yongzhao Xiao, Zheli Zhou, Rong Shan, Te Gao, Jiachen Zhu, Weiwen Liu, Yong Yu, and Weinan Zhang.
A survey of llm-based deep search agents: Paradigm, optimization, evaluation, and challenges, 2025.
URL [https://arxiv.org/abs/2508.05668](https://arxiv.org/abs/2508.05668).
- Yang et al. (2025)
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu.
Qwen3 technical report, 2025.
URL [https://arxiv.org/abs/2505.09388](https://arxiv.org/abs/2505.09388).
- Yang et al. (2018)
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning.
HotpotQA: A dataset for diverse, explainable multi-hop question answering.
In *Conference on Empirical Methods in Natural Language Processing (EMNLP)*, 2018.
- Yao et al. (2023)
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao.
React: Synergizing reasoning and acting in language models.
In *The Eleventh International Conference on Learning Representations*, 2023.
URL [https://openreview.net/forum?id=WE_vluYUL-X](https://openreview.net/forum?id=WE_vluYUL-X).
- Yu et al. (2025)
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang.
Dapo: An open-source llm reinforcement learning system at scale, 2025.
URL [https://arxiv.org/abs/2503.14476](https://arxiv.org/abs/2503.14476).
- Zeng et al. (2025)
Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al.
Glm-4.5: Agentic, reasoning, and coding (arc) foundation models.
*arXiv preprint arXiv:2508.06471*, 2025.
- Zheng et al. (2025)
Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu.
Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025.
URL [https://arxiv.org/abs/2504.03160](https://arxiv.org/abs/2504.03160).
- Zhou et al. (2025)
Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al.
Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese.
*arXiv preprint arXiv:2504.19314*, 2025.

## Appendix A The Use of Large Language Models

We utilized LLMs to aid and polish writing.

## Appendix B Ethical Considerations

Our training data collection process may inadvertently include personal information from publicly available sources. To mitigate privacy risks, we collect data exclusively from public webpages like Wikipedia, filtering out irregular websites and social media platforms to avoid overly private content. We will release our dataset under strict access controls, requiring formal approval for academic use only. Before public release, we will perform comprehensive anonymization, replacing names and other identifying information to ensure no real personal privacy is compromised. We acknowledge that our models could potentially be misused. To address this concern, we will implement a rigorous review process for all requests to access our open-source model weights, ensuring they are used solely for legitimate research and educational purposes.

## Appendix C Enhanced Tool Suite

Our agent operates with three core tools designed for fine-grained web exploration:

- 1.
Web Search takes text queries as input and returns document titles, URLs, and snippets from search results.
- 2.
Fetch retrieves webpage content given a URL. Unlike existing approaches that truncate content or apply external summarization, our fetch tool implements paginated browsing that mimics human web navigation, allowing the model to assess initial content and decide whether to continue reading or exit.
- 3.
Find enables in-page keyword search for lengthy webpages. The model can locate relevant information sections and the surrounding context before deciding which portions warrant detailed examination.

This design addresses a fundamental limitation where existing web agents suffer information loss through hard truncation or external summarization. By preserving complete information access while providing navigation flexibility, our tool suite enables fine-grained control over information gathering across extended interaction sequences.

## Appendix D Templates

Below are the templates we used.

## Appendix E Question Examples

We show several representative questions below. We present the question, the core entity of this question, and the answer to this question.

### E.1 Case Study

The following is a case trajectory of DeepMiner on BrowseComp.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. Why were Search-R1 and Synthetic Data RL omitted as baselines, given their direct methodological overlap (RL training on hard-QA dataset)?

2. In Figure 4, the reward and performance curves show persistent fluctuations without clear convergence, even after 80 training steps. Could the authors further explain the missing training steps results?
soundness: 2
strengths: 1. Sliding-window design is an effective context-efficiency solution. The method enables around 100 turns within 32 k tokens. Unlike static truncation/summarization, this mechanism scales gracefully with context length and maintains coherence across windows.

2. The empirical motivation is solid with preliminary analysis. Figure 2 clearly shows that without the sliding window, the number of incorrect trajectories increases steeply as context length grows, as tool outputs expand exponentially and squeeze assistant reasoning tokens.
summary: This paper presents a framework DeepMiner for training long-horizon LLM agents that can reason and search across extended multi-turn sessions under a fixed context window.
In particular, it introduces a sliding-window mechanism that omits outdated tool outputs while preserving the assistant’s reasoning trace, ensuring training–inference consistency through windowed sequence-level training.
The system is trained in two phases: a SFT stage using reverse-constructed trajectories from authentic web sources, followed by RL via sequence-level training with trajectory-level advantages.
weaknesses: 1. The reverse-constructed DeepMiner dataset is not open-sourced; details such as question samples, source selection, and verification criteria are missing. Reproduction and fair comparison are thus impossible.

2. The same trajectory-level reward is propagated to all windowed subsequences. This can over-credit early steps and under-credit later corrective reasoning.

3. Missing comparison to related baselines. The paper omits baselines such as Search-R1 [1] and Synthetic Data RL [2], both using RL for multi-hop QA.

4. Limited Ablations and Hyperparameter Studies. No analysis of different window sizes, sliding size, and base model sizes.

[1] Jin, Bowen, et al. "Search-r1: Training llms to reason and leverage search engines with reinforcement learning." arXiv preprint arXiv:2503.09516 (2025).

[2] Guo, Yiduo, et al. "Synthetic Data RL: Task Definition Is All You Need." arXiv preprint arXiv:2505.17063 (2025).

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: In Figure 4, the reward and BrowseComp performance seem to be continue improving at the end of the training. Why didn't authors train the model for longer steps?
soundness: 3
strengths: Quality:

The proposed method achieves significance performance improvement on BrowseComp, which is a very challenging benchmark. It is impressive authors' 32B model outperforms DeepSeek V3.1-671B model, which is an order of magnitude larger, on this benchmark. Authors run good ablations for the training method (context management and tool call budget) although there is no ablation on the data synthesis side.

Significance, Originality:

The management of context window across multi-hop trajectories has been explored in the prior work, for example https://arxiv.org/abs/2505.16421 , but to my knowledge, previous methods were more rudimentary and did not discuss its implication with GRPO-style monte carlo advantage estimation. This paper calls out the attention on the importance of context window management with GRPO-style training, which has the potential of being adopted as standard approach in the area.

Clarity:

The paper is clearly written and easy to follow.
summary: Authors propose a method for training Deep Research agents. There are two key contributions. The first is data contribution: authors propose a method of synthesizing complex questions with verifiable answers which elicit deep research capabilities. Authors also propose a methodology to compact the context window by maintaining tool results for a recent time window. GRPO is slightly extended to accommodate the fact that each turn depends on a different context from past turns. Compared to some of the previous open-source agents, authors' fine-tuning of Qwen3-32B performs better on BrowseComp and competitive on GAIA and Xbench-DS.
weaknesses: The idea of synthesizing training data for deep research with progressively evolving questions and fetching web data have been explored in many concurrent work, for example 
Websailor https://arxiv.org/abs/2507.02592 , Webshaper https://arxiv.org/abs/2507.1506 , DeepDive https://arxiv.org/abs/2509.10446 . While authors do refer these papers and compare against results from these papers, it is unclear whether the improvement is due to better synthesis or better context management, and if authors' method of synthesis is any better than previous methods, which design decisions led to how much improvement and why. Without an ablation separating the contribution of reverse-constructed data from dynamic context management, it is difficult to attribute the performance gain to one factor or the other.

While authors share their prompt for quality judgment in the appendix, many details of the data pipeline including exact prompts, question construction logic, and the LLM used to synthesize data are missing. This makes it difficult for other research groups to build upon results of this paper.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: The following questions are proposed based on the above weaknesses.
1. Regarding the claim that tool responses mainly influence only the model' s immediate next decision — could the authors elaborate how this conclusion was empirically derived? For example, were ablation or correlation analyses conducted to quantify the short-term versus long-term effects of tool responses on reasoning outcomes?
2. The paper employs a simple binary (0/1) correctness reward. Have the authors explored or considered more granular reward designs?
3. The paper mentions that training trajectories are segmented into multiple sub-sequences via sliding windows. Could the authors explain in detail how gradients are propagated within this setup?
4. Since the default sliding window configuration (size=3, step=2) is fixed in all experiments, have the authors conducted ablation or sensitivity analyses to evaluate how varying window sizes or strides impact performance, stability, or context retention efficiency?
5. As the reward design depends on LLM-as-judge evaluation, could the authors provide theoretical or empirical justification for its reliability? Has the team investigated possible bias, variance, or inconsistency in LLM-judged rewards compared with human or rule-based evaluation?
soundness: 2
strengths: 1. The paper tackles an important and timely challenge — enabling deep, long-horizon reasoning in open-domain search agents.
2. The proposed sliding-window mechanism offers a practical way to manage growing contexts without relying on external summarization.
3. The paper presents experiments across multiple challenging benchmarks (e.g., BrowseComp, XBench-DeepSearch, GAIA), showing consistent improvements over previous open-source systems.
summary: This paper presents DeepMiner, a training framework for long-horizon, multi-turn reasoning agents. The authors identify two main limitations in current multi-turn systems — insufficient task complexity and ineffective context management. To address these, DeepMiner introduces: (1) a reverse construction pipeline to generate complex, verifiable, multi-source question-answer pairs from authentic web data; (2) a dynamic context window with a sliding-window mechanism to preserve reasoning traces while selectively omitting older tool outputs. The method is implemented on Qwen3-32B, and after reinforcement learning (RL) training, the resulting DeepMiner-32B model achieves large gains on benchmarks such as BrowseComp-en, BrowseComp-zh, and XBench-DeepSearch, outperforming previous open-source agents by nearly 20 percentage points.
weaknesses: 1. The paper claims that tool responses mainly influence only the model' s immediate next decision, but it is unclear how this conclusion was obtained. No quantitative or ablation evidence supports this key assumption.
2. The reward design is overly simple (binary 0/1 correctness). More nuanced signal (e.g., step-wise or process-based reward) could lead to deeper policy learning.
3. The training mechanism that converts a full trajectory into multiple sub-sequences via sliding windows is conceptually interesting but not clearly specified. In particular, the paper does not explain how gradients are propagated across these truncated sequences and how batches are constructed to ensure consistency between local and global optimization.
4. The default sliding window configuration (size=3, step=2) is used throughout all experiments, but the paper does not include an ablation or sensitivity study exploring how different window settings affect performance, stability, or context retention.
5. The reward design relies on an LLM-as-judge evaluation, but the paper lacks discussion about the reliability and theoretical justification of using such models as verifiers in reinforcement learning.

---

confidence: 4
contribution: 3
initial_rating: 4
presentation: 3
questions: 1. Can authors provide a performance comparison on $BrowseComp$ between [DeepMiner Data + Vanilla Context] (e.g., truncating when the context limit is reached) and [DeepMiner Data + Dynamic Context]? This would be the only direct way to evaluate the contribution of the dynamic context window.

2. Why did authors choose to completely discard tool outputs instead of using a (potentially trainable) summarization module? While summarization adds complexity, discarding information entirely seems like a fundamental constraint for deep research tasks. How does your strategy handle tasks that require backtracking and comparison of early evidence?

    Besides, have you analyzed the failure cases of DeepMiner? How many failures are caused by the model needing an early tool output that was already discarded (e.g., needing content from turn 5 at turn 40)?
soundness: 3
strengths: - The authors correctly identify that existing multi-hop QA datasets (like HotpotQA) are insufficient for eliciting the deep reasoning, verification, and backtracking abilities required in long-horizon tasks. The proposed "reverse construction" and "obfuscation" strategies are an effective attempt at generating high-difficulty, high-fidelity training data, and the comparison in Table 3 validates the superiority of this data.

-  DeepMiner tackles a highly practical and critical problem in multi-turn search agents: context length limitation. Achieving nearly 100 interaction turns within a 32k window is an accomplishment. The substantial performance leap on `BrowseComp-en` by nearly 20 points demonstrates the combined potential of the new data and framework.
summary: This paper presents DeepMiner, a framework designed to address two core challenges in long-horizon search agents: insufficient training data complexity and context management limitations. The authors introduce a "reverse construction" method to generate complex, verifiable QA pairs from authentic web sources. Concurrently, they design a dynamic context management strategy that uses a sliding window to compress (discard) distant tool outputs while preserving the assistant's reasoning chain, enabling nearly 100 turns of interaction within a standard 32k context length.
weaknesses: - The paper lacks  ablation studies, which leaves its core claim—the effectiveness of the dynamic context window—unsubstantiated. How much of the massive performance boost (e.g., +20% on BrowseComp) is attributable to the superior training data versus the dynamic context window? The authors compare DeepMiner data to HotpotQA data in Table 3, but this only proves the importance of the data.

- The paper is missing a crucial comparison: [DeepMiner Data + Vanilla Context] vs. [DeepMiner Data + Dynamic Context]. Without this ablation, it is impossible to determine if the dynamic context window itself actually enhances reasoning capabilities, or if it merely serves as a tool to enable longer runs, with all capability gains stemming from the DeepMiner dataset.

- The context efficiency analysis in Table 2 is "training-free" and uses a completely different base model (GPT-OSS-120B). This makes its results difficult to correlate with the main training results of DeepMiner-32B (based on Qwen3-32B) and thus cannot serve as supporting evidence for the dynamic window's effectiveness.

- Compared to summarization-based methods, summarization, while lossy, at least retains some signal. This paper's method discards the information entirely. For complex tasks requiring long-range dependencies and fine-grained information retrieval, this "all-or-nothing" dropping strategy could lead to catastrophic failures in the reasoning chain.


Decision: Reject