Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools
Authors: Junde Wu, Jiayuan Zhu, Yuyuan Liu, Min Xu, Yueming Jin
Accepted at: acl25

Agentic Reasoning: A Streamlined Framework for Enhancing LLM Reasoning with Agentic Tools

Junde Wu1,2, Jiayuan Zhu1, Yuyuan Liu1, Min Xu3,4, Yueming Jin2∗,
1University of Oxford, 2National University of Singapore,
3Carnegie Mellon University, 4MBZUAI

Abstract

We introduce Agentic Reasoning, a framework that enhances large language model (LLM) reasoning by integrating external tool-using agents. Agentic Reasoning dynamically leverages web search, code execution, and structured memory to address complex problems requiring deep research. A key innovation in our framework is the Mind-Map agent, which constructs a structured knowledge graph to store reasoning context and track logical relationships, ensuring coherence in long reasoning chains with extensive tool usage. Additionally, we conduct a comprehensive exploration of the Web-Search agent, leading to a highly effective search mechanism that surpasses all prior approaches. When deployed on DeepSeek-R1, our method achieves a new state-of-the-art (SOTA) among public models and delivers performance comparable to OpenAI Deep Research, the leading proprietary model in this domain. Extensive ablation studies validate the optimal selection of agentic tools and confirm the effectiveness of our Mind-Map and Web-Search agents in enhancing LLM reasoning. Our code and data are publicly available.

<table>
  <tr>
    <th>Model</th>
    <th>Accuracy (%)</th>
  </tr>
  <tr>
    <td>GPT-4o†</td>
    <td>3.3</td>
  </tr>
  <tr>
    <td>Grok-2†</td>
    <td>3.8</td>
  </tr>
  <tr>
    <td>Claude 3.5 Sonnet†</td>
    <td>4.3</td>
  </tr>
  <tr>
    <td>Gemini Thinking†</td>
    <td>6.2</td>
  </tr>
  <tr>
    <td>OpenAI o1†</td>
    <td>9.1</td>
  </tr>
  <tr>
    <th>DeepSeek-R1</th>
    <th>9.4</th>
  </tr>
  <tr>
    <td>OpenAI o3-mini (medium)†</td>
    <td>10.5</td>
  </tr>
  <tr>
    <td>OpenAI o3-mini (high)†</td>
    <td>13.0</td>
  </tr>
  <tr>
    <th>Agentic Reasoning w/ R1</th>
    <th>23.8 (+14.4)</th>
  </tr>
  <tr>
    <th>Perplexity deep research †</th>
    <th>21.1</th>
  </tr>
  <tr>
    <th>OpenAI deep research †</th>
    <th>26.6</th>
  </tr>
</table>

† denotes proprietary models.

Table 1: On Humanity’s Last Exam, we achieved a remarkable 23.8% with DeepSeek-R1, marking a 14.4% improvement over the base model. This narrows the gap to the proprietary OpenAI Deep Research to just 2.8%, which depends on a stronger internal reasoning model.

DeepSeek-R1 that applying rule-based outcome rewards during training, such as evaluating whether a piece of code executes successfully, could yield remarkable reasoning capabilities equaling o1-level math and coding performance.

Although current reasoning methods excel in structured domains like math and code—where outcomes are easily verifiable—applying these techniques to less structured or knowledge-intensive tasks remains a significant challenge. As mentioned in DeepSeek-R1 (Team, 2024), not all problems benefit from formal reasoning approaches. Many fields, such as social sciences, ethics, or experiential disciplines, rely on abstract concepts, conventional wisdom, factual verification, understanding complex logical relationships, or moral reasoning. When models attempt to impose math- or coding-style reasoning onto such areas, they often produce flawed or overly rigid results. Developing approaches that account for these unique requirements is essential for advancing the appli-

1 Introduction

Recently, large reasoning models, such as OpenAI’s o1 (Jaech et al., 2024), Qwen-QwQ (Team), and DeepSeek-R1 (Team, 2024), have demonstrated impressive stepwise reasoning capabilities over long sequences through large-scale reinforcement learning. These advancements provide promising solutions to complex reasoning tasks (Wei et al., 2022; Lewkowycz et al., 2022; OpenAI) and have inspired foundational efforts to replicate o1-like reasoning patterns across a broader range of models (Qin et al., 2024; Huang et al., 2024; Zhang et al., 2024). It is recently revealed by
cability of reasoning model beyond their current domains.

Deep, thoughtful answers to open-ended questions often require extensive research, repeated verification, information retrieval, computational analysis, and the organization of complex logical relationships—steps fundamental to human reasoning. In this process, humans rely heavily on external tools, such as internet searches for gathering information, computational tools for quantitative analysis, or whiteboards and mind maps for organizing thoughts. This raises an intriguing question: can reasoning LLMs similarly leverage external tools to enhance their reasoning and tackle intensive knowledge work across diverse domains?

Previous efforts have attempted to integrate search or retrieval-augmented generation (RAG) into the reasoning process (Shao et al., 2024; Khaliq et al., 2024; Islam et al., 2024; Li et al., 2025), with notable examples including Gemini’s and OpenAI’s Deep Research. However, these models are proprietary, and their exact methodologies remain undisclosed. In contrast, open-source models primarily focus on retrieval and web-search integration during reasoning but still exhibit a notable performance gap compared to their closed-source counterparts.

We introduce Agentic Reasoning, a framework that enhances reasoning by integrating external LLM-based agents as tools. This approach allows LLMs to delegate specific tasks to auxiliary agents during the reasoning process, thereby improving their overall problem-solving capabilities. Through extensive experimentation with integrating various agents into the reasoning process, we identified three essential agents that prove highly effective for general reasoning across diverse problems. The Web-Search agent, which retrieves relevant information from the internet to supplement the model’s knowledge. The Code agent, capable of performing computational analyses and coding tasks to support quantitative reasoning. Finally, the memory agent, which we call Mind-Map, constructs knowledge graphs based on the reasoning context, enabling the organization of complex logical relationships in a manner similar to human mind mapping. Together, these agents enhance the model’s ability to tackle complex problems and do deep research with greater efficiency and precision.

We evaluated our model on general knowledge-intensive benchmarks requiring complex reasoning capabilities, categorized into two key areas: (1) solving expert-level questions and (2) conducting deep research on real-world expert-level tasks. For expert-level questions, we evaluate the model on Humanity’s Last Exam (Phan et al., 2025), a recently released benchmark assessing AI performance across a broad range of subjects. As shown in Table 1, we achieves a new high of 23.8% accuracy, marking a 14.4% improvement over the raw model and narrowing the open-source vs. closed-source gap to just 2.8% compared to the proprietary OpenAI Deep Research, which benefits from a stronger internal reasoning model. For real-world expert-level tasks, Agentic Reasoning was assessed by domain experts, who found that it effectively automated complex manual investigation. This underscores its potential to streamline labor-intensive processes and boost productivity in knowledge-intensive domains.

In brief, our contribution can be concluded as:

• We propose Agentic Reasoning, a streamlined framework that enhances reasoning by integrating external LLM-based agentic tools. We experimentally identify web-search, coding, and Mind-Map agents as three universally effective tools.

• We explore the design of the Web-Search agent and identify a strategy that outperforms previous search or RAG approaches.

• We develop a knowledge-graph-based Mind-Map to assist reasoning, improving the model’s ability to handle complex logic and maintain coherence in long reasoning chains.

• We evaluate our approach on expert-level problem-solving and deep research tasks, achieving new SOTA results across several benchmarks and surpassing prior methods in human evaluations.

2 Method

2.1 Agentic Reasoning Pipeline

Our core idea is to enhance LLM reasoning by integrating external LLM-based agents into the process. During reasoning, the model can call these agents as tools to assist in problem-solving while maintaining a structured memory to store its reasoning context. In the overall process, we deploy a Web-Search agent and a Code agent as problem-solving
Question
As of Q4 2024, A mid-sized retailer, X operates 50 stores across the southwestern U.S. and aims to reduce overstocking costs by 10% while still maintaining at least 98% product availability for its top-selling items during the upcoming holiday quarter. how can X Retail forecast next quarter’s optimal inventory levels per store to balance cost minimization against high service levels?

LLM Models
Reasoning...
I need to identify Relevant External Economic Indicators [Web-Search]
...
Result: 2.7% inflation and 109 CCI
To model demand volatility amid inflation and shifting consumer sentiment, I need to search historical sales under relevant economic indicators. [Web-Search]...
...
Then, we can use ARIMAX with external regressors to predict per-store sales [Code] ...
...
So putting this all together, the strategies would be [Mind Map]

Tools
Search
Query: US Q4 2024 Inflation and CCI
Searching...
...As of Q4 2024, the United States inflation rate stands at 2.7%, the Consumer Confidence Index (CCI) is at 109 ...
Search
Query: Use ARIMAX to predict Q4 per-store sales given...
Searching...
Result: The projected sales will be...
Code
SARIMAX(sales, exog=[cci, inflation], order=(1,1,1), seasonal_order=(0,1,1,4))
result = model.fit()
exog_future = pd.DataFrame([{"cci": [115], "inflation": [3.5]}, index=pd.Timestamp("2025-03-31")])
Forecast = ...

Figure 1: The overall workflow of Agentic Reasoning. Given a question, the reasoning LLM can invoke the Web-Search agent to retrieve external information, the Coding agent to perform quantitative computations, and the Mind-Map agent to structurally memorize the reasoning context, to provide a comprehensive solution.

tools, along with a knowledge-graph agent, called Mind-Map, to serve as memory.

Specifically, the reasoning LLM can dynamically determine when to call external agentic tools during its reasoning process. As shown in Figure 1, when needed, it embeds specialized tokens into its reasoning sequence, categorizing them as web-search tokens, coding tokens, or Mind-Map calling tokens. Alongside these tokens, the reasoning LLM generates a query as a message to the external agents. Upon detecting such a token, the reasoning process temporarily halts to extract the query and its reasoning context. These queries are then dispatched to the corresponding external agents. The agents would consider both the received query and the reasoning context to ensure the most pertinent results are returned to the main reasoning chain. These results are then reintegrated into the reasoning chain, allowing the model to continue its inference with an updated knowledge. This iterative retrieval-and-reasoning cycle continues as needed, enabling the model to dynamically refine its reasoning until it reaches a fully reasoned final answer.

2.2 Mind-Map Agent
We construct a Mind-Map to store and structure the real-time reasoning context of the reasoning model. This Mind-Map is built by transforming raw reasoning chains into a structured knowledge graph. Specifically, we use a graph-construction LLM to extract entities from the reasoning chain and identify semantic relationships between related entities, following a process similar to that used in GraphRAG (Edge et al., 2024).

The Mind-Map serves two primary functions. First, it clusters reasoning context into distinct groups and summarizes each of them. This is achieved by applying community clustering (Traag et al., 2019) on the knowledge graph and using an LLM to generate concise summaries for each group. Second, the knowledge graph can be queried with specific questions, such as “Who was Jason’s maternal great-grandfather?” Using standard RAG on the knowledge graph (Edge et al., 2024), we retrieve and return the relevant information to response the query.

These functions integrate the Mind-Map into two key aspects of the Agentic Reasoning process. First, it provides reasoning context to external tools, enabling them to generate more context-aware responses. The context is generated by synthesizing the summaries of each clustered group, performed by an LLM. Additionally, when the reasoning model encounters uncertainty or loses track in an extended reasoning process, it can query the Mind-Map as an external memory to retrieve relevant information and continue reasoning seam-
lessly. This ensures the model maintains a long reasoning chain across multiple breakdown tasks and tool calls without missing critical information.

2.3 Web-Search agent
A search agent is invoked to retrieve the most relevant documents from the web. It consists of four key components: query breakdown, a search service, a re-ranking service, and RAG.

When the reasoning model generates a web-search query, it is sent to the Web-Search agent, which first reorganizes it into one or more search-optimized queries suitable for search engines like Google or Bing. The process involves sending the LLM the original query along with the reasoning context retrieved from the Mind-Map, prompting it to generate suitable refined search queries. For example, given the original query "Search the external economic indicators" and the context "We are looking for the optimal investing strategy for a retailer in the U.S. in Q4 2024", the Web-Search agent would break it down into more specific queries such as "U.S. Q4 2024 inflation rate" and "U.S. Q4 2024 CCI". These queries are then sequentially sent to the search engine, which returns related web pages.

After we retrieved the web pages for each refined query, we apply a re-ranking model to rank web pages based on their alignment with the original query and context. The average relevance score of the top 10 pages is computed, and if it falls below a predefined threshold, the Web-Search agent will iterate back to the last step and further refine the search query.

Once reranking is complete, web pages with relevance scores above the threshold are stored, and RAG is applied on them to extract meaningful insights. Each refined query undergoes RAG to generate a natural language response. Finally, an LLM synthesizes these responses into a cohesive final snippet, based on both the original query and reasoning context. This processed snippet is then integrated into the main reasoning process, ensuring that external insights enhance logical flow without causing disruption.

2.4 Coding Agent
Instead of prompting the reasoning model to generate code directly, we find it more efficient to delegate coding tasks to a specialized coding LLM. The reasoning model sends the relevant context and query message to the coding LLM, which then writes the required code, executes it via a compiler, and returns the results. This approach ensures that the reasoning model remains focused on its core reasoning process without being disrupted by coding tasks, allowing for longer and more coherent reasoning chains. Specifically, we format the coding request as follows: "Write code to perform <code message from reasoning model> given the context <reasoning context from Mind-Map> to answer the query <user query>." The coding LLM is instructed to always return its output in natural language, ensuring seamless integration with the reasoning model.

Question
A 68-year-old male with a 12-year history of moderate-to-severe COPD (FEV₁ = 45% of predicted, PaO₂ = 58 mmHg, PaCO₂ = 48 mmHg) and newly diagnosed heart failure (left ventricular ejection fraction, LVEF = 35%) presents with worsening dyspnea and reduced exercise tolerance. how can lung function be improved without exacerbating heart symptoms?

Reasoning...
[Code] Target an Alveolar PAO2PAO2 of ~140 mmHg, anticipating an A-a gradient that might still yield an arterial PaO₂ ≥65≥65 mmHg using Alveolar Gas Equation: \( P_{aO_2} = F_{iO_2} \times (P_{atm} - P_{eCO_2}) - \frac{P_{eCO_2}}{R} \)

Thus, 28% FiO₂ might be a reasonable starting point. In practice, clinicians often start with 24–28% supplemental O₂ via a Venturi mask (or nasal cannula approximations) and then titrate to keep SpO₂ around 88–92% in COPD to avoid excessive respiratory drive suppression or severe hypercapnia.

...
[Web-search] Optimal PEEP levels in COPD with heart failure
Low-to-moderate PEEP (5–10 cm H₂O) is recommended to improve oxygenation without significantly reducing venous return in HF.

...
Administer ~28% FiO₂, consider mild PEEP/CPAP of 4–5 cm H₂O, increase alveolar ventilation modestly, and titrate diuretics to manage fluid overload without excessive preload reduction.

Figure 2: Case study on a complex medical decision-making problem.
Table 2: Performance comparison on GPQA dataset across Physics, Chemistry, and Biology.

<table>
  <tr>
    <th>Method</th>
    <th>Phy.</th>
    <th>Chem.</th>
    <th>Bio.</th>
    <th>All</th>
  </tr>
  <tr>
    <th colspan="5">Direct Reasoning</th>
  </tr>
  <tr>
    <td>QwQ-32B</td>
    <td>75.6</td>
    <td>39.8</td>
    <td>68.4</td>
    <td>58.1</td>
  </tr>
  <tr>
    <td>Llama3.3-70B</td>
    <td>54.7</td>
    <td>31.2</td>
    <td>52.6</td>
    <td>43.4</td>
  </tr>
  <tr>
    <td>DeepSeek-R1</td>
    <td>86.8</td>
    <td>56.1</td>
    <td>63.8</td>
    <td>71.5</td>
  </tr>
  <tr>
    <td>GPT-4o†</td>
    <td>59.5</td>
    <td>40.2</td>
    <td>61.6</td>
    <td>50.0</td>
  </tr>
  <tr>
    <td>o1†</td>
    <td>92.8</td>
    <td>64.7</td>
    <td>69.2</td>
    <td>78.0</td>
  </tr>
  <tr>
    <td>o3-mini-low†</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>70.6</td>
  </tr>
  <tr>
    <td>o3-mini-mid†</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>76.8</td>
  </tr>
  <tr>
    <td>o3-mini-high†</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>79.7</td>
  </tr>
  <tr>
    <th colspan="5">Retrieve/Search in Reasoning</th>
  </tr>
  <tr>
    <td>RAgent w/QwQ-32B</td>
    <td>76.7</td>
    <td>46.2</td>
    <td>68.4</td>
    <td>61.6</td>
  </tr>
  <tr>
    <td>RAgent w/DeepSeek-R1</td>
    <td>87.7</td>
    <td>58.2</td>
    <td>65.7</td>
    <td>72.9</td>
  </tr>
  <tr>
    <td>SearchO1 w/QwQ-32B</td>
    <td>77.9</td>
    <td>47.3</td>
    <td>78.9</td>
    <td>63.6</td>
  </tr>
  <tr>
    <td>SearchO1 w/DeepSeek-R1</td>
    <td>90.2</td>
    <td>61.3</td>
    <td>71.4</td>
    <td>74.6</td>
  </tr>
  <tr>
    <th colspan="5">Agentic Reasoning</th>
  </tr>
  <tr>
    <td>Ours w/QwQ-32B</td>
    <td><b>88.1</b></td>
    <td><b>58.3</b></td>
    <td><b>79.6</b></td>
    <td><b>69.7</b></td>
  </tr>
  <tr>
    <td>Ours w/DeepSeekR1</td>
    <td><b>94.5</b></td>
    <td><b>73.7</b></td>
    <td><b>80.5</b></td>
    <td><b>81.2</b></td>
  </tr>
</table>

3 Experiments

3.1 Implementation Details

In our experiments, we use DeepSeek-R1 as the primary reasoning models by default. For the Web-Search agent, query breakdown and RAG are handled by DeepSeek-V3 (Liu et al., 2024). We use a maximum of 32,768 tokens, temperature of 0.7, top_p of 0.8, top_k of 20, and a repetition penalty of 1.05 across all models for generation. We use Bing as the search engine, retrieving the top 20 most relevant pages. The re-ranking model is Cohere Rerank 3.5, with a top-10 average relevance score threshold of 0.7 to determine if iterative query refinement is needed, allowing a maximum of three iterations. Additionally, web pages with a relevance score above 0.7 are selected for RAG processing. For the Mind-Map agent, both knowledge graph construction and Graph-RAG retrieval are also performed using DeepSeek-V3. For the coding agent, we use claude-3.5-sonnet to generate code and Python 3.11 for execution. We report pass1 results by default.

3.2 Solving Expert-level Problems

Agentic Reasoning model is able to call external tools in its reasoning to solve expert-level problems, except Humanity’s Last Exam we previously mentioned, we further evaluate it on two datasets: GPQA dataset (Rein et al., 2023), a PhD-level multiple-choice science QA benchmark, and GAIA (Mialon et al., 2023), a benchmark for AI agents that requires a set of abilities such as reasoning, web browsing, and tool-use proficiency.

As shown in Table 2, applying Agentic Reasoning to a strong reasoning model like DeepSeek-R1 achieves a new SOTA, surpassing even the best proprietary model, o3-mini-high. Compared to the base model DeepSeek-R1, our method boosts overall performance by nearly 10%. Compared to previous search-in-reasoning approaches (Li et al., 2025; Islam et al., 2024), Agentic Reasoning demonstrates superior reasoning enhancement, outperforming Search-O1 by approximately 5% overall. Furthermore, we show that this method is generally effective across different reasoning models, such as QwQ (Yang et al., 2024), where it improves base model accuracy by over 10%.

On GAIA (as shown in Table 3), Agentic Reasoning establishes a new SOTA among all publicly available methods. Compared to OpenAI’s Deep Research, which leverages its proprietary high-performance reasoning models, our approach surpasses it on Level 1 and Level 2 tasks while narrowing the gap to 2.26% on Level 3. GAIA requires a combination of advanced reasoning, web browsing, and tool-use proficiency for successful completion. Our results demonstrate that Agentic Reasoning excels in handling complex tasks while maintaining strong generalization across diverse problem domains.

We also present a case study on a complex medical decision-making problem, as shown in Figure 2. The model autonomously executes code to compute the optimal \( FiO_2 \) (Fraction of Inspired Oxygen) for a patient, performs a web search to retrieve the most accurate PEEP (Positive End-Expiratory Pressure) value, and synthesizes both results to determine the best treatment plan.

<table>
  <tr>
    <th>GAIA</th>
    <th>Level 1</th>
    <th>Level 2</th>
    <th>Level 3</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>Langfun</td>
    <td>58.06</td>
    <td>51.57</td>
    <td>24.49</td>
    <td>49.17</td>
  </tr>
  <tr>
    <td>InspectReact</td>
    <td>67.92</td>
    <td>59.30</td>
    <td>30.77</td>
    <td>57.58</td>
  </tr>
  <tr>
    <td>h2oGPTe</td>
    <td>78.49</td>
    <td>64.78</td>
    <td>40.82</td>
    <td>65.12</td>
  </tr>
  <tr>
    <td><b>AgenticReasoning</b></td>
    <td><b>74.36</b></td>
    <td><b>69.21</b></td>
    <td><b>45.46</b></td>
    <td><b>66.13</b></td>
  </tr>
  <tr>
    <td><i>Open AI Deep Research</i> †</td>
    <td>74.29</td>
    <td>69.06</td>
    <td>47.60</td>
    <td>67.36</td>
  </tr>
</table>

Table 3: Performance comparison on GAIA across different levels.

3.3 Deep Research

We evaluate the deep research capability of our approach using the FreshWiki dataset (Shao et al., 2024), which curates high-quality, recent Wikipedia articles. The model is prompted directly with the topic and asked to generate the ar-
Table 4: Comparison with Human-Written Articles

<table>
  <tr>
    <th></th>
    <th>ROUGE-1</th>
    <th>ROUGE-L</th>
    <th>Entity Recall</th>
  </tr>
  <tr>
    <td>Direct Gen</td>
    <td>27.32</td>
    <td>13.13</td>
    <td>6.11</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>29.14</td>
    <td>14.23</td>
    <td>8.84</td>
  </tr>
  <tr>
    <td>RAgent</td>
    <td>30.04</td>
    <td>14.21</td>
    <td>9.08</td>
  </tr>
  <tr>
    <td>Search-O1</td>
    <td>41.56</td>
    <td>16.08</td>
    <td>12.88</td>
  </tr>
  <tr>
    <td>STORM</td>
    <td>47.93</td>
    <td>17.42</td>
    <td>15.43</td>
  </tr>
  <tr>
    <th>Ours</th>
    <th>54.10</th>
    <th>19.62</th>
    <th>18.77</th>
  </tr>
</table>

Table 5: Human Evaluation on Deep Research articles (Rate Range: 1-5).

<table>
  <tr>
    <th></th>
    <th>Interest Level</th>
    <th>Organization</th>
    <th>Relevance</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Direct Gen</td>
    <td>1.2</td>
    <td>1.6</td>
    <td>1.2</td>
    <td>1.7</td>
  </tr>
  <tr>
    <td>RAG</td>
    <td>1.4</td>
    <td>2.1</td>
    <td>1.9</td>
    <td>2.3</td>
  </tr>
  <tr>
    <td>RAgent</td>
    <td>1.6</td>
    <td>2.3</td>
    <td>1.6</td>
    <td>2.6</td>
  </tr>
  <tr>
    <td>Search-O1</td>
    <td>2.5</td>
    <td>2.8</td>
    <td>2.1</td>
    <td>3.2</td>
  </tr>
  <tr>
    <td>STORM</td>
    <td>2.9</td>
    <td>3.2</td>
    <td>2.9</td>
    <td>3.7</td>
  </tr>
  <tr>
    <td>Gemini-DR*</td>
    <td>2.7</td>
    <td>2.5</td>
    <td>2.3</td>
    <td>3.0</td>
  </tr>
  <tr>
    <th>Ours</th>
    <th>3.7</th>
    <th>4.6</th>
    <th>4.2</th>
    <th>4.1</th>
  </tr>
</table>

ticle. Evaluation covers article quality, assessed via ROUGE and entity recall. This task needs a comprehensive analysis of long-form generation while highlighting key challenges like bias transfer and factual consistency.

We also conduct an evaluation of Agentic Reasoning for deep research in open deep research tasks. A group of PhD-level experts in finance, medicine, and law were asked to formulate 15 to 30 professional research questions closely related to their respective fields. These questions were designed to require at least 20 minutes of in-depth research to answer comprehensively. There are in total 56 questions were collected. The experts would review the generated articles on interest level, organization, relevance, and coverage. More details are in the appendix.

We evaluate our method using the same underlying reasoning model, DeepSeek-R1, and compare it against various search-enhanced reasoning approaches, including RAG, RAgent (Islam et al., 2024), and Search-O1 (Li et al., 2025), as well as STORM (Shao et al., 2024), which employs a more complex agent-based workflow. Additionally, we benchmark our approach against the proprietary Gemini Deep Research 1 on deep research tasks. As shown in Tables 4 and 5, our results demonstrate that Agentic Reasoning consistently outperforms all RAG and search-based methods, as well as Gemini Deep Research, across all benchmarks. These findings highlight the effectiveness of structured reasoning and tool-augmented frameworks in enabling more advanced and efficient deep research.

3.4 Analysis

3.4.1 Ablation on Toolbox

We conduct experiments to explore the impact of integrating different tools in Agentic Reasoning and find that tool quality is far more important than quantity. Specifically, the combination of web search, coding, and Mind-Map agents proves to be the most effective across various tasks, including those requiring expert-level proficiency. As shown in Figure 3, we evaluated performance on GPQA using Hugging Face’s default agent toolbox with seven tools and LangChain with 109 tools. Surprisingly, adding more tools often degraded performance by increasing the risk of inappropriate tool selection. Many capabilities, such as translation or code interpretation, are already embedded within the reasoning model, making their external integration redundant. Moreover, inaccuracies in external tool outputs can negatively affect overall response quality.

Figure 3 also presents an ablation study on the three proposed tools in this paper. We tested different tool combinations to assess their individual contributions to agentic reasoning. Among single-tool deployments, web search performed the best, while coding and Mind-Map achieved comparable results. Notably, combining tools yielded a synergistic effect—web search + Mind-Map or web search + coding provided greater improvements than the sum of their individual gains. The best performance was achieved when integrating all three: web search, Mind-Map, and coding.

3.4.2 Ablation on Web-Search agent Design

Integrating web search into LLMs has been widely explored in recent research (Li et al., 2025; Lewis et al., 2020; Islam et al., 2024). In Agentic Reasoning, we investigate various web-search strategies to determine the most effective approach. Our ablation study primarily considers standard RAG and Knowledge Refinement, where retrieved sources are summarized for the response. Additionally, we incorporate Query Breakdown, Rerank, and Mind-Map Reasoning Context, key components in our Web-Search agent. Our findings reveal that Query Breakdown, Rerank, and Mind-Map Reasoning Context incrementally improve performance.

1OpenAI Deep Research experiments are currently restricted by a high paywall.
Figure 3: The ablation study examines the impact of different tools in reasoning. Green ones represent external toolboxes, red ones are combinations of our proposed tools. The blue line is the overall performance of the base reasoning model.

Surprisingly, Knowledge Refinement, which is effective when used solely with RAG, becomes ineffective when combined with our three adopted components. This decline is primarily due to its redundancy with Rerank, which serves a similar role but proves more effective in most cases. As a result, our final Web-Search agent includes RAG, Query Breakdown, Rerank, and Mind-Map Reasoning Context for optimal performance.

<table>
  <tr>
    <th></th>
    <th>RAG</th>
    <th>Knowledge Refinement</th>
    <th>Query Breakdown</th>
    <th>Rerank</th>
    <th>Mind-Map Reasoning Context</th>
    <th>GAPA</th>
  </tr>
  <tr>
    <td>Search-OI</td>
    <td>✓</td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td></td>
    <td>74.6</td>
  </tr>
  <tr>
    <td>Storm</td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td>72.7</td>
  </tr>
  <tr>
    <td></td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td></td>
    <td>73.1</td>
  </tr>
  <tr>
    <td></td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td>75.2</td>
  </tr>
  <tr>
    <td></td>
    <td>✓</td>
    <td></td>
    <td>✓</td>
    <td>✓</td>
    <td></td>
    <td>76.2</td>
  </tr>
  <tr>
    <td></td>
    <td>✓</td>
    <td></td>
    <td></td>
    <td></td>
    <td>✓</td>
    <td>75.8</td>
  </tr>
  <tr>
    <td>Agentic Reasoning</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
    <td>✓</td>
    <td>76.4</td>
  </tr>
</table>

Table 6: Comparison of different web-search approaches.

3.4.3 The Effect of Mind-Map
We have shown in Figure 3 that our quantitative results demonstrate that Mind-Map significantly enhances performance. In this section, we analyze its impact on reasoning in detail. Mind-Map proves particularly effective in maintaining long reasoning with tools and clarifying complex logical relationships.

We find that questions needs longer reasoning chains and more tool calls tend to be inherently more difficult, leading to lower accuracy, as shown in Figure 5. Examining the model’s reasoning process, we find that LLMs often struggle to maintain coherence over long reasoning with massive tool calls. They may deviate from user queries, repeatedly call tools in the same way, or revisit previous errors, all of which negatively impact performance.

![A tricky question that misleads most LLMs is correctly answered by us.](page_1092_1042_324_180.png)

Figure 4: A tricky question that misleads most LLMs is correctly answered by us.

We introduced the Mind-Map agent to help the model manage its reasoning memory, ensuring coherent long reasoning and reducing errors. As shown in Figure 5, this mechanism significantly improves performance, particularly on questions requiring long reasoning chains and more tool calls. The structured memory provided by the Mind-Map agent preserves prior reasoning steps, mitigating common pitfalls in extended reasoning tasks.

Mind-Map is also especially helpful for the tasks heavily rely on logic relationships. We find it helps to correctly answer tricky logic-based questions that frequently fool LLMs. A well-known example is a modified riddle: "The surgeon, who is the boy’s father, says 'I can’t operate on this child, he’s my son!' Who is the surgeon to the boy?" As shown in Figure 4, DeepSeek-R1 took 17 seconds to process this question but still produced the wrong answer, a failure also observed in models from the GPT and
Gemini series models. These models often fall for a political-correct corpus contaminated response, failing to recognize the obvious logical structure. However, in our Agentic Reasoning framework, the use of a Mind-Map allows the model to explicitly analyze the logical relationships between the entities [surgeon], [boy], and [father], leading to the correct answer.

This property also enables Mind-Maps to enhance deductive reasoning in strategic games. We tested our approach in Werewolf, a classic social deduction game where players assume hidden roles as either villagers or werewolves. Villagers aim to identify the werewolves through discussion, while werewolves deceive the group and eliminate players without being caught. To evaluate performance, we invited seven experienced Werewolf players, each with over five years of experience, to compete against our Agentic Reasoning model. The results show that our model achieved an impressive 72% win rate, significantly surpassing both the expected statistical win rate and human performance in our experiment. In contrast, without Mind-Map, the model’s win rate dropped to 36%. As the Mind-Map of the model’s reasoning process shown in Figure 6, Mind-Map proved crucial in helping the model track relationships between players based on their spoken arguments. By maintaining a structured memory of interactions, it more effectively identified deception strategies, anticipated voting behaviors, and optimized its own disguise tactics. This result highlights that Mind-Map is not only a tool for structured logic but also a powerful enabler of strategic reasoning in dynamic, high-stakes environments.

Figure 5: Mind-Map improves performance on questions need long reasoning.

4 Related Work

The concept of multi-agent collaboration in LLMs has gained attention with frameworks like Auto-GPT (Yang et al., 2023) and LangChain Agents (Pandya and Holia, 2023), allowing models to interact with external APIs, search engines, and computational environments. While these frameworks introduce modular reasoning, they often lack optimized task delegation and structured integration, reducing their effectiveness in long-chain reasoning tasks. Recent research on Hierarchical Planning with LLMs (Luo et al., 2023) and Task-Specific AI Agents (Wu et al., 2024) explores structured agent cooperation for problem-solving. However, these approaches still do not deeply integrate agent tools within reasoning chains and fail to systematically explore optimal agent combinations that maximize reasoning performance.

Previous studies focus a lot on integrating the search capability into LLMs. Recent agentic RAG systems(Khalil et al., 2024; Islam et al., 2024; Li et al., 2025) have enabled models to autonomously determine when and what knowledge to retrieve, enhancing their planning and problem-solving capabilities. Additionally, research has explored (Li et al., 2025) integrating Web-Search agent into the reasoning model, like QwQ (Team) demonstrating the potential of search augmentation in structured reasoning. However, existing approaches have primarily focused on single-agent enhancements, neglecting the potential synergy of multiple agentic tools. Moreover, prior works have yet to integrate such tools with state-of-the-art reasoning models like DeepSeek-R1 or OpenAI-O1, limiting their effectiveness in solving highly complex tasks.

![Mind-Map in playing werewolf game. The first round and the second round. Player8 is the model.](page_1012_120_410_312.png)

Figure 6: Mind-Map in playing werewolf game. The first round and the second round. Player8 is the model.

5 Conclusion

We introduced Agentic Reasoning, a framework that enhances LLM reasoning by integrating Mind-Map, web search, and coding. Our approach improves problem-solving and deep research capabilities, outperforming existing models in expert-level QA and real-world research tasks. Agentic
Reasoning outperforms existing methods in both quantitative benchmarks and human evaluations. Future work will explore task-specific tools integration and test-time computing to further enhance AI's reasoning capabilities.
6 Limitations

Despite the strong performance of Agentic Reasoning, several limitations remain that warrant further research and refinement.

Computational Overhead and Efficiency. Integrating multiple external agents, including web search, Mind-Map, and code execution, significantly increases computational costs and inference latency. While these components enhance reasoning depth, their sequential invocation introduces bottlenecks, limiting real-time applicability. Future work would explore techniques such as agent prioritization, caching strategies, or adaptive invocation mechanisms to optimize efficiency without sacrificing accuracy.

Reliance on External Knowledge Sources. The effectiveness of Agentic Reasoning depends on the quality of retrieved knowledge, particularly in web search. The system lacks built-in verification mechanisms to assess the credibility of sources, making it susceptible to misinformation or biased content. Developing trust-aware retrieval mechanisms, such as fact-checking agents or weighted source reliability scores, could mitigate this risk and improve robustness in knowledge-intensive domains.

Interpretability and Trustworthiness. While the Mind-Map agent provides structured reasoning memory, the overall decision-making process remains highly dependent on LLMs. This reliance introduces the risk of hallucinations, which can derail the entire reasoning process, especially in complex, multi-step tasks. In high-stakes domains such as medical AI or legal reasoning, even minor inaccuracies can lead to significant consequences. Ensuring reliability requires additional safeguards, such as fact-verification mechanisms, confidence estimation, or human-in-the-loop oversight, to mitigate the risks associated with LLM-driven reasoning.

Acknowledgments

Junde Wu is supported by the Engineering and Physical Sciences Research Council (EPSRC) under grant EP/S024093/1 and GE HealthCare. Jiayuan Zhu is supported by the Engineering and Physical Sciences Research Council (EPSRC) under grant EP/S024093/1 and Global Health R&D of Merck Healthcare, Ares Trading S.A. (an affiliate of Merck KGaA, Darmstadt, Germany), Eysins, Switzerland (Crossref Funder ID: 10.13039/100009945). Yueming Jin is supported by the Ministry of Education Tier 1 grant, NUS, Singapore (24-1250-P0001).

References

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130.

Zhen Huang, Haoyang Zou, Xuefeng Li, Yixiu Liu, Yuxiang Zheng, Ethan Chern, Shijie Xia, Yiwei Qin, Weizhe Yuan, and Pengfei Liu. 2024. O1 replication journey—part 2: Surpassing o1-preview through simple distillation, big progress or bitter lesson? arXiv preprint arXiv:2411.16489.

Shayekh Bin Islam, Md Asib Rahman, KSM Hossain, Enamul Hoque, Shafiq Joty, and Md Rizwan Parvez. 2024. Open-rag: Enhanced retrieval-augmented reasoning with open-source large language models. arXiv preprint arXiv:2410.01782.

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.

M Abdul Khalilq, P Chang, M Ma, Bernhard Pflugfelder, and F Miletic. 2024. Ragar, your falsehood radar: Rag-augmented reasoning for political fact-checking using multimodal large language models. arXiv preprint arXiv:2404.12065.

Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. 2024. A human-inspired reading agent with gist memory of very long contexts. arXiv preprint arXiv:2402.09727.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.

Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843–3857.

Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437.

Xusheng Luo, Shaojun Xu, and Changliu Liu. 2023. Obtaining hierarchy from human instructions: an llms-based approach. In CoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP).

Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2023. Gaia: a benchmark for general ai assistants. arXiv preprint arXiv:2311.12983.

OpenAI. Learning to reason with LLMs.

Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. 2023. Memgpt: Towards llms as operating systems.

Keivalya Pandya and Mehfuzha Holia. 2023. Automating customer service using langchain: Building custom open-source gpt chatbot for organizations. arXiv preprint arXiv:2310.05421.

Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenblatt, Oliver Zhang, Mantas Mazeika, Tung Nguyen, Daron Anderson, Imad Ali Shah, Mikhail Doroshenko, Alun Cennyth Stokes, Moobeen Mahmood, Jaeho Lee, Oleksandr Pokutnyi, Oleg Iskra, Jessica P. Wang, Robert Gerbic, John-Clark Levin, Serguei Popov, Fiona Feng, Steven Y. Fong, Haoran Zhao, Michael Yu, Varun Gangal, Chelsea Zou, Zihan Wang, Mstyslav Kazakov, Geoff Galgon, Johannes Schmitt, Alvaro Sanchez, Yongki Lee, Will Yeandon, Scott Sauers, Marc Roth, Chidozie Agu, Søren Riis, Fabian Giska, Saiteja Utpala, Antrell Cheatom, Zachary Giboney, Gashaw M. Goshu, Sarah-Jane Crowson, Mohinder Maheshbhavi Naiya, Noah Burns, Lennart Finke, Zerui Cheng, Hyunwoo Park, Francesco Fournier-Facio, Jennifer Zampese, John Wydallis, John B. Wydallis, Ryan G. Hoerr, Mark Nandor, Tim Gehrunger, Jiaqi Cai, Ben McCarty, Jungbae Nam, Edwin Taylor, Jun Jin, Gautier Abou Loume, Hangrui Cao, Alexis C Garretson, Damien Silco, Qiuyu Ren, Doru Cocloc, Pavel Arkhipov, Usman Qazi, Aras Bacho, Lianghui Li, Sumeet Motwani, Christian Schroeder de Witt, Alexei Kopylov, Johannes Veith, Eric Singer, Paolo Rissone, Jaeheok Jin, Jack Wei Lun Shi, Chris G. Wilcocks, Ameya Prabhu, Longke Tang, Kevin Zhou, Emily de Oliveira Santos, Andrew Putpasov Maksimov, Edward Vendrow, Kengo Zenitani, Joshua Robinson, Aleksandar Mikov, Julien Guilold, Yuqi Li, Ben Pageler, Joshua Vendrom, Vladyslav Kuchkin, Pierre Marion, Denis Efremov, Jayson Lynch, Kaiqu Liang, Andrew Gritsevskiy, Dakotah Martinez, Nick Crispino, Dimitri Zvonkine, Natanael Wildner Fraga, Saeed Soori, Ori Press, Henry Tang, Julian Salazar, Sean R. Green, Lina Brissel, Moon Twayana, Aymeric Dieuleveut, T. Ryan Rogers, Wenjin Zhang, Ross Finocchio, Bikun Li, Jinzhou Yang, Arun Rao, Gabriel Loisseau, Mikhail Kalinin, Marco Lukas, Ciprian Manolescu, Nate Stambaugh, Subrata Mishra, Ariel Ghislain Kemogne Kamdoum, Tad Hogg, Alvin Jin, Carlo Bosio, Gongbo Sun, Brian P Coppola, Haline Heidinger, Rafael Sayous, Stefan Ivanov, Joseph M Cavanagh, Jiawei Shen, Joseph Marvin Imperial, Philippe Schwaller, Shairapnesh Senthilkuma, Andres M Bran, Andres Algaba, Brecht Verbeken, Kelsey Van den Houte, Lynn Van Der Sypt, David Noever, Lisa Schut, Ilia Sucholutsky, Evgenii Zheltonozhskii, Qiaoche Yuan, Derek Lim, Richard Stanley, Shankar Sivarajan, Tong Yang, John Maar, Julian Wykowski, Martf Oller, Jennifer Sandlin, Anmol Sahu, Cesare Giulio Ardito, Yuzheng Hu, Felipe Meneguitti Dias, Tobias Kreiman, Kaivalya Rawal, Tobias Garcia Vilchis, Yuexuan Zu, Martin Lackner, James Koppel, Jeremy Nguyen, Danil S. Antonenko, Steffi Chern, Bingchen Zhao, Pierrot Arsene, Sergey Ivanov, Rafal Poświata, Chenguang Wang, Daofeng Li, Donato Crisostomi, Ali Dehghan, Andrea Achilleos, John Arnold Ambay, Benjamin Myklebust, Archan Sen, David Perrella, Nurdin Karparov, Mark H Inlow, Allen Zang, Kalyan Ramakrishnan, Danil Orel, Vladislav Poritski, Shalev Ben-David, Zachary Berger, Parker Whitfill, Michael Foster, Daniel Munro, Linh Ho, Dan Bar Hava, Aleksey Kuchkin, Robert Lauff, David Holmes, Frank Sommerhaege, Anji Zhang, Richard Moat, Keith Schneider, Daniel Pyda, Zakayo Kazibwe, Mukhwinder Singh, Don Clarke, Dave Hyun Kim, Sara Fish, Veit Elser, Victor Efren Guadarrama Vilchis, Immo Klose, Christoph Demian, Juwila Anantheswaran, Adam Zweiger, Guglielmo Albani, Jeffery Li, Nicolas Daans, Maksim Radionov, Václav Rozhoň, Vincent Ginis, Ziqiao Ma, Christian Stump, Jacob Platnick, Volodymyr Nevirkoyets, Luke Basler, Marco Piccardo, Nir Cohen, Virenra Singh, Josef Tkadlec, Paul Rosu, Alan Goldfarb, Piotr Padlewski, Stanislaw Barzowski, Kyle Montgomery, Aline Menezes, Arkil Patel, Zixuan Wang, Jamie Tucker-Foltz, Jack Stade, Declan Grabb, Tom Goertzen, Fereshteh Kazemi, Jeremiah Milbauer, Abhishek Shukla, Hossam Elgnaiany, Yan Carlos Levy Labrador, Hao He, Ling Zhang, Alan Givré, Hew Wolff, Gözdenur Demir, Muhammad Fayez Aziz, Youness Kaddar, Ivar Angquist, Yanxu Chen, Ellyott Thornley, Robin Zhang, Jiayi Pan, Antonio Terpin, Niklas Muennighoff, Hailey Schoelkopf, Eric Zheng, Avishy Carmi, Jainam Shah, Ethan D. L. Brown, Kelin Zhu, Max Bartolo, Richard Wheeler, Andrew Ho, Shaul Barkan, Jiaqi Wang, Martin Stehberger, Egor Kretov, Peter Bradshaw, JP Heimonen, Kaustubh Sridhar, Zaki Hossain, Ido Akov, Yury Makarychev, Joanna Tam, Hieu Hoang, David M. Cunningham, Vladimir Goryachev, Demosthenes Patramanis, Michael Krause, Andrew Redenti, David Aldous, Jesyin Lai, Shannon Coleman, Jiangnan Xu, Sangwon Lee, Ilias Magoulas, Sandy Zhao, Ning Tang, Michael K. Cohen, Micah Carroll, Orr Paradise, Jan Hendrik Kirchner, Stefan Steinerberger,
Maksym Ovchynnikov, Jason O. Matos, Adithya Shenoy, Michael Wang, Yuzhou Nie, Paolo Gior-dano, Philipp Petersen, Anna Szyber-Betley, Paolo Faraboschi, Robin Riblet, Jonathan Crozier, Shiv Hulasayamani, Antonella Pinto, Shreyas Verma, Prashant Joshi, Eli Meril, Zheng-Xin Yong, Allison Tee, Jérémy Andréoletti, Orion Weller, Raghav Singhal, Gang Zhang, Alexander Ivanov, Seri Khoury, Nils Gustafsson, Hamid Mostaghami, Kunvar Thaman, Qijia Chen, Tran Quoc Khánh, Jacob Loader, Stefano Cavalleri, Hannah Szlyk, Zachary Brown, Himanshu Narayan, Jonathan Roberts, William Alley, Kunyang Sun, Ryan Stendall, Max Lamparth, Anka Reuel, Ting Wang, Hannmeg Xu, Pablo Hernández-Cámara, Freddie Martin, Thomas Preu, Tomek Kobrak, Marcus Abramovich, Dominic Williamson, Ida Bosio, Ziyi Chen, Biró Bálint, Eve J. Y. Lo, Maria Inês S. Nunes, Yibo Jiang, M Saiful Bari, Peyman Kassani, Zihao Wang, Behzad Ansarinejad, Yewen Sun, Stephane Durand, Guillaume Douville, Daniel Tordera, George Balabanian, Earth Anderson, Lynna Kvistad, Alejandro José Moyano, Hsiao-yun Milliron, Ahmad Sakor, Murat Eron, Isaac C. McAlister, Andrew Favre D. O., Shailesh Shah, Xiaoxiang Zhou, Firuz Kamalov, Ronald Clark, Sherwin Abdoli, Tim Santens, Harrison K Wang, Evan Chen, Alessandro Tomaselli, G. Bruno De Luca, Shi-Zhuo Looi, Vinh-Kha Le, Noam Kolt, Niels Mündler, Avi Semler, Emma Rodman, Jacob Drori, Carl J Fossum, Luk Gloor, Milind Jagota, Ronak Pradeep, Honglu Fan, Tej Shah, Jonathan Eicher, Michael Chen, Kushal Thaman, William Merrill, Moritz Firsching, Carter Harris, Stefan Ciobăcă, Jason Gross, Rohan Pandey, Ilya Gusev, Adam Jones, Shashank Agnihotri, Pavel Zhelnov, Siranut Usawasutaskorn, Mohammadreza Moffayazi, Alexander Piperski, Marc Carauleanu, David K. Zhang, Kostiantyn Dobarskyi, Dylan Ler, Roman Leventov, Ignat Soroko, Thorben Jansen, Scott Creighton, Pascal Lauer, Joshua Duersch, Vage Taamazyan, Dario Bezzi, Wiktor Morak, Wenjie Ma, William Held, Tran Duc Huy, Ruicheng Xian, Armel Randy Zebaze, Mohanad Mohamed, Julian Noah Leser, Michelle X Yuan, Laila Yacar, Johannes Lengler, Katarzyna Olszewska, Hossein Shahrtash, Edson Oliveira, Joseph W. Jackson, Daniel Espinosa Gonzalez, Andy Zou, Muthu Chidambaram, Timothy Manik, Hector Haffenden, Dashell Stander, Ali Dasouqi, Alexander Shen, Emilien Duc, Bita Golshani, David Stap, Mikalai Uzhou, Alina Borisovna Zhidkovskaya, Lukas Lewark, Miguel Orbeogo Ro-dríguez, Mátyás Vincze, Dustin Wehr, Colin Tang, Shaun Phillips, Fortuna Samuele, Jiang Muzhen, Fredrik Ekström, Angela Hammon, Oam Patel, Faraz Farhidi, George Medley, Forough Mohammadzadeh, Madellene Penållor, Haile Kasahun, Alena Friedrich, Claire Sparrow, Rayner Hernandez Perez, Taom Sakal, Omkar Dhamane, Ali Khajegili Mirabadi, Eric Hallman, Kenchi Okutsu, Mike Battaglia, Mohammad Maghsoudimehrabani, Alon Amit, Dave Hulbert, Roberto Pereira, Simon Weber, Handoko, Anton Peristy, Stephen Malina, Samuel Albanie, Will Cai, Mustafa Mehkary, Rami Aly, Frank Reidegeld, Anna-Katharina Dick, Cary Friday, Jasdeep Sidhu, Hassan Shapourian, Wanyoung Kim, Mariana Costa, Hubeyb Gurdogan, Brian Weber, Harsh Kumar, Tong Jiang, Arunim Agarwal, Chiara Concelono, Warren S. Vaz, Chao Zhuang, Haon Park, Andrew R. Tawfeek, Daattavya Aggarwal, Michael Kirchhoff, Linjie Dai, Evan Kim, Johan Ferret, Yuzhou Wang, Minghao Yan, Krzysztof Burdzy, Lixin Zhang, Antonio Franca, Diana T. Pham, Kang Yong Loh, Joshua Robinson, Abram Jackson, Shreen Gul, Gunjan Chhablani, Zhehang Du, Adrian Cosma, Jesus Colino, Colin White, Jacob Votava, Vladimir Vin-nikov, Ethan Delaney, Petr Spelda, Vit Stritecky, Syed M. Shahid, Jean-Christophe Mourrat, Lavr Vetoshkin, Koen Sponselee, Renas Bacho, Floren-cia de la Rosa, Xiuyu Li, Guillaume Malod, Leon Lang, Julien Laurendeau, Dmitry Kazakov, Fatimah Adesanya, Julien Portier, Lawrence Hollow, Victor Souza, Yuchen Anna Zhou, Julien Degorre, Yiğit Yalın, Gbenga Daniel Obikoya, Luca Arnaboldi, Rai, Filippo Bigi, M. C. Boscá, Oleg Shumar, Kan-uar Bacho, Pierre Clavier, Gabriel Recchia, Mara Popescu, Nikita Shulga, Ngefor Mildred Tanwie, Dennis Peskoff, Thomas C. H. Lux, Ben Rank, Colin Ni, Matthew Brooks, Alesia Yakimchyk, Huanxu Liu, Olle Häggström, Emil Verkama, Hans Gund-lach, Leonor Brito-Santana, Brian Amaro, Vivek Vijpeji, Rynaa Grover, Yiyang Fan, Gabriela Poesia Reis e Silva, Linwei Xin, Yosi Kratish, Jakub Łucki, Wen-Ding Li, Sivakanth Gopi, Andrea Caciolai, Justin Xu, Kevin Joseph Scaria, Freddie Vargas, Farzad Habibi, Long, Lian, Emanuele Rodola, Jules Robins, Vincent Cheng, Tony Fruhauff, Brad Raynor, Hao Qi, Xi Jiang, Ben Segev, Jingxuan Fan, Sarah Martin-son, Erik Y. Wang, Kaylie Hausknecht, Michael P. Brenner, Mao Mao, Xinyu Zhang, David Avagian, Es-hawn Jessica Scipio, Alan Ragoler, Justin Tan, Blake Sims, Rebeka Plecnik, Aaron Kirtland, Omer Faruk Bodur, D. P. Shinde, Zahra Adoul, Mohamed Zakry, Ali Karakoc, Tania C. B. Santos, Samir Shamselddeen, Loukmame Karim, Anna Liakhovitskaia, Nate Res-man, Nicholas Farina, Juan Carlos Gonzalez, Gabe Maayan, Sarah Hoback, Rodrigo De Oliveira Pena, Glen Sherman, Elizabeth Kelley, Hodjat Mariji, Rasoul Pouriamanesh, Wentao Wu, Sandra Men-doza, Ismail Alarab, Joshua Cole, Danyelle Ferreira, Bryan Johnson, Mohammad Safdari, Liangti Dai, Siriphan Arthornthurasuk, Alexey Pronin, Jing Fan, Angel Ramirez-Trinidad, Ashley Cartwright, Daphiny Potmaier, Omid Taheri, David Outevsky, Stanley Stepanic, Samuel Perry, Luke Askew, Raúl Adrián Huerta Rodríguez, Ali M. R. Minissi, Sam Ali, Ricardo Lorena, Krishnamurthy Iyer, Arshad Anil Fasiludeen, Sk Md Salauddin, Murat Islam, Juan Gonzalez, Josh Ducey, Maja Somrak, Vasiliios Mavroudis, Eric Vergo, Juehang Qin, Benjamín Bor-bás, Eric Chu, Jack Lindsey, Anil Radhakrishnan, Antoine Jallon, I. M. J. McNiss, Pawan Kumar, Lax-man Prasad Goswami, Daniel Bugas, Nasser Heydari, Ferenc Jeannplong, Archimedes Apronti, Abdallah Galal, Ng Ze-An, Ankit Singh, Joan of Arc Xavier, Kanu Priya Agarwal, Mohammed Berkani, Bened-ito Alves de Oliveira Junior, Dmitry Malishev, Nicolas Remy, Taylor D. Hartman, Tim Tarver, Stephen Mensah, Javier Gimenez, Roselynn Grace Monte-
cillo, Russell Campbell, Anshkhaya Sharma, Khalida Meer, Xavier Alapont, Deepakkumar Patil, Rajat Mameshwari, Abdelkader Dendane, Priti Shukla, Sergei Bogdanov, Sören Möller, Muhammad Rehan Siddiqi, Prajvi Saxena, Himanshu Gupta, Innocent Enyekevwe, Ragavendran P V, Zienab EL-Wasif, Aleksandr Maksapetyan, Vivien Rossbach, Chris Harjadi, Mohsen Bahalooohoreh, Song Bian, John Lai, Justine Leon Uro, Greg Bateman, Mohamed Sayed, Ahmed Menshawy, Darling Duclosel, Yashaswini Jain, Ashley Aaron, Murat Tiriyakioglu, Sheeshram Siddh, Keith Krenek, Alex Hoover, Joseph McGowan, Tejal Patwardhan, Summer Yue, Alexandr Wang, and Dan Hendrycks. 2025. Humanity’s last exam. Preprint. arXiv:2501.14249.

Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. 2024. O1 replication journey: A strategic progress report–part 1. arXiv preprint arXiv:2410.18982.

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022.

Yijia Shao, Yucheng Jiang, Theodore A Kanell, Peter Xu, Omar Khattab, and Monica S Lam. 2024. Assisting in writing wikipedia-like articles from scratch with large language models. arXiv preprint arXiv:2402.14207.

DeepSeek Team. 2024. Deepseek-r1-lite-preview is now live: unleashing supercharged reasoning power.

Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown. november 2024. URL https://qwenlm.github.io/blog/qwq-32b-preview.

Vincent A Traag, Ludo Waltman, and Nees Jan Van Eck. 2019. From louvain to leiden: guaranteeing well-connected communities. Scientific reports, 9(1):1–12.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Junde Wu, Jiayuan Zhu, Yunli Qi, Jingkun Chen, Min Xu, Filippo Menolascina, and Vicente Grau. 2024. Medical graph rag: Towards safe medical large language model via graph retrieval-augmented generation. arXiv preprint arXiv:2408.04187.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.

Hui Yang, Sifu Yue, and Yunzhong He. 2023. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint arXiv:2306.02224.

Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. 2024. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884.

Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19724–19731.
7 Appendix

7.1 Human Evaluation Survey
Please assess each response generated by the model based on the following criteria. Provide your rating on a scale from 1 to 5, where 1 is the lowest and 5 is the highest. You may also leave optional comments to clarify your reasoning.

1. Interest Level (Int.)
   • How engaging and intellectually stimulating is the generated response?
   • Rating Scale: 1: Not engaging - fails to capture interest. 2: Somewhat uninteresting - lacks depth or novelty. 3: Neutral - informative but not particularly engaging. 4: Engaging - provides depth and insight. 5: Highly engaging - deep and thought-provoking.
   • Optional Comment: What aspects of the response contributed to or detracted from its interest level?

2. Organization (Org.)
   • How well-structured and logically organized is the response?
   • Rating Scale: 1: Very disorganized - hard to follow. 2: Somewhat disorganized - requires effort to understand. 3: Neutral - moderately structured but could be clearer. 4: Well-organized - logical and easy to follow. 5: Exceptionally structured - very clear and logically ordered.
   • Optional Comment: Are there any areas where the response could be better structured?

3. Relevance (Rel.)
   • How relevant is the response to the research question posed?
   • Rating Scale: 1: Not relevant - off-topic or misleading. 2: Somewhat relevant - partially addresses the question. 3: Neutral - addresses the question but with some tangents. 4: Mostly relevant - minor deviations but generally on point. 5: Highly relevant - fully addresses the question.
   • Optional Comment: Did the response stay on topic? If not, how did it deviate?

4. Coverage (Cov.)
   • How comprehensively does the response cover the question?
   • Rating Scale: 1: Superficial - lacks depth and critical information. 2: Somewhat incomplete - covers only basic aspects. 3: Neutral - adequate coverage but missing key details. 4: Mostly complete - only minor gaps. 5: Fully comprehensive - deeply covers all necessary aspects.
   • Optional Comment: Are there any areas where additional information would improve the response?

Thank you for your participation!

7.2 Additional Experiments
7.2.1 Ablation Study on Memory Strategies
We have conducted a comparison of several alternative memory strategies within our agentic reasoning framework to replace the Mind-Map module (Table 7). These include: no memory (None Mem), using raw reasoning content as memory (Raw Mem), as well as integrating existing methods such as Read-Agent (Lee et al., 2024), MemoryBank (Zhong et al., 2024), and MemGPT (Packer et al., 2023). We evaluated all approaches on the GAIA benchmark and found that our Mind-Map strategy consistently achieved the highest performance across all settings.

<table>
  <tr>
    <th></th>
    <th>Level 1</th>
    <th>Level 2</th>
    <th>Level 3</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>None Mem</td>
    <td>62.37</td>
    <td>46.54</td>
    <td>24.49</td>
    <td>46.18</td>
  </tr>
  <tr>
    <td>Raw Mem</td>
    <td>62.37</td>
    <td>47.80</td>
    <td>26.53</td>
    <td>47.84</td>
  </tr>
  <tr>
    <td>Read-agent</td>
    <td>64.28</td>
    <td>51.57</td>
    <td>27.70</td>
    <td>49.83</td>
  </tr>
  <tr>
    <td>MemoryBank</td>
    <td>68.41</td>
    <td>55.18</td>
    <td>32.65</td>
    <td>53.49</td>
  </tr>
  <tr>
    <td>MemGPT</td>
    <td>72.04</td>
    <td>66.70</td>
    <td>42.11</td>
    <td>65.12</td>
  </tr>
  <tr>
    <td>Mind-Map (ours)</td>
    <td><b>74.36</b></td>
    <td><b>69.21</b></td>
    <td><b>45.46</b></td>
    <td><b>66.13</b></td>
  </tr>
</table>

7.2.2 Efficiency Analysis
We conducted a comparison on 56 deep research questions, measuring both performance and time consumed. We compared our model against three related proprietary systems: Perplexity-Deep Research, Gemini-Deep Research, and GPT-Deep Research. As shown in Table 8, we can see our model achieves substantially better performance than Perplexity-Deep Research while being faster, and it outperforms Gemini-Deep Research with comparable latency. While slightly behind GPT-Deep Research in performance, our model runs significantly faster, and we believe the performance
Table 8: Compare performance and time consumed on 56 deep research questions.

<table>
  <tr>
    <th></th>
    <th>Avg. time consumed / per question</th>
    <th>Interest Level</th>
    <th>Organization</th>
    <th>Relevance</th>
    <th>Coverage</th>
  </tr>
  <tr>
    <td>Perplexity-deep research</td>
    <td>3.1 mins</td>
    <td>2.0</td>
    <td>2.1</td>
    <td>1.6</td>
    <td>2.1</td>
  </tr>
  <tr>
    <td>Gemini-deep research</td>
    <td>7.7 mins</td>
    <td>2.7</td>
    <td>2.5</td>
    <td>2.3</td>
    <td>3.0</td>
  </tr>
  <tr>
    <td>Ours</td>
    <td>6.8 mins</td>
    <td>3.7</td>
    <td>4.6</td>
    <td>4.2</td>
    <td>4.1</td>
  </tr>
  <tr>
    <td>GPT-deep research</td>
    <td>17.8 mins</td>
    <td>4.1</td>
    <td>4.8</td>
    <td>4.2</td>
    <td>4.5</td>
  </tr>
  <tr>
    <td>Human w/ DeepSeek R1.</td>
    <td rowspan="2">1h 48 mins</td>
    <td rowspan="2">4.5</td>
    <td rowspan="2">4.8</td>
    <td rowspan="2">4.8</td>
    <td rowspan="2">4.5</td>
  </tr>
  <tr>
    <td>Cursor and Web-Search (6 questions)</td>
  </tr>
</table>

gap is likely attributable to GPT-Deep Research being built on a more advanced underlying reasoning model.

7.2.3 Websearch and Coding as Agent

Using agentic tool calls rather than direct API calls offers several advantages:

1. Overcoming single-model token limitations:
By structuring the workflow agentically, the system can break free from the token generation limits of a single LLM. This enables it to produce longer, higher-quality reasoning chains than would be possible within the token budget of a single model call.

2. Managing uncertainty and reducing error propagation:
Agents can self-monitor and assign varying levels of confidence to their outputs. As noted earlier, this mechanism helps the reasoning model treat low-confidence outputs as tentative, thereby mitigating cascading errors across components.

For example, in the web-search agent, if insufficient information is retrieved to confidently answer a query, the agent may respond with something like:
“Given the limited knowledge retrieved, a possible answer might be... However, due to the lack of sufficient source information, additional data is needed to provide a more accurate response.”

This uncertainty is explicitly communicated back to the reasoning model, allowing it to treat the response as tentative, rather than relying on it as a final answer. In our experiments, we found that this self-awareness and feedback mechanism helps reduce cascading errors and improves overall robustness. We support this claim with an ablation study on the GAIA benchmark, comparing our agent-based system with a version that uses direct API calls for coding and web-search. Table 9 shows that the agent-based design significantly reduces errors and improves performance, especially on level-3 hard problems, validating our approach.

3. Task-specific model modularity:
The agentic design allows us to assign different LLMs to different tasks. For example, Claude-Sonnet tends to perform better on coding tasks, so we route the coding agent to use it specifically. Similarly, for tasks like summarizing web search results, we can use a lightweight, non-reasoning model to preserve efficiency. This modular setup allows for both improved performance and optimized resource usage by matching the best model to each subtask.

Table 9: Compare with a version that uses direct API calls for coding and web-search.

<table>
  <tr>
    <th></th>
    <th>Level 1</th>
    <th>Level 2</th>
    <th>Level 3</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>API Calling</td>
    <td>60.22</td>
    <td>46.54</td>
    <td>24.49</td>
    <td>47.18</td>
  </tr>
  <tr>
    <td>Agentic Tools</td>
    <td><b>74.36</b></td>
    <td><b>69.21</b></td>
    <td><b>45.46</b></td>
    <td><b>66.13</b></td>
  </tr>
</table>

Reviewer paper summaries: Agentic Reasoning is a framework designed to improve LLM reasoning by integrating external tool-using agents. It dynamically employs a Web-Search agent, a Code Execution agent, and a structured memory mechanism to handle complex tasks requiring deep research. A key component is the Mind-Map agent, which constructs a knowledge graph to keep track of context and logical connections, preserving coherence in prolonged reasoning chains. The authors also introduce an enhanced Web-Search agent, surpassing previous methods and achieving state-of-the-art public performance on DeepSeek-R1, closely matching top proprietary solutions like OpenAI Deep Research. Ablation studies confirm that both the Mind-Map and Web-Search agents contribute significantly to the gains in LLM reasoning, and the authors release their code and data publicly.

This paper proposes Agentic Reasoning designed to improve the reasoning capabilities of LLMs through the integration of external tools, such as web search and code execution and structured memory for deep research problems. A key component is the Mind-Map agent, which constructs a knowledge graph to maintain context and coherence in reasoning.  The paper also explores the Web-Search agent, resulting in a highly effective search mechanism. When used with DeepSeek-R1, this method achieves state-of-the-art performance among public models and performs comparably to OpenAI Deep.

This paper introduces Agentic Reasoning, a framework that integrates external tool-using agents with large language models (LLMs) to enhance reasoning capabilities. The framework combines three key agents: a Web-Search agent for retrieving information, a Code agent for computation, and a novel Mind-Map agent that constructs structured knowledge graphs to organize complex reasoning.

This paper proposes a framework for agentic reasoning that augments an LLM with three external tools - web search, code execution, and structured memory (called "Mind-Map"). The Mind-Map agent constructs a knowledge graph from the reasoning context, to store reasoning context and track logical relationships, ensuring coherence in long reasoning chains.

The authors evaluate Agentic Reasoning framework with DeepSeek-R1 and Qwen-QwQ, on benchmarks requiring expert-level knowledge and deep research, such as Humanity's Last Exam, GPQA, GAIA, and FreshWiki. Their results demonstrate marked improvements over the base reasoning LLM and close the gap with proprietary models like OpenAI's Deep Research and Gemini Deep Research, achieving state-of-the-art results among public models on several benchmarks. 

This is a strong paper with impressive empirical results even if the ideas are not entirely novel. The main weaknesses lie in the lack of detail on how and when the Mind-Map is queried to make use of stored information and potential latency issues due to the number of operations involved in answering questions.
Summary of strengths: 1.By combining web search, code execution, and structured memory, the framework flexibly addresses a wide range of complex reasoning tasks.

2.Constructing a knowledge graph helps maintain logical consistency in long chains of thought, especially when multiple tools are involved.

3.The improved Web-Search agent sets a new public benchmark on DeepSeek-R1 and approaches the level of leading proprietary models, validating the overall effectiveness of Agentic Reasoning.

**Strengths**

Use of mind map agents for structured knowledge representation and integrating with external tools with structured reasoning is novel

Paper presented thorough evaluation process across different type of tasks with expert level answering and deep research using both quantitative and qualitative 

The ablation studies show how each part works, especially the Mind-Map agent and the Web-Search agent. Finding good tools is more important than having many tools is insightful.

The paper showed impressive results, achieving new best scores on several difficult tests like Humanity's Last Exam, GPQA, and GAIA. The framework consistently does better than existing open-source methods and is even competitive with closed models like OpenAI Deep Research.

The authors openly discuss the limitations of their approach, such as higher computational costs, reliance on the quality of external knowledge, and potential issues with interpretability and trustworthiness.Appreciate openness

- The paper provides extensive ablation studies on different tool combinations and Web-Search agent designs
- Empirical results show clear performance improvements over baseline models

1. The paper builds a framework for agentic reasoning and comprehensively evaluates it on a set of very relevant benchmarks. The strong empirical results across multiple challenging benchmarks (Humanity's Last Exam, GPQA, GAIA) demonstrate the usefulness of the framework. The paper achieves state of the art on these benchmarks using open source models and closes the gap with proprietary models. 

2. The introduction of Mind-Map agent as a structured memory mechanism using knowledge graphs is a key innovation. It directly tackles the problem of maintaining coherence and context in long, complex reasoning chains involving multiple tool calls, a known limitation of current LLMs.
Summary of weaknesses: 1.the motivation of this paper is not clear, as integrating other agents as tools may be not effective than directly using the tools. It even makes the system more complex. More direct comparison experiments should be discussed and performed in this paper.

**Weaknesses:**

The use of multiple external agents increases the amount of computing power and time needed to get results. This could make the framework less practical 

The framework's effectiveness depends on the quality of external information it uses, especially from web searches. This is a concern because there aren't strong mechanisms in place to check the accuracy of the information.

Due to the presence of multiple components, errors that arise within the system tend to cascade and escalate rapidly. This means that a small initial error in one component can quickly spread and affect other components, leading to a much larger overall error in the system.

Reasoning and debugging multi-component and multi-agent systems is challenging

The example in figure 4 is interesting. My experiment with Gemini yielded the correct response, while ChatGPT and DeepSeek did not. Gemini's success suggests that the issue is more about alignment than the models' inability to analyze logical relationships. Therefore, a mind map may not be necessary

The paper acknowledges the limitations of compute, but does not present data or clearly explain the cost and benefit tradeoffs which makes it hard to validate the net benefits

The use of only 56 questions across the domains of finance, law, and medicine might not be statistically significant enough to draw definitive conclusions about the model's performance. A broader range of questions and a larger sample size for human evaluations could provide more robust and reliable evidence for validating the results.

- The core concept of agentic reasoning with LLMs has been extensively explored in prior work. The paper presents an interesting engineering improvement so foundamentally lack of novelty.
- The prompt templates (which are the main contirbutions) for the various agents are not included
- No analysis of computational overhead and efficiency tradeoffs (which i think is important), despite acknowledging this as a limitation.
- The ablation study on memory is not enough I think, apart from mindmap, there should be lots of other memory strategies, and  LLMs inherently lack of the ability to handle complex graph data ( by linearizing the data), so I think this part should have more evidence.

Could the authors expand on "Additionally, when the reasoning model encounters uncertainty or loses track in an extended reasoning process" - How will the model know that it has lost track of some information? 

Overall, the paper needs more details on how the Mind-Map agent is constructed and queried. Maybe showing step by step addition of entities into the Mind-Map as the inference proceeds, and more importantly, in what scenarios does the LLM query Mind-Map for information, if ever. 

There seem to be a lot of steps involved in the answer generation process, which raises some latency concerns. A comparison of how long it takes for this framework to respond to a set of questions compared to say OpenAI reasoning models will answer this concern. 

Reproducibility - The paper needs more implementation details to be reproducible, however this is a minor issue for this framework, since the authors mention that the code is publicly available.
Comments, suggestions and typos: Please refer to the weakness

Have you compared latency and compute cost with your framework with others?

Why is websearch and coding an agent and why is not a simple API call? Agentic aspect is not coming out very clearly

see above

Minor grammatical error, line 203: "we retrieve and return the relevant information to response the query".
Overall assessment scores: 3.5

2

2.5

4