DeepEyesV2: Toward Agentic Multimodal Model

## Abstract

Abstract Agentic multimodal models should not only comprehend text and images, but also actively invoke external tools, such as code execution environments and web search, and integrate these operations into reasoning. In this work, we introduce DeepEyesV2 and explore how to build an agentic multimodal model from the perspectives of data construction, training methods, and model evaluation. We observe that direct reinforcement learning alone fails to induce robust tool-use behavior. This phenomenon motivates a two-stage training pipeline: a cold-start stage to establish tool-use patterns, and reinforcement learning stage to further refine tool invocation. We curate a diverse, moderately challenging training dataset, specifically including examples where tool use is beneficial. We further introduce RealX-Bench, a comprehensive benchmark designed to evaluate real-world multimodal reasoning, which inherently requires the integration of multiple capabilities, including perception, search, and reasoning. We evaluate DeepEyesV2 on RealX-Bench and other representative benchmarks, demonstrating its effectiveness across real-world understanding, mathematical reasoning, and search-intensive tasks. Moreover, DeepEyesV2 exhibits task-adaptive tool invocation, tending to use image operations for perception tasks and numerical computations for reasoning tasks. Reinforcement learning further enables complex tool combinations and allows model to selectively invoke tools based on context. We hope our study can provide guidance for community in developing agentic multimodal models.

## 1 Introduction

Figure: Figure 1: Illustration of agentic multimodal models. (a) Existing models show unsatisfactory performance in real-world scenarios, showing clear limitations especially when perception, reasoning, and search must be tightly integrated. (b) A multi-step visual reasoning example requiring coordinated perception, search, and reasoning.
Refer to caption: x1.png

An agentic multimodal model should not only be capable of understanding text and images, but can also actively invoke tools (e.g., a code execution environment or a web search interface) and seamlessly integrate these operations into its advanced reasoning process. For example, as illustrated in Figure [1](https://arxiv.org/html/2511.05271v2#S1.F1) (b), when asked to identify the species of a flower in an image, an agentic multimodal model first crops the region containing that flower, then uses the origin image to search and determine the species. Although existing multimodal models demonstrate strong perception and interpretation abilities, they remain largely passive and lack the ability to autonomously invoke external tools, which is essential for agentic multimodal models. Tools enable explicit, verifiable operations on inputs (e.g., cropping, measuring, computation) and provide access to up-to-date, source-grounded knowledge, thereby improving accuracy, reducing hallucinations, and supporting traceable reasoning.

These tool-use capabilities can be categorized into two types: (i) Operation tools: Current models cannot perform complex operations on visual or numerical data, including fine-grained image manipulations (e.g., cropping, measuring) and quantitative computations. This limits their capacity to reason about detailed visual content or solve mathematical problems. (ii) Information retrieval tools: Models cannot proactively access up-to-date external knowledge, which often leads to outdated conclusions or statements without verifiable sources. Some recent works attempt to rely on a single tool. For example, as shown in Figure [1](https://arxiv.org/html/2511.05271v2#S1.F1) (b), DeepEyes zheng2025deepeyes uses cropping to achieve fine-grained perception, but due to the lack of information retrieval capability, DeepEyes cannot correctly determine the category based solely on its internal knowledge. In contrast, although MMSearch-R1 wu2025mmsearch can perform search, it lacks fine-grained perception, leading to retrieval failures. A substantial gap remains between existing approaches and truly agentic multimodal models. While o3 (o3) has explored “thinking with image” reasoning pattern that combines operations and search, how to realize such capabilities remains unclear.

To explore how to construct such agentic multimodal models, we introduce DeepEyesV2, which seamlessly integrates tool invocation within the dynamic reasoning loop. DeepEyesV2 actively decides when and how to invoke tools, enabling a dynamic process of evidence acquisition and verification. Then, tool outputs are iteratively incorporated into reasoning process, allowing model to refine its hypotheses, validate intermediate results, and ultimately arrive at more reliable and interpretable conclusions. In this work, we systematically investigate key aspects of building an agentic MLLM, including model training strategies, dataset curation, and evaluation protocols.

We first follow the setup of DeepEyes (zheng2025deepeyes) and apply reinforcement learning directly on Qwen2.5-VL (bai2025qwen2), but find that limited inherent tool-use capability prevents stable tool invocation. This highlights the need for a cold-start stage to establish reliable tool-use patterns. Thus, we curate a high-quality dataset that spans diverse scenarios, including perception, reasoning, and search tasks. After cleaning, we apply two filters: (i) difficulty filtering, retaining only questions unsolvable by the base model, and (ii) tool-benefit classification, keeping cases where tool use improves accuracy. Data are split into two subsets: tool-solvable examples for RL and harder unsolved cases for cold start, further augmented with long chain-of-thought trajectories. Supervised fine-tuning on this cold-start dataset enables the model to acquire basic tool-use patterns and deeper reasoning, after which RL further strengthens tool invocation. Notably, we rely only on two simple rewards, accuracy and format, without complex reward engineering (su2025pixel).

Figure: Figure 2: Case reasoning trajectory of DeepEyesV2. DeepEyesV2 seamlessly integrates code execution and web search within its iterative reasoning process. Notably, in the right case, the behavior of accessing webpages via code does not exist in cold start data and is spontaneously acquired during reinforcement learning.
Refer to caption: x2.png

DeepEyesV2 demonstrates strong synergistic capabilities across perception, search, and reasoning. However, existing benchmarks mostly focus on just one of these abilities and lack an integrated, cross-capability benchmark that can comprehensively evaluate all three. Therefore, we propose a new benchmark, called RealX-Bench. RealX-Bench emphasizes cross-capability integration, requiring models to attend to fine-grained visual regions, retrieve external evidence, and reason over multimodal context. As showm in Figure [1](https://arxiv.org/html/2511.05271v2#S1.F1) (a), current models perform well below human performance on RealX-Bench, revealing a substantial performance gap and underscoring RealX-Bench’s difficulty. Compared with current open-source models and models limited to using a single tool, DeepEyesV2 demonstrates powerful coordination across the three capabilities.

Besides, we evaluate DeepEyesV2 on benchmarks covering real-world understanding, mathematical reasoning, and search-intensive tasks.
DeepEyesV2 outperforms both general-purpose MLLMs and prior specific reasoning approaches. Specifically, on real-world understanding benchmarks, DeepEyesV2 surpasses even Qwen2.5-VL-32B in some benchmarks through effective tool use. On reasoning tasks, DeepEyesV2 shows preformance gains across multiple benchmarks, including $+7.1$ on MathVerse ($52.7\%$ accuracy). On search benchmarks, DeepEyesV2 delivers strong advantages, reaching $63.7\%$ on MMSearch (jiang2024mmsearch), far beyond the MMSearch-R1 (wu2025mmsearch) ($53.8\%$). These results demonstrate that by reliably invoking tools, DeepEyesV2 extends its comprehensive capabilities, achieving accurate and advanced reasoning.

We observe the task-dependent tool invocation patterns in DeepEyesV2. For perception tasks, DeepEyesV2 primarily uses image operations, such as cropping, to extract fine-grained visual details, whereas for reasoning tasks, DeepEyesV2 favors numerical analysis. Moreover, reinforcement learning can further enhances tool-use behavior, enabling more complex tool combinations and adaptive decision-making. DeepEyesV2 learns to selectively invoke tools based on the problem context, reflecting the emergence of autonomous, agentic reasoning.

Figure: Figure 3: Pipeline of DeepEyesV2. DeepEyesV2 invokes tools and incorporates execution results into subsequent reasoning steps, enabling iterative and tool-augmented multimodal inference.
Refer to caption: x3.png

The main contributions are summarized as follows:
(i) We introduce DeepEyesV2, an agentic multimodal model that unifies code execution and web search within a single reasoning loop, enabling reliable and complex reasoning.
(ii) We construct a carefully curated training corpus through rigorous data filtering and cleaning. The resulting dataset is diverse in task types, of appropriate difficulty, and explicitly designed to ensure the beneficial integration of tools. Based on this, we build both cold-start SFT data and RL data that complement each other.
(iii) Extensive experiments across real-world understanding, mathematical reasoning, and search-intensive benchmarks demonstrate the strong reasoning and tool-usage ability of DeepEyesV2.
(iv) We propose RealX-Bench, a comprehensive benchmark designed to evaluate real-world multimodal reasoning involving perception, search, and reasoning integration, providing a rigorous platform for assessing agentic multimodal intelligence.
(v) We analyze the dynamics of tool-use behavior in DeepEyesV2, revealing task-adaptive patterns. Besides, we also find reinforcement learning can enable more complex tool combinations and adaptive, context-aware tool invocation.

## 2 Related Works

Multimodal Large Language Models. The field of multimodal large language models (MLLMs) has witnessed rapid progress in recent years. Early efforts mainly focus on combining pretrained visual encoders with large language models through lightweight adapters or projection layers, enabling basic vision–language alignment and simple multimodal understanding (li2023blip; liu2023visual; liu2024improved; bai2023qwen; chen2024internvl). Subsequently, more powerful architectures such as Qwen2.5-VL (bai2025qwen2), LLaVA-OneVision (li2024llava), and InternVL3 (zhu2025internvl3), expand the training scale and integrated more diverse visual data, significantly improving performance on benchmarks of visual question answering, captioning, and general perception tasks. Recently, some OmniMLLMs (li2025baichuan; zhao2025r1; fu2024vita; jain2024ola; hong2025worldsense) are capable of processing a mix of modalities like speech, video, and images simultaneously. However, existing MLLMs remain largely passive: they can interpret multimodal inputs and generate answers, but lack the ability to actively invoke external tools for computation or knowledge retrieval, which limits their reliability in complex reasoning tasks.

Thinking with Images.
The paradigm of “Think with Image” is first introduced by o3 (o3), which demonstrated that multimodal models can interleave reasoning with iterative visual analysis, actively manipulating images to support step-by-step problem solving. Many works attempt to reproduce such capabilities. Most approaches (su2025pixel; lai2025mini; liu2025visual; fan2025grit; jiang2025vlm; zhang2025chain) adopt a two-stage training pipeline, where a cold-start phase is followed by reinforcement learning. In contrast, DeepEyes (zheng2025deepeyes) only adopts reinforcement learning alone and incentivizes the “Think with Image” behaviors, leading to strong reasoning performance. However, the majority of these efforts employ a rather limited tool set, typically restricted to region cropping for fine-grained perception. To improve generality, PyVision (zhao2025pyvision) and Thyme (zhang2025thyme) utilize code execution to enable more flexible visual operations. Despite this progress, these models remain constrained to image manipulation only, and are unable to handle knowledge-intensive questions where access to up-to-date external information is essential.

Figure: Figure 4: Pioneer Experiments reveal that existing multimodal models cannot directly acquire reliable tool use ability through RL, demonstrating the necessity of a cold start phase. The red dashed line represents tool calls number in a single rollout, and the blue solid line represents the averge response length.
Refer to caption: x4.png

Search-oriented Reasoning.
To mitigate the inherent knowledge limitations of large multimodal language models, a growing line of work explores augmenting them with external knowledge acquisition. Early approaches commonly adopt the retrieval-augmented generation (RAG) paradigm (song2025r1; jin2025search), where relevant information is retrieved from a pre-constructed knowledge base and fed into the model. While effective, this paradigm remains constrained by the static and finite nature of the underlying corpus. To overcome these limitations, more recent studies attempt to leverage online search to dynamically access broader and up-to-date information (zheng2025deepresearcher). Beyond purely textual queries, some efforts extend search into the multimodal domain, enabling retrieval of not only documents but also images, charts, or other media forms relevant to the task (wu2025mmsearch; wang2025vrag). These advances highlight the potential of search-augmented reasoning to complement perception and tool-use capabilities, ultimately broadening the scope of problems that multimodal models can effectively address.

## 3 DeepEyesV2

We explore how to construct agentic multimodal models from the perspectives of training strategy, dataset design, and evaluation. We begin in Section [3.1](https://arxiv.org/html/2511.05271v2#S3.SS1) by presenting the overall pipeline of DeepEyesV2, which integrates tool invocation into the reasoning loop. Then, we conduct pioneer experiments in Section [3.2](https://arxiv.org/html/2511.05271v2#S3.SS2) to reveal the limitations of existing models in reliably using tools, underscoring the necessity of a cold-start stage. After that, we describe the curation of a high-quality training dataset and the principles behind its construction in Section [3.3](https://arxiv.org/html/2511.05271v2#S3.SS3). Finally, building on the cold-start foundation, we apply a reinforcement learning stage to further enhance the efficiency and flexibility of tool-use behavior, which is described in Section [3.4](https://arxiv.org/html/2511.05271v2#S3.SS4).

### 3.1 Overall Pipeline

Similar to DeepEyes (zheng2025deepeyes), DeepEyesV2 is an agentic multimodal model, but with extended tool-use capabilities beyond simple cropping. In DeepEyesV2, programmatic code execution and web retrieval are treated as complementary and interleavable tools inside a single reasoning trajectory (see Figure [3](https://arxiv.org/html/2511.05271v2#S1.F3)). Given an image input and the corresponding user query, DeepEyesV2 first generates an initial reasoning plan, and explicitly determines whether this question can be solved directly through internal reasoning or requires tool invocation. If tool use is necessary, DeepEyesV2 emits executable Python code or issues web search queries. Code execution is carried out in a sandboxed environment and can produce structured outputs such as transformed images, numerical measurements, computed arrays, plots, or execution logs. Image queries using the original whole image are submitted via SerpAPI and return the top five visually matched webpages (each with a thumbnail and title). Text queries return the five most relevant webpages, along with titles and snippets. All tool outputs are converted into observations and appended to model’s context. DeepEyesV2 then thinks further in light of these observations and may plan further tool invocations (either additional code, further searches, or both), iterating this reasoning–tool–integration loop until a conclusive answer is produced.

DeepEyesV2 can dynamically choose, combine, and use tools as reasoning unfolds. This integration yields three main advantages: (i) it allows expanded and enhanced analytical capability through executable code; (ii) it enables active and real-time knowledge seeking by retrieving multimodal evidence from the web; and (iii) it supports iterative, interleaved multi-tool reasoning, in which code execution and search can be dynamically combined within a single trajectory, rather than being isolated modules. Together, these features position DeepEyesV2 as a more general, reliable, and extensible framework for multimodal reasoning.

### 3.2 Pioneer Experiments

To investigate whether MLLMs can directly acquire tool-use ability through reinforcement learning, we first conduct a pioneer experiment on Qwen2.5-VL (bai2025qwen2) following DeepEyes (zheng2025deepeyes). As shown in Figure [4](https://arxiv.org/html/2511.05271v2#S2.F4), during training, we observe that in the early stages model occasionally attempts to produce Python code, but these outputs are often buggy or fail to execute, indicating that existing MLLMs struggle to generate stable and reliable code. As training continues, model gradually abandons code generation and converges to producing only short reasoning chains followed by direct answers, thereby bypassing tool use. Then, to encourage tool invocation, we incorporate the tool usage bonus mechanism from DeepEyes, which explicitly rewards the generation of code. With this additional signal, model is indeed able to produce correct and runnable code in the early stages, suggesting that the mechanism can enforce coding ability. However, with continued training a new degeneration emerges: model’s behavior converged to emitting exactly one code block per query, and this single block typically consists of non-executable, placeholder comments rather than meaningful code, revealing the phenomenon of reward hacking. This pioneer experiment highlights that existing MLLMs cannot reliably learn complex tool use through direct RL alone, motivating the need for a cold start to bootstrap model’s tool invocation ability.

Figure: Figure 5: Statistics of RealX-Bench. (a) Domain distribution across five representative categories: Daily Life, Media, Sports, Knowledge, and Games. (b) Distribution of subsets classified by required abilities: perception, reasoning, search, and integration. These numbers may overlap because the challenges are not mutually exclusive. Integration denotes questions that are difficult across all three abilities simultaneously.
Refer to caption: x5.png

### 3.3 Training Data Curation

Data Collection.
Pioneer experiments have highlighted the necessity of constructing a high-quality dataset for supervised fine-tuning to explicitly guide model to learn how to generate executable code and perform tool invocations. Following DeepEyes (zheng2025deepeyes), we collect data in accordance with the following principles: (i) Diverse tasks and image distribution. We incorporate varied data to cover a wide range of multimodal challenges and visual components. (ii) Verifiability and structured format. All questions are reformulated into a structured, open-ended QA format to facilitate objective evaluation. We exclude examples that cannot be reliably verified, such as those with incorrect answers, ambiguous phrasing, or poor readability.
(iii) Appreciate difficulty. We exclude examples that the base model can easily solve and prioritize questions that remain challenging. (iv) Beneficial integration of tools. We categorize examples based on whether tool usage leads to correct answers. Cases where model can solve correctly using additional tool calls are reserved for reinforcement learning, whereas examples that remain unsolved even with tool assistance are used for cold start.

Specially, we curate data from three major categories: perception, reasoning, and search. Besides, we also include long Chain-of-Cot (CoT) reasoning data in cold start subset. Please refer to Appendix [A.1](https://arxiv.org/html/2511.05271v2#A1.SS1) for more details on data sources. All datasets are carefully cleaned, reformatted, and divided into subsets for cold start or reinforcement learning subsets. To ensure sufficient difficulty, we employ Qwen2.5-VL-7B (bai2025qwen2) as a baseline evaluator. For each question, model is prompted to generate 8 responses, and we retain only those instances where it answers correctly at most two times, thereby filtering out trivial cases. To further assess tool-use effectiveness, we prompt model to solve each question with tool invocation, again collecting 8 responses per instance, and categorize examples according to their success rate.

Trajectories Synthesis.
We construct cold start datasets by eliciting step-by-step trajectories from models (e.g., Gemini 2.5 Pro (comanici2025gemini), GPT-4o (hurst2024gpt), and Claude Sonnet 4 (claude4)). For each prompt, these models are prompted to produce detailed reasoning traces that explicitly include tool-invocation markers (e.g., code snippets). Each declared tool call is executed, and the returned outputs are fed back to the originating model, and model continues reasoning, potentially issuing further tool calls, until it produces a final answer. The entire interaction is recorded as a single trajectory. Only trajectories with correct final answers and error-free code are retained for high-quality cold-start data.

### 3.4 Agentic Reinforcement Learning

After cold-start training has equipped model with basic tool-use patterns, we adopt reinforcement learning to further enhance its ability to integrate tools in dynamic environment. Unlike SFT, which relies on learning from static trajectories, agentic RL places the model in an interactive environment where it must dynamically decide when and how to invoke tools in order to solve tasks. Following DeepEyes (zheng2025deepeyes), we employ a sparse and outcome-driven reward. The overall reward consists of two components: (i) accuracy reward $R_{acc}$, which evaluates whether the final answer matches the ground truth, and (ii) format reward $R_{format}$, which penalizes outputs that violate required formats. The total reward is defined as: $R=R_{acc}+R_{format}$.

## 4 RealX-Bench

Table: Table 1: Results on RealX-Bench.

Existing multimodal benchmarks such as MME-RealWorld (zhang2024mme), SEED-Bench (li2023seedbench), and MMSearch (jiang2024mmsearch) primarily evaluate isolated capabilities, for instance, perception, retrieval, or reasoning.
However, real-world multimodal understanding often demands coordination across multiple abilities.
Thus, we introduce RealX-Bench, a comprehensive benchmark that evaluates the coordinated interplay of perception, search, and reason in complex real-world scenarios.

### 4.1 Design Principles.

We define three core abilities, perception, search, and reasoning, as follows: perception is the capacity to recognize and locate relevant visual elements; search means finding the needed information from the web or provided resources; reasoning means combining evidence to reach the correct answer through clear, multi-step logic.
To comprehensively evaluate model’s coordinated interplay of these abilities, we construct RealX-Bench adheres to the following design principles. (i) Challenging. Each question is deliberately difficult. For perception, challenge means precise localization of subtle targets under clutter or occlusion. For search, it requires multi-hop evidence gathering. For reasoning, it involves multi-step logical composition with intermediate consistency checks. Each question is constructed to exhibit at least one difficulty dimension. (ii) Real-World. All questions are grounded in real-world scenarios and realistic content distributions, and are refined for semantic fidelity and practical relevance. (iii) Objectivity. Every question has a short, unique answer in a standardized format and can be automatically verified via programmatic checks, enabling efficient, reproducible, and scalable evaluation.

### 4.2 Benchmark Construction

The construction follows a four-stage workflow: data collection, QA annotation, difficulty and category labeling, and quality control. First, We collect openly available images and their corresponding user questions from the internet, which faithfully reflect real-world scenarios. These questions fully reflect real-world scenarios. We filter them for visual quality and content diversity to ensure high quality and broad coverage. Then, experts refine each question and answer to better suit formal contexts and to ensure fluent, coherent language. After that, annotators assign a difficulty label to each question (e.g., whether it is perception challenging) and tag the corresponding image category. Finally, quality control checks verify answer correctness and uniqueness for every QA pair.

### 4.3 Data Statistics.

RealX-Bench consists of 300 question–answer pairs spanning five representative real-world domains, and we show the data statics in Figure [5](https://arxiv.org/html/2511.05271v2#S3.F5).
Along the difficulty dimension, each question is annotated on three ability axes, perception, search, and reasoning, with non-mutually exclusive labels. Because difficulty can be coupled (e.g., a question may be both perception-challenging and require multi-hop search), these counts overlap. Notably, $24\%$ questions are simultaneously challenging across all three abilities. Compared with prior benchmarks that mainly assess a single capability in isolation, RealX-Bench enables evaluation of integrated performance across perception, search, and reasoning.

## 5 Experiments

Table: Table 2: Results on real-world & OCR & chart understanding Benchmarks.

Table: Table 5: Ablation study on cold start data. Perception and reason represent multi-turn agent data with code execution, respectively, while Long CoT refers to single-turn, purely text-based reasoning data. Long CoT refers to text-only reasoning data. For more details, please refer to Appendix [A.1](https://arxiv.org/html/2511.05271v2#A1.SS1).

### 5.1 Implementation Details

We conduct training in two stages: cold start SFT and reinforcement learning. The backbone model is Qwen2.5-VL-7B (bai2025qwen2). For SFT, we train with a batch size of 128 and a learning rate of $1\times 10^{-5}$. Model is optimized for 3 epochs using AdamW (loshchilov2017decoupled) optimizer with cosine learning rate decay. For RL, we adopt DAPO (yu2025dapo) as the optimization algorithm, with a batch size of 256 and 16 rollouts per prompt. The KL coefficient is set to $0.0$, and the maximum response length is capped at $16,384$ tokens. The learning rate is $1\times 10^{-6}$, and the upper and lower clip ratios are $0.30$ and $0.20$, respectively. We utilize VLMEvalKit (duan2024vlmevalkit) to conduct all the evaluation, except for RealX-Bench, so the performance of DeepEyes may be a little different from (zheng2025deepeyes).

### 5.2 Evaluation on RealX-Bench

We evaluate existing models and DeepEyesV2 on RealX-Bench to assess their ability to integrate perception, search, and reasoning, and results are shown in Table [1](https://arxiv.org/html/2511.05271v2#S4.T1).

Struggling to Integrate Perception, Search, and Reasoning. Even the best proprietary model achieves only $46.0\%$ accuracy, far below human performance. Moreover, current models exhibit severe limitations in coordinating all three skills; for example, Gemini’s accuracy on subsets ($27.8\%$) that require combining all three skills is much lower than its average accuracy ($46.0\%$).

Search Benefits. Incorporating search tools effectively improves accuracy, especially in scenarios that require search. Using both text and image search yields substantial performance gains. However, text-only search provides larger improvements than image-only search, suggesting that current models still have limited ability to integrate image-search results effectively.

DeepEyesV2 demonstrates better coordination. Compared with other open-source models and models that incorporate zooming tools, DeepEyesV2 achieves superior performance. In particular, on tasks that require coordination of all three capabilities, DeepEyesV2 far outperforms other models, highlighting its strong multi-skill coordination.

Figure: Figure 6: Tool distribution comparison. DeepEyesV2 demonstrates the task-specific tool-calling distribution across different tasks. Reinforcement learning leads to a distribution shift.
Refer to caption: x6.png

### 5.3 Results on Other Benchmarks

Real-World & OCR & Chart Understanding.
We evaluate DeepEyesV2 across three categories of benchmarks: real-world understanding, OCR, and chart understanding. For comparison, we include two kinds of models: (i) open-source general-purpose MLLMs, including LLaVA-OneVision (li2024llava), Qwen2.5-VL (bai2025qwen2), and InternVL3 (zhu2025internvl3); and (ii) grounded reasoning models, such as DeepEyes (zheng2025deepeyes) and Thyme (zhang2025thyme). DeepEyes performs fine-grained perception by cropping the target region, while Thyme manipulates images through executable code. Compared to base model Qwen2.5-VL-7B, DeepEyesV2 demonstrates substantial performance gains, and even surpasses Qwen2.5-VL-32B in some benchmarks (Table [4](https://arxiv.org/html/2511.05271v2#S5.T4)), highlighting the effectiveness of tool-augmented reasoning. Moreover, it consistently outperforms existing grounded reasoning models. These results indicate that dynamic tool invocation enables model to extract fine-grained details, thereby improving real-world scene comprehension.

Multimodal Reasoning.
We further evaluate DeepEyesV2 on mathematical reasoning benchmarks to assess its strong reasoning capability. As shown in Table [4](https://arxiv.org/html/2511.05271v2#S5.T4), we compare DeepEyesV2 against existing open-source MLLMs, such as Qwen2.5-VL (bai2025qwen2), text-only multimodal reasoning models, including MM-Eureka (meng2025mm), and grounded reasoning models, such as DeepEyes (zheng2025deepeyes) and Thyme (zhang2025thyme). DeepEyesV2 consistently outperforms these alternatives, and notably achieves stronger results than text-only multimodal reasoning models, underscoring the benefit of tool use for enhancing mathematical reasoning.

Online Searching.
To further examine the effectiveness of external information acquisition, we evaluate DeepEyesV2 on search-oriented benchmarks. These datasets encompass knowledge-intensive visual question answering, fact verification, and multimodal retrieval-based reasoning, all of which require models to go beyond perceptual understanding and actively retrieve external evidence. For comparison, we benchmark DeepEyesV2 against both general-purpose MLLMs such as Qwen2.5-VL, Gemini 2.5 Pro, and GPT4o, as well as models where search capability is incorporated (jiang2024mmsearch; geng2025webwatcher). As shown in Table [4](https://arxiv.org/html/2511.05271v2#S5.T4), DeepEyesV2 demonstrates superior search capabilities, achieving consistently higher accuracy across all benchmarks.

### 5.4 Analysis

Training Data.
To understand how training data influences the development of tool-use ability, we investigate the impact of different dataset compositions.

• Cold Start Data.
We conduct ablations on the SFT dataset (Table [6](https://arxiv.org/html/2511.05271v2#S5.T6)). Perception and reason represent multi-turn agent data with code execution, respectively, while Long CoT refers to single-turn, purely text-based reasoning data. Directly evaluating Qwen2.5-VL-7B brings a great performance drop and confirms that existing MLLMs lack robust tool-use ability. Training only on perception data helps perception benchmarks but not reasoning; training only on reasoning data yields limited or negative gains, showing perception and reasoning rely on distinct tool-use patterns, with reasoning being more complex and harder to master. Adding long CoT trajectories substantially enhances reasoning and tool use, demonstrating that stronger thinking ability directly facilitates better tool use. Combining perception, reasoning, and CoT data achieves the best overall results, highlighting the complementary benefits of diverse supervision and the value of long CoT for complex reasoning.

Overall, these results highlight two key factors of cold start data: (i) diversity, as perception and reasoning rely on different tool-use patterns and data with diverse tasks should be involved to improve generalization; and (ii) the inclusion of long CoT data, which strengthens reasoning and substantially improves tool use on complex tasks.

• RL Data.
We further conduct ablation studies on different subsets of RL data. Results are shown in Table [6](https://arxiv.org/html/2511.05271v2#S5.T6). When training with only perception data, model achieves clear improvements on image-understanding benchmarks, but its performance on mathematics and search tasks declines. A similar trend is observed when using only reasoning data, where reasoning-related benchmarks improve, but perception and search tasks degrade. In contrast, combining perception and reasoning data yields consistent gains across both categories, demonstrating their complementary nature. Finally, incorporating search data leads to significant improvements on retrieval-oriented benchmarks, resulting in balanced and robust overall performance. These results emphasize that data diversity is critical for reinforcement learning in agentic multimodal models.

In-Deep Analysis.
Then, we conduct an in-depth analysis of DeepEyesV2’s tool-use behavior after cold start and RL, comparing the two stages to better understand how training shapes and alters the model’s strategies for invoking tools.

• Tool Distribution.
To understand how the model leverages tools across various scenarios, we analyze tool-use distributions over eight benchmarks before and after reinforcement learning (Figure [8](https://arxiv.org/html/2511.05271v2#S5.F8)). DeepEyesV2 exhibits clear task-dependent preferences: in real-world perception tasks (V*), model mainly uses cropping to obtain fine-grained visual details; in OCR tasks (SEED-Bench-2-Plus), it additionally performs region marking and numerical computations; chart-related tasks (CharXiv) involve more arithmetic operations; reasoning benchmarks (MathVista, MathVerse) are dominated by mathematical computations for intermediate verification and final answers; and search tasks (MMSearch, InfoSeek) primarily invoke search tools.

Moreover, when comparing behaviors before and after RL, we observe a notable shift. After reinforcement learning, model tends to perform more numerical operations across multiple tasks, and begins to integrate image manipulation tools (e.g., cropping) with search in search benchmarks, indicating that RL helps model develop a more synergistic use of heterogeneous tools to solve complex queries.

• Adaptive Thinking.
We further investigate tool-use efficiency by measuring the proportion of questions where model invokes tools before and after RL. As shown in Figure [8](https://arxiv.org/html/2511.05271v2#S5.F8), prior to RL, model over-relies on tools, using them for most questions. After RL, however, tool invocation rate decreases significantly, showing that model learns adaptive reasoning: it solves problems directly when tools are unnecessary while still leveraging them when beneficial. Combined with Figure [9](https://arxiv.org/html/2511.05271v2#A1.F9), these results highlight that reinforcement learning improves both efficiency and flexibility, enabling the balance between textual reasoning and tool calls.

• Training Dynamic.
We further analyze model dynamics during RL by tracking response length, reward, and tool invocation frequency throughout training (Figure [8](https://arxiv.org/html/2511.05271v2#S5.F8)). The average number of tool calls steadily decreases over time; however, the variance remains large, indicating that model does not simply converge to a fixed number of tool invocations (e.g., one per query). Instead, model learns adaptive thinking: it selectively invokes tools when necessary, while handling simpler problems with minimal or no tool use. For more challenging queries, the number and complexity of tool calls remain high, reflecting flexible and task-aware strategies. Shorter response lengths further indicate more efficient reasoning, allocating detailed tool-based steps only when beneficial. Together, these findings highlight that reinforcement learning not only enhances tool-use effectiveness, while fostering diversity, complexity, and efficiency in reasoning.

## 6 Conclusion

In this work, we explore how to construct agentic multimodal models that can actively invoke tools and integrate them into reasoning, from the perspectives of training, dataset design, and evaluation. We introduce DeepEyesV2 and conduct a practical two-stage training pipeline: supervised fine-tuning on a curated dataset to establish robust tool-use patterns, followed by reinforcement learning to strengthen and adapt tool invocation. Our analysis reveals task-dependent tool-use behaviors, and reinforcement learning enables more complex, context-aware tool combinations. Extensive experiments across perception, reasoning, and search benchmarks demonstrate the strong reasoning ability of DeepEyesV2, highlighting the advantages of combining tool invocation with reasoning.

## Appendix A Appendix

### A.1 Training Data

For perception-oriented tasks, we include V* [wu2024v], ArxivQA [li2024multimodal], Pixmo Counting [deitke2024molmo], TallyQA [acharya2019tallyqa], and SeekWorld [seekworld2025], covering a wide range of scenarios such as object recognition, visual counting, and chart interpretation. For reasoning tasks, we adopt ReVisual [chen2025advancing] to provide complex reasoning problems, and additionally incorporate MathCoder [wang2023mathcoder] and Retool [feng2025retool] to supplement with executable code-based problem-solving examples. Besides, we also include long Chain-of-Cot (CoT) reasoning data in cold start subset. For search-related tasks, we employ MMSearch-R1 [wu2025mmsearch], which includes both image-based and text-based retrieval questions. We further include data from VGR [wang2025vgr], Chain-of-Focus [zhang2025chain], and VLM-R^3 [jiang2025vlm] to strengthen the reinforcement learning corpus.

We present the distributions of our cold start and RL data in Figure [9](https://arxiv.org/html/2511.05271v2#A1.F9). The cold start data is divided into four parts: perception, reasoning, search, and Long CoT, while the RL data includes perception, reasoning, and search.

Figure: Figure 9: Distribution of cold start and reinforcement learning data.
Refer to caption: x8.png

### A.2 Evaluation Protocol

we summarize the benchmarks and models we compare across different kinds of tasks. Benchmarks cover three main categories: real-world understanding, mathematical reasoning, and search-intensive tasks, capturing the diversity of challenges faced by agentic multimodal models.

Real-World & OCR & Chart Understanding.
For real-world understanding, we adopt V* [wu2024v], HRBench [wang2025divide], MME-RealWorld [zhang2024mme], and TreeBench [wang2025traceable]; for OCR, we use OCRBench [liu2024ocrbench] and Seed-Bench-2-Plus [li2024seed]; and for chart reasoning, we evaluate on CharXiv [wang2024charxiv] and ChartQA [masry2022chartqa]. For comparison, we include two kinds of models: (i) open-source general-purpose MLLMs, including LLaVA-OneVision [li2024llava], Qwen2.5-VL [bai2025qwen2], and InternVL3 [zhu2025internvl3]; and (ii) grounded reasoning models, such as Pixel-Reasoner [su2025pixel], DeepEyes [zheng2025deepeyes] and Thyme [zhang2025thyme].

Multimodal Reasoning.
We include MathVista [lu2023mathvista], MathVerse [zhang2024mathverse], MathVision [wang2024measuring], WeMath [qiao2024we], and LogicVista [xiao2024logicvista]. We compare DeepEyesV2 against existing open-source MLLMs, such as Qwen2.5-VL [bai2025qwen2], text-only multimodal reasoning models, including MM-Eureka [meng2025mm], ThinkLite [wang2025sota], VL-Rethinker [wang2025vl], and VLAA-Thinker [chen2025sft], and grounded reasoning models, such as DeepEyes [zheng2025deepeyes] and Thyme [zhang2025thyme]

Online Searching.
We compare DeepEyesV2 on FVQA-test [wu2025mmsearch], InfoSeek [chen2023can], MMSearch [jiang2024mmsearch], and SimpleVQA [cheng2025simplevqa]. We benchmark DeepEyesV2 against both general-purpose MLLMs such as Qwen2.5-VL [bai2025qwen2], Gemini 2.5 Pro [comanici2025gemini], and GPT4o [hurst2024gpt], as well as models where search capability is incorporated [jiang2024mmsearch, geng2025webwatcher].

### A.3 Tool Taxonomy

The tools can be categorized into three major classes:

1. Code Execution.
Code execution covers a set of operations that require Python-based execution. We further divide it into four subtypes:

- •
Crop: extract a specific region of the input image for fine-grained analysis.
[⬇](data:text/plain;base64,Y3JvcHBlZCA9IGltYWdlXzEuY3JvcCgodG9wLCBsZWZ0LCByaWdodCwgYm90dG9tKSkKCnBsdC5pbXNob3coY3JvcHBlZCkKcGx0LmF4aXMoJ29mZicpCnBsdC5zaG93KCk=)
cropped = image_1.crop((top, left, right, bottom))
plt.imshow(cropped)
plt.axis(’off’)
plt.show()
- •
Numerical Analysis: perform numerical computations, formula evaluation, or quantitative reasoning.
[⬇](data:text/plain;base64,aW1wb3J0IG1hdGgKaGVpZ2h0ID0gNjgKCncgPSBoZWlnaHQgLyBtYXRoLnRhbihtYXRoLnJhZGlhbnMoMzcpKQoKeCA9IHcgLyBtYXRoLnRhbihtYXRoLnJhZGlhbnMoNDYpKQpwcmludChmIncgPSB7d30iKQpwcmludChmInggPSB7eH0iKQ==)
import math
height = 68
w = height / math.tan(math.radians(37))
x = w / math.tan(math.radians(46))
print(f"w = {w}")
print(f"x = {x}")
- •
Mark: annotate or highlight regions of interest in the image to support reasoning.
[⬇](data:text/plain;base64,ZnJvbSBQSUwgaW1wb3J0IEltYWdlRHJhdwoKZHJhdyA9IEltYWdlRHJhdy5EcmF3KGltYWdlXzEpCgpib3ggPSAoNTAsIDUwLCAzMDAsIDIwMCkKY29sb3IgPSAoMjU1LCAwLCAwKQp0aGlja25lc3MgPSA4CgpkcmF3LnJlY3RhbmdsZShib3gsIG91dGxpbmU9Y29sb3IsIHdpZHRoPXRoaWNrbmVzcykKCnBsdC5pbXNob3coaW1hZ2VfMSkKcGx0LnNob3coKQ==)
from PIL import ImageDraw
draw = ImageDraw.Draw(image_1)
box = (50, 50, 300, 200)
color = (255, 0, 0)
thickness = 8
draw.rectangle(box, outline=color, width=thickness)
plt.imshow(image_1)
plt.show()
- •
Other: other manipulation operations such as rotation, enhancement, or resizing.
[⬇](data:text/plain;base64,ZnJvbSBQSUwgaW1wb3J0IEltYWdlRW5oYW5jZQoKZW5oYW5jZXIgPSBJbWFnZUVuaGFuY2UuQnJpZ2h0bmVzcyhpbWFnZV8xKQoKZmFjdG9yID0gMS41CgpicmlnaHRfaW1nID0gZW5oYW5jZXIuZW5oYW5jZShmYWN0b3IpCgpwbHQuaW1zaG93KGJyaWdodF9pbWcpCnBsdC5heGlzKCdvZmYnKQpwbHQuc2hvdygp)
from PIL import ImageEnhance
enhancer = ImageEnhance.Brightness(image_1)
factor = 1.5
bright_img = enhancer.enhance(factor)
plt.imshow(bright_img)
plt.axis(’off’)
plt.show()

2. Image Search.
Given an image query, we utilize SerpAPI to retrieve visually similar results from the web, returning candidate images with thumbnails.

3. Text Search.
Based on a textual query, we retrieve relevant webpages and provides both titles and snippets of content.

Figure: Figure 10: Error analysis.
Refer to caption: x9.png

### A.4 Error Analysis

We categorize the errors made by DeepEyesV2 into three main types (Figure [10](https://arxiv.org/html/2511.05271v2#A1.F10)). First, tool execution errors occur when the model generates a correct reasoning trajectory but fails during tool operation, such as cropping the wrong region or using incorrect search keywords. Second, tool selection errors arise when the model chooses an inappropriate tool for the task, for example selecting text search when an image search is required. Third, tool result analysis errors happen when the model correctly selects and executes a tool, but misinterprets or incorrectly analyzes the returned outputs. This categorization helps to identify the main sources of failure and guides future improvements in tool-invoked reasoning.

### A.5 Prompt

### A.6 More Cases

We show more cases in Figure[11](https://arxiv.org/html/2511.05271v2#A1.F11),[12](https://arxiv.org/html/2511.05271v2#A1.F12).

Figure: Figure 11: Case Study 1.
Refer to caption: x10.png

Figure: Figure 12: Case Study 2.
Refer to caption: x11.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: - How robust is DeepEyesV2’s tool invocation under noisy or adversarial tool outputs (e.g., failed code execution or irrelevant search results)?
- How does DeepEyesV2 compare to closed-source agentic systems like GPT-4o with “thinking with images” capabilities under controlled conditions?
- Is there any mechanism to control when not to invoke a tool to reduce unnecessary computational overhead?
soundness: 3
strengths: - The work clearly articulates the challenge of building agentic multimodal models and distinguishes itself from prior “thinking-with-image” paradigms by integrating multiple heterogeneous tools (code + search) in a unified reasoning loop.
- The proposed two-stage pipeline (cold-start + RL) is well-motivated and empirically justified, addressing the instability of direct RL for tool learning.
- DeepEyesV2 consistently outperforms both general-purpose and tool-augmented baselines, often matching or exceeding larger models in accuracy.
- The behavioral study of tool distribution and adaptive invocation provides convincing evidence that the model learns non-trivial reasoning strategies rather than merely following scripted patterns.
summary: The paper introduces DeepEyesV2, an agentic multimodal model designed to actively use external tools—such as code execution and web search—within its reasoning process. Unlike conventional multimodal large language models that passively interpret visual and textual inputs, DeepEyesV2 integrates tool invocation into a closed reasoning loop.
weaknesses: - The success of the approach relies heavily on the curated cold-start dataset. Details about data sources, annotation quality, and scalability are somewhat underexplored.
- Most benchmarks are vision-centric. It remains unclear how well the approach generalizes to other modalities such as audio, video, or 3D tasks.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: 1. Can you provide ablation studies showing performance gains from RL versus SFT only, and analyze the primary failure modes in tool invocation?
2. What are the quantitative metrics on tool call frequency, success rates, and computational overhead (training time and inference latency)?
3. Can the model generalize to new tools not seen during training, and if so, how much additional data is required?
4. What specific reward design prevents reward hacking, and what makes multimodal tool use fundamentally different from text-only approaches?
soundness: 2
strengths: 1. Good presentaion demonstrates task-adaptive tool invocation (image ops for perception, computation for reasoning)
Practical two-stage training approach (cold-start SFT + RL refinement)
2. Task-Adaptive Behavior: The finding that models learn to selectively invoke different tools based on task requirements (image operations for perception, computation for reasoning) is interesting and suggests genuine understanding rather than blind tool use.
summary: This paper introduces DeepEyesV2, an agentic multimodal model that extends beyond traditional perception and reasoning capabilities to actively invoke external tools such as code execution environments and web search APIs. The authors observe that direct reinforcement learning alone fails to induce robust tool-use behavior, motivating a two-stage training pipeline: (1) a cold-start stage using supervised fine-tuning to establish initial tool-use patterns, and (2) a reinforcement learning stage to refine tool invocation strategies. The paper curates a diverse, moderately challenging training dataset with examples where tool use is beneficial, and validates DeepEyesV2 across multiple benchmarks including visual understanding, mathematical reasoning, and search-intensive tasks. A key finding is that DeepEyesV2 exhibits task-adaptive tool invocation, selectively using image operations for perception tasks and numerical computations for reasoning tasks, with reinforcement learning enabling complex tool combinations.
weaknesses: 1. Insufficient analysis of what RL learns: The paper lacks detailed examination of how tool-use patterns evolve during training, what new behaviors emerge, and when the model makes mistakes in tool invocation. Learning curves and failure analysis would strengthen the claims.
2. Inadequate efficiency analysis: Missing quantitative data on tool call frequency, success rates, and computational overhead. 
3. Unclear generalization to novel tools: Evaluation limited to a fixed tool set. Whether the model can adapt to new tools or requires retraining remains unexplored.
4. Limited novelty in training paradigm: The two-stage SFT + RL approach is standard in LLM literature. The paper needs clearer articulation of what makes multimodal tool use fundamentally different or novel techniques beyond established methods.

---

confidence: 3
contribution: 3
initial_rating: 4
presentation: 4
questions: See weaknesses.
soundness: 4
strengths: - This paper is well written and easy to follow.
- The paper goes far beyond reporting final scores by providing deep insights into the learning dynamics.
  - A discovery of "Adaptive Thinking", where the SFT model over-relies on tools, and the RL stage teaches it the efficiency of when not to use tools .
- The performance is well, the authors validate DeepEyesV2 across a wide and diverse range of benchmarks, covering real-world understanding, mathematical reasoning, and search-intensive tasks.
summary: This paper introduces DeepEyesV2, an agentic multimodal model designed to actively invoke external tools like code execution and web search and integrate their outputs into its reasoning loop. The authors first demonstrate that direct reinforcement learning (RL) alone fails to teach robust tool use, often leading to reward hacking. To solve this, they propose a two-stage training pipeline: (1) a "cold-start" supervised fine-tuning (SFT) stage using a carefully curated dataset of difficult, tool-beneficial examples to establish reliable tool-use patterns , and (2) an RL stage to refine and adapt these skills. The experimental results showing DeepEyesV2 outperforms existing models on real-world understanding, mathematical reasoning, and search benchmarks. The analysis also reveals that the model learns task-adaptive tool invocation (e.g., image operations for perception, computation for reasoning) and that RL enables more complex, context-aware tool combinations and efficiency.
weaknesses: - A substantive weakness of the paper is its limited methodological novelty, as its core SFT + RL two-stage training paradigm and reasoning CoT data curation are well-established approaches for reasoning-based models, making the work feel more like a high-quality technical report than a novel research contribution.
- The paper correctly observes that the SFT model "over-relies on tools" but fails to investigate if this is merely a statistical artifact of the cold-start data's high tool-calling density. More critically, the claim that RL fosters "adaptive efficiency" by decreasing tool use is presented as a raw observation without explaining the underlying mechanism, especially since the simple reward function ($R = R_{acc} + R_{format}$) provides no explicit incentive for this emergent efficiency.
- While the paper provides strong analyses and experiments, it is questionable whether supervising only the correctness of the final answer can effectively ensure the accuracy and relevance of the tool-use process itself. For instance, if the model invokes an irrelevant or unnecessary tool but still manages to produce the correct final answer, it receives a high reward, indistinguishable from that of a model that used tools logically. This makes the reasoning process uninterpretable and unreliable.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Will the curated dataset be released? If not, the results would be hard to reproduce since this "carefully curated training corpus" is fundamental.

2. SerpAPI was used for image search, but it's unclear if any API was used for text search. Does it simply use Google search? If so, did you encounter rate limiting?
soundness: 4
strengths: 1. Extensive experiments using RL alone to post-train Qwen2.5-VL and the thorough analysis are interesting and insightful.

2. Comprehensive evaluation on 8 benchmarks and analysis. Interesting and insightful comparison between pre-RL and post-RL in terms of tool use patterns.

3. Strong results across benchmarks.
summary: This work presents a sound and well-executed study on building an agentic MLLM using SFT and RL, unifying code execution and web search. Through extensive experiments, the authors show that their work DeepEyesV2 presents significant gains over prior work. The analysis and ablation studies provided valuable insights into why direct RL on a base model doesn't elicit robust tool-use.
weaknesses: 1. Lack of comparison with a concurrent work WebWatcher, whose core methodologies seem very similar to this work. WebWatcher's performance is cited in this work but its methodological similarities are not discussed.

2. Unclear how much of the gain on search benchmarks comes from the use of SerpAPI. Do baselines and this work use the same set of search APIs?


Decision: Accept (Poster)