Title: Dr.Mi-Bench: A Modular-integrated Benchmark for Scientific Deep Research Agent
Primary area: datasets and benchmarks
Keywords: LLM, Agent, Benchmark, Deep Research

Abstract: The explosive growth in academic literature necessitates automated deep research (DR) agents, yet their evaluation remains a significant challenge. First, existing benchmarks often focus narrowly on retrieval while neglecting high-level planning and reasoning. Second, existing benchmarks favor general domains over the scientific domains that are the core application for DR agents. To address these gaps, we introduce **Dr.Mi-Bench**, a **M**odular-**i**ntegrated **bench**mark for scientific **DR** agents. Grounded in academic literature, our benchmark uses a human-annotated dataset of **200 instances** across **10 scientific domains**, including both research and review papers. Besides, we also propose a **M**odular-**i**ntegrated **Eval**uation Paradigm for **DR** Agents (**Dr.Mi-Eval**), a novel modular-integrated evaluation paradigm, which leverages the rich structure of academic papers to assess the core competencies of planning, retrieval, and reasoning through two complementary modes: an **end-to-end** evaluation for DR agents and an **isolated** evaluation for foundational LLMs as potential backbones. Experimental results reveal a fragmented performance landscape: agents exhibit specialized strengths but share critical weaknesses, most notably in performing the multi-source retrieval required for review-style tasks and performing consistently across diverse scientific fields. Moreover, improving high-level planning capability is the crucial factor for unlocking the reasoning potential of foundational LLMs as backbones. By exposing these actionable failure modes, Dr.Mi-Bench provides a diagnostic tool to guide the development of more reliable academic research assistants.

TL;DR: We formalize the workflow of deep research agents and propose Dr.Mi-Bench, a modular-integrated deep research benchmark for scientific deep research agent.