DR^{3}-Eval: Towards Realistic and Reproducible Deep Research Evaluation
Authors: name: Qianqian Xie, name: Qingheng Xiong, name: He Zhu, name: Tiantian Xia, name: Xueming Han, name: Fanyu Meng, name: Jiakai Wang, name: Zhiqi Bai, name: Chengkang Jiang, name: Zhaohui Wang, name: Yubin Guo, name: Yuqing Wen, name: Jiayang Mao, name: Zijie Zhang, name: Shihao Li, name: Yanghai Wang, name: Yuxiang Ren, name: Junlan Feng, name: Jiaheng Liu
Organization: NJU-LINK Lab

Deep Research Agents (DRAs) aim to solve complex, long-horizon research tasks involving planning, retrieval, multimodal understanding, and report generation, yet their evaluation remains challenging due to dynamic web environments and ambiguous task definitions. We propose DR^{3}-Eval, a realistic and reproducible benchmark for evaluating deep research agents on multimodal, multi-file report generation. DR^{3}-Eval is constructed from authentic user-provided materials and paired with a per-task static research sandbox corpus that simulates open-web complexity while remaining fully verifiable, containing supportive documents, distractors, and noise. Moreover, we introduce a multi-dimensional evaluation framework measuring Information Recall, Factual Accuracy, Citation Coverage, Instruction Following, and Depth Quality, and validate its alignment with human judgments. Experiments with our developed multi-agent system DR^{3}-Agent based on multiple state-of-the-art language models demonstrate that DR^{3}-Eval is highly challenging and reveals critical failure modes in retrieval robustness and hallucination control. Our code and data are publicly available.