Towards Personalized Deep Research: Benchmarks and Evaluations
Authors: name: Yuan Liang; name: Jiaxian Li; name: Yuqing Wang; name: Piaohong Wang; name: Motong Tian; name: Pai Liu; name: Shuofei Qiao; name: Runnan Fang; name: He Zhu; name: Ge Zhang; name: Minghao Liu; name: Yuchen Eleanor Jiang; name: Ningyu Zhang; name: Wangchunshu Zhou
Published: 2025-09-29 17:39:17

Abstract: Deep Research Agents (DRAs) can autonomously conduct complex investigations
and generate comprehensive reports, demonstrating strong real-world potential.
However, existing evaluations mostly rely on close-ended benchmarks, while
open-ended deep research benchmarks remain scarce and typically neglect
personalized scenarios. To bridge this gap, we introduce Personalized Deep
Research Bench, the first benchmark for evaluating personalization in DRAs. It
pairs 50 diverse research tasks across 10 domains with 25 authentic user
profiles that combine structured persona attributes with dynamic real-world
contexts, yielding 250 realistic user-task queries. To assess system
performance, we propose the PQR Evaluation Framework, which jointly measures
(P) Personalization Alignment, (Q) Content Quality, and (R) Factual
Reliability. Our experiments on a range of systems highlight current
capabilities and limitations in handling personalized deep research. This work
establishes a rigorous foundation for developing and evaluating the next
generation of truly personalized AI research assistants.