xbench: Tracking Agents Productivity Scaling with Profession-Aligned
  Real-World Evaluations
Authors: name: Kaiyuan Chen; name: Yixin Ren; name: Yang Liu; name: Xiaobo Hu; name: Haotong Tian; name: Tianbao Xie; name: Fangfu Liu; name: Haoye Zhang; name: Hongzhang Liu; name: Yuan Gong; name: Chen Sun; name: Han Hou; name: Hui Yang; name: James Pan; name: Jianan Lou; name: Jiayi Mao; name: Jizheng Liu; name: Jinpeng Li; name: Kangyi Liu; name: Kenkun Liu; name: Rui Wang; name: Run Li; name: Tong Niu; name: Wenlong Zhang; name: Wenqi Yan; name: Xuanzheng Wang; name: Yuchen Zhang; name: Yi-Hsin Hung; name: Yuan Jiang; name: Zexuan Liu; name: Zihan Yin; name: Zijian Ma; name: Zhiwen Mo
Published: 2025-06-16 16:16:14

Abstract: We introduce xbench, a dynamic, profession-aligned evaluation suite designed
to bridge the gap between AI agent capabilities and real-world productivity.
While existing benchmarks often focus on isolated technical skills, they may
not accurately reflect the economic value agents deliver in professional
settings. To address this, xbench targets commercially significant domains with
evaluation tasks defined by industry professionals. Our framework creates
metrics that strongly correlate with productivity value, enables prediction of
Technology-Market Fit (TMF), and facilitates tracking of product capabilities
over time. As our initial implementations, we present two benchmarks:
Recruitment and Marketing. For Recruitment, we collect 50 tasks from real-world
headhunting business scenarios to evaluate agents' abilities in company
mapping, information retrieval, and talent sourcing. For Marketing, we assess
agents' ability to match influencers with advertiser needs, evaluating their
performance across 50 advertiser requirements using a curated pool of 836
candidate influencers. We present initial evaluation results for leading
contemporary agents, establishing a baseline for these professional domains.
Our continuously updated evalsets and evaluations are available at
https://xbench.org.