ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research
Authors: name: Wanghan Xu, name: Shuo Li, name: Tianlin Ye, name: Qinglong Cao, name: Yixin Chen, name: Hengjian Gao, name: Yiheng Wang, name: Qi Li, name: Kun Li, name: Sheng Xu, name: Shengdu Chai, name: Fangchen Yu, name: Xiangyu Zhao, name: Zhangrui Zhao, name: Weijie Ma, name: Zijie Guo, name: Haoyu Zhou, name: Haoxiang Yin, name: Lixue Cheng, name: Chaofan Hu, name: Haoxuan Li, name: Lu Mi, name: Xuxuan Xie, name: Yifan Zhou, name: Ruizhe Chen, name: Zhiwang Zhou, name: Xingjian Guo, name: Yuhao Zhou, name: Xuming He, name: Shengyuan Xu, name: Xinyu Gu, name: Jiamin Wu, name: Mianxin Liu, name: Chunfeng Song, name: Fenghua Ling, name: Dongzhan Zhou, name: Shixiang Tang, name: Yuqiang Li, name: Mao Su, name: Peng Ye, name: Siqi Sun, name: Bin Wang, name: Xue Yang, name: Zhenfei Yin, name: Tianfan Fu, name: Guangtao Zhai, name: Wanli Ouyang, name: Bo Zhang, name: Lei Bai, name: Wenlong Zhang
Organization: Intern Science

AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation. Expert-curated multimodal rubrics decompose the target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery while leaving room for new discovery. We evaluate seven autonomous research (auto-research) agents under a unified protocol and seventeen native LLMs through the lightweight ResearchHarness. Current systems remain far from reliable re-discovery: the strongest autonomous agent, Claude Code, averages 21.5, and the strongest ResearchHarness LLM, Claude-Opus-4.7, averages 20.7, with an LLM frontier mean of only 26.5. Error analysis shows that failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core. ResearchClawBench provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.