AcademiClaw: When Students Set Challenges for AI Agents
Authors: name: Junjie Yu, name: Pengrui Lu, name: Weiye Si, name: Hongliang Lu, name: Jiabao Wu, name: Kaiwen Tao, name: Kun Wang, name: Lingyu Yang, name: Qiran Zhang, name: Xiuting Guo, name: Xuanyu Wang, name: Yang Wang, name: Yanjie Wang, name: Yi Yang, name: Zijian Hu, name: Ziyi Yang, name: Zonghan Zhou, name: Binghao Qiang, name: Borui Zhang, name: Chenning Li, name: Enchang Zhang, name: Feifan Chen, name: Feng Jian, name: Fengyin Sun, name: Hao Qiu, name: Hao Zheng, name: Haoran Zhu, name: Hongyu Liu, name: Jianbin Deng, name: Jiaxin Song, name: Jiaying Chi, name: Jiayou Shi, name: Jie Fang, name: Jinghui Zhong, name: Jingyu Zhou, name: Jinze Li, name: Junfeng Yi, name: Junyan Yu, name: Junzhi Xue, name: Ni Song, name: Pengyi Chen, name: Qi Chen, name: Quansheng Li, name: Rui Tao, name: Shenghai Gong, name: Shenhang Lu, name: Tianqi Shen, name: Tianxiang Zhu, name: Tiehan Kang, name: Tingyu Li, name: Wendi Wu, name: Xiao Shen, name: Xiao Zhou, name: Xiaotao Zhang, name: Xinrong Li, name: Xuankun Yang, name: Xun Zhang, name: Yan Li, name: Ye Lu, name: Yi Wang, name: Yibo Zhou, name: Yichi Zhang, name: Yihao Sun, name: Yijun Huang, name: Yixin Zhu, name: Yixuan Wu, name: Yuchen Sun, name: Yue Wu, name: Yuheng Sun, name: Yukun Li, name: Yutian Tu, name: Yuxuan Qin, name: Yuzhuo Wu, name: Zeyu Li, name: Zhengyu Lou, name: Zhenning Ran, name: Zizhu He, name: Pengfei Liu

Benchmarks within the OpenClaw ecosystem have thus far evaluated exclusively assistant-level tasks, leaving the academic-level capabilities of OpenClaw largely unexamined. We introduce AcademiClaw, a bilingual benchmark of 80 complex, long-horizon tasks sourced directly from university students' real academic workflows -- homework, research projects, competitions, and personal projects -- that they found current AI agents unable to solve effectively. Curated from 230 student-submitted candidates through rigorous expert review, the final task set spans 25+ professional domains, ranging from olympiad-level mathematics and linguistics problems to GPU-intensive reinforcement learning and full-stack system debugging, with 16 tasks requiring CUDA GPU execution. Each task executes in an isolated Docker sandbox and is scored on task completion by multi-dimensional rubrics combining six complementary techniques, with an independent five-category safety audit providing additional behavioral analysis. Experiments on six frontier models show that even the best achieves only a 55\% pass rate. Further analysis uncovers sharp capability boundaries across task domains, divergent behavioral strategies among models, and a disconnect between token consumption and output quality, providing fine-grained diagnostic signals beyond what aggregate metrics reveal. We hope that AcademiClaw and its open-sourced data and code can serve as a useful resource for the OpenClaw community, driving progress toward agents that are more capable and versatile across the full breadth of real-world academic demands. All data and code are available at https://github.com/GAIR-NLP/AcademiClaw.