RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
Authors: name: Tianxing Chen, name: Yue Chen, name: Zixuan Li, name: Junyuan Tang, name: Kailun Su, name: Haoran Lu, name: Weijie Wan, name: Baijun Chen, name: Songling Liu, name: Haowen Yan, name: Honghao Su, name: Zhiyang Dou, name: Kaixuan Wang, name: Dandan Zhang, name: Yunze Liu, name: Yan Qin, name: Qiwei Liang, name: Qiwei Wu, name: Zijian Lin, name: Wenwei Lin, name: Yuran Wang, name: Minghua He, name: Tianshu Wu, name: Ruihai Wu, name: Jingquan Zhou, name: Kai-Chong Lei, name: Haibao Yu, name: Yuanfeng Ji, name: Weiyang Jin, name: Guanyu Lin, name: Xiaofan Li, name: Qi Xiong, name: Renjing Xu, name: Zhongyu Li, name: Wenhao Chai, name: Enze Xie, name: Ziwei Wang, name: Yao Mu, name: Hao Dong, name: Wojciech Matusik, name: Mingyu Ding, name: Wenbo Ding, name: Ping Luo, name: Masayoshi Tomizuka
Organization: RoboDojo-Benchmark

Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real-world evaluation is costly, time-consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real-world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo-benchmark.com/.