BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark
Authors: Nikita Chernyadev
Nicholas Backshall
Xiao Ma
Yunfan Lu
Younggyo Seo
Stephen James
Venue: CoRL 2024 (2024)
Keywords: Bi-Manual Manipulation
Mobile Manipulation
Benchmark
TL;DR: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark & Learning Environment, for Reinforcement and Imitation Learning

Abstract: We introduce BiGym, a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching to complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports a variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views. To validate the usability of BiGym, we thoroughly benchmark the state-of-the-art imitation learning algorithms and demo-driven reinforcement learning algorithms within the environment and discuss the future opportunities.

BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark
Nikita Chernyadev" Nicholas Backshall" Xiao Ma"
Yunfan Lu Younggyo Seo Stephen James
Dyson Robot Learning Lab
We introduce BiGym, a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views. To validate the usability of BiGym, we thoroughly benchmark the state-of offte-of-the-ar imitation learning algorithms and demo-drives reinforcement learning algorithms within the environment and discuss the future opportunities. Project website: pettp::/chhrnyeee -iiggbtthhh io/bigym/
Abstract:
Keywords: Bi-Manual Manipulation, Mobile Manipulation, Benchmark
Erynn 1PI
J
Figure I: BiGym focuses on mobile manipulation with home assistance humanoids. We provide 40 tasks ranging from simple mobile target reaching to complex dishwasher manipulations. Specifically, each task comes with demonstrations recorded by human demonstrators and can be used to benchmark both imitation learning and reinforcement learning algorithms.
"quua contributions
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

Introduction
Machine learning benchmarks are of significant importance for measuring and understanding the progress of research algorithms. Examples of notable benchmarks include ImageNet [1] for image understanding, KITTI [2] for autonomous driving, and SQuAD for language- question answer- ing [3]. In robotics, prior benchmarks have greatly reduced the cost of iterating and developing algorithms. Examples include OpenAI Gym [4], DeepMind Control Suite [5], and MetaWorke [6]. However, all of these benchmarks focus on pure reinforcement learning (RL) with dense shaped rewards, limiting their application in long-horizon manipulation tasks where accurately defining reward functions is challenging.
While crafting reward is difficult, obtaining expert trajectories, such as those from human demon- strations, is relatively straightforward. This advantage has boosted the popularity of demonstration- driven methods within the robot learning community, manifesting as both imitation learning (IL.) [7, 8, 9, 10, 11, 12, 13] and demo-driven RL [14, 15, 16, 17]. To support the research of building demo-driven agents, RLBench [18] was created with a wide range of single-arm fixed manipulation tasks with expert demonstrations generated by motion planners. Using motion planners allows RLBench to generate a large amount of demonstration data purely in simulation, however, the output trajectories are often either unnatural due to the inherent randomness in sampling-ban planners, or have an unrealistically narrow trajectory distribution when compared noisy real-world human demonstrations. Moreover, the community-progress is beginning to plateau on large number of RLBBench tasks, in particular with recent 3D next-best pose agents [10, 11, 16, 17, 19, 20, 21].
These limitation highlights the need for a new benchmark which provides: (1) more natural demon- strations like those seen in real-world robot data and (2) a set of new challenging tasks where state-of-the-art algorithms are likely perform poorly. To this end, we present BiGym, a demo- driven mobile bi-manual manipulation benchmark with a humanoid embodiment. BiGym covers 40 visual mobile manipulation tasks, ranging from simple tasks like moving plates between drainers to interacting with articulated objects such as dishwashers (see Figure 1). Unlike prior humanoid benchmarks [22, 23] that focus only on RL with dense shaped reward functions, which may lead to undesired behaviors [24], we provide for each task only sparse rewards but with 50 demonstrations, allowing evaluation of both IL and RL algorithms. Additionally, compared to previous benchmarks that rely on expert demonstrations generated by planners [18], the human-collected demonstrations in BiGym are much more realistic and multi-modal (see Figure 3), better reflecting the trajectories of real-robot movements. Finally, BiGym considers locomotion and mobile bi-manual manipulation challenges separately; specifically, BiGym allows users to switch between the whole-body mode, which jointly considers locomotion and manipulation, and a bi-manual mode, which focuses on upper-body mobile manipulation while controlling the lower body with fixed controllers (see Fig- ure 2). This separation of action modes enables researchers to better investigate and benchmark the capability of various algorithms with different focuses, i.e., locomotion control and mobile bi-manual manipulation solely. Code for BiGym is available on our project website.
2 Related Works
With the rapid progress in robot learning algorithms, the role of benchmarks has become crucial as a tool understand the effect of various algorithmic design choices and compare algorithms in the same setup. There have been a series of benchmarks for complex manipulation tasks. Most of the existing benchmarks consider single-arm manipulation scenario. The IKEA furniture assembly environment [25], BEHAVIOUR [26], and Habitat [27] provide a wide range of long-horizon household object (mobile) manipulation tasks. They emphasise long-horizon planning capabilities but employ abstract low-level actions that overlook physical interactions. Some benchmarks focus on more realistic settings with physics interactions [18, 28, 29, 30, 31, 32] and mainly support training RL agents. Notably, James al. [18] provide APIS to generate expert demonstrations with motion planners. As result, it is widely used for benchmarking IL [10, 11, 21] and demo-driven RL algorithms [16, 17, 19, 20). Concurrent to work, RoboCasa [33] constructs realistic environments

---

Caml
Cam3
Cam2
Demo
VR
Body
Bi-Manual
BiGym Framework
(b) Multi-Modal Demonstrations of BiGym
Figure 2: (a) BiGym builds upon Unitree HI robot with RGB-D cameras the head, left wrist, and right wrist. We collect human demonstrations by tele-operating with VR devices. BiGym allows users to control the humanoid in either whole-body mode, which considers both locomotion and manipulation, or the bi- bi-manual mode, which simplifies the locomotion with a predefined controller for the lower-body. (b) BiGym provides human-collected multi-modal demonstrations for tasks, e.g., in reach. targht.ttttttmmmmc multi modal, the agent can finish the task by reaching the target with either the left right hand.
with human demonstrations, but only for single-arm tasks. Unlike these benchmarks that only consider a single-arm manipulation setup, BiGym provides a variety mobile bi-manual manipulation tasks.
Bi-manual manipulation benchmarks consider controlling two arms or floating dexterous hands interact with the environment [34, 35, 36). More recently, benchmarks for humanoid robots have been introduced. For instance, LocoMujoco [22] focuses on locomotion control of different types of humanoids with two arms, but does not include manipulation tasks. As a concurrent work, HumanoidBench [23] focuses on benchmarking RL algorithms with task-specific shaped rewards on 15 locomotion and 12 manipulation tasks. In contrast, BiGym supports benchmarking both II. and RL algorithms by providing 40 tasks with human-collecter demonstrations. These demonstrations exhibit realistic noisy trajectories that cover wider data distribution compared planner- generated demonstrations [18], thus enabling the evaluation that better reflects the real-world performance of algorithms. We provide detailed comparison across benchmarks in Table 1.
BiGym
We present BiGym, demo-driven mobile bi-manual manipulation benchmark. BiGym consists of 40 mobile bi-manual manipulation tasks, ranging from simple target reaching to complex dishwasher cleaning tasks. To evaluate IL and demo-driven RL algorithms in realistic scenario with noisy, multi- modal demonstrations, BiGym provides numan-collected demonstrations for all tasks. We describe which challenges BiGym presents (see Section 3.1), the simulation platform (see Section 3.2), and details on human demonstration datasets (see Section 3.3) and tasks (see Section 3.4).
3.1 Challenges of BiGym
We design BiGym to pose the following challenges:
Partial Observability. BiGym tasks are formulated a partially observable Markov decision process (POMDP) [37] with discrete time 2...... .2..... ,, continuous action a, hybrid-observation. which include visual observations o, and robot levelevel states s,, and reward rą- To achieve the task, the agent is required to learn belief b,, i.c., distribution over the environment states, given past partial

---

Benchmark
Mobile
Arms
Action Mode
Task Horizon
Demonstrations
Human Demo
# Tasks
MetaWorld [6]
X
J/ER
500
X
X
50
RLBench [18]
x
J/EE
100-
X
106
RoboSuite [41]
1/2
J/EE
9
LocoMujoco [22]
V
100-500 500
X
27
HumanoidBench [23]
/
2
500 1000
X
X
27
BiGym (ours)
J/J FB
1000- 7000
40
Table I: Comparison with widely used henchmarks. J: Joint position action mode that controls all the joint angles of the robot. EE: and-Effector action mode with low-level planners. FB: Floating base. "Although LocoMujoco considers humanoids, it only studies the locomotion tasks, rather than manipulation.
observations st)=1 and actions far 1 which is non-trivial given the curse of history and the curse of dimensionality of POMDPs [38, 39].
Complex Task Space. Mobile bi-manual manipulation introduces a much more complex task space compared to fixed single-arm settings. This is because, with the presence of dual arms and the mobility of the agent, there may exist multiple ways of solving single task. For example, to grasp cup the side of table and put it into the closed drawer, the robot can consider several different ways: (I) pick up the cup with one hand, pull the drawer with the other hand, and put the cup into the drawer; (2) pull the drawer with one hand, pick up the cup with the same hand, and put the cup into the drawer. The mobility of the robot also allows navigating to the target with different routes. result, even if a mobile robot has a comparable number of degrees of freedom to a fixed robot, the task space is highly multi-modal and significantly more complex. To test the capabilities of robot learning algorithms in such scenarios, BiGym offers wide range of tasks featuring complex task spaces and human-collected demonstrations with diverse modalities.
Long Mask-Horizon and Sparse Reward. In common household scenarios, the agent will need to perform long-horizon tasks, which are composed of a series of sub-tasks, which require both task-level planning and low-level motion planning. For example, to load plates into the dishwasher, the agent should correctly locate the plates, open the dishwasher, pull out the trays, accurately put the plates the trays, and finally, close the door. In addition, due to the complex task space, it is extremely difficult to properly define reward functions. If one can define such rewards, the agent may easily fall into local minimas by learning to exploit such sub-optimal shaped rewards. Thus, BiGym instead provides a set of sparsely-rewarde tasks along with noisy collanee demonstrations, to evaluate the performance II. and demo-driven RI. algorithms
more realistic setup.
Realistic Multi-Modal Demonstrations. In contrast to prior benchmarks that generate expert demon- strations with motion planners [18, 40), BiGym provides human-collected demonstrations that are highly noisy and multi-modal. Specifically, we design BiGym tasks to be solvable in multiple ways to induce a multi-modal demonstration distribution. For instance, in reach_targe target parrgt_multtt modal task, reaching the target can be achieved with either left or right hand, as shown in Figure 2(b). This design enables us to evaluate the capabilities of robot learning algorithms using more realistic demonstrations, rather than synthetic demonstrations consisting of unnatural trajectories (see Figure 3).
3.2 Simulation Platform
We build BiGym simulation environments based MuJoCo [42] (see Figure 2(a) for the illustration of the whole system). We brief the core design choices below and more details are in Appendix A.
Humanoid Body Configurations. We implement the platform with the Unitree HI robot given its publicly available model [43]. As the original H1 comes with no grippers, we attach an additional Robotiq 2F-85 gripper with an actuated wrist joint each arm. We note that it is easy to swap the parallel gripper with other dexterous manipulators, but we leave it for future study observe that parallel grippers are sufficient for current tasks.

---

TOOCHL mOpLLL moco
Wall www.suoooodddoo opon
(RACT Jurger
[MC Joate wwwpprroggoooooooooo cowd
(a) BiGym
(b) RLBench
Figure 3: Visualisations of arm wrist position distributions BiGym and RLBench. We visualise the wrist positions of both BiGym human collected trajectories on the reach_tar tarptt.rgettull aegee_mmlii_mme wall.cupboard.com open task, well the RLBench reach.targes and the put_ knife. snife_on_chooee on.chopinng board task. The trajectories BiGym are noisy, multi-modal but smooth in general, but the motion planner generated trajectories RLBBench are either straight lines or unnatural.
and the
from bigyn. enve. resch_.reeccharrrgge import ReachTarget
from bigyn. aet.aactiionmcec. import JointPositionActionnee from demonstrations demo_store import DemoStore from demonstrations utils import Metadata
ReachTarget action_mode-il aaaodd==oiinPeetttoootttoooooee
floating brue absolute-True,
11
12
semo_store DemoStore. gow.eeeeoooollloooooooo
demos demo_store... gem_ (Ketadata en
14
agent Agent agent ingest (demos)
16
18
obs, training. splsode_length - None, 100, ENV_TIME_LIKIT range (taaiiiinnnin) if episode_length =- 0: abs action agent obs, reward, terminated, truncated, info env atv.step(actionn agent add_to_buftee agent (p env.close()
19
21
env. env.reset()
23
reward, terminated, truncated, info)
25
26
Figure 4: Example usage of BiGym Eavironment for training reinforcement learning agent. Demonstra- tions are pulled from a remote store and cached locally. Users can also customise their action modes or use the off-thc-shelf JointPosit ietittionniiionii modes with either absolute delta actions.
with flags to switch between the bi-manual or whole-body action
Observation Spaces. As shown in Figure 2(a), we mount three cameras on the robot: the forehead, the left wrist, and the right wrist. Each camera can generate both RGB and depth observations, which supports a diverse types of algorithms which use either type of observation. As a result, the observation space is defined as O = {Inead, Tieft Iright Dread: Dieft Dright Sproprio where I is the RGB image, D is the depth image, and Sproprio is the proprioception state of the robot. Additional observations, e.g., the gripper poses and robot poses, can also be easily obtained if required.
Action Modes. It remains unclear to the robotics community what action modes are the best for complex embodiment in mobile bi-manual manipulation tasks. Thus in BiGym, we provide flexible configurations for users to customise the action modes they want to use, and leave the choice to the users. Specifically, we provide two off-the-sheli action modes: the whole-hody action mode and the bi-manual action mode, with either delta absolute actions. For the whole-body action mode, allow full control of the humanoid joints. This allows studying whole-body manipulation with locomotion. With the bi-manual action mode, we simplify the control by treating the lower-bodd body of the humanoid as an omni-directional floating base controlled by classic controllers. In this case, we can focus on upper-body bi-manual mobile manipulation skills.
Scenes. The scenes in BiGym are created from MuloCo MICF models using custom object- oriented API based dm_control [5]. All MJCF models provided in BiGym were created from publicly available 3D models. Many other 3D models were processed to be used in BiGym: meshes were decimated to reduce the total number of polygons, moving parts of articulated objects were
5

---

separated, required joints and actuators were added, and convex collision meshes were created. Currently, BiGym provides 46 high-quality assets that could be reused to facilitate the creation of new environments. In addition to rigid object models, BiGym offers a set of articulated models, such as a dishwasher and customisable kitchen modules.
APL The interface for the benchmark follows the standard Gymnasium APIs [44] for training IL and RL agents. A typical workflow of RL agents training is demonstrated in Figure 4.
3.3 BiGym Human Demonstration Datasets
One of the key design choices in BiGym is to provide a fixed number of human-collected demonstra- tions for each task. This allows BiGym benchmark to better reflect the challenges of real-world robot learning, which involves dealing with noisy, multi-modal demonstrations in contrast to synthetic demonstrations generated by motion planners [18]. We describe BiGym's 's human demonstration dataset collection and management system as below and more details are in the appendix.
Demo Collection Pipeline. We use VR² collect demonstrations by virtually tele-operating the HI robot in simulation (using the 6 DoF poses of the headset and controllers). The headset pose controls the position and orientation of the HI body, and the 6 DoF poses of the controllers used to operate the arms. We solve the inverse kinematics for each arm and then reorient the grippers to match the orientation of the respective controller.
Down-Sampling Demonstrations. The control frequency of demonstrations can significantly affect the horizon length of tasks and the ability to capture fine grained control, both of which can greatly affect task success [45]. To provide users with the flexibility in selecting action frequencies, capture demonstrations at the frequency of physics calculation (500 Hz) and provide the functionality to down-sample the demonstrations to a desired frequency (20-500 Hz).
Demonstration Management. To minimise the use of storage for saving demonstrations, we save lightweight demonstrations that only contain control signals (actions). We then provide a tool that enables users pull such lightweight demonstrations and replay them obtain full demonstrations with user-specified observations such RGB depth images. These demonstrations are cached in users' local storage so that users do not have to re-download replay the same demonstrations again.
Tools. BiGym provides two tools for demonstration management. The demo tool provides various demonstration-reltte functionalities such as downloading, deleting, verifying, replaying at different frequencies, converting, and re-recording demonstrations. The demo recorder enables users easily collect data by streamlining the process of recording demonstrations with VR.
3.4 BiGym Tasks
BiGym provides 40 high-quality tasks with human-collected demonstrations support the research on household mobile bi-manual manipulation. We describe the tasks and their configurations below.
Reach Target Tasks. Different from the standard reach target tasks in single-arm settings, we consider three variants of reach target tasks:
(1) reach_targes www.ttarreesssingle.ee The robot must use a specified wrist to reach a coloured target. modal: The robot must reach the target with either left or right wrist. This induces a multi-modal demonstration distribution of reaching the target with different hands.
(2) reach.target.mit
We expect the policy to understand this multi-modality during training.
(3) reach target duall In this task, the robot must reach two targets, one with each arm. The success criteria require both wrists to be aligned with corresponding targets. Once this criteria is met, the targets are highlighted to provide visual feedback.
Table-Top Manipulation Tasks. We then consider table-top manipulation which requires the robot to interact with rigid-body objects, e.g., plates and cups. The challenge here is that the robot should be
Valve Index VR headset. two controllers and two base stations

---

able to identify remote objects and perform manipulation tasks that require moving the base together with the arms. We introduce the following tasks:
(4) blocks: Move blocks across the table, and stack them in the target area.
move_plate: Move the plate between two draining racks.
move two plates: Move two plates simultancously from one draining rack to the other. flip-cup: Flip the cup, initially positioned upside down on the table, to an upright position.
(8) flip.cutlery: Take the cutlery from the static holder, flip it, and place it back into the holder.
In addition to simple single-object manipulation tasks, we introduce complex manipulation tasks that require interaction with articulated objects, where it is crucial to understand the object's kinematics to perform constrained motion planning. The scenarios include: Dishwasher and Kitchen Counter tasks.
Dishwasher Tasks. We consider set of tasks which require interactions with the articulated dishwasher, ranging from sliding trays to long-horizon unloading tasks:
(9) dishwasher.open: Open the dishwasher door and pull out all trays.
(10) di siwasher close: Push back all trays and close the door of the dishwasher.
(1I) di wwsswwweer.ppe..tteee
Pull out the dishwasher's 's trays with the door initially open. (12) wishwasher.cooseeeoe cwwse trays: Push the dishwasher's 's trays back with the door initially open.
Move plates from the rack to the lower tray of the dishwasher. (14) dishwasher load cups: Move cups from the table to the upper tray of the dishwasher.
(13) bishwasher__oad_pllate
(15) dishwasher_load cutlory. Move cutlery from the table holder to the dishwasher's 's cutlery basket. At the beginning of the episode, the dishwasher is open, with the lower tray pulled out. table rack. Move cups from the upper tray of the dishwasher to the table. (18) dishwasher unload cutlery: Move cutlery from the cutlery basket to tray on the table. A full task of unloading a plate: picking up the plate from dishwasher, placing this plate into the rack located in the closed wall cabinet, and closing the dishwasher and cupboard.
(16) battaasher_unllaae plates: plate: Move plates from the tray of the dishwasher
(17) wwwwwasher.uuload.oooo
(19) httpraaseer.ullod.plllteee
(20) di dishwasher unload cup long: A full task of unloading a cup: picking up the cup, placing it inside the closed wall cabinet, and closing the dishwasher and cupboard. (21) wwshwasher.unllee wwwllodddcuteryy llog: A full task of unloading a cutlery: picking up a cutlery, placing it into the cutlery tray inside the closed drawer, and closing the dishwasher and drawer.
Kitchen Counter Tasks. In addition, BiGym considers a more complex kitchen counter scenario with multiple challenging articulated objects, e.g., the cupboard, the drawer, etc. Similar to the dishwasher tasks, we provide a range of short and fong-horizon tasks as below:
(22) toen Open the top drawer of the kitchen cabinet.
(23) drawer.top.cloo close: Close the top drawer of the kitchen cabinet.
(24) drawers open all: Open all sliding drawers of the kitchen cabinet.
(25) drawers_close_all: Close all sliding drawers of the kitchen cabinet.
(26) wwwl_cupboard open: Open doors of the wall cabinet.
(27) wall_cupboard_cclsee Close doors of the wall cabinet.
(28) cupboards open all: Open all drawers and doors of the kitchen set.
(29) cupboards_cleeee cattstscclleeeeeeee aal: Close all drawers and doors of the kitchen set.
(30) take_cups: Take two cups out from the closed wall cabinet and put them on the table.
(31) put_cups: Pick up cups from the table and put them into the closed wall cabinet.
(32) box: Pick up a large box from the floor and place it on the counter.
(33) store box: Move a large box from the counter to the shelf the cabinet below.
(34) saucepan_to hob: Take the saucepan from the closed cabinet and place on the hob.
(35) store.kitchenware. Take all items from the hob and place them in the cabinet below.
(36) sandwich_toase Use the spatula to put the sandwich on the frying pan.
(37) sandwich flip: Flip the sandwich in the frying pan using the spatula.
(38) sandwich_remove Take the sandwich out of the frying pan.
(39) Store.groceria lower: Place a random set of groceries in the cabinets below the counter.
7

---

(40) store groceries upper: Place a random set of groceries in cabinets and shelves on the wall.
Reward Functions. We provide sparse rewards for all the tasks based on success detector: a reward of is given for reaching the successful criteria and otherwise. Details are available Appendix B.
4 Experiments
The contribution of the paper is BiGym. However, in this section, we aim to validate that current algorithms can attain some degree of performance on all BiGym tasks, even if minimal. To this end, we conduct experiments with both state-of-the-ar II. and demo-driven RI. algorithms. Specifically, we focus on the following seven general robot learning algorithms:
IL Algorithms. We aim to investigate how different policy representations contribute the final performance of the algorithms on BiGym, which provides highly noisy and multi-modal demonstra- tions. In pursuit of this goal, we consider the following algorithms: standard Behaviour Cloning (BC), Action Chunking Transformers (ACT) [9] which trains a transformer model [46] to predict sequence of actions, and Diffusion Policies /8] which trains diffusion model to approximate the expert action distribution. In particular, we do not benchmark against the popular 3D next-best pose agents [10, 11, 16, 17, 19, 20, 21] since they reply on heuristic- brased key-frame extraction methods which only apply to single fixed arms [17]; thus, they are currently applicable to the mobile bi-manual manipulation morphology.
RL Algorithms. We mainly consider demo-driven RL algorithms which support training with expert demonstrations. Specifically, we focus on off- aff-policy algorithms and offline RL algorithms that have demonstrated good capabilities online settings. We consider the following algorithms: DrQV2 [47], Advantage Weighted Actor-Critic (AWAC) [48], Implicit Q-Learning (IQL) [49], and Coarse-to-fiin Deep Q-Network (CQN) [50]. We note that BiGym tasks can be extremely challenging for RL algorithms due their sparse reward, partial observations, and complex dynamies. To provide reference for future studies, we provide the results of all the methods as-is with the common set of hyperparameters, instead of tuning their performance for individual BiGym tasks.
We provide experimental results and discussions in Appendix C.
5 Discussions
Limitations. Currently, BiGym has the following main limitations. Firstly, although BiGym has posed series of challenging tasks of whole-body mobile bi-manual manipulation, most tasks are still at the skill level, requiring less complex long-horizon task and motion planning, except for the long" tasks, e.g, unload cups long. To fully benchmark the capability of algorithms in an ultimate household environment, such tasks are necessary. In addition, BiGym only supports the Unitree HI robot the moment. More different embodiments will be included for future tasks.
Opportunities and Future Works. BiGym presents various future research opportunities, including but not limited to: (1) exploring better network architectures for approximating multi-modal noisy human demonstrations; (2) studying better belief estimation mechanisms for the POMDP in the mobile manipulation context; (3) investigating better collaboration modes between arms on mobile platforms; (4) whole-body motion planning which improve the efficiency and performance of mobile agents while navigating in cluttered environments.
Conclusion. We introduce BiGym, new and challenging benchmark for demo-driven mobile bi-manual manipulation. BiGym covers 40 challenging tasks of mobile bi-manual manipulation, ranging from simple target reaching to complex dishwasher manipulation tasks. Built upon the humamoid embodiment of Unitree HI robot, BiGym allows users to flexibly customise the action modes: the whole-body mode and the bi-manual mode. Furthermore, we provide multi-modal and noisy human-collected demonstrations for all BiGym tasks, exhibiting realistic trajectories compared to synthetic ones generated by motion planners. In experiments, validate the usability of BiGym by benchmarking state-of-the-art IL and RL algorithms.
8

---

Acknowledgements
Big thanks to the members of the Dyson Robot Learning Lab for discussions and infrastructure help: lain Haughton, Richie Lo, Sumit Patidar, Sridhar Sola, Mohit Shridhar, Eugene Teoh, Jafar Uruc, and Vitalis Vosylius.
References
[1] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
[2] A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recognition (CVPR), 2012.
[3] P. Rajpurkar, R. Jia, and P. Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint 1806.v8886.033888 2018.
[4] G. Brockman, V. Cheung, L. Pettersson, Schneider, J. Schulman, Tang, and W. Zaremba. Openai gym, 2016.
[5] Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, D. d. L. Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq, et al. Deepmind control suite. arXiv preprint 10Xivv/801.0000 2018 8.
[6] T. Yu, D. Quillen, Z. He, R. Julian, K. Hausman, C. Finn, and S. Levine. Meta-world: A benchmark and evaluation multi-task and meta reinforcement learning. In Conference on robot learning, pages 1094-1100. PMLR, 2020.
[7] S. James, A. J. Davison, and E. Johns. Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task. In Conference on Robot Learning, pages 334-343, PMLR, 2017.
[8] C. Chi, S. Feng. Du, Z. Xu, E. Cousineau, B. Burchfiel, and Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Proceedings of Robotics: Science and Systems (RS5), 2023.
[9] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023.
[10] M. Shridhar, L. Manuelli, and D. Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. Conference on Robot Learning, pages 785-799, PMLR, 2023.
[11] T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki. Act3d: Infinite resolution action detection transformer for robotic manipulation. arXiv preprint arXiv:2306. 17817, 2023.
[12] X. Ma, S. Patidar, I. Haughton, and S. James. Hierarchical diffusion policy for kinematics-aware multi-lask robotic manipulation. arXiv preprint arXiv:2403.03880 2024.
[13] V. Vosylius, Seo, Uruç, and S. James. Render and diffuse: Aligning image and action spaces for diffusion-based behaviour cloning. Proceedings of Robotics: Science and Systems (RSS), 2024.
[14] T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, Schaul, B. Piot, D. Horgan, Quan, A. Sendonaris, Osband, et al. Deep gelaanning from demonstrations, In Proceedings of the AAAI conference on artificial intelligence, 2018.
[15] J. Matas, S. James, and A. J. Davison. Sim-to-real reinforcement learning for deformable object manipulation. In Conference on Robot Learning, pages 734-743, PMLR, 2018.

---

[16] S. James and A. J. Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation. IEEE Robotics and Automation Letters, 7(2):1612-1619, 2022.
[17] James, Wada, T. Laidlow, and J. Davison. Coarse-to-iie q-attention: Efficient learning for visual robotic manipulation via discretisation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13739-13748, 2022.
[18] S. James, Z. Ma, D. R. Arrojo, and A. Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019 3026, 2020.
[19] S. James and P. Abbeel. Coarse-to-fine q-attention with tree expansion. arXiv preprint 12X4 2022.
[20] S. James and P. Abbeel. Coarse-to-fil q-attention with learned path ranking, arXiv preprint MXXiv:2044001571, 2022.
[21] A. Goyal, J. Xu, Y. Guo, V. Blukis, Y.-W. Chao, and D. Fox. Rvt: Robotic view transformer for 3d object manipulation. arXiv preprint arXiv:2306. 14896, 2023.
[22] Al-Hafez, G. Zhao, Peters, and Tateo. Locomujoco: comprehensive imitation learning benchmark for locomotion, arXiv preprint arXiv:2311 1.0496, 2023.
[23] C. Sferrazza, D.-M. Huang. X. Lin, Y. Lee, and P. Abbeel. Humanoidbench: Simu- lated humanoid benchmark for whole-body locomotion and manipulation. arXiv preprint a4Xiv:2403. 10506, 2024.
[24] D. Amodei, Olah, J. Steinhardt, Christiano, Schulman, and D. Mané. Concrete problems in ai safety. arXiv preprint
(666600665, 2016.
[25] Y. Lee, E. S. Hu, and J. Lim. Ikea furniture assembly environment for long-horizon complex manipulation tasks. In 202/ ieee international conference on robotics and automation (icra), pages 6343-6349, IEEE, 2021.
[26] C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, M. Lingelbach, J. Sun, et al. 1ehavior-lk: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pages 80-93, PMLR, 2023.
[27] A. Szot, A. Clegg, E. Undersander, Wijmans, Y. Zhao, Turner, N. Maestre, M. Mukadam, D. S. Chaplot, O. Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in neural information processing systems, 34:251-266 2021.
(28) O. Mees, L. Hermann, E. Rosete-Beas and W. Burgard. Calvin: A benchmark for language- conditioned policy learning for long-howe horizoo robot manipulation tasks. IEEE Robotics and Automation Letters, 7(3)7327-7734,, 2022.
[29] S. Chen, Y. Xu, C. Yu, L. Li, X. Ma, Z. Xu, and D. Hsu. Daxbench: Benchmarking deformable object manipulation with differentiable physics. arXiv preprint arXiv:2210.13066, 2022.
[30] X. Lin, Y. Wang, J. Olkin, and D. Held. Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. In Conference on Robot Learning, pages 432-443 PMLR. 2021.
[31] T. Mu, Z. Ling, F. Xiang, D. Yang, X. Li, S. Tao, Z. Huang, Z. Jia, and H. Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations, arXiv preprint arXiv:2107. 14483, 2021.
[32] M. Heo, Y. Lee, D. Lee, and J. Lim. Furniturebenc:: Reproducible real-world benchmark for long-horizon complex manipulation. arXiv preprint arXiv:2305.12822 2023.
10

---

[33] S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo. A. Joshi, A. Mandlekar, and Y. Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. In Robotics: Science and Systems (RSS), 2024.
[34] Y. Zhu, J. Wong, A. Mandlekar, R. Martin-Martin, A. Joshi, Nasiriany, and Y. Zhu, robo- suite: A modular simulation framework and benchmark for robot learning. arXiv preprini ar09v:2009.1 12293, 2020.
[35] Y. Chen, Y. Geng, F. Zhong, J. Ji, Jiang, Z. Lu, H. Dong, and Y. Yang. Bi-dexhands: Towards human-level bimanual dexterous manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[36] K. Zakka, P. Wu, L. Smith, N. Gileadi, Howell, X. B. Peng, S. Singh, Y. Tassa, Florence, A. Zeng, al. Robopianist: Dexterous piano playing with deep reinforcement learning. In 7th Annual Conference Robot Learning, 2023.
[37] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 1998.
[38] H. Kurniawati, D. Hsu, and W. S. Lee. Sarsop: Efficient point- pomdp planning by approximating optimally reachable belief spaces. In Robotics: Science and Systems, 2009.
[39] X. Ma, Karkus, D. Hsu, W. S. Lee, and N. Ye. Discriminative particle filter reinforcement learning for complex partial observations. arXiv preprint arXiv:2002.09888 2020.
[40] J. Gu, F. Xiang, X. Li, Z. Ling, X. Liu, T. Mu, Y. Tang, S. Tao, X. Wei, Y. Yao, et al. Maniskil12: unified benchmark for generalizable manipulation skills. arXiv preprint arXiv:2302.04659, 2023.
[41] Y. Zhu, J. Wong, A. Mandlekar, R. Martín-Martin, Joshi, Nasiriany, and Y. Zhu. robo- suite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009.122 12293, 2020.
[42] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026-5033, IEEE, 2012. doi: 10. (009/RROS.012..386661
[43] K. Zakka, Y. Tassa, and MuJoCo Menagerie Contributors. MuloCo Menagerie: collec- tion of high-quality simulation models for MuJoCo, 2022. URL http://github. com/ pegle-ddeeminnd/muuee mmwoo//mmaaeooeoiii..e... mmnnaeerie.
[44] M. Towers, J. K. Terry, A. Kwiatkowski, J. U. Balis, G. d. Cola, T. Deleu, M. Goulão, A. Kallinteris, A. KG, M. Krimmel, R. Perez- Vicente, A. Pierré, S. Schulhoff, J. Tai, A. T. Shen, and O. G. Younis. Gymnasium, Mar. 2023. URL 8127025.
org/record/
[45] L. X. Shi, A. Sharma, T. Z. Zhao, and C. Finn. Waypoint-based imitation learning for robotic manipulation. arXiv preprini arXiv:2307. 14.2077..336.... 2023.
[46] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, t. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[47] D. Yarats, R. Fergus, A. Lazaric, and L. Pinto. Mastering visual continuous control: Improved augmeeted reinforcement learning. arXiv preprint arXiv:2107.09645, 2021.
[48] A. Nair, A. Gupta, M. Dalal, and S. Levine. Awac: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.00 09359, 2020.

---

[49] I. Kostrikov, A. Nair, and S. Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2/ 10.06119, 2021.
[50] Y. Seo, Uruç, and S. James. Continuous control with coarse-to-fine reinforcement learning. arXiv preprint arXiv:2407.07787, 2024.
[51] A. Paszke, S. Gross, Massa, Lerer, J. Bradbury, G. Chanan, T. Killeen, Lin, N. Gimelshein, L. Antiga, al. Pytorch: An imperative style, high-ppeffrmance deep learning library. Advances in neural information processing systems, 32, 2019.
[52] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 278.2016.
[53] D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.60 2013.
[54] D. Hendrycks and K. Gimpel. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016.
[55] Ba, J. R. Kiros, and G. E. Hinton. Layer normalization. arXiv preprint 11Xiv.600.004550 2016.
[56] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[57] Y. Lu, K. Hausman, Y. Chebotar, M. Yan, E. Jang, A. Herzog, T. Xiao, A. Irpan, M. Khansari, D. Kalashnikov, et al. Aw-opt: Learning robotic skills with imitation and reinforcement at scale. arXiv preprint L..5422 2021.
12

---

A Additional Simulation Details
In this section, we provide additional details about BiGym.
Observation Spaces. For image observations, we allow users to specify the resolution of the images, where the default resolution x4 Higher resolution may allow learning better policies, but find the default value works across tasks. In the whole-body mode, the proprioception Squel Sgrip), where s 3 is the joint angle positions of robot, sqvel Re is the corresponding velocities, and Sgrip E I² the gripper opening amount of both grippers. On the contrary, the bi-manual mode greatly simplifies the locomotion by replacing the lower-body control with predefined controller, i.e., a floating base. This reduces the dimension of sib spos and sip to s bm E 18 and quel _2 Furthermore, an additional state Sbase (I, y, 2, 0) E IR is included in Sproprio indicate the position and orientation of the floating base As result, Jbm
sff state proprro E I sth
"proprio E RR44 {shm sbm $base; Sgrip)
Action Spaces. In the whole-body mode where the agent has the full control over the body, an action space Awb E R223 is defined as Awb = {Aarms; Atorso Agrip). where Aarms E 10 controls both arms, Ategs 10 controls the legs, Atorso IR. controls the main torso joints, and Agrip E g² controls the opening amount of grippers. instead of the leg joints. Therefore the action space becomes Abm R {Aarms Abase Agriph with Abase Rd controlling the delta actions (ōxr, 82, 68) of the base.
bi bi-manual mode, the user controls the floating base
Simulation Performance. We present the simulation speed in Figure 5. The benchmark was done on a headless server of NVIDIA L4 GPU and Intel Xeon Gold 6438Y+ CPU, in a single process. Benefiting from the highly optimised MoJoCo engine, BiGym runs at around 400FPS to 1400FPS depending on the number of cameras. The performance could be further improved by using parallel environments or MuJoCo XLA, which speeds up the execution with XLA just-in-time compilation.
1400.0
Environment Steps /s
Environment Steps /s
1200.0
1400.0
1000.0-
1200.0
800.0
1000.0
600.0
800.0
400.0
Whole-Body
Bi-Manual
No. of Cameras
Control method
(a) Environment Speed with Different # Cameras
(b) Environment Speed of Different Action Modes
Figure 5: The environment run speed of BiGym with (a) different number of cameras and (b) different action modes. In (a), we use the bi-m bi-manual control method for measuring the performance.
B Details of Task Success Detectors
In this section, we detail the definitions of all task success detectors.
Reach Target Tasks.
(1) reach reach_target pontt_ssnnee.e_ee The distance from the robot left wrist to the target is smaller than a tolerance value. The default tolerance value is 0.I.
(2) reach target multi modal: The distance from either the robot left wrist or the right wrist smaller than tolerance value. The default tolerance value is O.I.
(3) reach target dual: The distance from the left wrist and the right wrist to their corresponding goals smaller than tolerance value. The default tolerance value is O.1.
13

---

Toble-Top Manipulation Tasks.
(4) Stack_blockst The three blocks are stacked on each other, i.c. in collision with each other, in target region on the table.
(5) move.plate: The following conditions must be met: (a) the orientation of the plate is upright,
(b) the plate is not colliding with the table, (c) the plate is colliding with the rack and (d) the robot has released the plate from its gripper.
(6) move wew._tto-pp.. -wo-plates: Transfer two plates to the target rack and meet all conditions similar to the move plate task.
(7) flip_cup: The following criteria must be met: (a) The cup is in collision with the counter. (b)
The orientation of the cup is upright. (c) The robot has released the cup from its gripper.
(8) flip.cutlery: Similar to the flip_cup task, but cutlery is used instead.
Dishwasher Tasks.
(9) dishwasher.ope The joint angles of the dishwasher door and both trays are close to 1 with tolerance value. The default value 0.1.
(10) dishwasher.closee The joint angles of the dishwasher door and both trays are close 0 with tolerance value. The default value is 0.1.
(11) dishvasher_open. trrys: The joint angles of dishwasher trays are close to with tolerance value. The default value is 0.1.
(12) wwwsswawher_ccooe www.cclloecccccee .trays: The joint angles of dishwasher trays are close to 0 with tolerance value. The default value is 0.1.
(13) dishwasher _lad -oaddppateddppppppp_eeeeea
plates: All plates are in collision with the bottom tray of the dishwasher and the robot has released the plates from it's grippers.
(14) dishwasher load cups: All cups in collision with the middle tray of the dishwasher and the robot has released the cup from its gripper.
(15) dishwasher loaddolauutti wwl.cculeery. All cutlery are in collision with the dishwasher cutlery basket and the robot has released the cutlery from its gripper.
(16) dishwasher unload plates: All plates are moved from the bottom tray of the dishwasher the drainer on the table, and placed onto the rack positioned on the counter-top. (17) dishwasher www.unldd....o unload.cups: All cups are moved from the middle tray of the dishwasher the cabinet and in collision with the cabinet counter. Additionally, all cups released from the robot gripper.
(18) dishwasher unloaa_cuttery.. and are in collision with the tray.
cutlery: All cutlery are moved from the dishwasher basket to the tray
(19) dishwasher unload plate long: All conditions of dishwasher close and dishwasher wnw.add.a..omommm.mmmmmmmm.lm..a plaats must be met. Additionally, all plates are placed inside the unload_plates
wall cabinet. Finally, all joint angles the wall cabinet doors are close 0 with a tolerance. The default value is 0.1.
(20) dishwasher unload cup long: Similar to dishwasher unload plate.long but with cups.
Similar to wtwp:ashhe..mlooa..uteeeeeee but with cutlery and instead of the cabinet, the cutlery must be placed closed drawer.
(21) di wwwrrasher.ulllddcutlleeee.
Kitchen Counter Tasks.
(22) drawer crrawertoo_oom _top_open: The joint angle of the top drawer is close to 1 with tolerance value. The default value is 0.1.
(23) drawer top close: The joint angle of the top drawer is close to with a tolerance value. The default value 0.1.
(24) drawers oppnn all: The joint angles of drawers are close to I with a tolerance value. The default value 0.1.
(25) drawers close all: The joint angles of all drawers close to with tolerance value. The default value is 0.1.
(26) wall www.lccupoardd.oog The joint angle of two doors of the wall cupboard is close to 1 with tolerance value. The default value 0.1.
14

---

(27) wall cupboard close: The joint angle of two doors of the wall cupboard is close to 0 with tolerance value. The default value 0.1.
(28) cupboards open all: The joint angles of the two doors and all drawers of the kitchen set are close to with tolerance value. The default value 0.1. (29) supboards-close-..
The joint angles of the two doors and all drawers of the kitchen set are close 0 with tolerance value. The default value is 0.1.
(30) take_cups All cups are in collision with the counter on the table and the robot has released the cups from its gripper.
(31) put_.upps All cups are in collision with the cupboard shelf and the robot has released the cups from its gripper.
(32) pick_box: The box collision with the counter and the robot has released the box from its grippers.
(33) store_box. The box in collision with the shelf and the robot has released the box from its grippers.
(34) saucepan_to jooo The saucepan is in collision with the hob and the robot has released the saucepan from its grippers.
(35) store ki kitchenware: Both the saucepan and the pan are in collision with the shelf, and the robot has released the objects from its grippers.
(36) sandwich ch toast: All the following conditions must be met: (a) The sandwich is in collision with the pan. (b) The orientation of the sandwich is either up down. (c) The pan is in collision with the hob.
(37) sandwich Similar to sandwich.toase In addition the sandwich orientation must be flipped.
(38) sandwich remove: All the following conditions must be met: (a) The sandwich is in collision with the board. (b) The orientation of the sandwich is either up or down.
(39) store groceries lower: All items are in collision with the shelf of the cabinet below the counter. Additionally, all items are released from the robot gripper.
(40) store groceries upper: All items are in collision with the shelf of the cabinet the wall. Additionally, all items are released from the robot gripper.
C Experiments
C.1 Implementation Details
We implemented all algorithms using PyTorch [51].
ACT. Following the official implementation', we train ResNet-18 encoder [52] to extract visual features and a transformer model to predict sequence of actions. Inputs to the transformer model are multi-view image features and proprioceptive features from a conditional variational autoencoder (CVAE) [53]. During execution, we use receding horizon control for all tasks by training the policy output an action sequence of length 16 and executing only the first step the sequence. Following the official implementation, we enable temporal ensembling improve the smoothness of the policy.
Diffusion Policy. Our implementation of Diffusion Policy closely follows the official release To be consistent with ACT, we use Re as vision encoders for all camera observations. As discussed Chi et al. [8]. the Diffusion Policy susceptible to the choice of backbones and their parameters: UNet- ID backbone might outperform the causal transformer backbone certain tasks and vice versa. Thus, we benchmark both the UNet- ID backbone and the causal Transformer backbone, and report the highest achieved performance between them in our main results. In addition, for all Diffusion Policy variants, we use action sequence length of 16 and execution length of 1, which we find to achieve strong performance in general. Following ACT, we also enable temporal ensembling for Diffusson Policies, which we find to be crucial for stabilising the inference.
15

---

Other Baselines. For BC and demo-driven RL baselines, we adopt the same network architectures which consist of an CNN-based image encoder and a fully-connected output head. The image encoder encodes each camera image with 3 layers of CNNs, each has kernel size 3 and 32 channels. In between the layers, we use SiLU activation function [54] and layer normalisation [55]. We flatten the CNN features and concatenate with the proprioception states form the final observation feature vector. The head has 2 fully connected layers of dimension 512, and bottlenecks the output to dimension 64, After the bottleneck layer, we normalise the output with layer normalisation followed by tanh activation.
Training Details. We use a frame stack of 4, Adam optimiser [56] with a learning rate of 0.0001, and batch size of 256 for all IL and demo-driven RL algorithms. In addition, specifically for all RL algorithms, we follow AW-Opt [57] and keep the demonstration ratio for each batch be 50% by using a separate demonstration replay buffer. This helps the exploration of the agent during sparse reward settings. We run 150K training steps for IL algorithms and 100K steps for demo-driven RL methods. We observe that all algorithms converge after 100K steps and longer training does not give additional performance boost. All results are averaged over the last 3 checkpoints.
C.2 Results and Discussions
In Table 2, we provide the performance of IL and demo-driven RL methods on 40 BiGym tasks. Overall, we observe that BiGym tasks are challenging and pose a variety of unique and interesting challenges for future researches. We outline our observations as below:
The mobile manipulation of articulated or rigid-body objects is challenging for the state-of- the-art algorithms. BiGym has presented series of tasks that involve interactions with articulated or rigid-body objects, which typically require nigh-precision manipulation, e.g., move_two wereettwo-platee pplates, cupboards open all, and blocks. When coupled with the mobile base, these tasks become more challenging because (i) accurately measuring the grasping poses while moving is hard, and (ii) correctly estimating the posterior distribution of the states given partial history information of POMDP is difficult. For instance, while we observe that ACT and Diffusion Policy achieve the overall best performance across all tasks, they still struggle in the seemingly simple tasks, e.g., stack_blocks, which requires the agent to pick 3 cubes, and stack them to a target region located on the other side of the table. We believe a more robust system with stronger memory mechanisms to track the "beliefs", i.e., estimating the posterior distributions of the states, necessary to solve such challenging BiGym tasks.
The long-horizon tasks in BiGym requires both task and motion planning of the agent. BiGym introduces a series of long- horizon tasks, e.g., dishwasher aeshwasher.unooeee www.oodddppssoooooe and put_cups. All algorithms fail on these tasks. Intuitively, these tasks are composed of multiple sub-tasks, and the difficulty level of achieving these long-horizon tasks grows exponentially at the same time. As model-free agents, our baselines are not capable of performing task-level reasoning. Hierarchical methods [12] could work better policy representation for these tasks. We leave it for future study,
The complex policy space of BiGym requires carefully designed agent architectures. We observe that almost on all tasks, ACT and Diffusion Policy achieves superior performance to BC and demo- driven RL baselines. We hypothesize this is because both ACT and Diffusion Policy utilise powerful policy classes based on generative representation learning, i.e., CVAE and Diffusion models, and they also use expressive network architecture such as transformers or UNets. In contrast, BC and all demo-driven RL approaches use simple CNN + MLP architectures. It is likely that these weaker architectures struggle to deal with the complex multi-modal noisy demonstrations introduced in BiGym. We believe this can motivate future research on finding appropriate policy representations for mobile bi-manual manipulation.
Demo-driven RL approaches struggle with the complex task space and sparse reward in BiGym. We observe that demo-driven RL algorithms fail on most of the BiGym tasks. For instance, CQN [50]. which exhibits strong performance fixed single-arm demo-driven RL setups, fails to solve most of the BiGym tasks. It is notable that all RL algorithms only achieve non-zero success rates simple
16

---

Table 2: Success rates (%) of IL and demo-driven RL algorithms on 40 BiGym tasks, evaluated on 50 episodes. We report the results aggregated over the last three checkpoints.
Algorithms
Algorithms
Task
BC
ACT
DitPolicy
DrQV2
AWAC
IQI.
CQN
reach target single
66.01-0.0
100.0:0.0
61.3::5.8
100.0:04.
94.0422.0
82.0-5.3
92.7:11122
reach. reach.target. multi.nodal modal
75.34:2.3
98.74:1.2
63.3+3.1
53.3448.
69.342.3
teach.terget.dua.
23.3+22.3
90.7=112
19.3+3.1
24.0=2.0
77.3=6.11
48.7+20.2
40.0=10.6
stack.blocks
0.0:0..0
0.0:0.00
0.040.00
0.0:0.00
0.0:0..0
0.0::.00
0.0:0.00
nove.plate
2.7+1.2
30,0
20.0=2.0
0.0+0.0
0.0=0.0
0.0=0.00
0.7=1.2
move tvo plates flip_cup flip_cutlery
7.32233
11.3:1..0
12.0=:.0
0.0:0.00
0.0:0..0
0.0
0.0:0.0
0.0.10.0
2..31.12
6.0.1.00
0.010.00
0.0.:..0
1.3
0.0.1...
0.7+-1.2
22.0+2.00
1,341.2
0.0+00.0
0.7+1.2
L3++1.2
1.3+112
äishsasher.opon
6.0+-5.3
72.0+-45.0
4.04-..00
dishraabber.coose
84.7+20.0
100.0=0.0
99,3+1.2
0.0+0.0
0.0=0.0
0.0+0.0
0,0=0.0
dishwasher open trays
16.745.8
100.0=0.0
0.0+0.0
0.0:0.00
0.0=0.0
0,0=000
0.0=0.0
dishwasber close.traye
0.0:0.00
100.0=0.0
52.0118.3
2.012.00
0.0
0.0.0.0
0.000.0
dishwasher load.ppaa.
0.0:0..0
34.0
0.010.0
0.00000
0.0-0..
0.0
0.0
dishwaaber loaddoooddpp_pp cupa
0.0+0.00
46.0+-0.0
8.44500
00+00
0.0+000
0.0+0.0
0.000.
Llebwasber.load.cuuu
8.742.3
42.028.7
3.3423
0.000.0
0.0=0.0
0.0
0.0=0.0
hisbxasher.umooddpllee..
5.311.2
2.0:::.5
0.0:0.00
0.010.0
0.0=0.0
dishwasher wnw.add..ooaddooooo
9.3.14.2
15.3110.11
0.71112
0.711.2
0.7
0.0
0.010.0
dishsasher cutlery
1.3412
dishiiasber.uul wtw.dddppaeee.onnggg
0.0+0.0
0.7+:112
0.0+0.0
0.0+0.0
0.0=0.0
0.0=0.0
0.0=0.0
diebrasher.umlodd.cooo wwwpoaddddeooongooeeeeeeee
0.0:0.00
0.0+0.00
0.040.0
0.0:0..0
0.0=0..0
0.0=0.0
0.0=0.0
dishwasber www.:dierwwlodd..uueoo.e long
1.3+2.3
14.7=8.3
5.345.8
0.0+0.0
0.0=0.0
0.0=0.0
0,0=0.0
draver_top.oper
9,3+16.2
100.0=0.0
3.343.11
0.0+00.0
8.7++1..0
2.0+2.0
0.0=0.00
top close
100.0=0.0
100.0=0.0
100.0:0.00
100.0:10.0
100.0:0.00
100.0:0.0
100.0:0.0
dravers,ee open all dravers_clsse
10.7110.1
100.0=0.0
16.718.3
0.0:10.0
0.0:0.00
0.010.0
0.020.0
28.34
call_cupboard_ppe
22.0++31.2
97.3+112
100.0=0.0
27.3=5.0
12.0=17.3
9,3+223
0.0+0.0
vall vell.cupboard close cupboards open all
100.0=0.0
100.0=0.0
100.0=0.0
100.0=0.0
26.0=:1.6
97.344.6
70.0+20.0
5.31442
17.312144
0.0.10.0
0.0:0.00
0.0:0.0
0.0:0.0
0.0:0.0
cupboards.close_ll
63.3
0.7.11.2
1.312.3
0.010.0
0.0.0.0
0.010.0
0.0.0.0
cups
0.0+00.0
26.0+-2.0
5.342.3
00+00.
0.0-00
0.0+0.0
0.0+0.0
put.oups
3.3+2.33
30.0
0.7
0.000.0
0.0=0.0
0.0+0.0
0.0=0.0
pick-box
20.7::1.2
40.71:1.2
22.0=0.0
0.010.0
0.0:0.0
0.0+0.0
0.0=0.0
stere.box
8.7.3.11
13.313.11
0.0.1.00
0.0.0.00
0.0.0.0
0.010.0
0.010.0
saucepan
21.34-4.6
88.0-2.0
store.kitchenvare
0.0+0.00
2.7+22.3
0,7+12
0.0+0.0
0.0=0.0
0.0=0.0
0.0=0.0
sandwich.toast
5.3+1.2
30.716.11
10.0=0.0
0.0:0.00
0.0=0.00
0.0=0.0
0.0=0.0
sandrich.flip
0.0+0.00
32.0=2.0
4.7+12
0.7+112
0.0=0.0
0.0=0.0
0,0=0.0
sandwich remove
40.748.3
55,317.00
48.0=0.0
0.71112
0.0=0.00
0.0=0.0
0.0=0.0
grocegroeeriss...o lover
0.0:10.0
0.0.1...0
0.0.10.0
0.0.1..0
0.010.0
0.010.0
0.0.0.0
atoro.grocariens upppeerreee.ppperr
0.0+00.0
0.0++0.0
0.0+0.00
0.0+0.00
0.0+00.0
0.0+0.00
0.0+0000
Average
18.01.1.11
46.31.14
20.8.0..8
13.0.112
12.410.0
10.5
tasks with little interaction with the objects, e.g., reach target single and top drawer close, and completely fail to solve all the other tasks. We hypothesise this is because (i) the presence of mobile base makes it more difficult for agents to explore meaningful state. e.g. erroneous base turning action can easily cause robot to lose view of the objects. and (ii) RL agents struggle to learn value-functions on long-horizon BiGym tasks with sparse reward.
17


Reviews:
[{"review_id": "Tj06dhmiIu", "reviewer": "Reviewer_oipX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a mobile bimanual benchmark in simulation with 40 human-demonstrated skills (50 demos each) of ranging difficulty. The skills include reach target, table-top manipulation, dishwasher, and kitchen counter tasks. The skills are benchmarked using 3 imitation learning baselines and 4 RL baselines. The results demonstrate that ACT and diffusion policy imitation learning algorithms perform best but leave a lot of room for improvement, and even completely fail on some tasks that are long-horizon and/or involve mobile navigation and manipulation. The conclusions are we need algorithms with longer temporal memory and for RL potentially hierarchical approaches to deal with the sparse reward on long-horizon tasks.", "summary_of_recommendation": "Overall, I think the mobiel bimanual benchmark is a good addition to the research community. My main points of feedback are: include the experiments in the main paper, try out the couple of ideas suggested (temporal memory for imitation learning and hierarchical RL), and demonstrate that the skills from the simulation benchmark transfer in some way to hardware.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 3, "quality_of_limitations": 1, "questions_for_rebuttal": "In addition to the weaknesses listed above, I have the following questions for the authors.\n1. Is 50 demonstrations per skill enough, particularly for long horizon tasks? How was this number chosen?\n2. Is saving action-only demonstrations that can be replayed later to get the observations sufficient? During replaying, are the rollouts of the demonstrations the same or stochastically different?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CZoFniGMIM", "reviewer": "Reviewer_RTpp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors proposed BiGym -  a benchmark based on MuJoCo and learning environment for mobile bi-manual manipulation with 40 tasks, with 50 human-collected demos for each task. It provides visual inputs as RBGD with overhead and wrist camera views. The paper provides extensive benchmarking results of current SOTA RL and IL methods.", "summary_of_recommendation": "Great system paper with a new benchmark to push for mobile bi-manual manipulation, but more real-world result is needed to distinguish it from concurrent related works.", "technical_quality": 3, "clarity_of_presentation": 5, "potential_impact": 3, "robotics_focus": 3, "quality_of_limitations": 2, "questions_for_rebuttal": "- It would be very helpful for the authors to provide some evaluation experiments on sim2real transfer gap. For example, some of the table-top task setup could be replicated in real world, and some preliminary real-world eval results on the policy trained from simulation would be useful to provide insights on how physically transferable the policy trained from simulation physics to real world.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xc8FeqRZ8q", "reviewer": "Reviewer_JNyU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a simulated benchmark and demonstration dataset for bimanual manipulation tasks based on the UniTree H1 humanoid robot. The benchmark features a variety of bimanual tasks situated in a kitchen environment. It supports both full-body control and bimanual mode with a floating base. The paper also includes results in comparing popular behavior cloning methods and offline RL methods.", "summary_of_recommendation": "I'm willing to change my rating if the authors can adequately address my question regarding full-body control.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 4, "robotics_focus": 3, "quality_of_limitations": 2, "questions_for_rebuttal": "See \"weaknesses\"", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5ioRDPw4GL", "reviewer": "Reviewer_bdfR", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents a benchmark data set for (mobile) bimanual manipulation tasks. The data is collected from human demo with a comprehensive bimanual tasks.", "summary_of_recommendation": "I think this is a good paper for CoRL if my concerns are addressed.", "technical_quality": 4, "clarity_of_presentation": 5, "potential_impact": 3, "robotics_focus": 2, "quality_of_limitations": 1, "questions_for_rebuttal": "The bipedal movement seems very unrealistic, how are you going to resolve this issue?\nThe reason of selecting the tasks is unclear to me.\nElaboration on limitation?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Strengths:
- Interesting and timely topic of bimanual manipulation, with a rich set of tasks consisting of both bimanual and whole-body manipulation.
- Reviewers agree that the baselines and experiments are insightful, showing gaps in SoTA on the proposed benchmark. 

Weaknesses:
- Reviewers raise concerns about the whole-body motion demos and controller and its impact on the benchmark’s realism, especially sim2real. 
- The experiments and analysis should be in the main paper, instead of in the appendix. Details about the controller used for collection demonstrations, and a limitation section should also be added to the paper.

Post-rebuttal:

The key points addressed in the rebuttal include:
- Limitation section: Authors have added a limitations section to the paper.
- Experimental details: Authors have described details of the controller and teleoperation.
However, some concerns raised by reviewers around sim2real still remain.
Decision: Accept