ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills
Authors: Jiayuan Gu
Fanbo Xiang
Xuanlin Li
Zhan Ling
Xiqiang Liu
Tongzhou Mu
Yihe Tang
Stone Tao
Xinyue Wei
Yunchao Yao
Xiaodi Yuan
Pengwei Xie
Zhiao Huang
Rui Chen
Hao Su
Venue: ICLR 2023 poster (2023)
Keywords: object manipulation
benchmark
computer vision
robotics
reinforcement learning

Abstract: Generalizable manipulation skills, which can be composed to tackle long-horizon and complex daily chores, are one of the cornerstones of Embodied AI. However, existing benchmarks, mostly composed of a suite of simulatable environments, are insufficient to push cutting-edge research works because they lack object-level topological and geometric variations, are not based on fully dynamic simulation, or are short of native support for multiple types of manipulation tasks. To this end, we present ManiSkill2, the next generation of the SAPIEN ManiSkill benchmark, to address critical pain points often encountered by researchers when using benchmarks for generalizable manipulation skills. ManiSkill2 includes 20 manipulation task families with 2000+ object models and 4M+ demonstration frames, which cover stationary/mobile-base, single/dual-arm, and rigid/soft-body manipulation tasks with 2D/3D-input data simulated by fully dynamic engines. It defines a unified interface and evaluation protocol to support a wide range of algorithms (e.g., classic sense-plan-act, RL, IL), visual observations (point cloud, RGBD), and controllers (e.g., action type and parameterization). Moreover, it empowers fast visual input learning algorithms so that a CNN-based policy can collect samples at about 2000 FPS with 1 GPU and 16 processes on a regular workstation. It implements a render server infrastructure to allow sharing rendering resources across all environments, thereby significantly reducing memory usage. We open-source all codes of our benchmark (simulator, environments, and baselines) and host an online challenge open to interdisciplinary researchers.

Published as a conference paper at ICLR 2023
MANISKILL2: A UNIFIED BENCHMARK FOR
GENERALIZABLE MANIPULATION SKILLS
Jiayuan Gu f Fanbo Xiang", Xuanlin Li Zhan Ling" " Xiqiang Liu' Tongzhou Mu Yihe Tang" "* Stone Tao", Xinyue Wei' Yunchao Yao Xiaodi Yuan", Pengwei Xie?
Zhiao Huang', Rui Chen?, Hao Su' University of California San Diego, TTsinghua University
ABSTRACT
Generalizable manipulation skills, which can be composed to tackle long- horizon and complex daily chores, are one of the cornerstones of Embodied AL However, existing benchmarks, mostly composed of a suite of simulatable enviromments, are insufficient push cutting-edge research works because they lack object-level topological and geometric variations, are not based on fully dynamic simulation, or are short of native support for multiple types of manipulation tasks. To this end, present ManiSkill2, the next generation of the SAPIEN ManiSkill benchmark, to address critical pain points often encountered by researchers when using benchmarks for generalizable manipulation skills. ManiSkill2 includes 20 manipulation task families with 2000+ object models and 4M+ demonstration frames, which cover sitionary/mobiie-basee single/dual- arm, and rigid/soft-body manipulation tasks with 2D/3D-input data simulated by fully dynamic engines. It defines a unified interface and evaluation protocol to support a wide range of algorithms (c.g., classic sense-plan-act, RL, IL). visual observations (point cloud, RGBD), and controllers (e.g., action type and parameterization). Moreover, it empowers fast visual input learning algorithms that CNN-based policy can collect samples about 2000 FPS with 1 GPU and 16 processes on regular workstation. It implements a render server infrastructure to allow sharing rendering resources across all environments, thereby significantly reducing memory usage. We open-source all codes of our benchmark (simulator, environments, and baselines) and host an online challenge open to interdisciplinary researchers.
Figure I: ManiSkill2 provides a unified, fast, and accessible system that encompasses well-curated manipulation tasks (e.g., stationary/mooill-base single/dual-arm, rigid/soft-body).
't and indicate equal contribution (in alphabetical order). See Appendix I for contributions.
PProject website: htttppss///mmaiiikil..gg.iiiaaaa
'Codes: ttddes:/htts::////a/a http:://///tbbb...oo http://////ttttbcooooo////aa/eet httt::///athhb..cco////aaaallaaa
*Challenge wehsite: ottpfh/hhhttr///aaaeo.ooooooooor

---

Published as a conference paper at ICLR 2023
1 INTRODUCTION
Mastering human-like manipulation skills is a fundamental but challenging problem in Embodied AI, which is at the nexus of vision, learning, and robotics. Remarkably, once humans have learnt to manipulate category of objects, they are able to manipulate unseen objects (e-g., with different appearances and geometries) of the same category in unseen configurations (e.g., initial poses). We refer such abilities to interact with a great variety of even unseen objects in different configurations as generalizable manipulation skills. Generalizable manipulation skills are one of the cornerstones of Embodied AL, which can be composed to tackle Long-horizon and complex daily chores (Ahn et al.) 2022 Gu et al., 2022). To foster further interdisciplinary and reproducible research on generalizable manipulation skills, it is crucial to build a versatile and public benchmark that focuses on object-level topological and geometric variations well practical manipulation challenges.
However, most prior benchmarks are insufficient to support and evaluate progress in learning generalizable manipulation skills. In this work, we present ManiSkill2, the next generation of SAPIEN ManiSkill Benchmark (Mu et al., 202I). which extends upon fully simulated dynamics, large variety of articulated objects, and large-scale demonstrations from the previous version. Moreover, we introduce significant improvements and novel functionalities, as shown below.
1) A Unified Benchmark for Generic and Generalizable Manipulation Skills: There does not exist a standard benchmark to measure different algorithms for generic and generalizable manipulation skills. It largely due well-known challenges to build realistically simulated environments with diverse assets. Many benchmarks bypass critical challenges as a trade-off.
by either adopting abstract grasp (Ehsani al., 2021; Szot et al., 202T, Srivastava et al., 2022)
or including few object-level variations (Zhu et al. 2020; Yu et al. 2020 James et al. 2020)
Thus, researchers usually have to make extra efforts to customize environments due to limited functionalities, which in turn makes reproducible comparison difficult. For example, (Shridhar example,,hrridaa et al.
(2022) modified 18 tasks in James et al. (2020) to enable few variations in initial states. Besides, some benchmarks are biased towards single type of manipulation, e.g., 4-DoF manipulation in Yu ct al. (2020). To address such pain points, ManiSkill2 includes a total of 20 verified and challenging manipulation task families of multiple types (stationary/mooile-- base, single/dual-arm, rigid/soft- body), with over 2000 objects and 4M demonstration frames, to support generic and generalizable manipulation skills. All the tasks implemented in a unified OpenAl Gym (Brockman et al. 2016) interface with fully-simulated dynamic interaction, supporting multiple observation modes (point cloud, RGBD, privileged state) and multiple controllers. unified protocol is defined to evaluate a wide range of algorithms (e.g.. sense- conse-plan-act, reinforcement and imtation learning) on both seen and unseen assets as well as configurations. In particular, we implement a cloud-based evaluation system to publicly and fairly compare different approaches.
2) Real-time Soft-body Environments: When operating in the real world, robots face not only rigid bodies, but many types of soft bodies, such as cloth, water, and soil. Many simulators have supported robotic manipulation with soft body simulation. For example, MuJoCo (Todorov et al.) 2012) and Bullet Coumann & Bai (2016-202I) use the finite element method (FEM) to enable the simulation of rope, cloth, and elastic objects. However, FEM-based methods cannot handle large deformation and topologiaal changes, such scooping flour cutting dough. Other environments, like SoftGym (Lin et al., 2020) and ThreeDWorld (Gan et al. 2020), are based on Nvidia Flex, which can simulate large deformations, but cannot realistically simulate elasto-plastic material, material
e-g., clay. Plasticinel.as .ab (Huang et al.) (02T) deploys the continum--nechanics--ee point method (MPM), but it lacks the ability to couple with rigid robots, and its simulation and rendering performance have much room for improvement. We have implemented a custom GPU MPM simulator from scratch using Nvidia's 's Warp (Macklin 2022) JIT framework and native CUDA for high efficiency and customizability. We further extend Warp's 's functionality to support more efficient host-device communication. Moreover, we have supported a 2-way dynamics coupling interface that enables any rigid-body simulation framework to interact with the soft bodies, allowing robots and assets in ManiSkil12 to interact with soft-body simulation seamlessly. To our knowledge, ManiSkill2 is the first embodied Al environment to support 2-w coupled rigid-MPM simulation, and also the first to support real-time simulation and rendering of MPM material.
3) Multi-controller Support and Conversion of Demonstration Action Spaces: Controllers transform policies' action outputs into motor commands that actuate the robot, which define the action space of task. MMartin-Martin et al. 22019) Zhu et al. (2020) show that the choice of action
2

---

Published as a conference paper at ICLR 2023
space has considerable effects on exploration, robustness and sim2real transferability of RL policies. For example, task-space controllers are widely used for typical pick-and-place tasks, but might be suboptimal compared to joint-space controllers when collision avoidance (Szot et al. 2021 is required. ManiSkill2 supports a wide variety of controllers, e.g., joint-space controllers for motion planning and task-space controllers for teleoperation. A flexible system is also implemented to combine different controllers for different robot components, For instance, it is casy to specify velocity controller for the base, a task-space position controller for the arm, and joint-space position controller for the gripper. It differs from Zhu et al. (2020), which only supports setting a holistic controller for all the components. Most importandy, ManiSkill2 embraces unique functionality to convert the action space of demonstrations to a desired one. It enables us to exploit large-scale demonstrations generated by any approach regardless of controllers.
4) Fast Visual RL Experiment Support: Visual RL training demands millions of samples from interaction, which makes performance optimization an important aspect in environment design. Isaac Gym (Makoviychuk et al.] 2021) implements fully GPU-based vectorized simulator, but it lacks an efficient renderer. also suffers from reduced usability (e.g.. difficult to add diverse assets) and functionality (e.g., object contacts are inaccessible). EnvPool (Weng et al. (022)) batches environments by a thread pool to minimize synchronization and improve CPU utilization. Yet its environments need to be implemented in C++, which hinders fast prototyping (e-g., customizing observations and rewards). As a good trade-off between efficiency and customizability, our environments are fully scripted in Python and vectorized by multiple processes. We implement an asynchronous RPC-based render server-client system to optimize throughput and reduce GPU memory usage. We manage to collect samples with an RGBD-input PPO policy at about 2000 FPS with 1 GPU and 16 CPU processors on a regular workstation.
2 BUILDING ENVIRONMENTS FOR GENERALIZABLE MANIPULATION SKILLS
Building high-quality environments demands cross-disciplinary knowledge and expertise, including physical simulation, rendering, robotics, machine learning, software engineering, etc. Our workflow highlights a verification-die driven iterative development process, which is illustrated in Appendix[A.. I Different approaches, including task and motion planning (TAMP), model predictive control (MPC) and reinforcement learning (RL). can be used to generate demonstrations according to characteristics and difficulty of tasks, which verify environments byproduct.
2.1 HETEROGENEOUS TASK FAMILIES
ManiSkil12 embraces a heterogeneous collection of 20 task families. A task family represents family of task variants that share the same objective but are associated with different assets and initial states. For simplicity, we interchangeably use task short for task family. Distinct types of manipulation tasks are covered: rigid/soft-body, stationary/mobile-aass single/dual- In this section, we briefly describe 4 groups of tasks. More details can be found in Appendix C
Soft-body Manipulation: ManiSkill2 implements 6 soft-body manipulation tasks that require agents to move or deform soft bodies into specified goal states through interaction.
1) Fill: filling clay from a bucket into the target beaker;
2) Hang: hanging a noodle the target rod;
3) Excavate: scooping up a specific amount of clay and lifting it to a target height:
4) Pour: pouring water from a bottle into the target beaker. The final liquid level should match the
red line on the beaker.
deforming plasticine from an initial shape into a target shape; target shapes are generated by randomly pinching initial shapes, and are given RGBD images or point clouds from 4 views. writing a target character on clay. Target characters are given as 2D depth maps.
5) Pinch:
6) Write:
A key challenge of these tasks is to reason how actions influence soft bodies, egg., estimating displacement quantity or deformations, which will be illustrated in See S.2
Precise Peg-in-hole Assembly: Peg-in-hole assembly is representative robotic task involving rich contact. We include a curriculum of peg-in-hole assembly tasks that require an agent to place an object into its corresponding slot. Compared to other existing assembly tasks, ours come with two
'The FPS reported for rigid-body environments.
3

---

Published as a conference paper at ICLR 2023
noticeable improvements. First, tasks target high precision (small clearance) at the level of millimeters, as most day-to-day assembly tasks demand. Second, our tasks emphasize the contact- rich insertion process, instead of solely measuring position rotation errors.
1) PegInsertionSide: a single peg-in-hole assembly task inspired by MetaWorld (Yu et al. 2020). It involves agent picking up a cuboid-shaped peg and inserting it into a hole with a clearance of 3mm on the box. Our task is successful only if half of the peg is inserted, while the counterparts in prior works only require the peg head to approach the surface of the hole.
2) PlagCharger: a dual peg-in-hole assembly task inspired by RL.Bench (James et al. 2020), which involves an agent picking up and plugging charger into vertical receptacle. Our assets (the charger and holes on the receptacle) are modeled with realistic sizes, allowing a clearance of 0.5mm, while the counterparts in prior works only examine the proximity of the charger to a predefined position without modeling holes at all.
3) Assembling inspired by Transporter Networks (Zeg et al., 2020), this task involves an agent picking up and inserting shape into its corresponding slot on a board with 5 slots in total. We devise programmatic way to carve slots on boards given any specified clearance (e.g., 0.8mm), such that we can generate any number of physically-raalitte boards with slots. Note that slots in environments of prior works are visual marks, and there are in fact no holes on boards.
We generate demonstrations for the above tasks through TAMP. This demonstrates that it is feasible to build precise peg-in-hole assembly tasks solvable in simulation environments, without abstractions from prior works.
Stationary 6-DoF aeck-end-place: 6-DoF pick-and-place is a widely studied topic in robotics. At the core is grasp pose Mahler et al. (2016); Qin et al. (2020); Sundermeyer et al. (202T). In ManiSkil12, we provide a curriculum of pick-and-place tasks, which all require an agent to pick up an object and move it to a goal specified variations among objects call for generalizable grasp pose predictions.
3D position. The diverse topology and geometric
1) PickCube: picking up cube and placing it at a specified goal position;
2) StuckCuber picking up a cube and placing it on top of another cube. Unlike PickCube, the goal
placing position is not explicitly given; instead, it needs to be inferred from observations.
3) PickSingle YCB: picking and placing an object from YCB (Calli et al. 2015):
4) PicKSingleEGAD: picking and placing
object from EGAD (Morrison al. (020);
5) PickClutterYCB: The task is similar to PickSingle YCB, but multiple objects are present in a single
scene. The target object is specified by a visible 3D point its surface.
Our tack--ad-place tasks are deliberately designed to be challenging. For example, the goal position is randomly selected within a large workspace (30 x 50 x 50 cm²). It poses challenges to sense-plan -aase--la-----e act pipelines that do not take kinematic constraints into consideration when scoring grasp poses, as certain high-quality grasp poses might not be achievable by the robot.
Mobile/Stationary Manipulation of Articulated Objects: We inherit four mobile manipulation tasks from ManiSkill1 (Mu et al., (02T), which are PushChair, MoveBucket, OpenCabinet/ooor, and OpennabinetDrawwer We also add a stationary manipulation task, TurnFaucet, which uses a stationary arm to turn on faucets of various geometries and topology (details in Appendix C.3).
Besides the above tasks, we have one last task, AvoidObstacles, which tests the navigation ability of a stationary arm to avoid a dense collection of obstacles in space while actively sensing the scene.
2.2 MULTI-CONTROLLEER SUPPORT AND CONVERSION OF DEMONSTRATION ACTION SPACES
The selection of controllers determines the action space. ManiSkil12 supports multiple controllers, e.g., joint position, delta joint position, delta end-effector pose, etc. Unless otherwise specified, controllers in ManiSkil12 translate input actions, which are desired configurations (e.g.. joint positions or end-effector pose), to joint torques that drive corresponding joint motors to achieve desired actions. For instance, input actions to joint position, delta joint position, and delta end- effector pose controllers are, respectively, desired absolute joint positions, desired joint positions relative current joint positions, and desired SE) transformations relative to the current end- effector pose. See Appendix B for a full description of all supported controllers.
Demonstrations are generated with one controller, with associated action space. However, researchers may select an action space that conforms to a task but is different from the original one. Thus, exploit large-scale demonstrations, it is crucial to convert the original action space
4

---

Published as a conference paper at ICLR 2023
to many different target action spaces while reproducing the kinematic and dynamic processes in demonstrations. Let us consider pair of environments: a source environment with a joint position controller used to generate demonstrations through TAMP, and target environment with a delta end-effector pose controller for Imitation Reinforcement Learning applications. The objective is to convert the source action (sre(() each timestep & the target action (tg. By definition, the target action (delta end-effector pose) Orga (t) Tegs (t) Tigl (1), where T Tige and Tog (1) are respectively desired and current end-effector poses in the target environment. To achieve the same dynamic process in the source environment, we need match Tigi (t) with Tare (t), where Tar (t) is the desired end-effector pose in the source environment. Tarc (t) can be computed from the desired joint positions (Aaro (t) in this example) through forward kinematics (FK(-)). Thus, we have
Ougt (t) (t) Tigll (t) (t) Tegs (t) F FK (T)re Tigl
Note that our method is closed-loop, as we instantiate a target environment to acquire Tige (t). For comparison, an open-loop method would use Tare (4) and suffers from accumulated execution errors.
REAL-TIME SOFT BODY SIMULATION AND RENDERING
In this section, we describe our new physical simulator for soft bodies and their dynamic interactions with the existing rigid-body simulation. Our key contributions include: 1) highly efficient GPU MPM simulator;
an effective 2-way dynamic coupling method support interactions between our soft-body simulator and any rigid-body simulator, in our case, SAPIEN. These features enable us to create the first real-time MPM-based soft-body manipulation environment with 2- way coupling.
Rigid-soft Coupling: Our MPM solver is MLS-MPM (Hu et al. 2018) similar to PlasticineLab (Huang et al., 202T, but with a different contact modeling approach, which enables 2-way coupling with external rigid-body simulators. The coupling works by transferring rigid- body poses to the soft-body simulator and transferring soft-body forces to the rigid-body simulator. At the beginning of the simulation, all collision shapes in the external rigid-body simulator are copied to the soft-body simulator. Primitive shapes (box, capsule, etc.) are represented by analytical signed distance functions (SDFs); meshes are converted to SDF volumes, stored as 3D CUDA textures. After each rigid-body simulation step, we copy the poses and velocities of the rigid bodies to the soft-body simulator. During soft-body simulation steps, evaluate the SDF functions MPM particle positions and apply penalty forces to the particles, accumulating forces and torques for rigid bodies. In contrast, PlasticineLab evaluates SDF functions on MPM grid nodes, and applies forces to MPM grids. Our simulator supports both methods, but we find applying particle forces produces fewer artifacts such as penetration. After soft-body steps, we read the accumulated forces and torques from the soft-body simulator and apply them to the external digd-body simulator. This procedure summarized Appendix D.I] Despite being a simple 2- 2-way coupling method, very flexible, allowing coupling with any rigid-body simulator, so we can introduce anything from a rigid-body simulator into the soft-body environment, including robots and controllers.
The performance of our soft-body simulator is optimized in 4 aspects. First, the simulator is implemented in Warp. Nvidia's JIT framework to translate Python code to native C++ and CUDA. Therefore, simulator enjoys performance comparable to C and CUDA. Second, we have optimized the data transfer (e.g., poses and forces in the 2-way coupling) between CPU and GPU by further extending and optimizing the Warp framework: such data transfers are performance bottlenecks in other JIT frameworks such as Taichi (Hu et al.; 2019), on which PlasticineL.ab is based. Third, our environments have much shorter compilation time and startup time compared to PlasticineLab thanks to the proper use of Warp compiler and caching mechanisms. Finally, since the simulator is designed for visual learning environments, we also implement a fast surface rendering algorithm for MPM particles, detailed in Appendix D.2 Detailed simulation parameters and performance are provided in Appendix [.3] D.3]
Performance Optimization:
PARALLELIZING PHYSICAL SIMULATION AND RENDERING
ManiSkil12 aims to be general and user- friendlyy framework, with low barriers for customization and minimal limitations. Therefore, we choose Python as our scripting language to model environments, and the open-source, highly flexible SAPIEN (Xiang al. 2020) our physical
5

---

Published as a conference paper at ICLR 2023
Worker1
Simzation
Pictures
Render
D2-1
Elter Obee
Main
Simiiation
Get Other lise
Man
Worker 2
Simvatioon
Pictures
Render
Image D211
Get Other Ots
Raward
Marnn
Get Other Obx
flevard
Worker N:
Masieal Simuation
Pictanes
Take
Render
Dosy D231
Reward Get Other Cise
Dosyy
Maie
Plysica
Get Diner Usse
Rowaid
Man
Main
[handar) (Pande]
(handar)
Tako Poture Seceives (Throod Pook)
Com
Envecsment
Copy Image
Polkyy
Copy Action
Copy Action
Envenmment
Cosy State
Pattcy
Dugy Action
Copy Action
Step
Shep
GPU
(a)
(b)
Two pipelines for visual RL sample collection. (a) Sequential pipeline. (b) Our pipeline with asynchronous rendering and render server improves CPU utilization, reduces data transfer, and saves memory.
Figure 2:
simulator. The Python language determines that the only way effectively parallelize execution is through multi-process, which is integrated in common visual RL training libraries (Raffin et al.) 2021 Weng al., 202I). Under the multi-processs environment paradigm, we make engineering efforts to enable our environments to surpass previous frameworks by increased throughput and reduced GPU memory usage.
What are the Goals in Visual-Learning Environment Optimization? The main goal of performance optimization in a visual-learning environment to maximize total throughput: the number of samples collected from all (parallel) environments, measured in steps (frames) per second. Another goal to reduce GPU memory usage. Typical Python-based multi-process environments are wasteful in GPU memory usage, since each process has to maintain a full copy of GPU resources, e.g., meshes and textures. Given a fixed GPU memory budget, less GPU memory spent rendering resources means more memory can be allocated for neural networks.
Asynchronous Rendering and Server-based Renderer: Fig. 2(a) illustrates a typical pipeline (sequential simulation and rendering) collect samples from multi-process process environments. It includes the following stages: (1) do physical simulation on worker processes; (2) take pictures (update renderer GPU states and submit draw calls); (3) wait for GPU render to finish; (4) copy image observations to CPU; (5) compute rewards and get other observations (e.g., robot proprioceptive info); (6) copy images the main python process and synchronize; (7) copy these images to GPU for policy learning; (8) forward the policy network on GPU; (9) copy output actions from the GPU to the simulation worker processes.
We observe that the CPU is idling during GPU rendering (stage 3), while reward computation (stage 5) often does not rely on rendering results. Thus, we can increase CPU utilization by starting reward computation immediately after stage 2. We refer to this technique as asynchronous rendering.
Another observation is that images are copied from GPU CPU on each process, passed to the main python process, and uploaded to the GPU again. It would be ideal if we can keep the data on GPU at all times. Our solution is to use a render server, which starts a thread pool on the main Python process and executes rendering requests from simulation worker processes, summarized figure bb.. The render server climinates GPU-CPU-GPU image copies, thereby reducing data transfer time. It allows GPU resources share across any number of environments, thereby significantly reducing memory usage. It enables communication over network, thereby having the potential to simulate and render on multiple machines. It requires minimal API changes the only change to the original code base is to receive images from the render server. It also has additional benefits in software development with Nvidia GPU, which we detail in Appendix F.2
Comparison: We compare the throughput of ManiSkil12 with 3 other framework that support visual RL: Habitat 2.0 (Szot et al. 2021), RoboSuite 1.3 (Zhu al. 2020), and Isaac Gym (Makoviychuk et al. 202T). We build a PickCube environment in all simulators. We use similar physical simulation parameters (500Hz simulation frequency and 20Hz control frequency I. and we use the GPU
A fair comparison of different frameworks is still challenging their simulation and rendering differ in fidelity. We try our best match the simulation parameters among the frameworks.

---

Published as a conference paper at ICLR 2023
ManiSkill2 Server
ManiSkill2 Syne
Habitat 2.0
RoboSuite 1.3
Isaac Gym
ManiSkil12 Server
Habitat
#Eavs
2.0
Total FPS (rand. action)
2487=24
942+19 931+4
1275a10 1224+13
924 894+15 32
865+35
4.9G
6,4G
(nature CNN)
25322663
815=5
5.1G
12.9G
Optimal #Envs
64
32
64
512
64
5.8G
(OOM)
(a)
(b)
Table I: (a) Comparison of sample collection speed (FPS) PickCube across different frameworks. (b) GPU memory usage on multi-process environments with 74 YCB objects each.
pipeline for rendering- Images have resolution 128x128. All frameworks are given computation budget of 16 CPU cores (logical processors) of an Intel i9-9960X CPU with 128G memory and RTX Titan GPU with 24G memory. We with random actions and also with randomly initialized nature CNN (Mnih et al. 2015) as policy network. We test all frameworks on 16, 32, 64, 128, 256, 512 parallel environments and report the highest performance. Results are shown in Table a) (more details in Appendix E.3). An interesting observation is that our environment performs the best when the number parallel environments exceeds the number of CPU cores. We conjecture that this because execution on CPU and GPU are naturally efficiently interleaved through OS or driver-level schedulers when the requested computation far exceeds the available resources.
Additionally, demonstrate the advantage of memory sharing thanks our render server. We extend the PickClutterYCB environment to include 74 YCB objects per scene, and create the same setting in Habitat 2.0. As shown in Table (bb). even though we enable GPU texture compression for Habitat and use regular textures for ManiSkil12, the memory usage of Habitat grows linearly as the number of environments increases, while ManiSkill2 requires very little memory to create additional environments since all meshes and textures are shared across environments.
5 APPLICATIONS
In this section, we show how ManiSkil12 supports widespread applications, including sense-plan- act frameworks and imitation / reinforcement learning. In this section, for tasks that have asset variations, we report results on training objects. Results on held-out objects are presented in Appendix F Besides, we demonstrate that policies trained in ManiSkil12 have the potential to be directly deployed in the real world.
5.1 SENSE-PLAN-ACT
Sense- Lense-plan-act (SPA) is a classical framework robotics. Typically, a SPA method first leverages perception algorithms to build world model from observations, then plans sequence of actions through motion planning, and finally execute the plan. However, SPA methods are usually open- loop and limited by motion planning, since perception and planning independently optimized. In this section, we experiment with two perception algorithms, Contact-GraspNet (Sundermeyer al. 202T) and Transporter Networks (Zeng et al. 2020).
Crrsact-GraspNet for PickSingle YCB:: YCB: The SPA solution for PickSingleYCB is implemented as follows. First, we use Contact-GraspNet (CGN) to predict potential grasp poses along with confidence scores given the observed partial point cloud. We use the released CGN model pre- trained on ACRONYM (Eppner et al., 202T). Next, we start with the predicted grasp pose with the highest score, and try to generate plan to achieve it by motion planning. If no plan is found, the grasp pose with the next highest score is attempted until a valid plan found. Then, the agent executes the plan to reach the desired grasp pose and closes its grippers to grasp the object. Finally, another plan is generated and executed to move the gripper to the goal position.
For each of the 74 objects, conduct 5 trials (different initial states). The task succeeds if the object's center of mass is within 2.5cm of the goal. The success rate is 43.24%. There are two of low confidence (27.03% of trials), especially for objects (e.g., spoon and fork) that are small and thin; 2) predicted poses are of low grasp quality or unreachable (29.739 73.7 of trials), but with high confidence. See Appendix F. for examples.
main failure modes: 1) predicted poses
Transporter Network for AssemblingKits: We benchmark Transporter Networks (TPN) on our AssemblingKits. The original success metric (pose accuracy) used in TPN is whether the peg is

---

Published as a conference paper at ICLR 2023
Obs. Mode
PickCube
StackCube
Fill
Hang
Excavate
Pour
Pinch
Write
Point Cloud RGBD
0.22 + 0.06
0.04
0.02
0.45
0.04
0.35 = 0.
0.08
0.04
0.02
0.02
0.00 + 0.00
0.00
0.00
0.01 1: 0.02
0.00 4 0.00
0.62 1 0.07 0.20 = 0.12
0.21 + 0.04
0.00
0.00
0.00 + 0.00
0.00 + 0.00
Table 2: Mean and standard deviation of the success rate of behavior cloning on rigid-body and soft-body tasks. For rigid-body assembly tasks not shown in the table, success rate
Obs. Mode
||
PickCCbbe
StackCube
|PicSSingleYCB]
PoglnsSide
PlugCharger
AssemblingKits
TurFFaucet
AvoidObsttclees
Point Cloud 0.94 + 0.03 0.95 = 0.02
0.51 + 0.05
10.01 + 0.01 0.01 + 0.02
0.00 +:00000 0.00
0.04 + 0.03
0.00 + 0.00 0.00 + 0.00
RGBD
0.91 + 0.05
0.87
0.04
0.18
0.07
00.01 + 0.01
0.01 + 0.01
0.00 0.00
0.03 + 0.03
Table 3: Mean and standard deviation of success rates of DAPG+PPO on rigid-body tasks. Training budget is 25M time steps.
placed within Iem and 15 degrees of the goal pose. Note that our version requires pieces to actually fit into holes, and thus our success metric is much stricter. We train TPN from scratch with image data sampled from training configurations using two cameras, base camera and hand camera. To address our high-precision success criterion, we increase the number of bins for rotation prediction from 36 in the original work to 144. During evaluation, we employ motion planning to move the gripper to the predicted pick position, grasp the peg, then generate another plan to move the peg to the predicted goal pose and drop it into the hole. The success rate over 100 trials 18% following our success metric, and 99% following the pose accuracy metric of (Zeng et al. 2020 See Appendix [F2 for more details.
5.2 IMITATION & REINFORCEMENT LEARNING WITH DEMONSTRATIONS
For the following experiments, unless specified otherwise, we use the delta end-effector pose controller for rigid-body environments and the delta joint position controller for soft-body environments, and we translate demonstrations accordingly using the approach in Sec Visual observations are captured from a base camera and a hand camera. For RGBD- agents, we use IMPALA (Espeholt et al.) 2018) as the visual backbone. For point cloud-based agents, we use PointNet (Qi et al. 2017) as the visual backbone. into the end-effector frame, and for environments where goal positions are given (PickCube and PickSingle we randomly sample 50 green points around the goal position to serve as visual cues and concatenate them to the input point cloud. We run 5 trials for each experiment and report the mean and standard deviation of success rates. Further details are presented in Appendix F.3
addition, we transform input point clouds
We benchmark imitation learning (IL) with behavior cloning (BC) on our rigid and bodt-body tasks. All models are trained for 50k gradient steps with batch size 256 and evaluated on test configurations.
Imitation Learning:
Results are shown in Table 2 For rigid-body tasks, we observe low or zero success rates. This suggests that BC is insufficient to tackle many crucial challenges from our benchmark, such as precise control in assembly and large asset variations. For soft-body tasks, we observe that it is difficult for BC agents to precisely estimate action influence on soft body properties (e.g. displacement quantity or deformation). Specifically, agents perform poorly on Excavate and Pour, as Excavate is successful only if a specified amount of clay is scooped up and Pour is successful when the final liquid level accurately matches a target line. On the other hand, for Fill and Hang, which do not have such precision requirements (for Fill, the task is successful as long as the amount of particles in the beaker is larger than a threshold), the success rates are much higher. In addition, we observe that BC agents cannot well utilize target shapes to guide precise soft body deformation, as they are never successful Pinch Write. See Appendix P.5 for further analysis.
RL with Demonstrations: We benchmark demonstration--asse online reinforcement learning by augmenting Proximal Policy Gradient (PPO) (Schulman et al. 2017) objective with the demonstration objective from Cemonstration-Auumeene
Policy Gradient (DAPG) (Rajeswaran et al. 2017). Our implementation is similar to Dia et al. (2022), and further details are presented in Appendix F3 We train all agents from scratch for 25 million time steps. Due to limited computation resources, we only report results rigid-body environments.
8

---

Published as a conference paper at ICLR 2023
Deph Senser
Robot Am
Grypar Cu
Tp Cuoo &
Simulation
Real
(8)
(b)
(c)
Left: Simulation and real world setup for PickCube. Right: Motion planning execution results for Pinch in simulation and in the real world: (a) initial state; (b) letter "M";
Figure 3:
: (c) letter "S"
Results are shown in Table We observe that for pick-and-place tasks, point cloud-based agents perform significantly better than RGBD-based agents. Notably, on PickSingleYCB, the success rate is even higher than Contact-GraspNe with motion planning. This demonstrates the potential of obtaining single agent capable of performing general pick-and-plat aaacc across diverse object geometries through online training. We also further examine factors that influence point cloud-based manipulation learning in Appendix [ However, for all other tasks, notably the assembly tasks that
require high precision, the success rates are near zero, In Appendix F.7 we show that if we increase the clearance of the assembly tasks and decrease their difficulty, agents can achieve much higher performance. This suggests that existing RL algorithms might have been insufficient yet to perform highly precise controls, and our benchmark poses meaningful challenges for the community.
In addition, we examine the influence of controllers for agent learning, and we perform ablation experiments on PickSingleYCB using point cloud-based agents. When we replace the delta end- effector pose controller in the above experiments with the delta joint position controller, the success rate falls to 0.22+0.18. The profound impact controllers demonstrates the necessity and significance of our multi-controller conversion system.
5.3 SIM2REAL
ManiSkil12 features fully-simulated dynamic interaction for rigid-body and soft-body. Thus, policies trained in ManiSkil12 have the potential be directly deployed in the real world.
PickCube: We train a visual RL policy on PickCube and evaluate its transferability to the real world. The setup is shown in Fig. LLL which consists of Intel RealSense D415 depth sensor, 7DoF ROKAE xMate3Pro robot arm, Robotiq 24- 2- 2-finger gripper, and the target cube. We first acquire the intrinsic parameters for the real depth sensor and its pose relative to the robot base. We then build simulation environment aligned with the real environment. We train a point cloud-based policy for 10M time steps, where the policy input consists of robot proprioceptive information (joint position and end-effector pose) and uncolored point cloud backprojected from the depth map. The success rate in simulation is 91. ..0% We finally directly evaluate this policy in the real world 50 times with different initial states, where we obtain 60.0% success rate. We conjecture that the performance drop is caused by the domain gap in depth maps, as we only used Gaussian noise and random pixel dropout data augmentation during simulation policy training.
Pinch: We further evaluate the fidelity of our soft-body simulation by executing the same action sequence generated by motion planning in simulation and in the real world and comparing the final plasticine shapes. Results are shown in Fig. BiR, which demonstrates that our 2-way coupled rigid- MPM simulation is able reasonably simulate plasticine's deformation caused by multiple grasps.
CONCLUSION
To summarize, ManiSkil12 is unified and accessible benchmark for generic and generalizable manipulation skills, providing 20 manipulation task families, over 2000 objects, and over 4M demonstration frames. It features highly efficient rigid-body simulation and rendering system for sample collection to train RL agents, real-time MPM- based soft-body environments, and versatile multi-controller conversion support. We have demonstrated its applications in benchmarking sense- plan-act and imitation / reinforcement learning algorithms, and we show that learned policies on ManiSkil12 have the potential to transfer the real world.

---

Published as a conference paper at ICLR 2023
REPRODUCIBLITY STATEMENT
Our Benchmark, algorithms, and applications are fully open source. Specifically, we open source the ManiSkill benchmark as well as all simulators and renderers used to build it. We open source all code for demonstration generation and controller action space conversion. We open source the entire learning framework used train Imitation / Reinforcement Learning and Sense-Plane Plan-Act algorithms. We release all training assets used in ManiSkil12. Since we will hold a public challenge based ManiSkil12, code related to asset generation and cloud-based evaluation service cannot be released for fairness. Nonetheless, all results in this work are reproducible, and we welcome researchers to participate in our challenge.
REFERENCES
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang- Lec, Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, and Andy Zeng. Do as i can and not as i say: Grounding language in robotic affordances. In arXiv preprint arXiv:2204.01691, 2022.
Kent Beck, Mike Beedle, Arie Van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, al. Manifesto for agile software development, 2001.
Greg Brockman, Vicki Cheung. Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprini arXiv:1606.1 16t.600660.0040, 2016.
Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pp- 510-517. IEEE, 2015.
Hilko Cords and Oliver G. Staadt. Instant liquids: Fast screen- space surface generation for particle- based liquids. In Poster proceedings of ACM Siggraph/Eurographic Symposium Computer Animation, 2008
Erwin Coumans and Yunfei Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. [tttp://pybulltt..oom
Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kembhavi. and Roozbeh Mottaghi. Manipulathor: A framework for visual object manipulation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4495- 4504, 2021.
Clemens Eppner, Arsalan Mousavian, and Dieter Fox. Acronym: A large-scale grasp dataset based on simulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp- 6222-622 6227. IEEE, 2021.
Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, lain Dunning, et al. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures. International conference on machine learning. pp. 1407-1416. PMLR, 2018.
Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, et Threedworld: A platform for interactive multi-modal physical simulation. arXiv preprint 2004422007.04954, 2020.
Chuang Gan, Siyuan Zhou, Jeremy Schwartz, Seth Alter, Abhishek Bhandwaldar, Dan Gutfreund, Daniel LK Yamins, James J DiCarlo, Josh McDermott, Antonio Torralba, et al. The threedworld transport challenge: A visually guided task-and motion planning benchmark for physically realistic embodied ai. arXiv preprint arXiv:2103 14025, 2021.
10

---

Published as a conference paper at ICLR 2023
Jiayuan Gu, Devendra Singh Chaplot, Hao Su, and Jitendra Malik. Multi-skill mobile manipulation for object rearrangement. arXiv preprint arXiv:2209.02777, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Gruphics (TOG), 370:) 2018
Yuanming Hu, Tzu-Mao Li, Luke Anderson, Jonathan KeelanKellly.. and Frédo Durand. Taichi: language for high-performance computation on spatially sparse data structures. ACM Transactions Gruphics (TOG), 38(6): 2019.
Zhiao Huang. Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. Plasticinelab: A soft- boft-body manipulation benchmark with differentiable physics. arXiv preprint arXiv:2 00..033.31111 2021.
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019- 3026, 2020.
Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, and Hao Su. Improving policy optimization with seneralist-speciiliist learning, In International Conference on Machine Learning. pp. 10104-10119. PMLR, 2022.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. BRLhttp: 7/arxiv. wrt. //tpxiv.rggaab////// 11122 6980
Chengshu Li, Cem Gokmen, Gabrael Levine, Roberto Martin-Martín, Sanjana Srivastava, Chen Wang, Josiah Wong, Ruohan Zhang, Michael Lingelbach, Jiankai Sun, et al. Behavior-Ik: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In 6th Anmual Conference on Robot Learning. 2022.
Xingyu Lin, Yufei Wang. Jake Olkin, and David Held. Softgym: Benchmarking deep reinforcement learning for deformable object manipulation. In Conference on Robot Learning, 2020.
Minghua Liu, Xuanlin Li, Zhan Ling, Yangyan Li, and Hao Su. Frame mining: a free lunch for learning robotic manipulation from 3d point clouds. In 6th Annual Conference on Robot Learning, 2022. URL http::///ppeeeeeeeeeeeeeeeeeeeee
Miles Macklin. Warp: A high-performance python framework for gpu simulation and graphics. bttpps://iie http:/gtttuu.ccc.. wttp://i.w.ccm//iii/aa/aaaaaaaaa March 2022. NVIDIA GPU Technology Conference (GTC).
Jeffrey Mahler, Florian T Pokorny, Brian Hou, Melrose Roderick, Michael Laskey, Mathicu Aubry, Kai Kohlhoff, Torsten Kröger, James Kuffner, and Ken Goldberg. Dex-net 1.0: A cloud-based network of 3d objects for robust grasp planning using multi-armed bandit model with correlated rewards. In IEEE International Conference on Robotics and Automation (aciooon//CCC (ICRA), pp. 1957-1964. IEEE, 2016.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2 2108. 10470, 2021.
Roberto Martín-Martnn Martin, Michelle A Lee. Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg. Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1010-1017. IEEE, 2019.
11

---

Published as a conference paper at ICLR 2023
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human- level control through deep reinforcement learning. nature, 518(7540):529-533 2015.
Douglas Morrison, Peter Corke, and Jürgen Leitner. Egad! an evolved grasping analysis dataset for diversity and reproducibility in robotic manipulation. IEEE Robotics and Automation Letters, (3):4368-433 -4136884377,, 2020.
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Cathera Yang. Xuanlin Li, Stone Tao, Zhiao Huang. Zhiwei Jia, and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-666-6666 352-660, 2017.
Yuzhe Qin, Rui Chen, Hao Zhu, Meng Song, Jing Xu, and Hao Su. S4g: single-shot se (3) grasp detection in cluttered scenes. In Conference robot learning, pp. 53-65. PMLR, 2020.
Amodal single-view
Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1-8, 2021. URL http:///mmmr..ogg/ppr..ses/
20
Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv preprint arXiv:1709.1008: 2017.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint I 2017.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. arXiv preprint arXiv:2209.0545/) 2022.
Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Martin-Martin, Fei Xia, Kent Elliott Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, Karen Liu, et al. Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. Conference Robot Learning, pp. 477-490. PMLR, 2022.
Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, and Dieter Fox. Contact-graspnet:: Efficient 6-dof grasp generation in cluttered scenes. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pp. 13438-13444, IEEE, 2021.
Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in Neural Information Processing Systems, 34:251-266, 2021.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujaco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026-5033. IEEE, 2012. doi: (009//RO...01226366666
Xinyue Wei, Minghua Liu, Zhan Ling, and Hao Su. Approximate convex decomposition for 3d meshes with collision-aware concavity and tree search. ACM Transactions Graphics (TOG), 41(4)11-18, 2022.
Jiyy Weng, Huayu Chen, Dong Yan, Kaichao You, Alexis Dubureq, Minghao Zhang, Yi Su, Hang Su, and Jun Zhu. Tianshou: A highly modularized deep reinforcement learning library. arXiv preprint
2021.
12

---

Published as a conference paper at ICLR 2023
Jiayi Weng, Min Lin, Shengyi Huang, Bo Liu, Denys Makoviichuk, Viktor Makoviychuk, Zichen Liu, Yufan Song, Ting Luo, Yukun Jiang, et al. Envpool: A highly parallel reinforcement learning environment execution engine. arXiv preprint arXiv:2206.0 200.105800,, 2022.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
11097-11107, 2020.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference robot learning. pp. (094-1100) PMLR, 2020.
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, and Johnny Lee. Transporter networks: Rearranging the visual world for robotic manipulation. Conference Robot Learning (CoRL), 2020.
Yuke Zhu, Josiah Wong, Ajay Mandlekar, and Roberto Martín-Marte robosuite: A modular simulation framework and benchmark for robot learning. arXiv preprint arXiv:2009. 12293, 2020.
13

---

Published as a conference paper at ICLR 2023
A SYSTEM DESIGN FOR DEVELOPMENT AND EVALUATION
A.1
VERIFICATION-DEES DRIVEN ITERATIVE DEVELOPMENT
Task Creation
Robot Configuration
Initial States
Asset Creation
Success Metric
Development Timeline
Generation
Verification Methods
Reward Design
Observation
Task and
Model Predictive Control
Motion Planning
Reinforcement Learning
Imitation Learning
Demonstrations
Figure 4: The workflow to build environments for generalizable manipulation skills.
Our workflow, following agile software development (Beck al., 2001), highlights a verification-- driven iterative development process, Different approaches can be used to generate demonstrations according to characteristics and difficulties of tasks, which verifies environments as byproduct. The workflow is also designed to be scalable and affordable to continuous integration of assets and tasks. Fig 4 iillustaaees the workflow.
Our workflow consists of 3 stages: task creation, reward design, and observation configuration. The first stage focuses on building task essentials, including creating assets, (e-g-. convex decomposition (Wei et al. (022). texture baking), configuring robots, generating initial states, and defining success metrics. The second stage aims at prototyping shaped reward functions. The reward function is requisite for methods like Model-Predictive Control (MPC) and RL. The third stage addresses observation spaces. For instance, camera parameters and placements need to be tailored for tasks that observations contain adequate information. To collect demonstrations and verify environments, we employ one of or a mixture of 3 complementary approaches: task and motion planning (TAMP), MPC, and RI.
Different methods have their own advantages and disadvantages. TAMP is free of crafting reward, and is suitable for many stationary manipulation tasks like pick-and-place, but shows difficulty when tackling underactuated systems (e.g. pushing chairs and moving buckets in Mu et al.) (202I). MPC able to search solutions to difficult tasks given well-designed shaped rewards without training or observations. However, it is non-trivial to design a universal shaped reward for variety of objects. RL requires additional training and hyperparameter tuning, but more scalable than MPC during inference.
A.2 CLOUD BASED EVALUATION SYSTEM
To allow the community evaluate and benchmark together we build and provide a simple cloud based evaluation system. Users can register accounts and enter the benchmark and begin making submissions that solve our various tasks.
A key feature of the evaluation system is flexibility. This is increasingly important the number of unique approaches from heuristics, motion planners, and end-to-end RL solutions grow. Evaluation systems need the flexibility to allow all kinds of approaches be benchmarked. To this end, do the following.
User submissions are in the form of docker images, allowing to install any code and save any models necessary to solve various tasks. This makes programming on the user's side very flexible. The user only needs provide function that accepts observations and returns actions.
14

---

Published as a conference paper at ICLR 2023
We further give users the flexibility to configure the evaluation environment suit their needs before running the evaluation. For example, users can define a configuration function their solution that sets the observation mode (e.g. RGB-D Point Cloud), as well controllers (e.g. delta end- effector pose or joint position).
To benchmark submitted docker image, we simply pull it server and run the evaluation code that loads the user's ss solution. The results of the evaluation are then submitted to a database and displayed on pubic benchmark.
B DETAILS OF OUR COMPREHENSIVE CONTROLLER SUITE
Controllers are interfaces between policies and robots. Policies output actions to controllers, and controllers convert actions control signals drive the robot joints. ManiSkil12, the default robot being controlled is the Franka Emika, also known as Panda. The degree of freedom (DoF) of single Panda arm
B.1 TERMINOLOGY
1. Fixed Joint: A joint that cannot be controlled. The degree of freedom (DoF) is 0.
2. qpos (q): Controllable joint positions.
3. qvel (ÿ): Controllable joint velocities.
4. Target Joint Positions (7): Target positions of motors that drive each joint.
5. Target Joint Velocities (?): Target velocities of motors that drive each joint.
6. End-effector Position (p): The position of an end-effector.
7. End-effector Rotation (R): The rotation of an end-effector.
8. End-effector Target Position (p): The target position of an end-effector.
9. End-effector Target Rotation (R): The target rotation of an end-effector.
10. PD Controller: Control loop based r(t) (g(t() (()) + Ka(ū(t) i(t)). Kp (stiffness) and Ka (damping) are hyperparameters. T(t) denotes the torque (generalized force) of the motors.
11. Augmented PD Controller: Augmented PD controller: Passive forces (like gravity) are compensated for the PD controller.
12. Action (a): Input the controllers, and also output of the policy.
13. Tool Center Point (TCP): TCP a user defined coordinate frame, often relatively fixed to the robot end effector. For example, in our case, if the robot uses two-finger gripper. TCP is defined at the center point between the gripper's 's two fingers.
B.2 TARGET VS. NARGET CONTROLLERS
In our controller implementation, we have the notion of "target" and "non-target" controllers. For example, when we say deltu end-effector pose controller, new desired pose specified relative to the current end-effector pose. In contrast, if we say larget delta end-effector pose controller, the new desired pose is specified relative to the previous desired pose. Please read the next section for their formal definitions.
B.3 NORMALIZED ACTION SPACE
We design the action space of controllers conform to the preferences of users. As RL users generally prefer normalized action space, most controllers listed below will have a normalized action space [-1,1], 1], with few exceptions listed individually.
15

---

Published as a conference paper at ICLR 2023
B.4 DETAILS CONTROLLERS
B.4.1 ARM CONTROLLERS
arm pd joint pos (7-dim, unnormalized): 0z qt- The target joint velocities de are always 0. As this controller is suitable for motion planning, the action space of this controller is not normalized.
2. arm joint delta pos (7-dim):
---1-
3. arm
(7-dim): 0r = q: 4-
4. arm am-ep_eee_aa pdd_e_deettapps (3-dim): a₁ P: Pt-1. where Pt is the position of the end-effector timestep The controller then internally computes the fargetjoinn positions of the robot:
IK (pt, Rt), where (-,-) computes the joint positions from the end-effector's position and rotation before sending the joint positions the controller. Note that this controller only controls the position, but not the rotation, of the end-effector.
5. arm arm-pd pdeedddeeeee eargett pelta.pos delta (3-dim): a, = Pc P
6. arm wtwpppddeeelaa__eeeeeeeppeeeeeee arm pre- -e-_dddeedelltppoo controller with the addition of end-effector's rotation being passed in as input. Thus T; = To Ti-1, where I is the target transformation of the end-effector, and Ta the delta pose induced by the position and 3D rotation (represented compact axis-angle of the action.
(6-dim): This controller
very similar to the previous
7. arm am-ppdee__argge carget.ded pose (6-dim): T, Ta Ti-1-
8. arm.pd_joint_vel (7-dim): a₂ = -c- The stiffness value K, of this controller is always set 0.
9. arm podjoointpposs.s (14-dim): An extension of arm_pd joint joint_poss pos that supports target velocities input.
10. arm pd joint delta pos (14-dim): The delta variant of the arm pd joint pos vel controller.
B.4.2 GRIPPER CONTROLLER (1-DIM)
We use joint position control for the gripper, and we force the two gripper fingers to have the same target position.
B.5 EFFECTIVENESS CONVERSION DEMONSTRATION ACTION SPACES
In this section, exemplify the success rate of our demonstration conversion method by converting from the arm_pd joint_jinnaee ppos controller to the arm-pd _eteepd_e___e _deeta_poosee controller. demonstration is converted successfully if, following the same trajectory initialization and the converted actions, the task is successful at the last time step. We experiment on 4 representative tasks: PickSingleYCB, Assembling TurnFaucet. Write. For each task, we select 100 demonstrations randomly. The success rates for PickSingleYCB AssemblingKits Write, TurnFaucet are 99%, 98%, 100%, and 80%, respectively. Note that our policy generate demonstrations for TurnFaucet involves rich and inconsistent contact between the end-effector and the faucet handle (i.e., our policy uses the gripper to push the handle, rather than grasping and rotating it, which case force closure can lead to more consistent contact). Such polices be sensitive accumulated emrors during execution, which can result in task failure, although the actions converted by our demonstration conversion method attempt to reproduce motion faithfully.
C DETAILS OF OBSERVATIONS, TASK FAMILIES, DEMONSTRATIONS AND EVALUATION PROTOCOLS
Unless otherwise noted, all demonstrations are generated through TAMP. For evaluation, we employ two-stage setup. Final result is the average result from the two stages.
16

---

Published as a conference paper at ICLR 2023
C.1 SUPPORTED OBSERVATION MODES
We support most observation modes found in OpenAl gym (Brockman et al. 2016). The details of each observation mode are listed below.
1. state dict: Returns dictionary of states that contains robot proprioceptive information, ground truth object information (such as object poses), and task-specific goal information (if given). Visual observations (images and point clouds) are not included.
2. state: Returns the flattened version of state_dict.
3. rgbd: Returns rendered RGBI) images from all cameras, along with robot proprioceptive information and task-specific goal information (if given).
4. robor.o On top of rgbd, returns ground truth segmentation masks for the robot joints.
5. pointcloud: Returns fused point cloud from all cameras, along with robot proprioceptive information and task-specific goal information (if given).
6. pointcloud robot On top of pointcloud, returns ground truth segmentation masks for the robot joints.
Here, the robot proprioceptive information includes joint positions, joint velocities, the pose of the robot base, along with the pose of the gripper's tool center point if the robot uses a two-finger gripper.
Note that different from the previous version of ManiSkill, ManiSkil12 does include ground- truth segmentation in the default observation modes (rgbd or pointcloud). All visual-based experiments this paper do not leverage such privileged information. For example, specify which link of a faucet should be manipulated in TurnFaucet, we use its initial position instead of a ground- truth segmentation mask. Besides, also support observations modes (rgbd+robot ssgg pointcloud+ robottsege to provide the segmentation masks of robot links, which facilitates robotic applications and can be obtained the real world using the robot proprioceptive information.
C.2 PICK-AND-PLACE
PickCube
Objective: Pick up cube and move it goal position.
Success Metric: The cube is within 2.5 cm of the goal position, and the robot is static.
Demonstration Format: 1,000 successful trajectories.
Evaluation Protocol: 100 episodes with different initial joint positions of the robot and initial cube pose for each of stage and stage 2.
Task-specific Extra Observations: 3D goal position of the cube.
StackCube
Objective: Pick up a red cube and place onto green one.
Success Metrie: The red cube is placed top of the green stably and it is not grasped.
Demonstration Format: 1,000 successful trajectories.
Evaluation Protocol: 100 episodes with different initial joint positions of the robot and initial poses of both cubes for each of stage and stage 2.
Task-specific Extra Observations: None.
PickSingleYCB
Objective: Pick up a YCB object and move it to a goal position.
Success Metrie: The object within 2.5 em of the goal position, and the robot static,
Demonstration Format: 100 successful trajectories for each of the 74 YCB objects.
17

---

Published as a conference paper at ICLR 2023
Evaluation Protocol: In addition to the training objects, also use another confidential set of 40 objects from other sources as the test object set. For the two evaluation stages in total, for each object in the training set, we test 5 episodes with different seeds. For each object in the test set, we test 10 episodes with different seeds. Half the objects are evaluated in each stage.
Task-specific Extra Observations: 3D goal position of the object.
PicKSingleBGAD
Objective: Pick up an EGAD object and move to a goal position. The color for the EGAD object is randomized.
Success Metrie: The object within 2.5 em the goal position, and the robot statie.
Demonstration Format: trajectories for each of the 1,600 training objects sampled from EGAD. For certain objects where it's difficult to apply TAMP, we might provide less than 5 trajectories.
Evaluation Protocol: For this task, have held portion of the EGAD dataset. This held-out test dataset consists of 150 objects. During evaluation, in each stage, we evaluate trajectory for each of the 150 objects sampled from the training dataset and 2 trajectories for each of the 75 objects sampled from the held-out test dataset.
spec-ipeccfic Extra Observations: 3D goal position of the object.
PickClutterCCB
Objective: Pick up an object from a clutter of 4-8 YCB objects.
Success Metrie: The object is within 2.5 em of the goal position, and the robot is static.
Demonstration Format: A total of 4986 trajectories from the training object set.
Evaluation Protocol: addition to the training objects, also use another confidential of 40 objects from other sources as the test object set. For each evaluation stage, we test 100 episode configurations on the training object set and on the test object set.
Task-specific Extra Observations: 3D position of the object to pick up, and 3D position of the goal.
C.3
ASSEMBLY
AssemblingKits
Objective:
Insert an object into the corresponding slot on plate.
Success Metric: An object must fully fit into its slot, which must simultaneously satisfy 3 criteria: (1) height of the object center within 3mm of the height of the plate; (2) rotation error is within 4 degrees; (3) position
is within 2cm.
Demonstration Format:
We provide 1,720 trajectories in total. These trajectories are generated from over 300 kit configurations and 20 training shapes.
This task has held-out test dataset for evaluation. The test dataset features 20 shapes that are similar to the shapes the training set. provide samples for test assets in Fig. 5 In each evaluation stage, evaluate 100 sampled training episode configurations and 100 sampled test dataset configurations.
Evaluation Protocol:
Task-specific Extra Observations: 3D initial and goal position of the object be placed.
PeglnsertionSide
Objective: Insert peg into the horizontal hole in box.
Success Metrie: Half of the peg is inserted into the hole.
Demonstration Format: 1,000 successful trajectories.
Evaluation Protocol: 100 episodes with different initial joint positions of the robot, initial poses of the peg and box, the position and size of the hole for each stage and stage
Task-specific Extra Observations: None.
18

---

Published as a conference paper at ICLR 2023
FI
Figure 5: A sample plate with test assets.
PlugCharger
Objective: Plug charger into wall receptacle.
Success Metric: The charger is fully inserted into the receptacle.
Demonstration Format: 1,000 successful trajectories.
Evaluation Protocol: 100 episodes with different initial joint positions of the robot, initial poses of the charger and wall for each of stage 1 and stage
Task-specific Extra Observations: None.
C.4 MISCELLANEOUS TASKS
AvoidObstacles
Navigate the robot arm through a region of dense obstacles and move the end-effector to goal pose. The shape and color of dense obstacles are randomized.
Objective:
Success Metric: The end-effector pose is within 2.5 cm and 15 degrees of the goal pose.
Demonstration Format: 1976 trajectories for different layouts.
Evaluation Protocol: 100 episodes with different layouts of obstacles for each of stage 1 and stage
2.
Task-specific Extra Observations: The goal pose of the end-effector.
TurnFaucet
Objective: Turn faucet by rotating its handle.
Success Metric: The faucet handle has been turned past a target angular distance.
Demonstration Format: For most faucet models, we provide 100 trajectories per asset. For approximately 15 of the 60 training models were TAMP cannot find a solution, demonstrations are generated through MPC-CEM using our designed rewards.
Evaluation Protocol: This task has held-out test object set. For the two evaluation stages in total, we evaluate episodes for each of the 60 training objects and 17 episodes for each of the test objects. Half of the objects are evaluated in each stage.
specifc Extra Observations: The remaining angular distance to rotate the handle, the target handle position (since there can be multiple handles in a single faucet), and the direction to rotate the handle specified as 3D joint axis.
C.5 SOFT-BODY MANIPULATION
Fill
19

---

Published as a conference paper at ICLR 2023
Objective: Fill clay from bucket into the target beaker.
Success Metric: The amount of clay inside the target beaker > 90%; soft body velocity < 0.05.
Demonstration Format: 200 successful trajectories generated through motion planning.
Evaluation Protocol: 100 episodes with different initial rotations of the bucket and initial positions of the beaker for each of stage and stage 2.
Task spec-sc Extra Observations: Beaker position.
Hang
Objective:
Hang noodle on a target rod.
Success Metric: Part the noodle is higher than the rod; two ends of the noodle are on different sides of the rod; the noodle is not touching the ground; the gripper is open; soft body velocity < 0.05.
Demonstration Format: 200 successful trajectories generated through motion planning.
Evaluation Protocol: 100 episodes with different initial positions of the gripper and rod poses for each of stage and stage 2.
spec-fc Extra Observations: Rod position.
Excavate
Objective: Lift specific amount of clay to target height.
Success Metric: The amount of lifted clay must be within a given range; the lifted clay is higher than a specific height; fewer than 20 clay particles are spilled on the ground; soft body velocity
< 0.05.
Demonstration Format: 200 successful trajectories generated through motion planning.
Evaluation Protocol: 100 episodes with different bucket poses and initial heightmaps of clay for each of stage and stage 2.
Task-specific Extra Observations: Target clay amount.
Pour
Objective:
Pour liquid from bottle into beaker.
Success Metric: The liquid level in the beaker is within 4mm of the red line; the spilled water is fewer than 100 particles; the bottle returns to the upright position in the end; robot arm velocity
0.05.
Demonstration Format: 200 successful trajectories generated through motion planning.
Evaluation Protocol: 100 episodes with different bottle positions, the level of water in the bottle, and beaker positions for each stage and stage
Task-specific Extra Observations: Red line height.
Pinch
Objective:
Deform plasticine into target shape.
Success Metric: The Chamfer distance between the current plasticine and the target shape is less than 0.31, where the Chamfer distance between the initial shape and target shape.
Demonstration Format: 1556 successful trajectories generated through heuristie motion planning. Different trajectories correspond to different target shapes.
Evaluation Protocol:
50 episodes with different target shapes for each stage and stage
Task-speciff Extra Observations: RGBD point cloud observations of the target plasticine from 4 different views.
20

---

Published as a conference paper at ICLR 2023
Write
Objective: Write given character on clay. The target character is randomly sampled from an alphabet of over 50 characters.
Success Metric: The IoU (Intersection over Union) between the current pattern and the target character is larger than 0.8.
Demonstration Format: 200 successful trajectories generated through heuristic motion planning.
Evaluation Protocol: 50 episodes with different target characters for each of stage and stage 2. spec-sp Extra Observations: The depth map the target character.
C.6 MOBILE MANIPULATION
O PenCabinetDrawer
Objective: A single-arm mobile robot needs to open designated target drawer cabinet. The friction and damping parameters for the drawer joints are randomized.
Success Metric: The target drawer has been opened least 90% of the maximum range, and the target drawer is static.
Demonstration Format: 300 trajectories for each target drawer in the training object set. The training object set consists of 25 cabinets. Each cabinet could contain multiple drawers.
Evaluation Protocol: This task has held-out test object set (10 unseen cabinets). In the first stage, we evaluate 250 trajectories in total. Among these 250 trajectories, 125 levels are evenly distributed over 5 unseen objects in the test set, and the other 125 levels are evenly distributed over all objects in the training set. In the second stage, we evaluate another 250 trajectories. Similarly, 125 levels come from the training set and the other 125 levels from the 5 other unseen objects in the test (different from the test objects stage 1).
Task-specific Extra Observations: Since cabinet can contain several drawers, specify the target drawer by its initial center mass.
Cabnnnninttooor
Objective: A single-arm mobile robot needs open designated target door on a cabinet. The friction and damping parameters for the door joints are randomized.
Success Metrie: The target door has been opened least 90% of the maximum range, and the target door static.
Demonstration Format: 300 trajectories for each target door the training object set. The training object set consists of 42 cabinets. Each cabinet could contain multiple doors.
Evaluation Protocol: This task has held-out test object set (10 unseen cabinets). The evaluation protocol is similar OrenCabineDDrawer
Task-specific Extra Observations: Since one cabinet can contain several doors, we specify the target door by a segmentation mask.
PushChair
Objective: A dual-arm mobile robot needs to push a swivel chair to a target location on the ground (indicated by red hemisphere) and prevent it from falling over. The friction and damping parameters for the chair joints are randomized.
Success Metric: The chair is close enough (within 15 cm) to the target location, is static, and does not fall over.
Demonstration Format: 300 trajectories for each chair in the training object set. The training object set consists of 26 chairs.
Evaluation Protocol: This task has held-out test object set (10 unseen chairs). The evaluation
protocol is similar to OpenCabinetDraver
Task-specific Extra Observations: None.
21

---

Published as a conference paper at ICLR 2023
MoveBucket
Objective:
A dual-arm mobile robot needs to move bucket with a ball inside and lift onto platform.
The bucket is placed on or above the platform at the upright position, and the bucket is static, and the ball remains bucket.
Success Metric:
Demonstration Format: 300 trajectories for each bucket in the training object set. The training object set consists of 29 buckets.
Evaluation Protocol: This task has held-out test object set (10 unseen buckets). The evaluation protocol is similar to Oreeaeri
Taek-specific Extra Observations: None.
D SOFT-BODY DETAILS
D.1 SOFT-BODY SIMULATION AND WAY COUPLING ALGORITHM
The simulation and 2-way coupling algorithm are summarized in algorithm I
Algorithm Rigid MPM Simulation and Dynamic Coupling
initialize rigid scene
initialize soft scene
copy rigid body shapes and center of mass to soft scene
initialize renderer
for environment step do
execute ManiSkill2 controllers
for rigid step per environment step do
process ManiSkill2 substep
step rigid scene
copy rigid body poses to soft scene initialize force-torque buffers per rigid body
for soft step per rigid step do compute penalty forces
accumulate equivalent rigid-body forces and torques force-torque buffers MPM particle to grid (mass, momentum, forces)
MPM grid compute velocity
MPM grid particle (velocity)
integrate MPM particles end for
apply accumulated forces and torques on rigid bodies end for
copy rigid body states to SAPIEN renderer
copy MPM particles to SAPIEN renderer execute renderer
end for
D.2 SOFT-BODY RENDERING
To support visual learning, we extended SAPIEN's renderer to support rendering particle-based soft body. To render particle-based material, one common approach convert the particles triangle meshes using meshing algorithm such as marching cubes; PlasticineLab implements ray-tracing framework that renders spheres directly. Our approach screen-space splatting (Cords & Staadt, 2008), similar to Nvidia Flex's built-in renderer. We customize SAPIEN's shader to render soft- body particles as spheres, use a bilateral filter to smooth the depth buffer, then compute normal and lighting the smoothed soft-body depth. These implemented extra screen-space render passes. The effect of the smoothing filter is shown in figure Moreover, we customize Warp to support the transfer of partiele positions from simulation renderer with single GPU-GPU copy;
22

---

Published as a conference paper at ICLR 2023
this further reduces rendering latency. A concemn of the sereen-space splatting is the inconsistency across different views due to the use of sereen-space filters. However, in practice, by scaling the bilateral filter according to pixel distance from camera, the rendering results produced are visually consistent most the time.
(b)
Figure 6: Our particle renderer (a) without bilateral filter, and (b) with bilateral filter.
D.3 SOFT BODY DETAILS
We summarize the key parameters In table For all soft body environments with rendering, a single environment runs at around 17-18 FPS; 16 parallel environments single GPU gives overall 80- 84 FPS on single RTX Titan GPU (4x real time). This performance gain is mainly due to the sequential CPU-rigid-bor baddiddbody and GPU-soft- processor not able to submit CUDA kernels fast enough to keep up with the GPU. Therefore, it can potentially further optimized with vectorized simulation, which we leave as future work.
simulation. is also partially due to a single CPU
Grid Length
Particle Volume|
Density
Young's Modulus Poisson Ratio Yield Stress
0.005 to 0.015 6.2e-8/1.2e.
330 3000
le4/3e5
0.3
2e3/1e4
Table 4: Ranges for key parameters used in our MPM simulation. All fields are in SI units.
E PERFORMANCE OPTIMIZATION DETAILS
E. RENDER SERVER IMPLEMENTATION
Our render server implemented with the gRPC framework, which exchanges Protocal Buffers with the HTTP 2 protocol over networks or unix sockets. The server side is managed by a thread pool, listening client requests on multiple concurrent threads. For "take picture" request from a worker process, our implementation puts the task of updating GPU matrices and launching draw calls onto another thread and returns immediately back to the worker process. This ensures minimum waiting time on the worker side.
On the rendering server, all rendering resources are managed by central resource manager. Any resource loading request (c.g.. models, images, textures) must through the manager. The manager ensures only one copy of any resource is loaded onto the GPU, shared across potentially multiple scenes.
E.2 ADDITIONAL BENEFITS OF RENDER SERVER
In general, rendering resource sharing across processes is not well supported. Rendering frameworks like OpenGL and Vulkan are designed to be efficient in single-process multi-threaded environment. Resources and documentations on multi-process renderer are rarely provided. Therefore as developer, it is far easier to understand renderers in single process setting.
In addition, Nvidia's GPU profiler, Nsight System, is currently unable to profile Vulkan multiple processes Linux in experiments. Running the renderer in multiple processes makes it hard to understand GPU performance. Thus, rumning rendering in one main process is almost required for developer to optimize for hardware utilization.
23

---

Published as a conference paper at ICLR 2023
Sample collection speed with Nature CNN
Sample collection speed with random actions
Hublaraa sil
Baac
leac Opm
25000
MarthFl-Seve
Overall FPS
2000
Marbbr Wyw.en--yesoeol Sysc-lonk
Overall FPS
2030
Acsoduttee
Handod fordrr
1500
15:0
1000
1000
128 512
#Paraliel Environmntss
Parrlllel Environments
Figure Comparison of sample collection speed (FPS) with random actions and with Nature CNN-sampled actions across different frameworks and different numbers of parallel environments. "ManiSkill-SSnn-RReat " refers to ManiSkil12 with synchronous rendering and without render server. "ManiSkill2" refers to ManiSkill2 with asynchronous rendering but without render server. "MamiSkill2-Serrv" refers to ManiSkil12 with both asynchronous rendering and render server enabled. Some curves are not fully drawn beyond certain number of parallel environments due to performance drop ("ManiSkill2-Server & "Isaac Gym"), GPU out of memory ("RoboSuite" & "Mamiikiil2-SSnn-RRRe Renderr & "ManiSkill2"), crash ("Habitat 2.0").
E.3 MORE DETAILS ON SAMPLE COLLECTION SPEED COMPARISON
We use the PickCube environment to compare the sample collection speed between different simulators. The resulting total number of vertices of collision meshes is 1009 for ManiSkill2 and Habitat-2.0 and 1210 for IsaacGym and Rohosuite. The number of vertices of visual meshes is 69747 for all simulators. Each control step consists 25 physical simulation steps. We the GPU pipeline for rendering. All frameworks are given a budget of 16 CPU cores(logical processors) of Intel 19-9960X CPU with 128G memory and RTX Titan GPU with 24G memory.
To complement results in Table I] we plot further details about the relationship between the number of parallel environments and the policy sample collection speed across different frameworks in Figure 7 We adopt an agent that outputs random actions, along with a CNN-based agent that uses randomly-initializen nature CNN (Mnih al., (015)) as its visual backbone. We observe that ManiSkill2 with asynchronous rendering enabled (and without the render server) is already able to outperform the speed of other frameworks. With render server enabled, ManiSkil12 further achieves 2000+ FPS with 16 parallel environments single GPU.
F ADDITIONAL EXPERIMENT DETAILS, RESULTS, AND ANALYSIS
F.1 CONTACT-GRASPNET PickSingleYCB
We directly use the pretrained model provided by the original authors of Contact-GraspNe (Sundermeyer et al. (021). We exemplify successful trajectories by the model in Fig. 8 as well as common failure modes in Fig. g
(a)
(b)
(c)
(d)
Figure 8: Sampled frames demonstrating correct and successful grasp of can. Frame (a) shows the initial state; (b) shows the gripper approaching the predicted grasp from above; (c) shows the gripper grasping the can; (d) shows the robot moving the can towards green goal position.
24

---

Published as a conference paper at ICLR 2023
Figure 9: Examples of unsuccessful grasps, (a) shows an erroneous rotation prediction in grasp pose; (b) shows a correct rotation prediction in grasp pose, but the gripper is not close enough the object to grasp it; (c) shows reasonable grasp pose, but the gripper will slip away from the bottle upon finger closure due friction and bottle geometry; (d) shows a reasonable grasp pose that is not achievable through motion planning due to kinematic constraints of the robot arm.
F.2 TRANSPORTER NETWORK FOR AssemblingKits
Gripper Type
Success (ours) Success (Zeng all) Pos
5mm
2.5mm
Imm Rot < 44| 1'
||ot 0.25
Two-finger Gripper
0.18
0.98 0.93
0.98
0.80
0.30
0.96 0.91
0.48 0.48
0.22
Suction Gripper
0.15
0.89
0.66
0.18
0.22
Gripper Type
Success (ouns)
Success (Zeng et al.) Pos Smm Pos < 2.5mm Pos < 1mm Rot < 4" |ot < 1 Rot 0.25
Two-linger Gripper
0.18
0.99
0.99
0.82
0.31
0.96
0.48
0.24
Suction Gripper
0.14
0.96
0.92
0.66
0.17
0.94
0.52
0.31
Table 5: Success rate of Transporter Networks on our AssemblingKits task on training assets (up) and test assets (down). We also report the success rate based on the original paper, where trial is successful if the target piece is placed within Icm and 15 degrees of the goal position.
We adopt the official code from Zeng et al. (2020). To encourage precise rotation prediction, we increase the number of rotation prediction bins from 36 in the original work 144. We further benchmark with two grippers: suction gripper following the original work, and another two-finger gripper from Franka Panda. Our training dataset is generated from random initial configurations of training assets in AssemblingKits. More specifically, for each sampled initial configuration, capture RGBD images from the hand-camera and the base-camera, unproject them to 3D point clouds, fuse the point clouds, and render the top-down orthographic image to feed to the Transporter Network model. The ground truth labels, namely the object pick position and the goal pose to place the object, are directly obtained from the environment state information.
Table 5] the results. The success rate of Transporter Network following our success criterion (which requires pieces to fully fit in holes) is lot lower than following the metric in the original paper. We observe that failure modes are mainly due to imprecise rotation/position prediction for placing the target piece.
F.3 DETAILED SETUP FOR IMITATION LEARNING & RL FROM DEMONSTRATIONS
In ManiSkil12, our demonstrations are provided using the joint position controller. Before train demonstration based agents on rigid & soft-body tasks, we first use the approach in Sec 2.2 translate the provided demonstrations into the delta end-effector pose controller for rigid-body environments, and into the delta joint position controller for soft-body environments. We then filter the translated trajectories, such that only successful trajectories are used for agent learning.
For RGBD-based agents, we use IMPALA (Espeholt et al. 2018) as the visual backbone, and we concatenate images captured from the base camera and the hand camera as visual input. Image resolution is 128x128. For point cloud-based agents, use PointNet (Qi et al.) 2017) the visual backbone. We first obtain a single fused point cloud by transforming point clouds from different cameras into the robot-base frame and concatenating the points together. We then remove the ground using height clip and randomly downsample the point cloud to 1200 points. For rigid- body environments, also transform the point cloud (along with robot proprioceptive information
25

---

Published as a conference paper at ICLR 2023
Obs. Mode
PickSingleYCB
AssemblingKits
TurnFaucet
AvoidObstacles
Point Cloud
0.36
0.06
0.00 + 0.00
0.01 + 0.01
0.00 1 0.00
RGBD
0.08 0.03
0.00 0.00
0.01 = 0.01
0.00 + 0.00
Table 6: Mean and standard deviation of success rates of DAPG+PPO on rigid-body tasks on held- test objects. Training budget is 25M time steps.
and goal position, if given) into the end-effector frame. In addition, for environments where goal positions are given (PickCube and PickSingleY we randomly sample 50 green points around the goal position to serve as visual cues and concatenate them to the input point cloud.
To train demonstration-basel agents, for rigid-body environments, we use 1000 demonstration trajectories, except environments that have multiple assets (PickSingleYCB: 7300 trajectories; TurFFaucet 4500 trajectories; AssemblingKits: 1700 trajectories). For soft-body environments, we use 200 demonstration trajectories (except Pinch with 1550 trajectories).
In this work, our demonstration-basse Policy Gradient (PPO) (Schulman al., (017) and Demonstration-AAgmmeee Policy Gradient (DAPG) [Rajeswaran et al.) (017). We adopt the training objective modified from Jia et al. (2022). Here, we use Pn and P= denote the distribution of demonstration transitions and online environment rollout transitions, respectively. We can then write the overall policy loss follows (value loss omitted here):
online RL algorithm is modified from Proximal
" CCLIP(() B( min To(als)
To.a (als)
(0) = Bec,a)~ [Ta[ss [(e(a))((To(((a))
(DAPG+PPO(O) = LCLIP (0) + w (0)
PM
We set w 0.1 0.995 where is the epoch count for PPO. PPO epoch is defined as online environment sampling steps followed by policy and value network updates.
For further details about networks and algorithm hyperparameters, see Appendix F.8
F.4 RESULTS FOR DAPG+PPO ON HELD-OUT OBJECT SETS
In Section 5.2 for tasks that have asset variations, presented evaluation results on the training object set. In this section, we present results on the held-out object set. See Table 6 for details.
F.5 FURTHER ANALYSIS OF IMITATION LEARNING ON SOFT-BODY TASKS
We observe that is difficult for BC agents to accurately estimate the influence of an action on fine- grained soft body properties, such as displacement quantity and deformation. For example, both Fill and Pour tasks require robot agent to move soft body objects (clay or liquid) into target container, but Fill has a much higher success rate. An underlying cause is that Fill allows the robot agent to put all of the clay into the beaker while Pour requires higher precision i.e. the final liquid level must match the target line. Therefore, agents need to precisely control the bottle tilt angle in order to precisely control the amount of liquid poured into the beaker. Similarly, for Excavale, agents must perform fine judgments on how deep they should dig order to scoop up a specified amount clay. On the other hand, Hang does not require agent to perform high accuracy measurements, is easier for agents to succeed.
In addition, notice that BC agents cannot well utilize target shapes to guide precise soft body deformation. Specifically, for Pinch and Write that require shape deformation, the BC models have very poor performance. As shown in Fig. I0 the robot learns the motion of pinching and has made some progress toward the goal, but not good enough. Similarly, the robot agent has learned to draw some patterns, but they are not close enough the target character.
26

---

Published as a conference paper at ICLR 2023
Behaviour Cloning Results
Goals
Behaviour cloning examples for Pinch and Write tasks. BC models have learned to make some progress towards the goals but not enough meet the success conditions.
Figure 10:
Original
Delta Joint Position Controller Robot Base Frame Point Cloud Remove Visual Goal Cues
0.51 + 0.05
0.22 + 0.18
0.00 + 0.00
0.16 + 0.07
Table 7: Ablations PickSingle (training object set) for point cloud- basudd agents trained with PPOG+PPO. The "original" result refers to the result in Table which uses the delta end-effector pose controller, transforms inputs point clouds into the robot's end-effector frame, and (for pick-and- place tasks where goal position is given) appends 50 green points sampled around the goal position the input point clouds serve visual cues.
F.6 MORE RESULTS ON POINT CLOUD-BASER MANIPULATION LEARNING
In this section, examine factors that influence point cloud-based manipulation learning to complement the results in Section 5.2 Results are shown in Table 7 addition to our prior observation that choices of controllers have significant effect performance,
also observe that (1) selecting good coordinate frames to represent input point clouds could be crucial for agents' success, which corroborates the findings in Liu et al. (2022): (2) adding proper visual cues could benefit point floud-based agent learning.
F.7 MORE ANALYSIS ON ASSEMBLY TASKS
we observed that agents trained with Imitation Learning or Reinforcement Learning perform poorly on many tasks that require high precision, such the assembly tasks. We further analyze sources of difficulty from these tasks by training agents on easier versions of these tasks where the clearance threshold is significantly increased. Results are shown Table We observe that when we increase clearance threshold and decrease task difficulty, some agents are able to achieve lot higher performance. However, even in this case, many agents still do not perform well. We conjecture that this is due to many other challenging factors, such as occlusions of the target slots when agents attempt to insert a peg or charger.
In Section 5.2
F.8 NETWORK ARCHITECTURES AND HYPERPARAMETERS FOR IL & RL
In this section, we present the specific network architectures and algorithm hyperparameters used for Section 5.2 Here define "state vector" the concatenation of proprioceptive robot state information and task-specific goal information (if given).
For IMPALA (Espcholt et al. 2018), the visual backbone is similar ResNet10 (He et al. 2016), with hidden size in all layers equal to 64, normalizations removed, and the first convolution layer modified to have kernel size 4 and stride 4. Features from the last layer are projected 384-
27

---

Published as a conference paper at ICLR 2023
Observation Mode |PogInsertionSide((X (xx)
PegInsertionSide(11x PlugCharger(1x)
PlugCharger(10xs
Point Cloud
0.00 + 0.00
0.01 T 0.01
0.00 1 0.00
0.01 0.01
RGBD
0.00 + 0.00
0.00 0.00
0.00 0.00
0.00 0.00
Observation Mode |egglnsertionSide(1X Ix) PeglnsertionSide((())) PlugCharger(1x)
PlugCharger(10x)
Point Cloud RGBD
0.01 T: 0.02 0.01 + 0.01
0.74 T1 0.10
0.01 + 0.01
0.02 0.02
0.05 0.03
0.01 + 0.01
0.29 0.07
Table 8: Analysis of IL & demoontratioonbusse RL on assembling tasks. We control the difficulty of tasks by changing clearance configurations. Top: Behavior Cloning. Bottom: DAPG+PPO. Ix=default clearance (3mm for PegInsertionSide and 0.5mm for PlugCharger); 10x 10 times default clearance.
dimensional vector before being concatenated with the state vector. For PointNet (Qi et al. 2017),
the hidden layer sizes are [64, 128, 512] before maxpooling over the number-of-poine dimension. We do not spatial normalization layers, and we add layer normalization to the output of each intermediate layer before maxpooling.
For both BC and DAPG+PPO, we use learning rate of 3e-4 for training and optimize with Adam KKingma Ba 2015). For DAPG+PPO, the actor and critic networks share the same visual backbone. In addition, when training point cloud-based policies, we initialize the linear layer right before the policy and value head output to be zero. We find this helpful for stabilizing point cloud- based policy learning. Hyperparameters are shown in Table
Hyperparameters
Value
Discount (?)
0.95
in GAE
0.95
PPO clip range
0.2
Max KL
0.1
Gradient norm clipping
0.5
Entropy
0.0
Number of samples per PPO step
20000
Number of samples per minihatch
300
Number of critic warmup epochs Number of PPO update epochs Critic loss coefficient Demonstration loss coefficient
4
2
0.5
0.1
0.995
Recompute advantage after each PPO update epoch
True
Reward normalization Advantage normalization
True
True
Only use critic loss to update visual backbone Reset environment upon suceess during policy rollout
True
False
Table Hyperparameters for DAPG+PPO.
28

---

Published as a conference paper at ICLR 2023
G COMPARISON WITH OTHER BENCHMARKS FOR ROBOTIC MANIPULATION
Benchmark
ManiSkill2
|EEHAVIO IK
Tahitat 2
SancGyn
Manipuh/The
RL.Bench
TDW
Grusp implemeniaaion
Physical
Abstract
Abstract
Physical
Abstract
Physical Procedural
Physical
Abstract
Absiract
#Demo trajoctories
30k
~2000
Procedural
Multi-controllrr support
Yes
Unknown
Ves
No
No
No
Yes
Yes
No
Visual RL/IL baselines
Full
Limited
Full
No
Full
No
No
Na
No
#Ohject models
21144
3324
YCB
150
80
10
28
112
#Scenes
306
105
30
10S
Ray-tracing support
Kuafo
Omniverrs
NVISII
Unity
Domain randomization
Partin
Unknown
Yes PhysX 5
No Unity
No
Yes
Yes V-REP
No
Rigid-hody simulation
SAPIEN
Omniverse
Bullet
Mujoco
Mujoco
Unity
Soft-body simulation
Warp-MPM
Omniverse
Table 10: Comparison with other existing benchmarks for robotic manipulation. The information of each benchmark based on its major focus. For example, all the simulation backends of these benchmarks support physically implemented grasp, but some of them focus on high-level actions and thus use abstract grasp for benchmarking algorithms. Multi-controller support measures whether benchmark has implemented multiple controllers and provided interfaces to select one for each task.
Table 10 compares ManiSkill2 with other existing benchmarks for robotic manipulation. ManiSkill2 features aarge-scale demonstrations for every task, a great variety of objects, multi-controller support and conversion of demonstration action spaces, and focus fully physically implemented grasp. We invested significant efforts to select, fix, and re-model objects and integrate them to our task families, generate demonstrations with fully physical grasping, and perform large-scale visual manipulation benchmarking. Through these processes, we carefully verify all of our tasks. The low success rates on many tasks demonstrate that our benchmark poses interesting and challenging problems for the community.
We are actively working on current limitations of ManiSkill2: photorealism, domain randomization, and scene-level variation. First, the simulation backend (SAPIEN) of ManiSkill2 supports ray- tracing renderer (Kuafu). However, photorealism usally achieved at the cost of speed. For example, BEHAVIOR-IK. Li et al. (2022) can only achieve 60 FPS with Nvidia Omniverse and require high-end GPUs. In this work, we focus on physically realistic short-horizon and low- level visuomotor manipulation. We have experimentally supported ray-tracing rendering for use cases like generating data offline for supervised learning or fine-tuning policy learned on non- photorealistic data. Second, we have included domain randomization for physical parameters and visual appearance for some of tasks. We will provide tutorials for users to customize domain randomization according to their cases. Third, We are introducing mobile manipulation tasks similar to those in ManipularTHHe THOR ArmPointNav (Ehsani et al., (202)) and Habitat 2.0 (Szot et al.) 2021 but with physically implemented base movement and grasp. Those tasks will involve scene- level variations and demand advanced navigation abilities.
SSince BEHAVIOR- IK (Li ct al.
2022 is not public yet, some detailed information unknown.
"For Habitat 2.0 Szot et al. 2021 refer its Home Assistant Benchmark.
'For IsaacGym MMkkvvyychke et al.
2021). we refer to its 4 manipulation tasks in the original paper: Franka
Cube Stacking, Shadow Hand, Allegro Hand, and TriFinger.
FFor Manipular THOR [Ehsani et al. 2021), we refer to ArmPointNav task.
7 For TDW (Gan et al, 2020), we refer to its Transport Challenge (Gan et al. 2021
Asssets like boards with slots AssemblingKits, feasible layouts of obstacles for AvoidObstacles, and
target characters for Write are not included. All these assets require offline generation.
"ManiSkil12 has included some domain randomization for physical parameters (e.g.. joint damping and friction of PushChair, OpenCabinettDoor, and OpenCabinetDrawery and visual appearance (e.g., colors PickSingle&GGAD and PandaAvoidOOsstaelen))
10 ManiSkill2 implements MPM-based soft-body simulator based on Warp Macklin 2022 Deformable objects like cloth not supported yet.
29

---

Published as a conference paper at ICLR 2023
H CONTRIBUTIONS
Designed and implemented ManiSkil12 infrastructure: Jiayuan Gu, Fanbo Xiang, Rui Chen
Designed rigid-body environments and collected demonstrations: Jiayuan Gu, Xuanlin Li, Xiqiang Liu, Tongzhou Mu
Designed soft-body environments and collected demonstrations: Fanbo Xiang, Xinyue Wei
Designed and implemented server-based asynchronous rendering: Fanbo Xiang, Zhan
Ling, Jiayuan Gu
Designed and implemented the soft-body simulator: Fanbo Xiang, Xinyue Wei, Xiaodi Yuan
Conducted sense-plan-act experiments: Stone Tao, Xiqiang Liu, Jiayuan Gu
Conducted RL/IL experiments on rigid-body environments: Yunchao Yao, Xuanlin Li, Zhan Ling
Conducted II. experiments on soft-body environments: Yihe Tang, Xinyue Wei, Zhan Ling
Conducted real-world experiments: Rui Chen, Pengwei Xie
Implemented the cloud-based evaluation system: Stone Tao, Fanbo Xiang
Managed or advised the project: Hao Su, Rui Chen, Zhiao Huang
30


Reviews:
[{"review_id": "RiEuDITglHL", "reviewer": "Reviewer_zjV2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose ManiSkill2, a benchmark for robotic manipulation. It builds upon ManiSkill 1.0 and has better support for different types of gripper-based manipulations, uses significantly more interactive objects, runs much faster, and includes soft-body simulations. The authors show experiments on a few tasks, 4+ million frames of gripper-based interactions, which can be used for imitation learning, and include pickup object sim2real results.", "summary_of_the_review": "The engineering effort that went into this is quite impressive to support such a wide variety of agent-based manipulations. However, the work is pretty incremental, focusing on a few clear next steps from the previous work in ManiSkill 1.0. I would have also liked the authors to start moving towards more realistic task setups that a robotic agent might actually encounter in the real-world. Currently, the agents are just operating with isolated objects on a grayscale background, and this isn't practically useful for most tasks we want robots to perform.", "strength_and_weaknesses": "### Strengths\n* The extent of the manipulation capabilities and support is great.\n* The number of objects used for a manipulation benchmark is quite impressive.\n* The performance benchmarks and simulation speeds are impressive. I appreciate all the engineering effort that went into this paper!\n* The sim2real results are neat.\n\n\n### Weaknesses\n\n* **W1. Lack of discussion on objects.** The authors claim they have 2000+ object models, a significant increase over the 162 objects in ManiSkill 1.0. However, there isn't much of a discussion on where these objects come from. Where do they come from? Are they going to be released (and under what license)? What is the distribution of the object categories? To what extent are they interactive/articulated?\n\n* **W2.** The backgrounds and setup of the scenes are quite weak. In comparison to ManipulaTHOR, Habitat 2.0, and iGibson 2.0 for manipulating different objects, which are in household environments, the gray backdrop behind the objects makes the environments feel like a toy environment. It would be much better if the manipulation of objects happened in the context of other objects (e.g., when turning on a faucet, the agent is always in a bathroom/kitchen), when opening the cabinet, the agent is always in a kitchen or bedroom, etc. It doesn't seem like the tasks are very realistic to what one might want a robot to do in the real-world. The one task used for sim2real in this paper is merely picking up a fairly visually distinct cube.\n\n* **W3.** I wish the paper was formatted a bit differently, and instead focused mostly on experiments in the main part of the paper, instead of discussing technical details in sections 3 and 4. The main part of the paper doesn't seem like much of the content in these sections is necessary for most users of ManiSkill2 to read. It seems like much of this could have gone into the appendix.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EjS5ALOcIHX", "reviewer": "Reviewer_fwz3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new benchmark for robotic manipulation that supports various types of manipulation tasks with improved efficiency. The main contributions are:\n- Rich task collections, assets, and demonstrations.\n- Multi-controller support and action space conversion.\n- Efficient 2-way coupling rigid-MPM simulation.\n- Efficient parallelization for visual RL training.", "summary_of_the_review": "I recommend acceptance based on the benchmark novelty and the engineering efforts for simulation efficiency. In my opinion, this paper has no major flaws, and I believe there is enough motivation for the community to use this benchmark to facilitate future research.", "strength_and_weaknesses": "Strengths:\n- Variety of the manipulation tasks: stationary/mobile-base, single/dual-arm, rigid/soft-body, different action spaces and controllers. These, in combination, provide richer families of tasks than previous benchmarks.\n- Good software engineering techniques for efficient learning with multi-physics and visual input.\n\nWeaknesses:\n- Lack of different grippers and arms support - only Panda arm and two-finger/suction grippers are implemented. To facilitate dexterous manipulation and more sim-to-real transfer, it is important to support more industrial arms (IIWA, UR5e, etc.) with multi-finger grippers.\n- Limitations are not presented in the paper. It would be important to understand what are the pros and cons of this benchmark compared to existing ones in a clear manner (e.g. having a table showing the comparison with popular benchmarks on multiple dimensions/features). For example, Maniskill2 does not seem to support photorealism, domain randomization, cloth simulation, scaling to thousands of parallel environments (like Isaac Gym), etc., which exist in some of the previous benchmarks.\n\nQuestions:\n- While claiming Maniskill2 is the first to support real-time simulation and rendering of MPM material, I don't see concrete statistics supporting this. What is the speed comparison against naive MPM simulation or PlasticineLab?\n- Is two-way rigid-soft coupling necessary for manipulation tasks in Maniskill2? Though I believe it is more realistic (compared to one-way coupling), for manipulation purposes, the rigid grippers are usually much heavier than the soft materials being manipulated. In this case, maybe one-way coupling is realistic enough and more efficient. It would be necessary to showcase where the two-way coupling is a must for some manipulation tasks.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "4NrhLnlG5r", "reviewer": "Reviewer_RgH8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a new benchmark environment for manipulation tasks. The environment supports both traditional rigid-body manipulation tasks and soft-body tasks, while unified interface allows a wide range of algorithms. Results show that the asynchronous rendering and render server approach provides an improved FPS for CNN-based policy learning.", "summary_of_the_review": "This work proposes several improvements to the current simulator for manipulation. The improvements in rigid-body soft-body interface and simulation performance, along with the variety of supported tasks, is of great value. This work's weakness includes omitted details from different sections and an issue in observation. Overall, I am leaning towards acceptance.\n", "strength_and_weaknesses": "Strength:\n1.\tThe proposed environment supports a wide range of observation and controller types and supports mainstream algorithms in sense-plan-act, RL and IL frameworks.\n2.\tThe tasks include those from previous works and some newly proposed ones, covering a wide aspect of challenges in manipulation. The support for soft-body tasks is of great importance.\n3.\tThe asynchronous rendering approach and render server implementation allow improved rendering performance and reduced memory usage.\n\nWeaknesses:\n1.\tAll the proposed manipulation tasks are based on robotic arms and lack support for other actuators (e.g. Dexterous hands).\n2.\tSince different simulators have different parameters that may drastically affect the overall performance, please include the details of parameters for each simulator (e.g. Did you use the GPU pipeline in IsaacGym or just used the CPU pipeline, the substeps for physics simulation, the number of the total vertex in the simulation). \n3.\tThe performance comparison is carried out on RGBD input settings. The performance of pure state input and point-cloud input is not reported.\n4.\tThe effectiveness and accuracy of demonstration conversion are neither discussed in detail nor measured quantitatively.\n5.\tPoint-cloud observations in some tasks contain ground truth segmentation, which cannot be easily obtained in the real world. This barrier for sim-to-real transfer could be avoided with a better observation design.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster