RoboHive: A Unified Framework for Robot Learning
Authors: Vikash Kumar
Rutav Shah
Gaoyue Zhou
Vincent Moens
Vittorio Caggiano
Abhishek Gupta
Aravind Rajeswaran
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Robot Learning

Abstract: We present RoboHive, a comprehensive software platform and ecosystem for research in the field of Robot Learning and Embodied Artificial Intelligence. Our platform encompasses a diverse range of pre-existing and novel environments, including dexterous manipulation with the Shadow Hand, whole-arm manipulation tasks with Franka and Fetch robots, quadruped locomotion, among others. Included environments are organized within and cover multiple domains such as hand manipulation, locomotion, multi-task, multi-agent, muscles, etc. In comparison to prior works, RoboHive offers a streamlined and unified task interface taking dependency on only a minimal set of well-maintained packages, features tasks with high physics fidelity and rich visual diversity, and supports common hardware drivers for real-world deployment. The unified interface of RoboHive offers a convenient and accessible abstraction for algorithmic research in imitation, reinforcement, multi-task, and hierarchical learning. Furthermore, RoboHive includes expert demonstrations and baseline results for most environments, providing a standard for benchmarking and comparisons. Details: https://sites.google.com/view/robohive

wtw.://tss.ooollloooooo/ee//eeee RoboHive: A Unified Framework for Robot Learning
Vikash Kumar Rutav Shah", Gaoyue Zhou", Vincent Moens", Vittorio Caggiano Jay Vakil Abhishek Gupta>s,, Aravind Rajeswaran
Washington', UC Berkeley", CMU7, UT Austin', OpenAl", GoogleAI Meta-AI
Abstract
We present RoboHive, a comprehensive software platform and ecosystem for re- search in the field of Robot Learning and Embodied Artificial Intelligence. Our platform encompasses diverse range of pre-existing and novel environments, including dexterous manipulation with the Shadow Hand, whole-arm manipulation tasks with Franka and Fetch robots, quadruped locomotion, among others. Included environments are organized within and cover multiple domains such as hand ma- nipulation, locomotion, multi-task, multi-agent, muscles, etc. comparison to prior works, RoboHive offers a streamlined and unified task interface taking depen- dency on only minimal set of well-maintaiie maintained packages, features tasks with high physics fidelity and rich visual diversity, and supports common hardware drivers for real-world deployment. The unified interface of RoboHive offers convenient and accessible abstraction for algorithmic research in imitation, reinforcement, multi-task, and hierarchical learning. Furthermore, RoboHive includes expert demonstrations and baseline results for most environments, providing standard for benchmarking and comparisons. Details: https://ssites, htas://sies.google..ommm. how.gelcccc/iee///ooovveeee?
Introduction
Recent years have witnessed unprecedented breakthroughs Artificial Intelligence (AI), par- ticularly in the domains of game playing [1, 2].
protein folding [3], and language modeling [4]. Comparatively, the progress in robot learning has been slow. This slower pace can partially be attributed to Moravec's paradox [5]. which posits that sensorimotor behaviors are intrinsi- cally harder for Al agents than high-level cogni- tive tasks. Simultaneously. another crucial issue demands our attention: significant hindrance also lies in the convoluted software frameworks for robot learning and the lack of universally recognized benchmarks. This increases the bar- rier for entry, limits rapid prototyping, and re- stricts the influx of ideas. Unlike fields such as computer vision or natural language processing, where benchmarks and datasets are standard- ized, the landscape of robotics remains more fragmented.
Rabobtive
Mustle
Figure 1: A subset of RoboHive's task suites. [ín clock- wise order] Multi-Task Suite: Environments facilitating multiple tasks once. Arms Suite: Diverse arms (w/ grippers) exposed tabletop manipulation tasks. Hand Suite: Diverse hands exposed to tasks requiring dex- terity. Locomotion Suite: Diverse collection of legged locomotion tasks. Myo Suite: Task collections with musculoskeletal agents. Deformable Suite: Tasks lection with deformable objects.
Addressing this gap, we introduce RoboHive, cohesive ecosystem tailored for robot learning. As dual-function platform, RoboHive operates benchmarking suite and a research tool. It delivers plethora of environments, precise task
Conrespondence vikaehplus@@mail.com
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

Adroit (original)
Franka Kitchen (onginal)
Adroit (RoboHive)
Kitchen (RoboHivv)
Figure 2: RoboHive presents environments with rich visual diversity to aid research in visual generalization and places high emphasis on physical realism to facilitate the real-world transfer.
definitions,, and stringent evaluation criteria, supporting diversity of learning paradigms like re- inforcement, imitation, and transfer learning. This facilitates efficient exploration and prototyping for researchers. Additionally, RoboHive equips users with teleoperation capabilities and hardware integration
integration, enabling scamless transition between physical robots and their simulated counterparts. With RoboHive, our goal is to bridge the gap between the current state robot learning and its potential for growth.
The primary contribution of our work is the development and open sourcing of the RoboHive, unified framework for robot learning. The key features of RoboHive include:
1. Environment Zoo: RoboHive features extensive and diverse collection of environments, covering broad spectrum of research areas. These environments encompass wide range of manipulation tasks involving both free and articulated objects, dexterous in-hand manipulation, locomotion with bipedal and quadrupedal robots, and even manipulation with musculoskeletal arm-hand models. Our simulated environments, powered by MuJoCo [6]. provide fast physics simulation and are designed to ensure high level of physical realism.
2. Unified Robot Class and out-of-box hardware support: RoboHive introduces a unified Robot- Class abstraction that seamlessly interfaces with both simulated and physical robots using sim- hooks and hardware-hooos respectively. This unique capability enables researchers to effortlessly interface with robotic hardware, and port their results from simulation to reality by simply modify- ing a single flag.
3. Teleoperation Support and Expert Dataset: RoboHive ve offers teleoperation capabilities out-of- the-box through various modalities, including a keyboard, 3D space mouse, and virtual reality controllers. Collected using this feature, we are open-sourcing RoboSer one of the largest real-world manipulation datasets collected via human-teleope human-teleoperation covering 12 skills across multiple kitchen tasks. These teleoperation features and datasets are particularly valuable to researchers engaged in imitation learning, offline learning, and related fields.
4. Physics Fidelity and Visual Diversity: To expose the next research frontier in real- real-world robotics, RoboHive emphasizes tasks with high physics fidelity and rich visual diversity surpassing previ- ous benchmarks. We incorporate complex assets, rich textures, and improved scene composition, aligning visuomotor control research with real-world visual complexities. Furthermore, RoboHive natively supports visual domain randomization and scene layout randomization in several envi- ronments, enhancing the versatility of visual perception, while also providing realistic and rich physical content.
Metrics and Baselines: RoboHive establishes clear and concise metrics for evaluating algorithm performance across all environments. The framework provides user-friendly gym-like API [7] for seamless integration with learning algorithms, ensuring accessibility for a wide range of researchers and practitioners. Moreover, in collaboration with TorchRL [8] and mjRL [9], RoboHive includes comprehensive baseline results for commonly studied algorithms within the research community, offering a benchmark for performance comparison and analysis.
2 Relationship to Prior Work
Benchmarks play crucial role in guiding research progress. In the realm of robot learning, simulated state-based benchmarks like OpenAl Gym [7] and dm-control [10] have played a pivotal role in leading the development cycles of numerous algorithms and applications [11, 12, 13, 14, 15, 16].

---

However, with the advancements in the field, benchmarks also need to evolve to calibrate the difficulty and rigor of the problems (to avoid overfitting), as well as expose the next research frontiers. For instance, the progression from MNIST towards CIFAR-10 in favor of ImageNet significantly influenced the evolution of deep learning for computer vision [17, 18, 19, 20, 21]. Similarly, RoboHive aims to provide an enhanced set of comprehensive and challenging benchmarks representative of the next frontiers of research in the current robot learning landscape.
Unlike other fields, like NLP and vision, where research questions are well-defined and often homogeneous, open questions in the field of Robot learning comprise of diverse and heterogeneous sets: such as (bipedal/quadrupeda)) locomotion, (legged/wheeled arial//errestrillaqquutt navigation, (on/off-roo road) driving, (rigid/deformable, in- an-hand/fulll-ae boody) manipulation. Given this diversity, the field relies on cluster of benchmarks each with unique strengths. Despite this, unification can facilitate the cross- cross-pollinans polsinoolinntoon of ideas that can significantly catalyze progress.
Towards this goal, RoboHive offers comprehensive ecosystem for robot learning, serving as both a diverse collection of benchmarks and a comprehensive research toolkit with out-of-the-box support for common robotic hardware drivers, teleoperation capabilities, one of the largest real-world robotics datasets, etc. to name few.
In terms of benchmarking, RoboHive presents a diverse collection of environments and tasks under a unified framework that includes domains like dexterous and tabletop manipulation, legged locomotion, musculoskeletal agents, deformable objects, multi-task, auutt problem statements, as well as real-world benchmarks in locomotion and manipulation. This is in contrast to existing benchmark clusters like [7, 10] which primarily consist of state-based simple toy domains unrepresentative of realistic robotics challenges. RoboHive tasks are designed with attention physical realism which provides an advantage over the [22, 23, 24, 25] cluster of benchmarks, which trades off physical accuracy for large-scale scenes and photorealism. While there are efforts that adhere to physical realism [26, 27, 28] they are often domain-specific. RoboHive greatly surpasses their task and visual diversity, and covers multiple domains. The work that bears the closest resemblance to ours is IsaacGym [29]. which features photo-realistic rendering, and GPU-accelerated physics. However, these advantages come at the cost of physical realism and demand a high level of expertise for development. This can be particularly challenging for research investigations that frequently require customization and rapid prototyping.
Lastly, several environments in RoboHive have been adapted from prior works and are made available with the permission of the original authors. These environments were not originally designed and released as benchmarks [30, 31, 32, 33), but have emerged as such over time. Wide adoption without support led various customization by the community leading to fragmentation and challenges when comparing results across different versions. In the spirit of the evolution of such well well-proven benchmarks, RoboHive adopts these environments and presents them within a single framework. The refreshed versions of these environments feature rich visual diversity (Figure 2) to meet the growing needs for visual generalization in robot learning, clear evaluation metrics, and baselines to facilitate further adoption by the community. The associated datasets have also been reparsed, checked for reproducibility, and are being made available with RoboHive. The unification of these tasks under RoboHive ve not only brings them together under single framework but also deprecates the unsupported dependencies making the entire framework be instantiated via pip install robohive.
3 Design Philosophy & Framework Overview
The primary goal of RoboHive is the development and open-sourcing of a unified framework to support research in robot learning. Alongside wide collection of carefully designed tasks, RoboHi ve provides an ecosystem to bridge the gap between algorithmic advancements in our benchmarks and real-world results. An overview of the different components of Robollive is outlined in Figure 3. We provide additional details on these components and the rationale behind their fundamental design principles in this section.
3.1
Abstract Robot Class for Simulation & Hardware Backends
At the core of RoboHive is abstract robot class, depicted in Figure 4, which provides unified abstraction for both simulator and hardware backends. The abstract robot class exposes unified APIs for various sensors and robots across both simulation and hardware, thereby abstracting lowelevel
3

---

Foundation Models
Wa BiGnA VCT, Affordance Ohiigg
Task Collections
observations,
Augmentation Models
Divenion taee
Openai Gymeeet
Nonvstalgnaryy tooks
Domain
Portia Qhtervatiitty rupport
AI
Human Agents
Sparoel demos
Burcced deserationmmeoon
Agents
Simulations
Powored by MudeDoo Highatesical Rim Wousily Bync
Robot Models
actions
Seeecr Action Modes Nols model
Hardwares
Figure RoboHive's versatility stems from modular design. The four primary modules are- task-collections that contain multiple environment suites, agents take contain human as well as algorithmic actors, a unified robot that seamlessly bridges simulation and reality, and a foonddaioo/augmmnatiiii module that supplements RoboHivecapabilit with pre-trained models and sim2real paradigms.
implementations and enabling the switch between simulation and the real world by toggling just single flag. For live demo of this functionality, please see the linked video Currently, we support a diverse collection commonly studied robots across several morphologies as outlined below.
Hands: Adroit [34], Shadow [33]. Allegro [35],
Robot
MPL [36], D'Hand [37], D' Manus [38]
Arms: Franka [31], Sawyer [37], Fetch [39]
Quadrupeds: Spot Mini [40], MIT-Cheetah [41],
(MuLkO
Pre-seckaged
D'Kitty [32]
Bipeds: Darwin [42], Atlas [43] Musculo-skeletal: MyoHand [44]. MyoElbow [44]
Figure 4: Overview of RoboHiie'ss robot class
that presents unified framework to seamlessly work between simulation and physical hardware
Simulation Since the broader vision of RoboHive is to facilitate progress in real- world robot learn- ing, all simulation models are designed with care toward alignment with real- world considerations. open-source
To ensure high degree of physical realism, build RoboHive top of the MuloCo physics engine [6]. Furthermore, RoboHive natively supports domain randomization, sensor noise, and delays, which can all be easily toggled through base configuration file (Figure 4).
Hardware RoboHive provides out-of-the box support for various types of robot hardware. also exposes base class for users to extend per the hardware configurations they have access to. The robot-hardware class exposes robots in the real world via respective drivers. The robol class supports multiple control modalities such position, velocity, and torque control, and can be easily customized using a configuration file. Hardware and sensor drivers and hooks for common robots are natively supported for easy deployment and transfer of results from simulation the real world.
3.2 Environments
RoboHive exposes environments through the widely adopted OpenAI API. All the environments are organized into suites, a visual depiction of which is provided in Figure 5. They span from whole arm and dexterous manipulation of free and articulated objects to locomotion agents varied terrains, to control of detailed musculoskeletal models. These environments are useful across different algorithmic approaches such as visual imitation learning, model-free, and model-basee based RL.
For the observation space, RoboHive environments expose observations through abstract sensors for both simulation and the real world. Sensors can include low-dimension states (simulation-only) proprioception, and cameras (RGBD). We also natively support common transformations of the visual observations. This includes visual domain randomization [45] of textures and lighting conditions,

---

Manipulation
Table-Tor Top
Myo
Deformable
ROBEL
In-hand
Arms
6
Multi-Task
Multi-Agent
8
Locomation
Figure 5: RoboHive Framework hosts a wide collection of environments of varying complexity (state/visual observations, fully/partially observable. sparse/dense rewards, etc) across multiple domains. Each column represents task family within RoboHive and highlights subset of its tasks.
well embedding the visual observation using pre-trained visual models [46, 47, 48, 49, 50). During registration, RoboHive environments can be fully customized using a base configuration file, equip the robot with various sensors, augmentations, and visual foundation models.
RoboHive also explicitly decouples rewards and success criteria. In RL, common to use rewards to judge task's 's performance. This complicates relative comparisons developing reward functions (cither by hand or through learning) that can generate desired robot behaviors is itself core component of the robot learning field. Thus, RoboHive ships with an independent success criteria per environment that can be used for evaluations, as opposed to reporting rewards. We provide basic dense reward functions for all tasks and facilitate Finally, RoboHive also supports fully vectorized/batched rollouts in simulation for fast environment sampling, along with named dictionaries for sensors, observations, and reward/success metrics to enhance interpretability and to facilitate debugging.
to improve provide new reward functions.
TaskSuite Overview: Next we provide brief overview of various task suites present RoboHive.
Hand Manipulation Suite This task suite contains a collection of hands. The set of tasks is inspired by [51] and presents a collection where the arm and hand joints need be coordinated to solve tasks. Tasks in this set typically have high action space and face challenges in temporally co-ordaining small movements of the fingers with large movements the arms.
Table- Top Suite This task suite contains collection of tasks involving arms and grippers interact- ing with small-scale objects. Tasks like stacking blocks, pouring, zipping, pushing, and pick-place common in this task suite.
5

---

Myo Suite MyoSuite contains a unique collection of tasks involving musculoskeletal systems. This collection is inspired by [44] and challenges policies with third-order actuation involving muscle dynamics and high dimensional actuation space of MyoHand. Tasks of varying difficulty levels are present - ppssing pointing, key turn, baoding balls, etc.
Deformable Manipulation Suite Our deformable suite contains set of tasks involving flexible material. MuJoCo is primarily a rigid body simulator but has support for few deformable objects such as ropes and soft bodies. The tasks in this suite involve interacting with ropes, granular medium, soft sponges, etc. These tasks need to be solved from visual inputs as the state space of the deformable objects is hard to specify.
Robel Suite ROBEL suite was introduce in [32] and consists of well defined manipulation and locomotion tasks using D'Claw and D' Kitty low cost robots with parallel instantiation in real world. In-hand Manipulation Suite In hand manipulation suite is primarily inspired by the task set presented in [33]. Tasks in this set predominately involve in-hand reorientation of objects such as die, baoding balls, pencils, etc. Catastrophic failures due to the loss of control and thereby the object pose a primary challenge in solving this suite.
Arm Manipulation Suite Arm manipulation suite is similar in spirit to the tabletop suite but requires tasks that require careful attention to the morphology of the arms. Tasks in this set involve interaction with large objects (such as appliances) where self-collisions and collision with objects are of concern.
Multi-Task Suite With one of the primary motivations behind RoboHive being generalization, this suite is of key importance. This suite is organized as a collection of tasks where an agent can have shared experiences that can be generalized between tasks. This task suite builds off from the task set introduced in Franka-kitchen [31]) and table scene (as introduced in [52])
Multi-Agent Suite To facilitate investigation in multi-agent interactive behaviors, we present a
novel suite in RoboHive. Tasks in this suite build from tasks presented in [32], [53]. Tasks objective
include interacting with passive objects, as well as other agents.
Locomotion Suite RoboHive also features a wide collection of locomotion agents and a variety of locomotion tasks. Both agents and tasks are inter- changealeee this suite. Task objectives vary from simple reaching tasks to maneuvering around obstacles as well as object interaction.
Classical Suite Finally, no task collection is complete without set of classical problems While such problems are well represented in existing benchmarks [7, 10], RoboHive hosts set that exposes unique challenges such as non-holonomic control, over actuation, etc.
3.3 Agents
By exposing environments through the OpenAl Gym API, RoboHive makes it convenient to de- velop various agents/algorithms that can help the robots accomplish various tasks. In the realm of reinforcement learning, by instantiating an environment (through a base configuration file) with an observation space and reward as described in Section 3.2, along with a robot's native action space, any standard RL algorithm applicable to an MDP/POMDP setting can be used to train agents. Along with TorchRL [8] and mjRL. [9], we provide an initial set of baseline implementations with policies effeective in both simulations( well the real worldZ Code samples of these im- plementations along with baseline results for various environments can be directly accessed from https://sites. http://iites.goggl..oooo wwwpgll..oo/viiww/obbeooo/
Besides online RL, robot learning encompasses other classes of algorithmic approaches to train agents, such as offline RL and imitation learning. These approaches however require starter dataset from the environment with at least a few successful trajectories. To facilitate the collection of such datasets, RoboHive provides out-of-the-box teleoperation support for most environments through a variety of user interfaces like keyboard, 3D space mouse, and VR controllers. This teleoperation capability is shared across both simulation and the real world. Please see this short live demo that demonstrates the case of telcoperation capabilities shipped with RoboHive in under 2 minutes, right from pip installing the package.
4 Capabilities and Utilities of the RoboHive Framework
We believe that RoboHi ve has several capabilities and components that are of broad use to various communities related to robot learning. We outline several of the salient capabilities and utilities of RoboHive in this section.

---

4.1 RoboHive as repository of environments and agents
Environments RoboHive naturally provides unified framework for environments and agents that can power research in robot learning. By providing a large diversity and complexity of robotics simulation assets and tasks, fast and physically realistic simulation models, and a modular framework that is easy-to-use and extend, RoboHive ve naturally plugs into the ecosystem meeting research needs in reinforcement learning and imitation learning settings, Given the need for more complex tasks and visually rich environments for studying visual generalization, and the huge space of users who rely on MuJoCo for their simulation needs 2, RoboHive provides the ecosystem necessary to both develop learning algorithms as well as deploy them on various robots both in simulation as well as real-world studies. By matching environments with real-world considerations and by providing both simulation and hardware backends and hooks, RoboHive can considerably reduce the barrier of entry to robot learning and enable researchers primarily interested in algorithm development to more easily deploy their ideas on robots. RoboHive provides this flexibility without sacrificing speed and accuracy. To demonstrate the performance of RoboHive ve, we provide the environment throughput for several task suites in Table (See Table 3 for full details).
Table I:
This table illustrates the environment data collection performance single machine using 32 processes distributed across A100 GPUs. The data collection process utilizes TorchRL's collector classes in an asynchronous manner, following a "first ready first served" approach. Each batch of data (500 steps/batch, 800K total) was collected on a separate process, employing a random policy, and transformed to a floating point tensor and resized to [84 84) pixels. The results presented in this figure are based on three different seeds, and the reported values indicate the average and standard deviation across seeds and environments, Multi-task suite rendering was achieved over 4 cameras with a resolution of [256 x 256] pixels, Arm and Hand suites had 3 cameras with a resolution of [244 x 244] pixels. The Myo suite does not support rendering over dedicated cameras and was excluded from this analysis. The decision to employ TorchRL as the backend for data collection is justified by the substantial superiority of these results compared to those achieved using single-process multiprocessed solutions offered by most other frameworks, as reported more details [8].
Task Family
Samples or Frames per second
State observations
Proprioception + Visual observations
Arm Manipulation
7473 = 1237
993 + 63
Hand manipulation
8785 + 665
1713 + 24
Multi-task
4430 + 6
346 + 0.4
Myo
7366 + 3509
n/a
Agents ve supports two types of behavioral agents: (1) human agents (teleoperation), and (2) algorithmic agents (c.g. policies trained with reinforcement learning or imitation learning).
Human Agents (teleoperation): RoboHive ve natively supports various input devices or interfaces using which humans can interact with veenvrrmees Currently, supported interfaces include keyboards, game controllers, DoF 3D connection space mice, VR controllers, and the versatile CyberGlove. These interfaces can be used to control both simulated real robots to accomplish various tasks through a teleoperation setup, and can also serve to generate datasets that can power various offline RL and imitation learning algorithms.
Algorithmic Agents: In addition to human agents, RoboHive also boasts a collection of pre-trained policies (with state as well as visual imputs) from well-studied algorithm families. These policies have been trained through partnering with torchRL and mjRL, and can be readily downloaded and used for interacting with the environment, generating datasets for offline RL or IL, or as a baseline to compare the performance of new algorithms. Code samples for training these policies and the pre-trained policies can be accessed from the project website
4.2
Sim and Real Counterparts
To easily study transfer between simulation and reality, RoboHive makes it seamless to toggle between simulation and reality. Effective transfer between simulation and reality requires bridging the reality gap. RoboHive does this by providing peatooooooreaaisticc rendering, accurate physics modeling, and attention to detail in real-world factors such as latency, actuator, sensing delays, etc. Figure 6 (video Z) showcases a task-policy deployed simulations as well real-world via the sim2real bridge across two domains.
multiple simulation engines now natively support importing MuJoCo models
7

---

Sim
Real
Real
Figure 6: Rigorous measures are taken to capture physical realism RoboHive's simulations models. Thanks to Robohive's s robot class policies trained simulations easily transfer over to the real world. Depicted in the figure are the synchronized deployment of policies trained in simulation to both simulation and the real world for representative manipulation task with Franka (left) and locomotion task with D' Kitty [32] (right).
4.3 Teleoperation and Dataset Collection
Since a variety of robot learning methods such offline reinforcement learning, imitation learning. and finetuning with reinforcement learning rely on meaningfully collected datasets, RoboHive provides both pre-collected datasets which we refer to as RoboSer and teleoperation interfaces for users to collect their own data. The key factors here are the size and diversity of the pre-collected datasets and the ease of use of the teleoperation interface that facilitates dataset collection (DemoLZ) both in simulation as well as the real world.
In addition refreshed versions prior datasets corresponding to the environments we adopted, RoboSet also features one of the largest (still growing) spen-source real-world robotics datasets released with commodity hardware covering over 12 skills across over 30 tasks in multiple kitchen scenes, We mark the full composition of RoboSer in Table 2. We provide more details on RoboSer in Appendix A, the data collection process in subsection A.1, reproducibility statement subsection A.2, and access and maintenance commitments in subsection A.3.
Table 2: RoboSer data compositions across various domains and sources
Domain
Trajs
Tasks
World
Visuals
Source
Real kitchen
28,700
40
Real
cam
Human TeleOp [36]
Bin Manipulation
70,000
Real cam
Script+Policy [54]
Franka kitchen
600
Sim
cam
Human TeleOp [31]
Adroit
25
Sim
cam
Human TeleOp [30]
Franka kitchen
75
5
Sim
cam
Expert Policy [55]
Adroit
75
Sim
cam
Expert Policy [55]
Kitty
75
Sim
cam
Expert Policy [55]
4.4 Datasets and Environments for Visual Imitation Learning and Offline RL
As described above, the ability to directly interact with the tasks has facilitated the collection of a rich dataset in both simulation and the real world. With this dataset, then benchmark variety of visual imitation learning (summarized below) and offline RL techniques (see [56] for details).
ROBEL (expert::: Locomotion Suite
FK1-v4/expert): Kitchen
1.0
R.3940
300
RSMDD
88.50
88150
Success Rale
VCI-Lenge
Proproccpttve
Success Rata
Propfocopt.ve
VCI-wage
State
20
DiGityStand
DKilywok
McroOpen
Knob1on
Knot2off Environment Task
StoorOsen
LowerOzen
Environment Task
Figure 7: Robel and Kitchen task family: Expert trajectories trained using NPG are used to collect the dataset for behavior cloning consisting of 75 trajectories per task. Each visual baseline in the Kitchen task family is averaged over 3 seeds x 3 camera angles 25 trajectory rollouts. For the Robel Task family the results are averaged over 3 seeds x 25 trajectory rollouts.
Learning from Visual Observations Learning behaviors directly from sensory inputs, especially visual, is challenging due to the high-dimensional, noisy sensory space. Various approaches have been

---

HMS(human): Hand Manipulation Suite
FK1 FK1-v4(human): Kitchen
100-
109
Success Rale
80-
VCI
Propriceppiiee
Success Rate
80
Proprocapiive
Large
Stata
Slate
Expert Poliy
Eapert Paity
Envrronmen Task
Environment Task
Figure 8: Hand Manipulation and Kitchen task family: Human teleoperation is used to collect the dataset for behavior cloning consisting of 25 trajectories per task for HMS [30] whereas for kitchen tasks, we collect 100 trajectorics for cach task in Kitchen Task Family (FK1)v4). Each visual baseline in both task familics averaged over 3 seeds x camera angles x 25 trajectory rollouts for robust evaluations.
proposed that learn low-dimensional, compact representations of the input images and then use these representations to learn the task. We benchmark the performance of various state-of-the-ar visual representation learning algorithms including RRL [46], R3M [48], and VCI [50] on the challenging Kitchen domain of Multi-Task, Locomotion Tasks on D' Kitty from ROBEL Task suite, and Adroit domain of the Hand Manipulation [Figure 7, Figure 8] task suites from RoboHive using the dataset outlined in Table 2. We compare these visual baselines with an oracle policy trained using privileged 'State' information. Moreover, we add another baseline that is blind to the image observations and uses the robot's proprioceptive information to solve the task, providing minimum acceptable performance of the visual baselines on the benchmark. While expert demonstrations via an 'Expert Policy' feasible in simulation and easier to test models, accessing such Expert Policy in the real world is difficult. Alternatively, human-collected demonstrations possessed with multimodal action distribution via teleoperation is widely used in the real world. We benchmark the visual baselines on both types of datasets: Expert Demonstrations (Figure 7) and Human Teleoperated Dataset (Figure 8). The performance of the demonstration trajectories shown with the label 'Expert Policy,' which consists of rollouts from expert policy in Figure 7 and human telcoperated trajectories in Figure 8. To the right precedent in the community for a fair and robust comparison of different basclines, we average each method across multiple seeds, and camera angles for all our results.
We observe wide gap between state-of-the-at visual baselines and policy learned using the privileged state information ("State") [Pink] using both expert and human telnp-eteeeperrteed datasets, highlighting the need for learning better and more robust sensory-motor representations, including visual observations. Surprisingly, the tasks in the locomotion domain (ROBEL in with D' Kitty (Figure 7) perform worse using any visual representation baselines than a proprioceptive-only baseline [Red], highlighting the challenge of learning with diverse visual observations. Moreover, the larger performance gap between tasks in the Kitchen suite using human telcoperated (Figure 8) and dataset collected using expert policy (Figure 8) demonstrates the additional challenges in learning policies that can model multimodal action distributions. Finally, RoboHive also includes dexterous Hand Manipulation Suite (Figure 8) where solving tasks with few demonstrations (25 demonstrations in this suite for each task) using behavior cloning can be challenging (e.g.. Pen and Relocate). These results underpin the breadth of visual diversity and task-level generalization in Robolivv''s 's task collections often overlooked in prior works.
4.5 Well packaged and maintained ecosystem
RoboHive offers versatile simulation and execution tool that caters various tasks and domains by consolidating all environments under a common base class. The consistent constructor and method signatures make it one-stop solution. The library's s ability seamlessly integrate environments of diverse natures through a single entry point highlights its versatility.
The library is equipped with a continuous integration workflow that tests each environment construc- tion after each contribution, which ensures that the package is fully functional at any point in time. RoboHive is also well packaged and can be instantiated using a pip install robohive.
Because a library is only as good as its documentation, RoboHive package features curated and extensive documentation of its features that spans across multiple media. It offers an installation guideLZ, getting started guideLZ, getting started video with simulation as well as real tele-operation demo comprehensive tutorials detailed documentation[2, environment descriptions(3, video

---

showeaseL of task performance, trained baselines for different task families, issue tracking[ discussion forums and support commitments. Depth of RoboHive capabilities can be best appreciated by visiting our webpage page
At the time of writing, the library has been actively maintained for about five years, is mature enough to be used by broad community of researchers, and has already facilitated a diverse set of results. Given its long-lasting maintenance, we expect voboHiveto be actively maintained by its authors as well as the open-source community for the foreseeable future.
4.6 Reproducibility
RoboHive's basclines are and will keep on being tracked across various versions of the software to check for performance regressions. In each RoboHive's 's releases we provide detailed releases notes with individual runs and performance comparison across versions with at least 3 seeds for a selected set of environments and algorithms.
5 Limitations
With a number of recent publications ([48, 49, 57, 50, 58]) leveraging RoboHive, the framework is witnessing a steady adoption in the community. Owing the scope of the framework, we do have a few limitations. (1) RoboHive provide trained agents only for few algorithmie families that are most commonly used in the field in that corresponding task family. We hope that open-sourcing and community engagements will help us make the coverage more comprehensive. (2) While RoboHive has a wide coverage of task families, time, especially towards aerial and navigation suites. (3) RoboHive's robot class supports a diverse set of commonly used hardware but it's not representative. Manipulation hardware support outweighs locomotion, navigation, and aerial hardware support.
hope that the community will help enrich the ecosystem
Acknowledgments
Given the broad scope of RoboHive and its design flexibility that makes it compatible with a diverse family of algorithms, the true potential of the framework can only be realized with community-wide participation. Hence we are open soen-sou the entire framework. We hope the RoboHive ecosystem provides firm footing for those who are getting started with the field well those who are at the forefront of research and development. We sincerely acknowledge the help and support of the authors and contributors of the original work upon which RoboHive builds. RoboHive wouldn't 't have been possible without their original contributions, and their help in interfacing their original work to the RoboHive ecosystem.
10

---

References
[1] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che, Julian Schrittwieser, loannis Antonogiou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game go with deep neural networks and tree search. Nature, 529:484 2016.
[2]
Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemystaw Dębiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint 1912.
2019.
[3] Mihaly Varadi, Stephen Anyango, Mandar Deshpande, Sreenath Nair, Cindy Natassia, Galabina Yordanova, David Yuan, Oana Stroe, Gemma Wood, Agata Laydon, al. Alphafold protein structure database: massively expanding the structural coverage of protein-sqqeene space with high-accuracy models. Nucleic acids research, 50(D1):D43--D44, 2022.
[4] OpenAI. Chat gpt, 2023.
[5] Moravec' paradox. Moravec's paradox.
[6] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In IROS, 2012.
[7] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. 2016.
[8] Albert Bou, Matteo Bettini, Sebastian Dittert, Vikash Kumar, Shagun Sodhani. Xiaomeng Yang. Gianni De Fabritiis, and Vincent Moens. TorchRL: A data-driven decision- making library for PyTorch. 2023.
[9] University of Washington Seattle Movement Control Lab. A library of rl algorithms for continuous control tasks.
[10] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint
2018.
[11] Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Volodymir Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, lain Dunning, Shane Legg, and Koray Kavukcuoglu. Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures, 2018.
[12] Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised represen- tations for reinforcement learning. In International Conference on Machine Learning, pages 1639. PMLR, 2020.
[13] Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004. /3649, 2020.
[14] Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning, pages 2555-2565. PMLR, 2019.
[15] Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning explore via self-supervised world models. In International Conference on Machine Learning, pages 8583-8592. PMLR, 2020.
[16] Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Ried- miller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pages 4479,, PMLR, 2018
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[18] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks, 2018.
[19] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks, 2017.

---

[20] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021.
[21]
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer, 2022.
[22] Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub. Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9339-9347, 2019.
[23] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05444
2017.
[24] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pages 1-16. PMLR, 2017.
[25] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based interactive environment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[26] Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Abhishek Joshi, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint arXiv:2009. 12293, 2020.
[27] Stephen James, Zicong Ma. David Rovick Arrojo, and Andrew Davison. Rlbench: The robot learning benchmark & learning environment. IEFE Robotics and Automation Letters, 5(2):3019-3026 2020.
[28] Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference robot learning, pages 1094-1100 PMLR, 2020.
[29] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo. Michelle Lu, Kier Storey. Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108. 2021.
[30] Aravind Rajeswaran*, Vikash Kumar* Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. In Proceedings of Robotics: Science and Systems
(RSS), 2018.
[31] Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving long horizon tasks via imitation and reinforcement learning. Conference on Robot Learning (CoRL), 2019.
[32] Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, and Vikash Kumar. ROBEL: RObotics BEnchmarks for Learning with low-cost robots. In Conference Robot Learning (CoRL), 2019.
[33] Anusha Nagabandi, Kurt Konoglie, Sergey Levine, and Vikash Kumar. Deep Dynamics Models for Learning Dexterous Manipulation. In Conference on Robot Learning (CoRL), 2019.
[34] Vikash Kumar. Manipulators and Manipulation in high dimensional spaces. PhD thesis.
University of Washington, Seattle, 2016
[35] Wonik robotics. Allegro hand, 2023.
[36] Vikash Kumar and Emanuel Todorov. Mujoco haptix: A virtual reality system for hand manipu- lation. In 2015 JEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pages 657-663. IEEE, 2015.
[37] Abhishek Gupta, Justin Yu, Tony Z Zhao, Vikash Kumar, Aaron Rovinsky, Kelvin Xu, Thomas Devlin, and Sergey Levine. Reset-free reinforcement learning via multi-task learning: Learning dexterous manipulation behaviors without human intervention. In 202/ IEEE International Conference Robotics and Automation (ICRA), pages 6664-6671. IEEE, 2021.
12

---

[38] Raunaq Bhirangi, Abigail DeFranco, Jacob Adkins, Carmel Majidi, Abhinav Gupta, Tess Hellebrekers. and Vikash Kumar. All the feels: A dexterous hand with large area sensing. arXiv preprini arXiv:2210. 15658, 2022.
[39] Matthias Plappert, Marcin Andrychowicz, Alex Ray. Bob McGrew, Bowen Baker, Glenn Powell, Jonas Schneider, Josh Tobin, Maciek Chociej, Peter Welinder, et al. Multi-goal reinforcement learning: Challenging robotics environments and request for research. arXiv preprint
2018.
[40]
Boston Dynamics. Spot mini, 2023.
[41] MIT. Mini cheetah.
[42] Igor Mordatch, Kendall Lowrey, and Emanuel Todorov. Ensemble-cio: Full-body dynamic mo- tion planning that transfers to physical humanoids. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5307-5314. IEEE, 2015.
[43] Tom Erez, Kendall Lowrey, Yuval Tassa, Vikash Kumar, Svetoslav Kolev, and Emanuel Todorov. An integrated system for real-time model predictive control of humanoid robots. In Humanoid Robots (Humanoids), 2013 13th IEEE-RAS International Conference on, pages 292-299. IEEE, 2013.
[44] Vittorio Caggiano, Huawei Wang, Guillaume Durandau, Massimo Sartori, and Vikash Kumar. Myosuite - a contact-rich simulation suite for musculoskeletal motor control. https:// github. com/ awcebokrrseearchhmgg...... wwtpch//myostteeeeeeeeree 2022.
[45] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojeiech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RS/ international conference intelligent robots and systems (IROS), pages 23-30. IEEE, 2017.
[46] Rutav Shah and Vikash Kumar. Rrl: Resnet as representation for reinforcement learning. In International Conference Machine Learning. PMLR, 2021.
[47] Simone Parisi, Aravind Rajeswaran, Senthil Purushwalkam, and Abhinav Gupta. The unsur- prising effectiveness of pre-trained vision models for control. In International Conference on Machine Learning, pages 17359-17371. PMLR, 2022.
[48] Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203. /2601, 2022.
[49] Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training. arXiv preprint arXiv:2210.0000
2022.
[50] Arjun Majumdar, Karmesh Yadav, Sergio Arnaud, Yecheng Jason Ma, Claire Chen, Sneha Silwal, Aryan Jain, Vincent-Pierre Berges, Pieter Abbeel, Jitendra Malik, Dhruv Batra, Yixin Lin. Olcksandr Maksymets, Aravind Rajeswaran, and Franziska Meier. Where are we in the search for an artificial visual cortex for embodied intelligence? 2023.
[51] Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine. Learning Complex Dexterous Manipulation with Deep Reinforce- ment Learning and Demonstrations. In Proceedings of Robotics: Science and Systems (RSS), 2018.
[52] Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. Conference on Robot Learning (CoRL), 2019.
[53] Ofir Nachum, Michael Ahn, Hugo Ponte, Shixiang Gu, and Vikash Kumar. Multi-agent manipulation via locomotion using hierarchical sim2real. arXiv preprint 1xXvv:190.00222 2019.
[54] Patrick Lancaster, Nicklas Hansen, Aravind Rajeswaran, and Vikash Kumar. Visuo-motor world models for real-world robot manipulation. https://arii.oor/////233001.1 p42t 2023.
[55] Sham M Kakade. A natural policy gradient. In NIPS, 2002.
[56] Gaoyue Zhou, Liyiming Ke, Siddhartha Srinivasa, Abhinav Gupta, Aravind Rajeswaran, and Vikash Kumar. Real world offline reinforcement learning with realistic data source. arXiv preprint 220006220 2022.
13

---

[57] Zhao Mandi, Homanga Bharadhwaj. Vincent Moens, Shuran Song, Aravind Rajeswaran, and Vikash Kumar. Cacti: A framework for scalable multi-task multi-scene visual imitation learning. arXiv preprini arXiv:22/2.05711 2022.
[58] Siddharth Karamcheti, Suraj Nair, Annie Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, and Percy Liang. Language-driven representation learning for robotics. arXiv preprini 1xX1:2300.12766, 2023.
14

---

Appendix
A Datasets
RoboHive offers extensive support for diverse range of datasets known RoboSet. These datasets are designed to facilitate pre-training and offline learning research. With the support of multiple input devices that users can interact with RoboHive's environments, we obtain datasets that encompass both human-collectee data and expert datasets acquired via trained policies.
The dataset is structured in the form of trajectories, capturing essential information at each time step. These trajectories comprise observations, actions, rewards, RGB visuals from multiple camera views, and other relevant environmental information. The richness of information within RoboSer makes suitable for research endeavors across various domains and topics, including but not limited to offline learning like imitation learning and offline reinforcement learning, visual generalization, and policy learning generalization. All datasets within RoboSet are in HDF5 format, which is suitable for organizing large and complex hierarchical data.
In the subsequent sections, we introduce the composition of RoboSet, the data collection process, and provide insights into our data access and maintenance plan.
A.1 Collection Process
In this section, we introduce the data collection process of our 1) expert dataset and 2) human dataset. For expert datasets, we use a trained task-specific NPG policy for the target task and roll out 25 trajectories in the environment each for three different trained agents. The expert dataset also contains failure trajectories.
Our human dataset is collected through human teleoperation using Puppet [36]. During the collection process, a human teleoperator uses an HTC Vive headset and controller to control the robot in an end effector space. We subsequently replay and parse the trajectories in each target environment collect task-relevant information. The human trajectories in RoboSet are mostly successful.
Since some of the RoboHive environments are built upon existing environments such Adroit and Franka Kitchen, it is natural to integrate datasets collected from the original environments. For instance, human dataset of the hand manipulation suite is adapted from the human trajecto- ries collected from the DAPG project [51]. We replay the original trajectories into RoboHive's corresponding environments. This enables us to reuse datasets from prior work effectively while supporting information that wasn't 't contained in the original dataset, for example, RGB observations. By integrating this exteroception information alongside the state-based proprioception ohservations, RoboHive becomes a comprehensive testbed for conducting research on visual generalization as well. To ensure the quality and validity of the replayed trajectories, we provide reproducibility report in subsection A.2.
A.2 Reproducibility
The human dataset used in RoboHive's 's hand manipulation suite is derived from the original dataset from [51]. In order to incorporate this dataset into RoboHive, we replay the robot's actions from the original trajectories within the corresponding environment of RoboHive. During this process, we record the observations, rewards, and other task-relevant information.
To ensure the validity and reproducibility of the replayed trajectories, we perform thorough comparison between the 100 original trajectories and the replayed trajectories. Figure 9 depicts the histogram of the norm of the difference between the last states of each original trajectory and the replayed trajectory. Notably, we observe that the trajectories converge to the same environment state within approximation bounds after replay, with the final states being visually indistinguishable. This alignment performance indicates that the replayed trajectories maintain the same level of quality as the original dataset. As result, the validity of past results and findings remains intact within the RoboHive version the task. Please refer to our Wiki Page for further information and visualizations.
15

---

70
60
# Trajectories
40
30
20-
10
0.1
0.3
0.4
0.5
Last State Difference
Figure Comparison of Last State Discrepancy between Original and Replay Trajectories
A.3 Access & Maintenance
To ensure convenient and efficient access for users, we have open-sourced Ro- boSel https://sites.goolleccom wtw.glec.o/eeee///ooooeeeeeee example trajectories are available on our Wiki Page.
The complete list of datasets and
To utilize RoboSet effectively, users are expected to download the dataset of their task of interest, and then they can employ their preferred training method train policies using this data. To evaluate the performance of these trained policies, users are encouraged to perform 25 rollouts in the environment. Lastly, users can compare their methods to our baseline experiments outlined in subsection 4.4.
The availability of both expert and human datasets in RoboSet makes it suitable for various learning approaches, encompassing only imitation learning methods that rely high-quality data, but also offline reinforcement learning methods and any techniques that leverage multimodal or play data. Moreover, the wealth of visual data within RoboSel opens up avenues for visual pre-training and generalization. Importantly, all datasets in RoboSel adhere to a consistent format, mitigating the overhead associated with transitioning between different task suites and environments. Looking ahead, we anticipate that RoboSet will continue to expand in terms of both quantity and diversity, ensuring its ongoing relevance and value to the research community.
B Environment sampling performeance
Table 3 illustrates the environment data collection performance on single machine using 32 processes distributed across 8 A100 GPUs. The data collection process utilized TorchRL's collector classes in asynchronous manner, following "first ready first served" approach. Each batch of data (500 steps/batch, 800K total) was collected on a separate process, employing a random policy, and transformed to floating point tensor and resized to [84 x 84] pixels. The results presented in this figure are based on three different seeds, and the reported values indicate the average and standard deviation across seeds. Multi-task suite rendering
achieved over 4 cameras with resolution of [256 x 256] pixels, Arm and Hand suites had cameras with resolution of [244 x 244] pixels. The Myo suite does not support rendering
dedicated cameras and excluded from this analysis. The decision to employ TorchRI. as the backend for data collection is justified by the substantial superiority of these results compared to those achieved using single-process process multiprocessed solutions offered by most other frameworks, as reported more details [8].
16

---

Table Full details of collection speed for all environments considered on per-environment basis
Task Family
Env name
Samples or Frames per second
Arm Manipulation
FrankaPiodPlaceFieeeew
6174.0235
www.eiiccccaaaaeaaonoooo lawdeeaaan__vdddv __2d-v0
928.9412
FrankaPick SaannkaPiccPlaceRauie Lacceennoonm-vv
6127.2433
Frankal Push
7338.9487
FrankaP PushRandom sorirddoomydvvdv v2d-v0
994.2035
Franka PannkkaassRanndome
7382.3643
FrankaReach Fixed-vv
8906.9175
1055.2824
Franka BuunkaRRachRamdom:e
8911.4261
Hand manipulation
door v2d-v1
1681.459
doorvl
9124.0997
hammer canmerrv2d-vl hammer-vl
1722.2912
7826.282
pen_v2d-vl
1711.8543
pen-vl
8871,5027
relocate_v2cr v2dvv1
1737.4508
relocate-vl
9317.9963
Multi-task
FKI OnRandom-v4
346.0061
FKI _i__Knobll
4436.5437
FKI
346.8642
FK1 Knob20fffandom--vi
4434.0908
FKI LdoorOpenRandom
346.5334
FKI MicrooOpenRandommww
4424.7308
FKI_ Sd__SdoorOpenRandom.. _22d-v442
346.0869
FK1 SdaorOOpenRndoom-y
4423.86
Myo
myoElbowPose http:/owwoo///MMeMeaeeeeendoo
11022. 1737
wroElbooPPoos//DMMRnndnnoow
10998.8446
myoFinngerReachhRanddmo---
10959.5896
myoofondDDieeeerrienttamdddd
6617.4297
7118.316
myoHandPen
7890.5196
ExyottandReuchRanddmss
6915.9179
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
[Yes] We provide extensive details of our claims throughout the paper with plots and references. External links to our website and video demonstrations are also provided where appropriate
(b) Did you describe the limitations of your work?
[Yes] See section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] Our paper doesn't 't involve any dataset problem involving humans or public
places. We expect no negative impact to society
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments (e.g. for benchmarks)...
17

---

(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or URL)?
[es]] See subsection 4.6.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] subsection 4.4
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] subsection 4.4
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] A set of few shared machines available within academic settings
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Table Table 2.
(b) Did you mention the license of the assets? [Yes] The license information is linked to project page we refer to subsection 4.3. The license of dataset is CC-BY 4.0.
(c) Did you include any new assets either the supplemental material or as URL? [Yes] See subsection A.3.
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [Yes] We have taken explicit consent from the authors and are releasing back with the same license as original.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] Our datasets contain robot trajectories they don't contain personally identifiable information or offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
18


Reviews:
[{"review_id": "FHf4vL4drD", "reviewer": "Reviewer_DRrC", "summary": "", "questions": "", "limitations": "1. A detailed explanation for sim-to-real needs to be added in the paper. For example, how the framework integrates the delays and dynamic errors between simulation and physical robots. \n2. In the visual observations, it is meaningful to compared this framework with previous approach in terms of the simulation speed instead of simply comparing between different tasks inside the framework. \n3. For the supplementary material, it would be great if the authors include a detailed documentation to use the material and put them onto the public repo. \n", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. This paper introduces a unified framework for learning tasks in robotics filed.\n2. Various tasks have been implemented to demonstrate the effectiveness of this framework.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "2Lahbnx3Bj", "reviewer": "Reviewer_UW1F", "summary": "", "questions": "", "limitations": "The authors did a nice job discussing RoboHive's limitations in its limited offering of trained agents for specific algorithmic families and an uneven representation in task and hardware support.", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "This paper contributes to the robot learning field by introducing a comprehensive and diverse framework that bridges gaps in existing benchmarks, and integrates both physical and simulated robotics. It provides a unified, accessible platform, thus promoting collaboration, standardization, and rapid prototyping across diverse areas of robot learning.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "qVtwg5iwlT", "reviewer": "Reviewer_iKLG", "summary": "", "questions": "", "limitations": "yes", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. Integrate existing robot learning tasks and coding under a unified architecture for researchers to use and reproduce.\n2, Provide the interface of robot teleoperation that can collect demonstration, which is not supported by some original environments.\n3, Provides higher physics fidelity and rich visual diversity content, improving the performance of sim2real.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "PJ2WeZPEP7", "reviewer": "Reviewer_NtXK", "summary": "", "questions": "", "limitations": "The Limitation section is well described", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "The RoboHive is large set of tools for development, training and evaluation of wide variety of robotic tasks. it offers separate suites for each of the subfields. The teleoperation methods covers most frequent methods to collect expert trajectories.There is also wide variety of objects and environments. The author put a lot of effort to convert various expert trajectories into one format. I also like the modularity of the system.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: Broadly speaking, this paper presents a potentially useful framework for the exploration of simulated tasks in the robotics arena. It integrates a number of distinct tasks with a well-known physics simulator and provides a comparatively simple entry point to starting work on sim-to-real robotics efforts. Although this work is similar to other efforts to build robotic simulation environments, the suite of tasks and efforts to ensure physical fidelity are noteworthy. However, the scientific novelty of the benchmark itself is somewhat limited. There are also areas in which the paper itself could be improved, especially with the addition of more technical detail; the presentation of the paper in a benchmarks track does not in and of itself mean that technical content is inappropriate. More discussion of the limitations of the simulator, especially limitations that are inherent to MuJoCo, would be informative. The authors are encouraged to take reviewers' detailed suggestions into account when preparing future versions of the paper.

Pros:
+ The framework for learning tasks in robotics is in an important area, and RoboHive supports a wide range of tasks that could in turn support sim-to-real development in robotics.
+ The (claimed) simple to begin working with mechanism for robot-based learning has the potential to improve accessibility for this kind of research.
+ Allows for the collection of demonstrations of tasks, which is a difficult problem hat some competitor technologies do not support.
+ A wide variety of objects and environments is represented.

Cons:
- Technical and implementation content could be more detailed. Papers in the benchmarks track present things at a high level, yes, but it is still a technical track.
- Limitations of the toolkit should be made clearer, especially wrt. simulation speed and the accuracy of the underlying simulator.
- Some discussion of the parameter tuning required when using the toolkit would be informative.
- RoboHive offers a significant possible set of tasks, but is not in and of itself scientifically novel. More demonstration of effective sim-to-real performance would help clarify the contribution.
Decision: Accept (Poster)