SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Vikash Kumar $^{\iota \kappa \lambda \gamma \phi *}$ , Rutav Shah $^{\delta}$ , Gaoyue Zhou $^{\gamma}$ , Vincent Moens $^{\phi}$ , Vittorio Caggiano $^{\phi}$ , Jay Vakil $^{\phi}$ , Abhishek Gupta $^{\lambda \zeta \iota}$ , Aravind Rajeswaran $^{\iota \lambda \kappa \zeta \phi}$
U.Washington $^{\iota}$ , UC Berkeley $^{\lambda}$ , CMU $^{\gamma}$ , UT Austin $^{\delta}$ , OpenAI $^{\kappa}$ , GoogleAI $^{\zeta}$ , Meta-AI $^{\phi}$
https://sites.google.com/view/robohive
We present RoboHive, a comprehensive software platform and ecosystem for research in the field of Robot Learning and Embodied Artificial Intelligence. Our platform encompasses a diverse range of pre-existing and novel environments, including dexterous manipulation with the Shadow Hand, whole-arm manipulation tasks with Franka and Fetch robots, quadruped locomotion, among others. Included environments are organized within and cover multiple domains such as hand manipulation, locomotion, multi-task, multi-agent, muscles, etc. In comparison to prior works, RoboHive offers a streamlined and unified task interface taking dependency on only a minimal set of well-maintained packages, features tasks with high physics fidelity and rich visual diversity, and supports common hardware drivers for real-world deployment. The unified interface of RoboHive offers a convenient and accessible abstraction for algorithmic research in imitation, reinforcement, multi-task, and hierarchical learning. Furthermore, RoboHive includes expert demonstrations and baseline results for most environments, providing a standard for benchmarking and comparisons. Details: https://sites.google.com/view/robohive
Recent years have witnessed unprecedented breakthroughs in Artificial Intelligence (AI), particularly in the domains of game playing [1, 2], protein folding [3], and language modeling [4]. Comparatively, the progress in robot learning has been slow. This slower pace can partially be attributed to Moravec's paradox [5], which posits that sensorimotor behaviors are intrinsically harder for AI agents than high-level cognitive tasks. Simultaneously, another crucial issue demands our attention: a significant hindrance also lies in the convoluted software frameworks for robot learning and the lack of universally recognized benchmarks. This increases the barrier for entry, limits rapid prototyping, and restricts the influx of ideas. Unlike fields such as computer vision or natural language processing, where benchmarks and datasets are standardized, the landscape of robotics remains more fragmented.
Addressing this gap, we introduce RoboHive, a cohesive ecosystem tailored for robot learning, as a benchmarking suite and a research tool. It
Figure 1: A subset of RoboHive's task-suites. [in clockwise order] Multi-Task Suite: Environments facilitating multiple tasks at once. Arms Suite: Diverse arms (w/ grippers) exposed to tabletop manipulation tasks. Hand Suite: Diverse hands exposed to tasks requiring dexterity. Locomotion Suite: Diverse collection of legged locomotion tasks. Myo Suite: Task collections with musculoskeletal agents. Deformable Suite: Tasks collection with deformable objects.
*Correspondence to vikashplus@gmail.com
Figure 2: RoboHive presents environments with rich visual diversity to aid research in visual generalization and places high emphasis on physical realism to facilitate the real-world transfer.
definitions, and stringent evaluation criteria, supporting a diversity of learning paradigms like reinforcement, imitation, and transfer learning. This facilitates efficient exploration and prototyping for researchers. Additionally, RoboHive equips users with teleoperation capabilities and hardware integration, enabling a seamless transition between physical robots and their simulated counterparts. With RoboHive, our goal is to bridge the gap between the current state of robot learning and its potential for growth.
The primary contribution of our work is the development and open sourcing of the RoboHive, a unified framework for robot learning. The key features of RoboHive include:
1. Environment Zoo: RoboHive features an extensive and diverse collection of environments, covering a broad spectrum of research areas. These environments encompass a wide range of manipulation tasks involving both free and articulated objects, dexterous in-hand manipulation, locomotion with bipedal and quadrupedal robots, and even manipulation with musculoskeletal arm-hand models. Our simulated environments, powered by MuJoCo [6], provide fast physics simulation and are designed to ensure a high level of physical realism.
2. Unified Robot Class and out-of-box hardware support: RoboHive introduces a unified RobotClass abstraction that seamlessly interfaces with both simulated and physical robots using sim-hooks and hardware-hooks, respectively. This unique capability enables researchers to effortlessly interface with robotic hardware, and port their results from simulation to reality by simply modifying a single flag.
3. Teleoperation Support and Expert Dataset: RoboHive offers teleoperation capabilities out-of-the-box through various modalities, including a keyboard, 3D space mouse, and virtual reality controllers. Collected using this feature, we are open-sourcing RoboSet - one of the largest real-world manipulation datasets collected via human-teleoperation covering 12 skills across multiple kitchen tasks. These teleoperation features and datasets are particularly valuable to researchers engaged in imitation learning, offline learning, and related fields.
4. Physics Fidelity and Visual Diversity: To expose the next research frontier in real-world robotics, RoboHive emphasizes tasks with high physics fidelity and rich visual diversity surpassing previous benchmarks. We incorporate complex assets, rich textures, and improved scene composition, aligning visuomotor control research with real-world visual complexities. Furthermore, RoboHive natively supports visual domain randomization and scene layout randomization in several environments, enhancing the versatility of visual perception, while also providing realistic and rich physical content.
5. Metrics and Baselines: RoboHive establishes clear and concise metrics for evaluating algorithm performance across all environments. The framework provides a user-friendly gym-like API [7] for seamless integration with learning algorithms, ensuring accessibility for a wide range of researchers and practitioners. Moreover, in collaboration with TorchRL [8] and mjRL [9], RoboHive includes comprehensive baseline results for commonly studied algorithms within the research community, offering a benchmark for performance comparison and analysis.
2 Relationship to Prior Work
Benchmarks play a crucial role in guiding research progress. In the realm of robot learning, simulated state-based benchmarks like OpenAI Gym [7] and dm-control [10] have played a pivotal role in leading the development cycles of numerous algorithms and applications [11, 12, 13, 14, 15, 16].
However, with the advancements in the field, benchmarks also need to evolve to calibrate the difficulty and rigor of the problems (to avoid overfitting), as well as to expose the next research frontiers. For instance, the progression from MNIST towards CIFAR-10 in favor of ImageNet significantly influenced the evolution of deep learning for computer vision [17, 18, 19, 20, 21]. Similarly, RoboHive aims to provide an enhanced set of comprehensive and challenging benchmarks representative of the next frontiers of research in the current robot learning landscape.
Unlike other fields, like NLP and vision, where research questions are well-defined and often homogeneous, open questions in the field of Robot learning comprise of diverse and heterogeneous sets: such as (bipedal/quadrupedal) locomotion, (legged/wheeled, aerial/terrestrial/aquatic) navigation, (on/off-road) driving, (rigid/deformable, in-hand/full-body) manipulation. Given this diversity, the field relies on a cluster of benchmarks each with unique strengths. Despite this, unification can facilitate the cross-pollination of ideas that can significantly catalyze progress.
Towards this goal, RoboHive offers a comprehensive ecosystem for robot learning, serving as both a diverse collection of benchmarks and a comprehensive research toolkit with out-of-the-box support for common robotic hardware drivers, teleoperation capabilities, one of the largest real-world robotics datasets, etc. to name a few.
In terms of benchmarking, RoboHive presents a diverse collection of environments and tasks under a unified framework that includes domains like dexterous and tabletop manipulation, legged locomotion, musculoskeletal agents, deformable objects, multi-task, multi-agent problem statements, as well as real-world benchmarks in locomotion and manipulation. This is in contrast to existing benchmark clusters like [7, 10] which primarily consist of state-based simple toy domains unrepresentative of realistic robotics challenges. RoboHive tasks are designed with attention to physical realism which provides an advantage over the [22, 23, 24, 25] cluster of benchmarks, which trades off physical accuracy for large-scale scenes and photorealism. While there are efforts that adhere to physical realism [26, 27, 28] they are often domain-specific. RoboHive greatly surpasses their task and visual diversity, and covers multiple domains. The work that bears the closest resemblance to ours is IsaacGym [29], which features photo-realistic rendering, and GPU-accelerated physics. However, these advantages come at the cost of physical realism and demand a high level of expertise for development. This can be particularly challenging for research investigations that frequently require customization and rapid prototyping.
Lastly, several environments in RoboHive have been adapted from prior works and are made available with the permission of the original authors. These environments were not originally designed and released as benchmarks [30, 31, 32, 33], but have emerged as such over time. Wide adoption without support led to various customization by the community leading to fragmentation and challenges when comparing results across different versions. In the spirit of the evolution of such well-proven benchmarks, RoboHive adopts these environments and presents them within a single framework. The refreshed versions of these environments feature rich visual diversity (Figure 2) to meet the growing needs for visual generalization in robot learning, clear evaluation metrics, and baselines to facilitate further adoption by the community. The associated datasets have also been repaired, checked for reproducibility, and are being made available with RoboHive. The unification of these tasks under RoboHive not only brings them together under a single framework but also deprecates the unsupported dependencies making the entire framework be instantiated via pip install robohive.
3 Design Philosophy & Framework Overview
The primary goal of RoboHive is the development and open-sourcing of a unified framework to support research in robot learning. Alongside a wide collection of carefully designed tasks, RoboHive provides an ecosystem to bridge the gap between algorithmic advancements in our benchmarks and real-world results. An overview of the different components of RoboHive is outlined in Figure 3. We provide additional details on these components and the rationale behind their fundamental design principles in this section.
3.1 Abstract Robot Class for Simulation & Hardware Backends
At the core of RoboHive is an abstract robot class, depicted in Figure 4, which provides a unified abstraction for both simulator and hardware backends. The abstract robot class exposes unified APIs for various sensors and robots across both simulation and hardware, thereby abstracting low-level
Figure 3: RoboHive's versatility stems from its modular design. The four primary modules are - task-collections that contain multiple environment suites, agents take contain human as well as algorithmic actors, a unified robot that seamlessly bridges simulation and reality, and a foundation/augmentation module that supplements RoboHive capabilities with pre-trained models and sim2real paradigms.
implementations and enabling the switch between simulation and the real world by toggling just a single flag. For a live demo of this functionality, please see the linked video. Currently, we support a diverse collection of commonly studied robots across several morphologies as outlined below.
- Hands: Adroit [34], Shadow [33], Allegro [35], MPL [36], D'Hand [37], D'Manus [38]
Arms: Franka [31], Sawyer [37], Fetch [39]
- Quadrupeds: Spot Mini [40], MIT-Cheetah [41], D'Kitty [32]
Bipeds: Darwin [42], Atlas [43]
- Musculo-skeletal: MyoHand [44], MyoElbow [44]
Figure 4: Overview of RoboHive's robot class that presents a unified framework to seamlessly work between simulation and physical hardware
Simulation Since the broader vision of RoboHive is to facilitate progress in real-world robot learning, all simulation models are designed with care toward alignment with real-world considerations. To ensure a high degree of physical realism, we build RoboHive on top of the now open-source MuJoCo physics engine [6]. Furthermore, RoboHive natively supports domain randomization, sensor noise, and delays, which can all be easily toggled through a base configuration file (Figure 4).
Hardware RoboHive provides out-of-the-box support for various types of robot hardware. It also exposes a base class for users to extend as per the hardware configurations they have access to. The robot-hardware class exposes robots in the real world via respective drivers. The robot class supports multiple control modalities such as position, velocity, and torque control, and can be easily customized using a configuration file. Hardware and sensor drivers and hooks for common robots are natively supported for easy deployment and transfer of results from simulation to the real world.
3.2 Environments
RoboHive exposes environments through the widely adopted OpenAI Gym API. All the environments are organized into suites, a visual depiction of which is provided in Figure 5. They span from whole arm and dexterous manipulation of free and articulated objects to locomotion agents on varied terrains, to control of detailed musculoskeletal models. These environments are useful across different algorithmic approaches such as visual imitation learning, model-free, and model-based RL.
For the observation space, RoboHive environments expose observations through abstract sensors for both simulation and the real world. Sensors can include low-dimension states (simulation-only), proprioception, and cameras (RGBD). We also natively support common transformations of the visual observations. This includes visual domain randomization [45] of textures and lighting conditions,
1 Manipulation
4 In-hand
2 Table-Top
3 Myo
4 Deformable
7 Multi-Agent
5 ROBEL
Figure 5: RoboHive Framework hosts a wide collection of environments of varying complexity (state/visual observations, fully/partially observable, sparse/dense rewards, etc) across multiple domains. Each column represents a task family within RoboHive and highlights a subset of its tasks.
5Arms
as well as embedding the visual observation using pre-trained visual models [46, 47, 48, 49, 50]. During registration, RoboHive environments can be fully customized using a base configuration file, to equip the robot with various sensors, augmentations, and visual foundation models.
RoboHive also explicitly decouples rewards and success criteria. In RL, it is common to use rewards to judge a task's performance. This complicates relative comparisons as developing reward functions (either by hand or through learning) that can generate desired robot behaviors is itself a core component of the robot learning field. Thus, RoboHive ships with an independent success criteria per environment that can be used for evaluations, as opposed to reporting rewards. We provide basic dense reward functions for all tasks and facilitate users to improve or provide new reward functions. Finally, RoboHive also supports fully vectorized/batched rollouts in simulation for fast environment sampling, along with named dictionaries for sensors, observations, and reward/success metrics to enhance interpretability and to facilitate debugging.
TaskSuite Overview: Next we provide a brief overview of various task suites present in RoboHive. Hand Manipulation Suite This task suite contains a collection of hands. The set of tasks is inspired by [51] and presents a collection where the arm and hand joints need to be coordinated to solve tasks. Tasks in this set typically have a high action space and face challenges in temporally co-ordaining small movements of the fingers with large movements of the arms.
Table-Top Suite This task suite contains a collection of tasks involving arms and grippers interacting with small-scale objects. Tasks like stacking blocks, pouring, zipping, pushing, and pick-place are common in this task suite.
Myo Suite MyoSuite contains a unique collection of tasks involving musculoskeletal systems. This collection is inspired by [44] and challenges policies with third-order actuation involving muscle dynamics and high dimensional actuation space of MyoHand. Tasks of varying difficulty levels are present - posing, pointing, key turn, baoding balls, etc.
Deformable Manipulation Suite Our deformable suite contains a set of tasks involving flexible material. MuJoCo is primarily a rigid body simulator but has support for a few deformable objects such as ropes and soft bodies. The tasks in this suite involve interacting with ropes, granular medium, soft sponges, etc. These tasks need to be solved from visual inputs as the state space of the deformable objects is hard to specify.
Robel Suite ROBEL suite was introduced in [32] and consists of well defined manipulation and locomotion tasks using D'Claw and D'Kitty low cost robots with parallel instantiation in real world. In-hand Manipulation Suite In hand manipulation suite is primarily inspired by the task set presented in [33]. Tasks in this set predominately involve in-hand reorientation of objects such as die, baoding balls, pencils, etc. Catastrophic failures due to the loss of control and thereby the object pose a primary challenge in solving this suite.
Arm Manipulation Suite Arm manipulation suite is similar in spirit to the tabletop suite but requires tasks that require careful attention to the morphology of the arms. Tasks in this set involve interaction with large objects (such as appliances) where self-collisions and collision with objects are of concern.
Multi-Task Suite With one of the primary motivations behind RoboHive being generalization, this suite is of key importance. This suite is organized as a collection of tasks where an agent can have shared experiences that can be generalized between tasks. This task suite builds off from the task set introduced in Franka-kitchen [31]) and table scene (as introduced in [52])
Multi-Agent Suite To facilitate investigation in multi-agent interactive behaviors, we present a novel suite in RoboHive. Tasks in this suite build from tasks presented in [32], [53]. Tasks objective include interacting with passive objects, as well as other agents.
Locomotion Suite RoboHive also features a wide collection of locomotion agents and a variety of locomotion tasks. Both agents and tasks are inter-changeable in this suite. Task objectives vary from simple reaching tasks to maneuvering around obstacles as well as object interaction.
Classical Suite Finally, no task collection is complete without a set of simple-to-understand classical problems. While such problems are well represented in existing benchmarks [7, 10], RoboHive hosts a set that exposes unique challenges such as - non-holonomic control, over actuation, etc.
3.3 Agents
By exposing environments through the OpenAI Gym API, RoboHive makes it convenient to develop various agents/algorithms that can help the robots accomplish various tasks. In the realm of reinforcement learning, by instantiating an environment (through a base configuration file) with an observation space and reward as described in Section 3.2, along with a robot's native action space, any standard RL algorithm applicable to an MDP/POMDP setting can be used to train agents. Along with TorchRL [8] and mjRL [9], we provide an initial set of baseline implementations with policies effective in both simulations  $\boxed{ \begin{array}{r l} \end{array} }$  as well as the real world  $\boxed{ \begin{array}{r l} \end{array} }$ . Code samples of these implementations along with baseline results for various environments can be directly accessed from https://sites.google.com/view/robohive  $\boxed{ \begin{array}{r l} \end{array} }$ .
Besides online RL, robot learning encompasses other classes of algorithmic approaches to train agents, such as offline RL and imitation learning. These approaches however require a starter dataset from the environment with at least a few successful trajectories. To facilitate the collection of such datasets, RoboHive provides out-of-the-box teleoperation support for most environments through a variety of user interfaces like keyboard, 3D space mouse, and VR controllers. This teleoperation capability is shared across both simulation and the real world. Please see this short live demo that demonstrates the ease of teleoperation capabilities shipped with RoboHive in under 2 minutes, right from pip installing the package.
4 Capabilities and Utilities of the RoboHive Framework
We believe that RoboHive has several capabilities and components that are of broad use to various communities related to robot learning. We outline several of the salient capabilities and utilities of RoboHive in this section.
4.1 RoboHive as a repository of environments and agents
**Environments** RoboHive naturally provides a unified framework for environments and agents that can power research in robot learning. By providing a large diversity and complexity of robotics simulation assets and tasks, fast and physically realistic simulation models, and a modular framework that is easy-to-use and extend, RoboHive naturally plugs into the ecosystem meeting research needs in reinforcement learning and imitation learning settings. Given the need for more complex tasks and visually rich environments for studying visual generalization, and the huge space of users who rely on MuJoCo for their simulation needs $^{2}$ , RoboHive provides the ecosystem necessary to both develop learning algorithms as well as deploy them on various robots both in simulation as well as real-world studies. By matching environments with real-world considerations and by providing both simulation and hardware backends and hooks, RoboHive can considerably reduce the barrier of entry to robot learning and enable researchers primarily interested in algorithm development to more easily deploy their ideas on robots. RoboHive provides this flexibility without sacrificing speed and accuracy. To demonstrate the performance of RoboHive, we provide the environment throughput for several task suites in Table 1 (See Table 3 for full details).