SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 1: This table illustrates the environment data collection performance on a single machine using 32 processes distributed across 8 A100 GPUs. The data collection process utilizes TorchRL's collector classes in an asynchronous manner, following a "first ready first served" approach. Each batch of data (500 steps/batch, 800K total) was collected on a separate process, employing a random policy, and transformed to a floating point tensor and resized to  $[84 \times 84]$  pixels. The results presented in this figure are based on three different seeds, and the reported values indicate the average and standard deviation across seeds and environments. Multi-task suite rendering was achieved over 4 cameras with a resolution of  $[256 \times 256]$  pixels, Arm and Hand suites had 3 cameras with a resolution of  $[244 \times 244]$  pixels. The Myo suite does not support rendering over dedicated cameras and was excluded from this analysis. The decision to employ TorchRL as the backend for data collection is justified by the substantial superiority of these results compared to those achieved using single-process or multiprocessed solutions offered by most other frameworks, as reported in more details in [8].
Task Family	Samples or Frames per second
State observations	Proprioception + Visual observations
Arm Manipulation	7473 ± 1237	993 ± 63
Hand manipulation	8785 ± 665	1713 ± 24
Multi-task	4430 ± 6	346 ± 0.4
Myo	7366 ± 3509	n/a
Agents RoboHive supports two types of behavioral agents: (1) human agents (teleoperation), and (2) algorithmic agents (e.g. policies trained with reinforcement learning or imitation learning).
Human Agents (teleoperation): RoboHive natively supports various input devices or interfaces using which humans can interact with RoboHiveenvironments. Currently, supported interfaces include keyboards, game controllers, 6 DoF 3D connection space mice, VR controllers, and the versatile CyberGlove. These interfaces can be used to control both simulated real robots to accomplish various tasks through a teleoperation setup, and can also serve to generate datasets that can power various offline RL and imitation learning algorithms.
Algorithmic Agents: In addition to human agents, RoboHive also boasts a collection of pre-trained policies (with state as well as visual inputs) from well-studied algorithm families. These policies have been trained through partnering with torchRL and mjRL, and can be readily downloaded and used for interacting with the environment, generating datasets for offline RL or IL, or as a baseline to compare the performance of new algorithms. Code samples for training these policies and the pre-trained policies can be accessed from the project website.
4.2 Sim and Real Counterparts
To easily study transfer between simulation and reality, RoboHive makes it seamless to toggle between simulation and reality. Effective transfer between simulation and reality requires bridging the reality gap. RoboHive does this by providing near-photorealistic rendering, accurate physics modeling, and attention to detail in real-world factors such as latency, actuator, sensing delays, etc. Figure 6 (video) showcases a task-policy deployed in simulations as well real-world via the sim2real bridge across two domains.
${}^{2}$  multiple simulation engines now natively support importing MuJoCo models
Figure 6: Rigorous measures are taken to capture physical realism in Robohive's simulations models. Thanks to Robohive's robot class policies trained in simulations easily transfer over to the real world. Depicted in the figure are the synchronized deployment of policies trained in simulation to both simulation and the real world for a representative manipulation task with Franka (left) and a locomotion task with D'Kitty [32] (right).
4.3 Teleoperation and Dataset Collection
Since a variety of robot learning methods such as offline reinforcement learning, imitation learning, and finetuning with reinforcement learning rely on meaningfully collected datasets, RoboHive provides both pre-collected datasets - which we refer to as RoboSet and teleoperation interfaces for users to collect their own data. The key factors here are the size and diversity of the pre-collected datasets and the ease of use of the teleoperation interface that facilitates dataset collection (Demo) both in simulation as well as the real world.
In addition to refreshed versions of prior datasets corresponding to the environments we adopted, RoboSet also features one of the largest (still growing) open-source real-world robotics datasets released with commodity hardware covering over 12 skills across over 30 tasks in multiple kitchen scenes. We mark the full composition of RoboSet in Table 2. We provide more details on RoboSet in Appendix A, the data collection process in subsection A.1, reproducibility statement in subsection A.2, and access and maintenance commitments in subsection A.3.
Table 2: RoboSet data compositions across various domains and sources
Domain	# Trajs	# Tasks	World	Visuels	Source
Real kitchen	28,700	40	Real	4 cam	Human TeleOp [36]
Bin Manipulation	70,000	4	Real	4 cam	Script+Policy [54]
Franka kitchen	600	4	Sim	4 cam	Human TeleOp [31]
Adroit	25	4	Sim	3 cam	Human TeleOp [30]
Franka kitchen	75	5	Sim	4 cam	Expert Policy [55]
Adroit	75	4	Sim	3 cam	Expert Policy [55]
D’Kitty	75	3	Sim	4 cam	Expert Policy [55]
4.4 Datasets and Environments for Visual Imitation Learning and Offline RL
As described above, the ability to directly interact with the tasks has facilitated the collection of a rich dataset in both simulation and the real world. With this dataset, we then benchmark a variety of visual imitation learning (summarized below) and offline RL techniques (see [56] for details).
Figure 7: Robel and Kitchen task family: Expert trajectories trained using NPG are used to collect the dataset for behavior cloning consisting of 75 trajectories per task. Each visual baseline in the Kitchen task family is averaged over 3 seeds  $\times$  3 camera angles  $\times$  25 trajectory rollouts. For the Robel Task family the results are averaged over 3 seeds  $\times$  25 trajectory rollouts.
Learning from Visual Observations Learning behaviors directly from sensory inputs, especially visual, is challenging due to the high-dimensional, noisy sensory space. Various approaches have been
Figure 8: Hand Manipulation and Kitchen task family: Human teleoperation is used to collect the dataset for behavior cloning consisting of 25 trajectories per task for HMS [30] whereas for kitchen tasks, we collect 100 trajectories for each task in Kitchen Task Family (FK1-v4). Each visual baseline in both task families is averaged over 3 seeds  $\times$  3 camera angles  $\times$  25 trajectory rollouts for robust evaluations.
proposed that learn low-dimensional, compact representations of the input images and then use these representations to learn the task. We benchmark the performance of various state-of-the-art visual representation learning algorithms including RRL [46], R3M [48], and VC1 [50] on the challenging Kitchen domain of Multi-Task, Locomotion Tasks on D'Kitty from ROBEL Task suite, and Adroit domain of the Hand Manipulation [Figure 7, Figure 8] task suites from RoboHi ve using the dataset outlined in Table 2. We compare these visual baselines with an oracle policy trained using privileged 'State' information. Moreover, we add another baseline that is blind to the image observations and uses the robot's proprioceptive information to solve the task, providing a minimum acceptable performance of the visual baselines on the benchmark. While expert demonstrations via an 'Expert Policy' are feasible in simulation and easier to test models, accessing such Expert Policy in the real world is difficult. Alternatively, human-collected demonstrations possessed with multimodal action distribution via teleoperation is widely used in the real world. We benchmark the visual baselines on both types of datasets: Expert Demonstrations (Figure 7) and Human Teleoperated Dataset (Figure 8). The performance of the demonstration trajectories is shown with the label 'Expert Policy,' which consists of rollouts from expert policy in Figure 7 and human teleoperated trajectories in Figure 8. To set the right precedent in the community for a fair and robust comparison of different baselines, we average each method across multiple seeds, and camera angles for all our results.
We observe a wide gap between state-of-the-art visual baselines and policy learned using the privileged state information ('State') [Pink] using both expert and human-teleoperated datasets, highlighting the need for learning better and more robust sensory-motor representations, including visual observations. Surprisingly, the tasks in the locomotion domain (ROBEL in with D'Kitty (Figure 7) perform worse using any visual representation baselines than a proprioceptive-only baseline [Red], highlighting the challenge of learning with diverse visual observations. Moreover, the larger performance gap between tasks in the Kitchen suite using human teleoperated (Figure 8) and dataset collected using expert policy (Figure 8) demonstrates the additional challenges in learning policies that can model multimodal action distributions. Finally, RoboHive also includes dexterous Hand Manipulation Suite (Figure 8) where solving tasks with few demonstrations (25 demonstrations in this suite for each task) using behavior cloning can be challenging (e.g., Pen and Relocate). These results underpin the breadth of visual diversity and task-level generalization in RoboHive's task collections often overlooked in prior works.
4.5 Well packaged and maintained ecosystem
RoboHive offers a versatile simulation and execution tool that caters to various tasks and domains by consolidating all environments under a common base class. The consistent constructor and method signatures make it a one-stop solution. The library's ability to seamlessly integrate environments of diverse natures through a single entry point highlights its versatility.
The library is equipped with a continuous integration workflow that tests each environment construction after each contribution, which ensures that the package is fully functional at any point in time. Robohive is also well packaged and can be instantiated using a pip install robohive.
Because a library is only as good as its documentation, RoboHive package features a curated and extensive documentation of its features that spans across multiple media. It offers an installation guide, a getting started guide, a getting started video with simulation as well as real tele-operation demo, comprehensive tutorials, detailed documentation, environment descriptions, a video
showcase of task performance, trained baselines for different task families, issue tracking, discussion forums, and support commitments. Depth of RoboHive capabilities can be best appreciated by visiting our webpage page - https://sites.google.com/view/robohive.
At the time of writing, the library has been actively maintained for about five years, is mature enough to be used by a broad community of researchers, and has already facilitated a diverse set of results. Given its long-lasting maintenance, we expect RoboHiveto be actively maintained by its authors as well as the open-source community for the foreseeable future.
4.6 Reproducibility
RoboHive's baselines are and will keep on being tracked across various versions of the software to check for performance regressions. In each RoboHive's releases we provide a detailed releases notes with individual runs and performance comparison across versions with at least 3 seeds for a selected set of environments and algorithms.
With a number of recent publications ([48, 49, 57, 50, 58]) leveraging RoboHive, the framework is witnessing a steady adoption in the community. Owing to the scope of the framework, we do have a few limitations. (1) RoboHive provide trained agents only for a few algorithmic families that are most commonly used in the field in that corresponding task family. We hope that open-sourcing and community engagements will help us make the coverage more comprehensive. (2) While RoboHive has a wide coverage of task families, we hope that the community will help enrich the ecosystem over time, especially towards aerial and navigation suites. (3) RoboHive's robot class supports a diverse set of commonly used hardware but it's not representative. Manipulation hardware support outweighs locomotion, navigation, and aerial hardware support.
Given the broad scope of RoboHive and its design flexibility that makes it compatible with a diverse family of algorithms, the true potential of the framework can only be realized with community-wide participation. Hence we are open-sourcing the entire framework. We hope the RoboHive ecosystem provides a firm footing for those who are getting started with the field as well as those who are at the forefront of research and development. We sincerely acknowledge the help and support of all the authors and contributors of the original work upon which RoboHive builds. RoboHive wouldn't have been possible without their original contributions, and their help in interfacing their original work to the RoboHive ecosystem.
[57] Zhao Mandi, Homanga Bharadhwaj, Vincent Moens, Shuran Song, Aravind Rajeswaran, and Vikash Kumar. Cacti: A framework for scalable multi-task multi-scene visual imitation learning. arXiv preprint arXiv:2212.05711, 2022.
[58] Siddharth Karamcheti, Suraj Nair, Annie S Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, and Percy Liang. Language-driven representation learning for robotics. arXiv preprint arXiv:2302.12766, 2023.
RoboHive offers extensive support for a diverse range of datasets known as RoboSet. These datasets are designed to facilitate pre-training and offline learning research. With the support of multiple input devices that users can interact with RoboHive's environments, we obtain datasets that encompass both human-collected data and expert datasets acquired via trained policies.
The dataset is structured in the form of trajectories, capturing essential information at each time step. These trajectories comprise observations, actions, rewards, RGB visuals from multiple camera views, and other relevant environmental information. The richness of information within RoboSet makes it suitable for research endeavors across various domains and topics, including but not limited to offline learning like imitation learning and offline reinforcement learning, visual generalization, and policy learning generalization. All datasets within RoboSet are in HDF5 format, which is suitable for organizing large and complex hierarchical data.
In the subsequent sections, we introduce the composition of RoboSet, the data collection process, and provide insights into our data access and maintenance plan.
A.1 Collection Process
In this section, we introduce the data collection process of our 1) expert dataset and 2) human dataset. For expert datasets, we use a trained task-specific NPG policy for the target task and roll out 25 trajectories in the environment each for three different trained agents. The expert dataset also contains failure trajectories.
Our human dataset is collected through human teleoperation using Puppet [36]. During the collection process, a human teleoperator uses an HTC Vive headset and controller to control the robot in an end effector space. We subsequently replay and parse the trajectories in each target environment to collect task-relevant information. The human trajectories in RoboSet are mostly successful.
Since some of the RoboHive environments are built upon existing environments such Adroit and Franka Kitchen, it is natural to integrate datasets collected from the original environments. For instance, our human dataset of the hand manipulation suite is adapted from the human trajectories collected from the DAPG project [51]. We replay the original trajectories into RoboHive's corresponding environments. This enables us to reuse datasets from prior work effectively while supporting information that wasn't contained in the original dataset, for example, RGB observations. By integrating this exteroception information alongside the state-based proprioception observations, RoboHive becomes a comprehensive testbed for conducting research on visual generalization as well. To ensure the quality and validity of the replayed trajectories, we provide a reproducibility report in subsection A.2.
The human dataset used in RoboHive's hand manipulation suite is derived from the original dataset from [51]. In order to incorporate this dataset into RoboHive, we replay the robot's actions from the original trajectories within the corresponding environment of RoboHive. During this process, we record the observations, rewards, and other task-relevant information.
To ensure the validity and reproducibility of the replayed trajectories, we perform a thorough comparison between the 100 original trajectories and the replayed trajectories. Figure 9 depicts the histogram of the norm of the difference between the last states of each original trajectory and the replayed trajectory. Notably, we observe that the trajectories converge to the same environment state within approximation bounds after replay, with the final states being visually indistinguishable. This alignment in performance indicates that the replayed trajectories maintain the same level of quality as the original dataset. As a result, the validity of past results and findings remains intact within the RoboHive version of the task. Please refer to our Wiki Page for further information and visualizations.
Figure 9: Comparison of Last State Discrepancy between Original and Replay Trajectories
A.3 Access & Maintenance
To ensure convenient and efficient access for users, we have open-sourced RoboSet https://sites.google.com/view/robohive/RoboSet. The complete list of datasets and example trajectories are available on our Wiki Page.
To utilize RoboSet effectively, users are expected to download the dataset of their task of interest, and then they can employ their preferred training method to train policies using this data. To evaluate the performance of these trained policies, users are encouraged to perform 25 rollouts in the environment. Lastly, users can compare their methods to our baseline experiments outlined in subsection 4.4.
The availability of both expert and human datasets in RoboSet makes it suitable for various learning approaches, encompassing not only imitation learning methods that rely on high-quality data, but also offline reinforcement learning methods and any techniques that leverage multimodal or play data. Moreover, the wealth of visual data within RoboSet opens up avenues for visual pre-training and generalization. Importantly, all datasets in RoboSet adhere to a consistent format, mitigating the overhead associated with transitioning between different task suites and environments. Looking ahead, we anticipate that RoboSet will continue to expand in terms of both quantity and diversity, ensuring its ongoing relevance and value to the research community.
B Environment sampling performance
Table 3 illustrates the environment data collection performance on a single machine using 32 processes distributed across 8 A100 GPUs. The data collection process utilized TorchRL's collector classes in an asynchronous manner, following a "first ready first served" approach. Each batch of data (500 steps/batch, 800K total) was collected on a separate process, employing a random policy, and transformed to a floating point tensor and resized to  $[84 \times 84]$  pixels. The results presented in this figure are based on three different seeds, and the reported values indicate the average and standard deviation across seeds. Multi-task suite rendering was achieved over 4 cameras with a resolution of  $[256 \times 256]$  pixels, Arm and Hand suites had 3 cameras with a resolution of  $[244 \times 244]$  pixels. The Myo suite does not support rendering over dedicated cameras and was excluded from this analysis. The decision to employ TorchRL as the backend for data collection is justified by the substantial superiority of these results compared to those achieved using single-process or multiprocessing solutions offered by most other frameworks, as reported in more details in [8].
Table 3: Full details of collection speed for all environments considered on a per-environment basis
Task Family	Env name	Samples or Frames per second
Arm Manipulation	FrankaPickPlaceFixed-v0	6174.0235
FrankaPickPlaceRandom_v2d-v0	928.9412
FrankaPickPlaceRandom-v0	6127.2433
FrankaPushFixed-v0	7338.9487
FrankaPushRandom_v2d-v0	994.2035
FrankaPushRandom-v0	7382.3643
FrankaReachFixed-v0	8906.9175
FrankaReachRandom_v2d-v0	1055.2824
FrankaReachRandom-v0	8911.4261
Hand manipulation	door_v2d-v1	1681.4599
doorv1	9124.0997
hammer_v2d-v1	1722.2912
hammer-v1	7826.282
pen_v2d-v1	1711.8543
pen-v1	8871.5027
relocate_v2d-v1	1737.4508
relocate-v1	9317.9963
Multi-task	FK1_Knob1OnRandom_v2d-v4	346.0061
FK1_Knob1OnRandom-v4	4436.5437
FK1_Knob2OffRandom_v2d-v4	346.8642
FK1_Knob2OffRandom-v4	4434.0908
FK1_LdoorOpenRandom_v2d-v4	346.5334
FK1 MICROOpenRandom-v4	4424.7308
FK1_SdoorOpenRandom_v2d-v4	346.0869
FK1_SdoorOpenRandom-v4	4423.86
Myo	myoElbowPose1D6MExoRandom-v0	11022.1737
myoElbowPose1D6MRandom-v0	10998.8446
myoFingerReachRandom-v0	10959.5896
myoHandDieReorientRandom-v0	6617.4297
myoHandKeyTurnRandom-v0	7118.316
myoHandPenTwirlRandom-v0	7890.5196
myoHandReachRandom-v0	6915.9179
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
[Yes] We provide extensive details of our claims throughout the paper with plots and references. External links to our website and video demonstrations are also provided where appropriate
(b) Did you describe the limitations of your work?
[Yes] See section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] Our paper doesn't involve any dataset or problem involving humans or public places. We expect no negative impact to society
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See subsection 4.6.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] subsection 4.4
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] subsection 4.4
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] A set of few shared machines available within an academic settings
(a) If your work uses existing assets, did you cite the creators? [Yes] See Table Table 2.
(b) Did you mention the license of the assets? [Yes] The license information is linked to the project page we refer to in subsection 4.3. The license of the dataset is CC-BY 4.0.
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes] See subsection A.3.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] We have taken explicit consent from the authors and are releasing it back with the same license as original.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] Our datasets contain robot trajectories so they don't contain personally identifiable information or offensive content.