Cross-Embodiment Dexterous Grasping with Reinforcement Learning

## Abstract

Abstract Dexterous hands exhibit significant potential for complex real-world grasping tasks. While recent studies have primarily focused on learning policies for specific robotic hands, the development of a universal policy that controls diverse dexterous hands remains largely unexplored.
In this work, we study the learning of cross-embodiment dexterous grasping policies using reinforcement learning (RL). Inspired by the capability of human hands to control various dexterous hands through teleoperation, we propose a universal action space based on the human hand’s eigengrasps. The policy outputs eigengrasp actions that are then converted into specific joint actions for each robot hand through a retargeting mapping. We simplify the robot hand’s proprioception to include only the positions of fingertips and the palm, offering a unified observation space across different robot hands.
Our approach demonstrates an 80% success rate in grasping objects from the YCB dataset across four distinct embodiments using a single vision-based policy. Additionally, our policy exhibits zero-shot generalization to two previously unseen embodiments and significant improvement in efficient finetuning.
For further details and videos, visit our project page .

## 1 Introduction

Robotic dexterous grasping (Bicchi, ; Duan et al., ) has been studied for decades, establishing a foundation for embodied agents to interact with the world through robotic hands. Recent advances have focused on grasping various objects using multi-fingered dexterous hands, employing data-driven methods (Qin et al., ; Mandikal & Grauman, ; Liu et al., ) or reinforcement learning (RL) (Xu et al., ; Wan et al., ; Wu et al., ), with real-world deployment of the learned policies (Qin et al., ; Agarwal et al., ). However, existing approaches typically learn policies tailored to specific dexterous hands, such as ShadowHand. Developing a policy for a new embodiment often necessitates substantial data collection or costly simulations and hyper-parameter tuning in RL.

In this paper, we aim to develop a cross-embodiment dexterous grasping policy (CrossDex) that is applicable to various dexterous hands. Cross-embodiment learning leverages shared structural features among different robots to acquire generalized skills, thus facilitating enhanced generalization to new robotic embodiments. Recent research has utilized these shared structures to learn cross-embodiment policies for different robot arms (Chen et al., ), robots with varied morphologies (Liu et al., ; Hejna III et al., ), and embodiments for navigation and manipulation (Yang et al., ; Doshi et al., ).
However, applying cross-embodiment learning to diverse dexterous hands remains largely unexplored and presents several challenges. Dexterous hands feature high degrees-of-freedom (DoFs) and vary in the number of fingers and DoFs, which complicates the unification of the action space for control across different robots. For instance, it is not straightforward to map a control command for a 22-DoF, 5-fingered ShadowHand to a 16-DoF, 4-fingered LEAP Hand (Shaw et al., ). Moreover, even if actions could be aligned among different hands, their varied sizes and shapes make it challenging to implement a single policy in contact-rich grasping tasks.
For example, Patel & Song () develops a policy for LEAP Hands with varied link morphology using graph neural networks (Scarselli et al., ), but this approach struggles to generalize between LEAP Hand and Allegro Hand, which, despite their structural similarities, differ in the joint limits of DoFs and shapes of links.

We take inspiration from how humans teleoperate robot hands to tackle this challenge. Humans are not only adept at manipulating everyday objects with their own hands but are also capable of transferring such manipulation skills to various robotic dexterous hands. In teleoperation systems (Li et al., ; Handa et al., ; Qin et al., ), by observing the robot hand and simultaneously adjusting their own hand poses, human operators can remotely control any robotic hand to grasp objects without needing prior knowledge of the specific robot hand or extensive practice.
We adopt a human-like policy that can execute actions within the space of human hand poses to “teleoperate” different dexterous hands. In our approach, we use eigengrasps (Ciocarlie et al., ) of the MANO (Romero et al., ) hand model as the unified action space, which efficiently compresses 45-dimensional hand poses into low-dimensional eigenvectors. These output hand poses are then converted into each dexterous hand’s joint positions through a retargeting algorithm (Handa et al., ), providing commands for the controller. We propose training neural networks to replace the optimization-based retargeting process, significantly improving the training speed.
To unify the hand’s proprioception within the observation space, we eliminate the hand-specific joint positions and adopt the 3D positions of fingertips and palms. These positions are crucial in grasping tasks (Handa et al., ) and are consistent across different embodiments.

We train the policy to grasp objects from the YCB dataset (Calli et al., ) using reinforcement learning within the IsaacGym (Makoviychuk et al., ) simulation environment. Following the teacher-student framework (Jia et al., ; Wan et al., ), we first train individual state-based policies for each object using PPO (Schulman et al., ), and then distill these into a vision-based policy using DAgger (Ross et al., ) to grasp all objects. To train each policy, we use four different dexterous hands in parallel environments, each attached to a 6-DoF robot arm with its base fixed on a table.
Experimental results demonstrate that our CrossDex policy outperforms baseline methods on the four training hands and two hands not seen during training. CrossDex achieves high success rates in grasping YCB objects and exhibits zero-shot generalization to unseen embodiments with a single policy. Additionally, the policy shows a significant increase in learning efficiency on unseen embodiments and objects via finetuning.

Figure: Figure 1: We propose CrossDex, learning a cross-embodiment policy for dexterous grasping. The learned RL policy can grasp diverse objects with a variety of dexterous hands and transfer to hands not seen during training.
Refer to caption: x1.png

Our contributions are summarized as follows:

- •
We propose cross-embodiment training for robotic dexterous grasping, marking a step towards a universal grasping policy.
- •
We adopt techniques from teleoperation to construct a universal action space and observation space for policy learning, enabling transfer across various dexterous hands.
- •
Our experimental results demonstrate the superior training performance and generalization capabilities of CrossDex.

## 2 Related Work

Dexterous grasping. Grasping is a fundamental skill that enables robotic manipulators to interact with the real-world environment. Multi-fingered dexterous hands (Pons et al., ; Kappassov et al., ; Shaw et al., ), in contrast to traditional parallel grippers (Yi et al., ; Hu et al., ; Fang et al., ), offer advanced hardware capabilities. However, the complexity of using dexterous hands to grasp objects brings significant challenges.
To address how to grasp, a substantial number of studies focus on generating grasping poses (Zhu et al., ; Shao et al., ; Li et al., ; Liu et al., ) and constructing grasping datasets (Wang et al., ; Chao et al., ).
Real-world applications require not only static grasping poses but also closed-loop grasping policies that can execute complete trajectories.
Recent studies utilize learning from demonstrations (Qin et al., ; Mandikal & Grauman, ; Qin et al., ) and deep reinforcement learning (RL) (Xu et al., ; Wan et al., ; Wu et al., ; Qin et al., ; Kannan et al., ) to develop dexterous grasping policies. RL offers distinct advantages due to its independence from human supervision and its scalability to numerous objects (Xu et al., ).
In this paper, we study cross-embodiment dexterous grasping using RL, marking a significant step towards a universal policy.

Cross-Embodiment Learning lays the foundation for developing generalized agents by utilizing data across various embodiments. In the field of embodiment transfer (Niu et al., ), leveraging data from source embodiments to learn target embodiment policies facilitates data-efficient transfer learning. Techniques include modeling mappings between embodiments (Zhang et al., ; Chen et al., ), learning invariant representations (Zakka et al., ; Salhotra et al., ; Wang et al., ), and employing hierarchical learning to share high-level policies (Hejna et al., ).

Another line of research focuses on multi-robot training to develop universal, generalizable policies. Studies by Hejna III et al. (); Liu et al. (); Patel & Song () explore generating varied morphologies and utilizing graph neural networks (Scarselli et al., ) to learn embodiment representations. Yu et al. (); Bousmalis et al. () use Transformers (Vaswani et al., ) to adapt policies to different embodiments in-context. Recent efforts (Doshi et al., ; Yang et al., ; Xu et al., ) aim to train a universal policy that accommodates embodiments with diverse functionalities, such as combining manipulation and navigation tasks.

However, existing research has not yet developed a universal policy for various dexterous hands. GET-Zero (Patel & Song, ) learns a policy for in-hand reorientation tasks using different morphological variants of LEAP Hand (Shaw et al., ), but this method cannot generalize across different types of dexterous hands. Our research is the first to introduce a universal grasping policy for diverse dexterous hands, achieving zero-shot generalization on hands not seen during training.

Hand Motion Retargeting is primarily studied within the context of robotic teleoperation (Antotsiou et al., ; Qin et al., ; Fu et al., ), where detected human hand poses are transformed into target joint positions of robotic hands. Practical approaches include direct mapping (Fu et al., ), supervised learning (Li et al., ), and optimization for energy functions (Antotsiou et al., ; Handa et al., ; Sivakumar et al., ). DexPilot (Handa et al., ), a notable optimization-based method for dexterous hand teleoperation, explicitly considers the relationships between fingertip positions. In our research, we integrate DexPilot with an RL policy, enabling the control of various dexterous hands with a single policy.

## 3 Preliminaries

### 3.1 Reinforcement Learning for Universal Dexterous Grasping

We consider a tabletop grasping scenario where a dexterous hand is mounted on a 6-DoF robot arm with its base fixed (see Figure 1). The goal is to grasp and lift an object initially placed on the table.

We assume that tasks differ in the dexterous hand $h\in\mathcal{H}$ and the object $\omega\in\Omega$, where $\mathcal{H}$ denotes the embodiment space and $\Omega$ denotes the object set. Each task, characterized by a unique combination of an embodiment and an object, is formulated as a Partially Observable Markov Decision Process (POMDP) $M_{h,\omega}=\langle\mathcal{O},\mathcal{S},\mathcal{A},\mathcal{T},R,\mathcal
{U}\rangle$, which represents the observation space $\mathcal{O}$, the state space $\mathcal{S}$, the action space $\mathcal{A}$, the transition dynamics $\mathcal{T}(s_{t+1}|s_{t},a_{t})$, the reward function $R(s_{t},a_{t})$, and the observation emission function $\mathcal{U}(o_{t}|s_{t})$. At each timestep $t$, the agent observes $o_{t}\in\mathcal{O}$ and takes an action $a_{t}\in\mathcal{A}$, then receives a reward $r_{t}=R(s_{t},a_{t})$. The environment then transitions to the next state $s_{t+1}\sim\mathcal{T}(s_{t+1}|s_{t},a_{t})$. The objective of cross-embodiment dexterous grasping is to maximize the expected return across all tasks $\sum_{h\in\mathcal{H},\omega\in\Omega}\mathbb{E}\left[\sum_{t=0}^{T-1}\gamma^{
t}r_{t}\right]$, where $T$ is the time limit and $\gamma$ is a discount factor.

Existing literature (Qin et al., ; Xu et al., ; Wu et al., ) on dexterous grasping focuses on using a single dexterous hand ($|\mathcal{H}|=1$) with $m$ DoFs and $n$ fingers to grasp any object. The observation $o\in\mathcal{O}$ consists of: (1) Robot proprioception, including the joint positions of the arm $\bm{J}^{a}\in\mathbb{R}^{6}$, the joint positions of the hand $\bm{J}^{h}\in\mathbb{R}^{m}$, and the positions of the fingertips and palm $\bm{x}^{h}\in\mathbb{R}^{(n+1)\times 3}$; (2) Object perception, which includes the object pose $\bm{b}\in\mathbb{R}^{7}$ consisting of the object position and quaternion in simulation and a visual code $\bm{v}^{\omega}$ representing the static shape of the object. In the real world, however, we access object information through visual observations. We choose to use the object point cloud $\bm{p}\in\mathbb{R}^{N\times 3}$, which contains $N$ points captured by cameras.
The action $a\in\mathcal{A}$ includes the target joint positions for the arm $\hat{\bm{J}}^{a}$ and the target joint positions for the hand $\hat{\bm{J}}^{h}$, which are passed to a PD controller for joint torque control.
The objective is to learn a vision-based policy $\pi_{\phi}^{V}\left(a_{t}|\bm{J}^{a}_{t},\bm{J}^{h}_{t},\bm{x}^{h}_{t},\bm{p}_
{t},a_{t-1}\right)$ parameterized by $\phi$ to maximize the expected return across all objects.

Training a vision-based policy for all objects directly using RL encounters optimization challenges inherent in multi-task RL (Yu et al., ), as well as the high-dimensionality of point cloud observations, which can slow down the learning process. Recent works (Jia et al., ; Xu et al., ; Wan et al., ) propose a teacher-student framework with curriculum learning to address these issues. We adopt a simplified approach from these studies. First, the object set is clustered into several groups, and a state-based policy $\pi^{S}_{\psi_{i}}\left(a_{t}|\bm{J}^{a}_{t},\bm{J}^{h}_{t},\bm{x}^{h}_{t},\bm
{b}_{t},\bm{v}^{\omega},a_{t-1}\right)$ parameterized by $\psi_{i}$ is trained using RL for each group $i$. Then, all state-based policies $\{\pi_{\psi_{i}}^{S}\}$ are distilled into a single vision-based policy $\pi^{V}_{\phi}$ using DAgger (Ross et al., ), an online imitation learning algorithm.
In our research, we leverage this framework to address the challenge of learning a universal vision-based dexterous grasping policy for various embodiments and objects.

### 3.2 Hand Pose Modeling and Retargeting

The main challenge in cross-embodiment learning arises from the misalignment of hand joint positions $\bm{J}^{h}$ that occurs in both observations and actions among different dexterous hands. This divergence results from differences in dimensions, joint limits, and the functionality of each joint across various embodiments.
Then, can we find a shared representation of hand joint positions across these embodiments and establish mappings between them?

The literature on hand teleoperation (Handa et al., ; Qin et al., ) suggests that human hand poses can effectively represent poses for dexterous hands. A human hand pose is represented by $(\theta,\beta)$, where $\theta\in\mathbb{R}^{48}$ denotes the 16 axis angles of the hand joints and wrist, while $\beta\in\mathbb{R}^{10}$ denotes the hand shape. Using the MANO hand model (Romero et al., ), we can derive the 3D positions of 21 keypoints on the hand from the hand pose: $\bm{x}^{M}=\text{MANO}(\theta,\beta),\bm{x}^{M}\in\mathbb{R}^{21\times 3}$.
The human hand pose can then be mapped to the joint positions of any dexterous hand through a retargeting process. We adopt optimization-based retargeting methods that optimize $\bm{J}^{h}_{t}$ at each timestep $t$:

$$ $\displaystyle\min_{\bm{J}^{h}_{t}}S\left(f^{h}(\bm{J}^{h}_{t}),\bm{x}^{M}_{t} \right)+\|\bm{J}^{h}_{t}-\bm{J}^{h}_{t-1}\|^{2}$ (1) $\displaystyle\text{ s.t.}\quad\bm{J}^{h}_{lower}\leq\bm{J}^{h}_{t}\leq\bm{J}^{ h}_{upper},$ (2) $$

where $f^{h}$ is the forward kinematics function of the dexterous hand, providing the positions of the fingertips and palm. $\bm{J}^{h}_{lower}$ and $\bm{J}^{h}_{upper}$ represent the joint limits of the dexterous hand, and $S$ is a predefined function that measures the similarity between the robot hand pose and the human hand pose. For example, DexPilot (Handa et al., ) evaluates the distances of the relative positions for each pair of fingertips of the robot hand and the human hand within $S$. This problem can be solved using quadratic programming algorithms.
Thus, we can leverage the human hand pose $(\theta,\beta)$ as a universal representation for dexterous hand poses.

Figure: Figure 2: CrossDex employs a unified observation and action space to facilitate the learning of a universal policy across various dexterous hands. Rather than relying on joint angles specific to each hand, our policy utilizes the positions of the fingertips and palm to discern the spatial relationship between the hand and the object. Actions are represented using eigengrasps from the MANO hand model, which are mapped to position targets of each hand’s PD controller through a retargeting process. This design, akin to teleoperation, enables consistent control across different dexterous hands. The policy is trained using reinforcement learning within a cross-embodiment simulation environment built on IsaacGym. To learn a vision-based policy, we substitute the object pose in this pipeline with the object’s point cloud.
Refer to caption: x2.png

## 4 Method

In this section, we present the proposed method, CrossDex, which utilizes shared representations of actions and observations to learn a cross-embodiment policy for dexterous grasping. Figure 2 provides an overview of our approach.

### 4.1 Eigengrasp Actions

We propose using human hand poses as a universal action interface for dexterous hands. The output human hand pose is converted to target joint positions for each dexterous hand through retargeting. Since a human hand pose can be translated into similar grasping poses across various dexterous hands, these actions have the potential to generalize across embodiments, sharing similar trajectories for grasping each object.

However, the dimensionality of human hand poses is significantly higher than the DoFs of most dexterous hands, which can make RL less efficient. Additionally, because the joints of human hands do not move independently and the limits of each joint are unknown, defining the policy’s output space becomes challenging. We aim for the policy’s output space to be a compact subspace of human hand poses that contains meaningful and natural poses. To achieve this, we adopt eigengrasps (Ciocarlie et al., ) to compress human hand poses.

Given a dataset $\mathcal{D}=\{\theta_{i}\}$ capturing human hand poses in daily life, where each $\theta_{i}$ represents the axis angles of a hand pose (with $\beta=0$ to disregard the variability in shapes), we apply PCA to the dataset and retain the first-$k$ eigenvectors, $\{\bm{e}_{i}\}_{i=1}^{k}$, known as eigengrasps. Each eigengrasp represents a distinctive hand pose, and we can generate diverse, novel hand poses by linearly combining these eigengrasps.
In addition to the arm actions, our cross-embodiment policy outputs $k$-dimensional hand actions $\bm{w}=(w_{1},\cdots,w_{k})$, which represent a weighted sum of eigengrasps: $\theta=\sum_{i=1}^{k}{w_{i}\bm{e}_{i}}$. The hand pose $\theta$ is then converted into keypoint positions using MANO and subsequently mapped to the target joint positions of each dexterous hand via DexPilot retargeting.

Agarwal et al. () also used eigengrasps to improve RL for dexterous manipulation. While their work applies eigengrasps in the LEAP Hand’s action space, we leverage eigengrasps of human hand poses as a universal action interface, enabling cross-embodiment control through retargeting.

### 4.2 A Unified Observation Space

The hand joint positions $\bm{J}^{h}$ within the observation space, which require embodiment-specific interpretation, are likely to hinder the policy when adapting to new embodiments with unknown proprioception structures. Therefore, we simplify the observation by discarding the hand joint positions, retaining only the fingertips and wrist positions $\bm{x}^{h}$. These positions are also used in the retargeting process, as shown in Equation 1, providing shared hand state representations across different embodiments. Notably, fingertips and palm positions are crucial for grasping tasks, as they enable the agent to effectively infer actions and value based on the relative positions of the object and the hand.

At this point, we establish the structures for cross-embodiment policies. Our state-based policy takes the form $\pi^{S}_{\psi}\left(\hat{\bm{J}}^{a}_{t},\bm{w}_{t}|\bm{J}^{a}_{t},\bm{x}^{h}_
{t},\bm{b}_{t},\bm{v}^{\omega},\hat{\bm{J}}^{a}_{t-1},\bm{w}_{t-1}\right)$, while the vision-based policy takes the form $\pi_{\phi}^{V}\left(\hat{\bm{J}}^{a}_{t},\bm{w}_{t}|\bm{J}^{a}_{t},\bm{x}^{h}_
{t},\bm{p}_{t},\hat{\bm{J}}^{a}_{t-1},\bm{w}_{t-1}\right)$, where $\bm{w}_{t}\in\mathbb{R}^{k}$ denotes the eigengrasp action.

### 4.3 Multi-Task Learning

Following the teacher-student framework, we divide the entire task space of embodiments and objects, $\mathcal{H}\times\Omega$, into groups. We learn state-based policies individually for each group, then distill these policies into a vision-based policy. Our experimental observations, detailed in Section 5.2, suggest that our policy architecture benefits from co-training across embodiments, exhibiting slightly enhanced stability and comparable performance when trained simultaneously on all embodiments compared to individual training on each dexterous hand. Therefore, we choose to divide only the object set for training the state-based policies.

We establish cross-embodiment simulation environments using IsaacGym (Makoviychuk et al., ) to train policies. During the teacher learning phase, we train a state-based policy $\pi^{S}_{\psi_{\omega}}$ for each object $\omega$ using PPO (Schulman et al., ) within parallel environments that iterate through all dexterous hands. In the distillation phase, we set up parallel environments for every combination of objects and hands. Here, we iteratively use the vision-based policy $\pi_{\phi}^{V}$ to collect trajectories. Actions in these trajectories are then relabeled using the learned teacher policies $\{\pi^{S}_{\psi_{\omega}}\}_{\omega\in\Omega}$, and we optimize $\phi$ with behavior cloning.

While the parallel environments in IsaacGym run at a frame rate of tens of thousands of fps, the retargeting process, which involves iterative optimization, runs at only 300 fps, significantly slowing down the training process. Fortunately, due to the continuous nature of the retargeting mappings from the human hand to dexterous hands, we propose to use neural networks to accelerate the retargeting process. For each dexterous hand $h$, we use MANO and the retargeting algorithm to label all samples $\theta_{i}\in\mathcal{D}$ from the dataset of human hand poses with the corresponding joint positions of the dexterous hand $\bm{J}^{h}_{i}$. Then, we train a neural network $P_{\xi}^{h}$ parameterized by $\xi$ to fit the mapping from $\theta_{i}$ to $\bm{J}^{h}_{i}$. These trained retargeting networks $\{P_{\xi}^{h}\}_{h\in\mathcal{H}}$ enable batch computations within parallel environments, enhancing overall training efficiency.

Due to the limited number of dexterous hand embodiments available, we incorporate some randomization to enrich the embodiment space for training, aiming at enhancing the robustness and transferability of the policy. Specifically, we add Gaussian noise to the origin position of the joint connecting the hand and the arm, which encourages the policy to adapt to variations in the hand attachment.

We summarize our training process in Appendix A. Further implementation details can be found in Appendix B.

## 5 Experiments

### 5.1 Tasks Setup

We evaluate CrossDex on 45 daily objects from the YCB dataset (Calli et al., ) and 6 dexterous hands, with URDFs provided by Ding et al. (). We use four of these dexterous hands in training and the remaining two – a 16-DoF, 4-fingered LEAP Hand and a 12-DoF, 5-fingered Inspire Hand – are reserved for testing the model’s generalization capabilities. In simulation, we implement a unified reward to train across all objects and dexterous hands:

$$ $r^{total}=r^{dis}+r^{height}+r^{xy}+r^{success},$ (3) $$

where $r^{dis}$ encourages to minimize the distance between the fingertips and the palm to the object, $r^{height}$ encourages to lift the object, $r^{xy}$ discourages horizontal object displacement, and $r^{success}$ indicates successful task completion. Success is defined as maintaining the object at a specific height for 30 steps while keeping the hand close to the object. The episode ends either when the task is completed or the object falls off the table. Detailed descriptions of the tasks and simulation settings are provided in Appendix B.

To obtain eigengrasps and train the retargeting networks, we use the GRAB dataset (Taheri et al., ), which includes 1.6M frames depicting human hands interacting with various objects. For RL and DAgger, we deploy 8192 parallel environments that encompass combinations of all the required objects and the four training hands. Additional training details can be found in Appendix B.

### 5.2 Cross-Embodiment Learning

**Table 1: Comparison of grasping success rates for YCB objects between CrossDex and baseline methods. The ‘Training hands’ columns show average performance on the four hands provided during training, while the ‘Unseen hands’ columns show average zero-shot performance on two hands not seen during training. ‘State’ denotes state-based RL policies, and ‘Vision’ denotes vision-based policies after multi-task distillation.**
| Method | Training hands<br>(State) | Training hands<br>(Vision) | Unseen hands<br>(State) | Unseen hands<br>(Vision) |
| --- | --- | --- | --- | --- |
| MT-Raw-OA | 0.914 | 0.782 | 0.054 | 0.162 |
| MT-Raw-A | 0.823 | 0.728 | 0.272 | 0.210 |
| MT-Raw-O | 0.884 | 0.779 | 0.046 | 0.145 |
| CrossDex | 0.885 | 0.800 | 0.391 | 0.352 |

To evaluate the performance of our method, we compare it against multi-task RL methods for cross-embodiment learning. Highlighting our principal technical contribution of unifying observations and actions for dexterous hands, we introduce MT-Raw-OA. This baseline trains policies using the complete proprioception of the robot and the original hand actions of target joint positions. To align the dimensions of hand joint positions across different hands, we pad them to a fixed dimension with zeros. To diminish the ambiguity in such observations and actions, we include a one-hot label of the embodiment within observations. Similarly, MT-Raw-A uses original actions with our unified observations and MT-Raw-O uses original observations but integrates our proposed eigengrasp actions. All baselines are trained using the same teacher-student framework as our method.

Table 1 presents the success rates of these approaches on YCB objects. Our vision-based policy surpasses all baselines on both training hands and unseen hands, demonstrating its superior performance. According to Figure 5 in the Appendix, our vision-based policy achieves success rates greater than $60\%$ on 42 of the 45 objects across the four training hands. This performance highlights its capability to effectively control various embodiments with a single policy. CrossDex particularly excels in zero-shot performance on new hands, showcasing unique transferability of the learned skills across different embodiments.

MT-Raw-OA and MT-Raw-O demonstrate high performance in state-based training. However, their performance decreases a lot when distilled into a vision-based policy. The primary reason is that these approaches incorporate embodiment labels in observations during state-based training, which makes the policy to rely on this information. During distillation, to enhance generalization across embodiments, we remove these robot labels in the vision-based policy, resulting in less informative observations. Consequently, on unseen hands, these two methods perform worse with state-based policies than with vision-based policies, as the state-based policies utilize the specific unseen label of the new robot, whereas vision-based policies discard this information to enhance generalization.

We notice that even when using the raw action space (MT-Raw-OA and MT-Raw-A), the policy generalizes to unseen hands, albeit with a lower success rate. This generalization is facilitated by the careful alignment of the URDFs for all dexterous hands, ensuring that the arrangement of joint position elements (from the thumb to the little finger) and the corresponding movements (opening the hand with increasing values and closing with decreasing values) are consistent. This manual alignment underperforms our method that leverages human hand eigengrasps to effectively align embodiments.

All methods achieve success rates exceeding $80\%$ during state-based training, demonstrating the effectiveness of jointly training various embodiments. Figure 3 illustrates the training curves of CrossDex alongside those from training each dexterous hand individually.
We observe that cross-embodiment training slightly enhances training efficiency and stability compared to individual training, while achieving comparable performance. This observation supports our approach of co-training across embodiments rather than dividing the embodiment space for individual training.

### 5.3 Embodiment Transfer

**Table 2: Finetuning performance of CrossDex compared to baselines on the unseen LEAP Hand. We evaluate the finetuning of state-based policies and the multi-task vision-based policy for YCB objects. Additionally, we evaluate multi-task finetuning on unseen objects using 55 objects from the GRAB dataset. Training curves for these experiments can be found in Appendix C.2.**
| Method | YCB<br>(5 objects, state) | YCB<br>(multi-task, vision) | GRAB<br>(multi-task, vision) |
| --- | --- | --- | --- |
| No-Pretrain | 0.758$\pm$0.122 | 0.436$\pm$0.159 | 0.313$\pm$0.373 |
| MT-Raw-OA | 0.701$\pm$0.002 | 0.417$\pm$0.007 | 0.651$\pm$0.007 |
| MT-Raw-A | 0.798$\pm$0.002 | 0.390$\pm$0.005 | 0.655$\pm$0.006 |
| MT-Raw-O | 0.708$\pm$0.014 | 0.385$\pm$0.003 | 0.616$\pm$0.018 |
| CrossDex | 0.872$\pm$0.013 | 0.643$\pm$0.009 | 0.740$\pm$0.009 |

Figure: Figure 3: Comparison of RL training curves: cross-embodiment learning vs. individual training on the ShadowHand.
Refer to caption: x3.png

The zero-shot performance on unseen hands, as shown in Table 1, highlights the cross-embodiment tranferability of the learned policy. Such capabilities not only allow for the direct deployment of the policy to new hands but also facilitate efficient finetuning by treating the policy as a pre-trained model for universal grasping skills across all embodiments.
We conduct RL finetuning experiments using a simple yet effective approach based on PPO. We initialize the finetuning policy $\pi$ with the pre-trained CrossDex policy $\pi_{0}$, and during each training iteration, we update $\pi$ using a weighted sum of the surrogate loss from PPO and the KL divergence to the pre-trained policy, $D_{\text{KL}}\left(\pi||\pi_{0}\right)$, to mitigate forgetting.

The results are presented in Table 2 and detailed training curves are provided in Appendix C.2. These results include finetuning state-based policies on the unseen hand, finetuning the multi-task vision-based policy on the unseen hand with all objects, and finetuning the vision-based policy to learn 55 unseen objects from the GRAB dataset (Taheri et al., ). We observe that CrossDex offers a superior pre-trained policy compared to other methods, excelling in all finetuning tasks involving an unseen hand. Notably, while a policy learned from scratch (No-Pretrain) struggles in multi-task learning, finetuning the CrossDex policy proves to be a promising approach to learn tasks across all objects simultaneously. Furthermore, finetuning approaches demonstrate enhanced stability compared to learning from scratch, as evidenced by the reduced variation and smoother training curves.

### 5.4 Ablation Study

Figure: Figure 4: Abaltion study on the eigengrasp action space. “MANO” refers to using raw axis angles of the MANO hand model as actions. “$k$-E” refers to using the first-$k$ eigengrasps as actions. Results show average success rates across five YCB objects.
Refer to caption: x4.png

**Table 4: Ablation study on embodiment randomization and the inclusion of embodiment-specific observations. “Rand.” refers to applying randomization to the hand mount position. “Obs.” refers to including one-hot robot labels and the randomized positions in the observations.**
| Rand. | Obs. | Training hands<br>(State) | Training hands<br>(Vision) | Unseen hands<br>(State) | Unseen hands<br>(Vision) |
| --- | --- | --- | --- | --- | --- |
| ✗ | ✓ | 0.834 | 0.738 | 0.044 | 0.146 |
| ✗ | ✗ | 0.836 | 0.735 | 0.259 | 0.209 |
| ✓ | ✓ | 0.840 | 0.726 | 0.040 | 0.187 |
| ✓ | ✗ | 0.885 | 0.800 | 0.391 | 0.352 |

We conduct ablation studies to assess the impact of key components in CrossDex, particularly focusing on the unified actions and observations.

Retargeting Methods: In our main results, we use DexPilot retargeting to map eigengrasp actions to robot joint positions. To evaluate it, we compare DexPilot with two other optimization-based retargeting methods by training state-based policies on 5 randomly chosen YCB objects. As shown in Table 3, all retargeting approaches yield similar training and test performance, which underscores the robustness of CrossDex to the choice of retargeting algorithm.

Eigengrasp Actions: We explore the importance of using eigengrasps and the impact of varying the number of eigenvectors $k$. Results presented in Figure 4 indicate that modifying $k$ from 1 to 36 leads to consistent training and test performance, highlighting our method’s insensitivity to changes in $k$. However, CrossDex-1-E shows decreased training performance, likely due to the limited capacity of the action space. CrossDex-MANO, which uses the 45-dimensional axis angles of the human hand as actions, performs worse on zero-shot adaptation than the configurations using varied numbers of eigengrasps.

Embodiment Randomization and Observations: Table 4 investigates whether applying embodiment randomization or adding embodiment-specific information to observations affect performance. Incorporating randomization appears to improve performance across both training and test hands, suggesting greater generalizability of the policy. However, including embodiment specific observations increases the performance gap between state-based and vision-based policies on training hands, as these observations benefit state-based policies but are unavailable for vision-based policies. Moreover, these observations lead to poorer generalization to unseen hands, supporting our argument for unifying observations for cross-embodiment learning.

### 5.5 Real-World Experiments

We establish a real-world setup using a RealMan RM65 robot arm paired with a LEAP Hand. The object point cloud is captured using three Intel RealSense D435i RGBD cameras. We provide video records of the real-world test on our [project page](https://sites.google.com/view/crossdex).

## 6 Conclusion and Limitations

In this paper, we propose CrossDex to address the challenges in cross-embodiment learning for dexterous grasping. Our main technical contributions include unifying hand actions with human hand eigengrasps to control various dexterous hands, utilizing pre-trained neural networks for efficient hand pose retargeting, and proposing a shared embodiment-unaware observation space to enhance generalization. Our experimental results demonstrate that CrossDex facilitates robust training compared to individual embodiment training, successfully controls various hand embodiments with a single policy, and effectively transfers to unseen embodiments through zero-shot generalization and finetuning.

Our work has some limitations that can be addressed in future research. First, we train policies on only four dexterous hands due to limited access to dexterous hand models. Future work should include a wider range of dexterous hand embodiments, which is likely to improve embodiment generalization. Additionally, while this work focuses on grasping tasks, there are many real-world tasks for robotic hands – such as in-hand reorientation, dynamic handover, and functional grasping – that remain unexplored in the literature of cross-embodiment learning. Future work could extend CrossDex to these tasks.

## Appendix A Algorithm

Figure: Algorithm 1 CrossDex training process.

## Appendix B Implementation Details

### B.1 Tasks Setup

For our experiments, we use objects from the YCB dataset (Calli et al., ) to evaluate CrossDex. We exclude objects that cannot be modeled as a rigid bodies, objects composed of more than 30 convex hulls after VHACD decomposition (to fit within the IsaacGym simulation), and objects that are either too large or too flat, necessitating functional grasping rewards (Wu et al., ) outside the scope of our study. The resulting dataset comprises 45 objects, representing a diverse array of everyday items with varied geometries, such as cups, scissors, and boxes. The full object list is available in Figure 5. For testing the finetuning performance on unseen objects, we use 55 objects from the GRAB dataset (Taheri et al., ). For state-based finetuning and certain ablation studies, we select five random objects from the YCB dataset, including mustard bottle, mug, spoon, softball, and cup_j.

Our reward function, adapted from UnidexGrasp++ (Wan et al., ), is defined as follows:

$$ $r^{total}=r^{dis}+r^{height}+r^{xy}+r^{success}.$ $$

Here, $r^{dis}$ penalizes the weighted L2 distance between the object center and the fingertips and palm:

$$ $r^{dis}=-2\times\text{average_fingertips_object_distance}-\text{palm_ object_distance}.$ $$

The $r^{height}$ term encourages lifting the object to a specific height:

$$ $r^{height}=\begin{cases}0,~{}~{}\text{if }\text{average_fingertips_object_ distance}\geq 0.12~{}~{}\text{and}~{}~{}\text{palm_object_distance}\geq 0.15 \\ 0.9-2|H-0.6|+(H-0.6)+\frac{1}{|H-0.6|+1},~{}~{}\text{otherwise},\end{cases}$ $$

where $H$ denotes the current height of the object, starting from an initial height of 0.3 meters.

$r^{xy}$ aims to minimize horizontal displacement of the object:

$$ $r^{xy}=-0.3\times\|\text{object_xy}-\text{object_initial_xy}\|.$ $$

Finally, $r^{success}$ awards 200 points upon task completion, defined as maintaining $|H-0.6|\leq 0.05$ and either $\text{average_fingertips_object_distance}\leq 0.12$ or $\text{palm_object_distance}\leq 0.15$ for $n$ steps. $n$ is set to 30 in test and we increase it to 60 in training to enhance policy robustness.

### B.2 Embodiments

We use six dexterous hand models provided by Ding et al. (), allocating four for training and two for test.

The training embodiments consist of:

- •
ShadowHand: a 5-fingered hand with 22 DoFs.
- •
Allegro Hand: a 4-fingered hand with 16 DoFs.
- •
Schunk SVH Hand: a 5-fingered hand with 20 DoFs.
- •
Ability Hand: a 5-fingered hand with 10 DoFs.

For testing, we use:

- •
Leap Hand: a 4-fingered hand with 16 DoFs.
- •
Inspire Hand: a 5-fingered hand with 12 DoFs.

Refer to Figure 9 for a visualization of the six dexterous hands.

Regarding the robot arm, we employ the 6-DoF RealMan RM65 arm. The base of the arm is mounted on the side panel of the table, and the dexterous hand is fixed to the end-effector, which aligns our hardware configuration.

### B.3 Training Retargeting Networks

The retargeting networks are designed to map 45-dimensional axis angles of fingers to joint positions for each dexterous hand. For simplicity, the 3-dimensional wrist rotation and the shape parameters of the human hand are set to zero.
Each retargeting network is a four-layer MLP with hidden layers of 512 dimensions and ReLU activation functions. Each network is trained for 200 epochs, minimizing the MSE loss. Figure 9 visualizes the performance of the learned retargeting networks.

### B.4 Training Policies

Our codebase for RL and DAgger is based on UniDexGrasp (Xu et al., ). For each state-based policy, we use five-layer MLPs for both the actor and the critic, featuring hidden layers with dimensions $[1024,1024,512,512]$ and using ELU activation functions. For the vision-based policy, we use a simplified PointNet (Qi et al., ) backbone incorporating two 1D convolution layers and two MLP layers to process the object point cloud, which has dimensions of $512$. Both the actor and the critic share the output of this backbone.

In state-based RL, we train the policies using PPO across 40,000 iterations within parallel simulations of 8,192 environments. Hyperparameters are provided in Table 5. For vision-based distillation, we use DAgger for 20,000 iterations within parallel simulations of 16,384 environments. Hyperparameters are provided in Table 6.
In our finetuning experiments, state-based finetuning is conducted for 10,000 iterations using 8,192 parallel environments, while vision-based multi-task finetuning is conducted for 20000 iterations using 16,384 parallel environments. Our experiments within environments of 8,192 can be completed on a single NVIDIA RTX 4090 GPU. For the larger scale experiments involving 16,384 environments and PointNet backbones, we use a single NVIDIA A100 GPU.

**Table 5: Hyperparameters of PPO.**
| Name | Symbol | Value |
| --- | --- | --- |
| Parallel rollout steps per iteration | – | 8 |
| Training epochs per iteration | – | 5 |
| Minibatch size | – | 16384 |
| Optimizer | – | Adam |
| Learning rate | $\eta$ | 3e-4 |
| Discount factor | $\gamma$ | 0.96 |
| GAE lambda | $\lambda$ | 0.95 |
| Clip range | $\epsilon$ | 0.2 |

**Table 6: Hyperparameters of DAgger.**
| Name | Symbol | Value |
| --- | --- | --- |
| Parallel rollout steps per iteration | – | 8 |
| Training epochs per iteration | – | 5 |
| Minibatch size | – | 4096 |
| Optimizer | – | Adam |
| Learning rate | $\eta$ | 3e-4 |

## Appendix C Additional Results

### C.1 Success Rates Per Object

Figure 5 shows the success rates achieved by the learned vision-based policy for each YCB object.

Figure: Figure 5: Average success rates for each YCB object across various embodiments.
Refer to caption: x5.png

### C.2 Training Curves

Figure 6, 7, and 8 illustrate the reward vs. training iteration curves for the finetuning experiments.

Figure: Figure 6: Training curves depicting state-based finetuning performance on five YCB objects using the unseen LEAP Hand.
Refer to caption: x6.png

Figure: Figure 7: Training curves depicting multi-task vision-based finetuning performance on all YCB objects using the unseen LEAP Hand.
Refer to caption: x7.png

Figure: Figure 8: Training curves depicting multi-task vision-based finetuning performance on the 55 unseen GRAB objects using the unseen LEAP Hand.
Refer to caption: x8.png

### C.3 Visualization of Eigengrasp Actions and Retargeting

Figure 9 displays samples from the eigengrasp action space and the corresponding joint positions of various dexterous hands predicted by our retargeting networks.

Figure: Figure 9: Visualization of hand poses for some eigengrasp actions and the corresponding poses of the six dexterous hands as predicted by our retargeting neural networks.
Refer to caption: x9.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: - The authors used four dexterous hands for training and would like to include more different hands in the future. It would be interesting to investigate the impact of the number of hands used for training on the learning performance, e.g., generalizability. 
- The dexterous hands for training and testing consist of either 4-fingered or 5-fingered configurations with different DoFs. What would be the impact if for example one or more 3-fingered hand is also included for training?
- The authors randomized the origin position of the joint connecting the hand and the arm to increase robustness and transferability of the policy. What would be the impact of extending this to randomize additional hand parameters?
soundness: 3
strengths: - The authors conducted a comprehensive benchmark and ablation study. 
- The presentation of the paper is clear and well-structured.
summary: This paper learns a unified vision-based reinforcement learning policy via teacher-student learning to control various dexterous hands for grasping different objects. The authors propose a unified observation space across various hands, and a universal action space inspired by human hand teleoperation. The learned policy can generalize to unseen dexterous hands in a zero-shot manner.
weaknesses: - Only LEAP Hand was tested in the real-world experiments.
- Some failure cases should be included in the video.
- Joint position of the arm is included for training policies, which might affect the generalizability to different arms.
- More design choices should be analyzed regarding the impact on generalizability. See the questions below.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: For policy observation, how should different numbers of fingers be handled? From Figure 2, it seems the authors simply consider four fingers of the hand and ignore the others. If the paper only considers anthropomorphic hands with more than four fingers, this should be clearly stated.

Instead of predicting the eigengrasp and performing retargeting mapping, what is the performance of directly predicting fingertip positions?

When evaluating on unseen hands, does the retargeting mapping network use new hand data for training? If this is the case, I believe there should also be a comparison with baselines using those data.

Can you clarify how “MT-Raw-A” makes predictions on the unseen hands? From my understanding, it outputs joint positions for the unseen hands. But what if the unseen hand has a different number of joints compared to the training hands?

Minor questions and suggestions:
* The YCB dataset contains more than 70 objects. Why were only 45 YCB objects used? What was the criterion for selection?
soundness: 3
strengths: This paper presents a clear story on training policies with multiple embodiments. The paper is well-written and illustrates the motivation from human teleoperation well.

The design of the action space (i.e., a linear combination of eigengrasp coefficients) is interesting.

The idea of using a neural network to approximate the retargeting is also interesting.

The authors also show that the policy can successfully transfer to the real world by training a student policy with vision as the input.
summary: This paper presents a method for learning a unified policy for dexterous grasping that can transfer to unseen hand morphologies. It achieves this by first designing an observation space that contains only fingertip positions and palm pose, which can be shared between different hands. Then, the action space is based on the eigengrasp from the MANO hand model. By using the shared observation and action space, the authors can train a reinforcement learning policy on four different hands and transfer the learned policy to new hands.
weaknesses: From my understanding, the observation space contains four fingertip positions. It is not discussed how to deal with three or five fingers, or if the method is specifically designed for anthropomorphic hands. If this is an intrinsic assumption or limitation, it should be discussed in the main paper.

The authors design several ablation experiments to show the importance of the proposed observation and action space. However, there is still one design missing: What if we train a policy that directly predicts the fingertip positions and palm positions while keeping the observation space the same as the proposed method? The fingertip positions could then be converted to joint positions (independently for each finger) using inverse kinematics. This can also generalize to new hands as long as we have the kinematics structure of the new hand.

The retargeting mapping network needs training for each morphology, which means that when evaluating a new, unseen hand morphology, it requires extra data from the target hand. I’m concerned that this is not a fair comparison to the baselines, which does not use these data.

It is also concerning that increasing the number of eigengrasp does not significantly improve performance (Figure 4). This might indicate that the learned grasping actions are quite similar and may be limited for future applications such as functional grasping.

The real experiments are also limited; only three successful trials are demonstrated.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: - What exactly are the observations used by the visual policies, and how are they transferred to the real robot system? In lines 809-810 of the appendix, you mention that the vision policies are trained using DAgger in 16,384 parallel environments, which is double the number of environments used for state-based PPO training. Generating visual observations for such a large number of environments seems particularly challenging, especially given that prior works using student-teacher learning to distill vision policies in Isaac Gym have used two orders of magnitude fewer environments (https://arxiv.org/abs/2211.11744, https://arxiv.org/abs/2111.03043). You also state that "the object point cloud p ∈ R^(N×3), which contains N points captured by cameras." Is this captured by one camera per environment? If so, at what resolution, and how does the object point cloud differ from the point cloud obtained directly from the camera?
- Has a quantitative evaluation been conducted on real-robot deployment?
- In lines 346-347 it is stated that "The episode ends either when the task is completed or the object falls off the table". Are episodes also terminated after a certain number of time-steps?
- Is the object shape or ID represented to the state-based policies? If not, is the grouping of objects intended to reduce variability between them enough to allow learning a single grasping policy? If so, how is this grouping performed, and what are the characteristics of the resulting groups? A clearer explanation of the grouping process and its effectiveness in handling object diversity would help clarify how the policy generalizes across different objects.
- In Figure 3, the training performance of cross-embodiment learning is compared to individual training with the Shadow Hand. Is there a reason to specificallly use the Shadow Hand here, rather than the mean of the performance of all single-embodiment policies?
soundness: 2
strengths: **Relevance of the Problem**
- The huge diversity of embodiments presents a significant challenge in embodied AI, making research on cross-embodiment learning highly relevant.

**Method**
- Leveraging pre-trained neural networks to approximate the optimization-based retargeting process across different embodiments is a intuitive approach to ensure efficient, parallelized training.

**Presentation**
- The writing is clear, and the logical flow of the paper effectively outlines studied problem and proposed solution, making it intuitive to follow.
- The visual presentation in Figures 1 and 2 is well-executed, providing a clear overview of the simulation setup and the input-output mapping from observations through policy actions to target joint positions.
summary: This paper introduces CrossDex, a method for learning cross-embodiment dexterous grasping by defining a unified, embodiment-independent observation and action space. The action space utilizes eigengrasps of the human hand based on the MANO hand model. To enable efficient training, pre-trained neural networks replace the optimization-based retargeting process from eigengrasps to the joint angles of various embodiments. The policies are trained on grasping tasks using objects from the YCB dataset across different robot hands and evaluated for transferability to unseen embodiments. Additionally, visual policies operating on object point clouds are distilled using DAgger.
weaknesses: **Method and Evaluation**
- The baselines used in the evaluations are ablations of the proposed method and do not appear to be particularly strong for this task.
- Training on 4 embodiments and transferring to 2 novel ones is indeed quite limited, as acknowledged in the limitations section. However, attributing this limitation to 'restricted access to dexterous hand models' is not a compelling justification. Incorporating a simulation with a broader range of dexterous hand models and exploring how the performance of cross-embodiment policies scales with an increasing number of embodiments would add significantly value to the study.
- Details on how the vision policies are trained including camera resolution and a definition of the object point cloud are missing.


**Results and Claims**
- In section 5.3 it is stated that the cross-embodiment transferabiliy of the learned policy allows for direct deployment of the policy to new hands. This statement is not supported by the performance drop of over 50% between seen and unseen embodiments oberved for CrossDex in Table 1.
- In lines 254-255 you state that the "the dimensionality of human hand poses is significantly higher than the DoFs of most
dexterous hands, which can make RL less efficient." However, this high-dimensionality does not seem to be a challenge according to the results in Table 1, where policies using the joint-based action space outperform or perform simlarly to CrossDex in the PPO training stage. Further, related work has shown good performance with PPO in parallized simulators even with very high-dimensional action spaces (https://arxiv.org/abs/2305.12127).
- The comparison to baseline methods does not convincingly support the claim that using eigengrasps is an effective approach to solving this problem. Baseline policies based on joint-level control are not expected to transfer well across embodiments with different and unknown joint limits, especially when there is no information about the function of each joint. Using a baseline that directly controls keypoints in the MANO hand model, rather than using eigengrasps, would be a more appropriate baseline to validate this claim. This approach would more naturally align with the task of positioning the fingers into stable grasping configurations on an object’s surface. Moreover, Figure 9 in the Appendix highlights substantial differences in the resulting joint configurations for the same eigengrasp across different embodiments. For instance, in the first row of the figure, the second hand shows three fingers touching at the tips, whereas in the fifth hand, there is a significant gap between the fingers. These very different grasp configurations raise questions about the reliability of grasp transfer under such variation.
- In the ablation on 'Eigengrasp Actions,' Figure 4 examines the effect of varying the number of eigengrasps on performance it is inferred that the method is relatively insensitive to this change. However, the finding that comparable performance can be achieved with just a single eigengrasp raises important questions. This outcome may reflect a characteristic of the problem being studied, which might not require the nuanced control offered by a multi-fingered hand, rather than demonstrating a strength of the proposed method itself.


**Minor Points on Writing**
- In line 483, "affect" should be "affects".


***

Overall, the fact that baselines not specifically designed for cross-embodiment transfer still achieve non-negligible success raises doubts about the suitability of the proposed approach. Moreover, a success rate below 40% on unseen embodiments does not justify the claim of direct transferability. This calls into question the overall feasibility of the framing. Additionally, the observation that a single eigengrasp performs comparably well on the task suggests that the task itself may not be sufficiently challenging. More complex manipulation problems are needed to meaningfully assess the impact of the proposed action space.

Meta-review: The authors study the important problem of cross-platform dexterous grasping. Their goal is to learn a grasping policy via RL that works on multiple hands, which they propose to do using a unique universal action space based on the human hand eigengrasps. These can be predicted and translated into robot commands, which means they can be applied to new robots in a zero-shot manner.

Strengths:
The authors do a comprehensive set of ablations to better characterize their method
Reviewers (hjEm, B2t5) agree that it's well written and structured
Interesting method, with a clever universal action space and neural nets for retargeting
Sim-to-real transfer demonstration on a new hand

Weaknesses:
No quantitative sim to real experiments
Limited generalization (potentially due to relatively few hands existing)
Baselines could be stronger
It would be useful to better characterize the variety of grasping strategies being used - success rates do seem somewhat low on unseen hands, and may not improve with more eigengrasps

Decision: Accept (Poster)