Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation
https://sites.google.com/view/implicitaugmentation/home
[Figure]
Abstract: Dexterous robotic hands have the capability to interact with a wide variety of household objects to perform tasks like grasping.
However, learning robust real world grasping policies for arbitrary objects has proven challenging due to the difﬁculty of generating high quality training data.
In this work, we propose a learning system ( ISAGrasp ) for leveraging a small number of human demonstrations to bootstrap the generation of a much larger dataset containing successful grasps on a variety of novel objects.
Our key insight is to use a correspondenceaware implicit generative model to deform object meshes and demonstrated human grasps in order to generate a diverse dataset of novel objects and successful grasps for supervised learning, while maintaining semantic realism.
We use this dataset to train a robust grasping policy in simulation which can be deployed in the real world.
We demonstrate grasping performance with a four-ﬁngered Allegro hand in both simulation and the real world, and show this method can handle entirely new semantic classes and achieve a 79% success rate on grasping unseen objects in the real world
Introduction
Human hands are powerful tools for manipulating a wide range of objects.
Our goal is to build dexterous robotic manipulators that can robustly and adeptly interact with human-centric environments.
However, robustly controlling a dexterous hand remains challenging due to its high dimensional state and action space and its multi-modal contact dynamics.
Recent work has used deep reinforcement learning algorithms to learn complex tasks with dexterous manipulators [1, 2, 3, 4, 5, 6].
These methods often require careful reward engineering and environment design, and often lack generalization, struggling with robust real world deployment.
An alternative paradigm involves collecting a large labelled dataset and using supervised learning (imitation learning).
This has shown signiﬁcant success with parallel jaw grippers [7, 8, 9] and suction cups [9, 10].
However, imitation learning methods have proven difﬁcult to scale to multi-ﬁngered robots due to the burden of human data collection.
To scale robust policy learning to dexterous grasping, we require techniques that can leverage a small amount of human effort to learn robust, general grasping policies.
This can be done by extrapolating a small number of human demonstrations to generate an abundance of data that interacts with diverse objects and is successful at grasping objects under real world dynamics.
The key question is —how do we generate this type of diverse training data?
In this work, we propose Implicit Shape Augmentated Grasping (ISAGrasp), shown in Figure 1, a learning system that leverages a correspondence-aware generative model [11] to extrapolate a small number of human demonstrations to a large dataset of realistic objects and their corresponding grasps.
In particular, we directly perform deformations on implicit 3-D shape representations of various objects in a learned latent space.
The representation of point-wise shape deformations allows us to generate transformed grasps for novel objects that can be made successful with a small amount of active interaction in simulation.
In doing so, implicit shape augmentation allows us to grow from a small dataset of human demonstration data for dexterous grasping to a signiﬁcantly larger and more diverse dataset with novel object shapes and dynamically successful grasps.
Given the dataset constructed by implicit shape augmentation, we can now perform large scale supervised learning.
In this work, we represent a policy as predicting a pre-grasp and a ﬁnal pose from pointclouds of the target object, with intermediate motion being performed with standard motion planning libraries.
We show empirically that a policy learned via supervised learning on the dataset constructed by
ISAGrasp is able to generalize widely across different objects in simulation and the real world.
We demonstrate the efﬁcacy of this pipeline on the rescaled YCB instances, ShapeNet and GoogleScans objects in simulation and achieve a 79% success rate on 22 unseen objects in the real world.
In summary, our contributions are (1) we propose a novel system, ISAGrasp , that is able to generate both a wide variety of objects and the corresponding dexterous grasps from a few human demonstrations.
(2) We show that the augmented dataset can be used for learning point-cloud based control policies that are able to robustly grasp a large variety of novel objects in simulation (3)
We demonstrate the efﬁcacy of the proposed pipeline by deploying it in the real world.
(4) We analyze the proposed pipeline through several ablation studies in simulation.
ISAGrasp: Dexterous Grasping Policies via Implicit Shape Augmentation Latent Vector
[Figure]
Oi To learn robust grasping policies that can operate in the real world for grasping novel objects of various shapes, we propose ISAGrasp —a framework for supervised learning of robust and general-
[Figure]
d g =
N (cid:88) i
= (cid:107) (cid:126) a r
i − s r (cid:126) a h i (cid:107) 2 , (1) d c =
N (cid:88) i
= (cid:107) (cid:126)
c r i − (cid:126) c h
i (cid:107) 2 , (2) d r = G ( M r , M h ) , (3) arg min ( [Equation] r [Equation]
f r [Equation] r [Equation]
[Equation]
izable grasping policies from successful grasps on a large variety of objects generated from a small set of human provided expert demonstrations.
Given a pointcloud of an object, we model a grasping policy as predicting (1) a pre-grasp pose T that controls robot hand’s translation T t and rotation T q , and (2) a ﬁnal pose G f that controls the 16-DoF ﬁnger pose that can ﬁrmly grab the object.
ISAGrasp assumes access to a set of labelled human grasping demonstrations, can be used to learn grasping policies via supervised learning.
However, to learn truly robust and general grasping policies, this dataset must be grown to a much more diverse set of objects and successful grasps.
Generating a multi-ﬁngered grasping dataset of diverse yet realistic objects is a non-trivial problem [12, 13, 14].
We approach this problem by leveraging a correspondence-aware, deformation-based generative model to widely augment the set of human provided demonstrations, which can then be used to learn robust and general policies via supervised learning.
An overview of our ISAGrasp system is shown in Figure 2, and we detail each component below.
2.1 Human-Robot Retargeting
ISAGrasp ﬁrst collects N human hand-arm demonstrations D
= { τ h 0 , τ h 1 , . . .
, τ hN } using motion capture, where each demonstration τ i = { ( h i 0 , o i 0 ) , ( h i 1 , o i 1 ) , . . .
, ( h iT , o iT ) } is a trajectory of hand pose h t and object pose o t .
These demonstrations are then “retargeted” to a 22-DoF ﬂoating Allegro hand [15] in simulation.
Note that this is a nontrivial problem since these demonstrations are in the morphology of the human hand and are typically not functional when directly mapped to a robot hand using standard Inverse Kinematics [16].
We formulate the retargeting objective as a non-linear optimization problem.
First, to allow a robot to grasp in a similar pose to a human demonstration, we use the same cost function proposed in
DexPilot [17], shown in Eq. 1, where s r is the ratio between the sizes of the robot and human hands, and each (cid:126) a r i and (cid:126) a h
i is a displacement vector between ﬁnger tips and the palm of the robot and human hands, shown as blue in Figure 3.
While Eq.
1 encourages grasp shape similarity, we additionally minimize the differences between the relative poses to the object, as captured by the vectors between ﬁngers and the object center for the human grasp, (cid:126) c r i , and the retargeted grasp, (cid:126) c h i (see Eq.2 and orange lines in Fig. 3).
Finally, to orient the robot palm similar to the human hand, we add Eq. 3, which optimizes the minimum geodesic distance G in SO(3) between the rotation matrix of the human palm M h and the robot palm M r .
The ﬁnal retargeting goal is to ﬁnd the 22 DoF conﬁguration that minimizes the weighted sum of these three terms, which results in a candidate grasp G r for each object in a human provided dataset.
As we describe in Section 2.3, the retargeted grasps are then reﬁned via a rejection sampling step and used to generate the dataset with successful, dynamically consistent retargeted grasps.
However, since this dataset is typically quite limited, we next outline how to augment the dataset with novel objects and grasps via implicit shape augmentation.
2.2 Implicit Shape Augmentation
Given a small set of object point clouds and successful retargeted grasps, G r , we build a large scale augmented dataset of novel objects and their corresponding successful grasps G d , using a correspondence-aware implicit generative model, DIF-Net [11].
To recap at a high level, DIF-Net aims to learn an implicit representation of 3D shapes as a scalar ﬁeld.
More speciﬁcally, DIF-Net represents a 3D shape via a instance agnostic template implicit ﬁeld (which is common for all shapes of a particular category and represent the common features of a category), together with a latent conditional 3D deformation ﬁeld (that perturbs this template ﬁeld) which allows the shapes to be adapted to every particular object instance while maintaining semantic 3-D structure.
Different novel but realistic shapes can be generated by sampling different latent vectors α and generating object deformations on known object classes using the learned deformation and correction ﬁelds, while maintaining semantics.
This model naturally provides dense correspondences across object instances since different object instances are pointwise deformations on the same template.
We use this generative model in two ways: (1) leverage the ability to sample a variety of object instances by sampling latent vectors α to generate various novel objects via deformations and (2) the resulting dense correspondences allow us to estimate dynamically consistent grasps from human demonstrations to novel generated objects.
Speciﬁcally, we use the latent conditional deformation ﬁeld from a pretrained DIF-Net to generate novel instances and dynamically consistent grasps.
To generate novel objects, we sample latent vectors α from a Gaussian that conditions a latent conditional deformation ﬁeld.
This deformation ﬁeld generates point-wise deformations of particular object meshes chosen from the set of human demonstrations to generate novel objects.
Second, to estimate the new grasps G d for the deformed objects, we ﬁnd N reference points on the original mesh that are close to the root position t r of the robot hand and compute the position t o of the robot relative to a local coordinate system O i , centered at each reference point i : t oi =
O − 1 i · t r .
Once the object is deformed, we compute the corresponding coordinate system O di on the deformed meshes using the same deformation ﬁeld, to estimate the new grasp location t d using the average of the local offsets: t d = 1 N Σ Ni =1 ( O di · t oi ) .
In this way, the dense correspondences obtained via the DIF-Net are directly useful in generating grasps for novel, deformed object instances.
[Figure]
Figure 5 visualizes how objects and correspondences deform with different sampled latent vectors.
We highlight the reference points on the original mesh, and their correspondences on deformed meshes (purple circles).
The objects are deformed into a variety of realistic shapes while maintaining the original semantic structures and grasping correspondences.
Using the new grasp location t d , together with retargeted orientation q r and ﬁnger pose f r , we can construct new grasps G d ( t d , q r , f r ) .
While G d are not guaranteed to be successful grasps, they provide good starting points for the local search procedure described next.
2.3 Grasp Reﬁnement for Dynamics Consistency
Given transformed grasps G d , we use pose perturbation with rejection sampling to generate successful and dynamically consistent grasping poses.
Since our shape augmentation model transforms successful grasps via the 3D deformation ﬁeld, we found that only a small amount of perturbation is typically needed to ﬁnd successful grasps for the deformed models using rejection sampling.
In particular, we sample local perturbations δ t , δ r and δ f from a uniform distribution, and add these perturbations to the translation t d , rotation q r and ﬁnger joints f r of the transformed poses G d .
We evaluate success of each perturbed grasp G p using a physics simulator (Pybullet [18]) and add domain randomization to save robust successful grasps and objects to the training dataset.
[Figure]
N o N p N f
N t
Using above methods, we can generate a large and successful dexterous grasping dataset with a variety of novel objects, and then perform supervised policy learning, as described in the next section.
2.4 Policy Learning via Supervised Learning
As described in the beginning of Section 2, we model the grasping problem as predicting a pre-grasp pose T and a ﬁnal pose G f given an object pointcloud observation.
We perform a standard empirical risk minimization procedure on the above-mentioned dataset as an architecture and use a network consisting of PointNet++ SA modules [19] as a feature extractor.
Instead of only feeding the raw point cloud to the network, we found signiﬁcant improvements by appending object points with additional information regarding the alignment between the robot hand and the local object surface.
In particular, we use the object point cloud p , the surface normal at each point (cid:126) N o , the normal of the table surface (cid:126)N t , and the hand facing direction (cid:126)N f and pointing direction (cid:126)N p to deﬁne the following feature vectors: f ( p ) =( p x , p y , p z , ( (cid:126)N o · (cid:126)N t ), ( (cid:126)N o · (cid:126)N f ), ( (cid:126)N o · (cid:126)N p ), ( (cid:126)N f · (cid:126)N t ).
We append these vectors to each point in the point cloud.
We found these features provide a compact description of alignment between the robot hand and the object, and using the relative vector alignments via the pairwise dot products allows us to improve the ﬁnal grasping performance.
The network outputs a 3-dim translation and 4-dim quaternion, which are used to deﬁne the pregrasp pose T .
Additionally, the network predicts a 16-dim ﬁnger poses, which is used to deﬁne ﬁnal pose
G f .
Figure 6 shows our network architecture details.
We provide training details in Appendix A.
System Details
Initial Dataset Construction.
We extracted human demonstrations from the DexYCB dataset [20], which contains mocap sequences D human of humans hand poses q iT picking up 20 YCB objects [21] with poses o iT .
We picked 10 demonstrations per object, resulting in 200 demonstration in total.
Appendix B 8.2 further explains our dataset choice.
ISAGrasp Implementation Details.
We use a pretrained DIF-Net [11] to augment objects into a variety of novel shapes.
First, we sample a 128-dim latent vector from a Gaussian distribution ∼ N ( µ, σ 2 ) , where µ = 0 and σ = 0 .
Second, to estimate corresponding new grasps G d , we choose N = 20 closest points on the object surface as reference points and compute corresponding grasp location t d (Section2.2).
To obtain successful deformed grasps, we apply rejection sampling by uniformly sampling δ t ∈ [ − 0 .
02 m, 0 .
02 m ] for translation t d , and δ r ∈ [ − 0 .
5] radians for rotation q r , and apply the same perturbation δ f ∈ [ − 0 .
1 , 0 . 1] radians for ﬁnger joints.
Hardware Setup.
We deploy our policy to a robotic platform that has 23 actuators across a KUKA
LBR iiwa 7 R800 robot arm and a Wonik Robotics Allegro robotic hand, and use two cameras to provide necessary point cloud information.
Details are provided in Appendix B.
Experiments
Through our experimental evaluation, we aim to answer the following questions:
Does ISAGrasp generate realistic novel objects based on a small set of scanned objects?
Does ISAGrasp allow for easy generation of dynamically consistent grasps on novel objects through grasp transformation and reﬁnement?
Do policies learned on the dataset produced by ISAGrasp show improved robustness and generalization on unseen objects?
How does the training perform with different input features.
[Table]
We address these questions through a study in a PyBullet [18] simulation [Table] followed by a real world experimental evaluation using the robotic system described in Section 3.
We provide additional details of baselines in appendix B and more analysis on elements of the ISAGrasp in Appendix C.
Evaluation Metrics.
We choose three unseen datasets with an increasing complexity:
RescaledYCB, ShapeNet [22] [Table] and GoogleScans [23] (see appendix B).
In particular, RescaledYCB contains 65 rescaled YCB objects, ShapeNet contains 200 unseen objects from ”Can”, ”Bottle”, ”Mug” and ”Bowl” categories [Table] and GoogleScans objects contains 200 everyday objects.
We place objects randomly on the table, and evaluate performance with 5 sets of object mass and friction.
The success is deﬁned as when the object is above the table by 10cm.
Baselines.
Table 1 additionally compares the performance of our method to other baselines — (1)
Randomly generate grasps around the object (Random) (2) use random baseline with rejection sampling to create successful dataset and train a policy (3) perform a predeﬁned grasp where robot always grasps from the top (Heuristic).
(4) train with Heuristic baseline with rejection sampling (5) train with Graspit!, an optimization-based grasp planner from prior work[24].
In addition, we train compare RL baselines: (6) one with a dense reward function using PPO[25]
(7) One using an RL method (DAPG + DR) that combines an imitation learning objective via behavior cloning with reinforcement learning [26]. (8) train with domain randomization, but no shape augmentation (DexYCB + DR - ISA) (9) train with no shape augmentation or domain augmentation (DexYCB - DR - ISA) in order to understand the impact of shape augmentation and the impact of domain randomization. Please ﬁnd further details of these baselines in the Appendix B. [Table]
4.1 Simulation Results.
We ﬁrst evaluate the efﬁcacy of the ISAGrasp system on learning dexterous grasping policies in simulation.
Table 1 shows the overall success rate of our method ISAGrasp on three unseen datasets in simulation.
ISAGrasp (bottom row) is able to achieve 74% success rate on rescaled unseen YCB objects even the policy is only trained on augmented shapes.
In addition, our method achieves 74% and 70% success rate on ShapeNet and GoogleScans objects.
We describe analysis on how the various baselines perform in details in Appendix B.
4.2 Real World Experiments.
Next, we evaluated the grasping policy learned in simulation by ISAGrasp, directly in the real world.
We perform directly simulation to reality transfer of the learned policy as described in Section 2.4.
We evaluate our policy on 22 unseen real world daily objects (see Appendix Figure 17) and evaluate each object 5 times with random poses.
We report number of success on each object in Table 2.
On average, the policy is able to achieve 79% success rate on real world evaluation on novel objects.
We observe several failure cases: (1) if the object is more transparent (Box1: container box), the pointclouds are incomplete and the network is less robust.
(2) objects which require more careful grasping (Power drill) often have lower success rate.
These show that we can leverage policies trained in simulation directly for real world grasping using ISAGrasp and that the robustness and generalization properties transfer from simulation to the real world.
4.3 Ablations and Analysis
We ﬁrst provide some insights into the impact of various design decisions in ISAGrasp below:
Impact of using correspondence-aware generative models: To understand how important us-
[Figure]
ing the dense correspondences provided by DIF-Net are for generating successful grasps on novel objects, we compare DIF-Net [11] with a correspondence-agnostic model, shapeGAN [14] by showing their generated meshes and the reﬁnement rate for obtaining successful grasps.
Reﬁnement rate refers to the ratio of grasps that can be successfully reﬁned via rejection sampling to the total number of proposed grasps.
We use this metric to compare the efﬁciency of creating stable grasp datasets generated using different approaches.
Shown in Figure 8 (Left), DIF-Net can smoothly deform the original shapes and generate more realistic shapes.
We conduct 50 times of perturbation with rejection sampling on 100 objects generated by both methods.
Figure8 (Right) shows reﬁnement rate ( R : random grasps without human demonstrations, D : using original human demonstrations without new grasps G d estimation, Corr : using new grasps G d ).
Initialized with correspondence-guided grasps, DIF-Net (Corr) achieves 76% reﬁnement rate while ShapeGAN(D) achieves below 30% with the same number of reﬁnements.
This indicates the importance of transferring grasps through a deformation based transformation for novel objects.
Impact of Input Representation:
To understand the choice of input representation for supervised learning, we compare feature choices used as input for training policies, evaluated on objects from the ShapeNet and GoogleScans datasets.
We observe that using pointcloud p , surface normal vector (cid:126)N o , Robot vectors (cid:126)N f and (cid:126)N p performs best.
Related Work
[Table]
Manipulation with Dexterous Hands.
To control a dexterous hand, prior work has investigated including planning with analytical models [27] and online trajectory optimization [28].
However, these methods assume accurate dynamics models and robust state estimates, which are difﬁcult to obtain in complex real-world manipulation.
Learning-based approaches particularly with deep reinforcement learning (RL) [29, 30, 5, 31, 32, 6, 2] have been investigated.
Despite the progress, training deep RL models remains challenging due to high sample complexity and reward engineering.
Although this issue has been mitigated by incorporating human demonstrations [26, 33, 3, 34, 1], these methods are still faced with a major challenge in scalability[3, 2].
Prior work has collected demonstrations kinesthetically [33], through VR interfaces [26], or using motion capture (mocap) solutions [35, 36, 37, 3], which are often limited in size. In comparison, the focus of our work allows policies to generalize to novel, unseen shapes, which could be easily combined with pipelines for improvement with RL with demonstrations [3, 1, 38, 26].
Data Augmentation and Robustness.
Data augmentation has been used traditionally in vision tasks where images are cropped, rotated, normalized, etc [39, 40, 41, 42, 41, 42] to improve generalization and model robustness.
In contrast, our work aims to generate novel objects with varying physical dynamics and does not simply learn invariant behavior but learns different grasping behavior for different objects.
In a similar vein, domain randomization is used in robotics, where predeﬁned parameters such as lighting, camera are randomized during training[43, 44] in order to learn a policy that is invariant to these parameters.
However, this randomization does not aid with generalization to novel object shapes.
More recently, methods have aimed to generate novel objects shapes programmatically [13], or use a generative model to create new objects in an adversarial setting [45].
This is.
motivated similarly, but differs in being for parallel jaw grasping problems and not generating dynamically consistent grasping behavior via correspondences.
Limitations
Challenging objects: it’s more challenging for our policy to succeed when the object is large or too ﬂat. Since our shape augmentation is built on dexYCB dataset, it becomes more challenging if the test object is too different from the training objects, or requires a more speciﬁc way of grasping. The shape augmentation also does not cover objects of widely varying scales.
See visualizations on challenging objects in Appendix D. Real world experiments: The method assumes access to a fairly complete point cloud.
It will not succeed in scenarios with heavy amounts of occlusion or noise in the point clouds. Functional grasping: Currently the method does not do dexterous and functional grasps, and is only designed to lift the object. The utility of a dexterous hand is perhaps best utilized with functional grasps, but the current system does not optimize for this directly. Accounting for kinematics:
The current system does not account for the kinematics of potentially hitting the table when the hand is mounted on a full arm setup. This should be accounted for as we build on this work in the future.
Conclusion
We present ISAGrasp, a novel system for learning dexterous grasping policies in the real world.
ISAGrasp leverages a data augmentation approach that bootstraps a small number of human demonstrations with a large dataset with diverse and novel objects and grasps.
By using a correspondenceaware generative model, we can deform original object shapes and generate dynamically consistent new grasps.
We create a large and diverse grasping dataset and train a policy via supervised learning that can then be deployed in simulation and the real world on grasping novel objects, achieving over 75% success rate on grasping novel object instances in the real world. Acknowledgement
We thank Mohit Shridhar for providing helpful feedback on our initial draft.
We thank Aaron Walsman for providing additional helps on ﬁgures and the draft.
We are also grateful for Vikash Kumar,
Aravind Rajeswaran, Jason Ma and Mandi Zhao for discussions on running RL baseline experiments.
Part of this work was done while Qiuyu Chen was an Intern at NVIDIA.
The work was also funded in part by an Intel gift.