Learning to Regrasp by Learning to Place
Abstract:
In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses.
Regrasping is needed whenever a robot’s current grasp pose fails to perform desired manipulation tasks.
Endowing robots with such an ability has applications in many domains such as manufacturing or domestic services.
Yet, it is a challenging task due to the large diversity of geometry in everyday objects and the high dimensionality of the state and action space.
In this paper, we propose a system for robots to take partial point clouds of an object and the supporting environment as inputs and output a sequence of pick-and-place operations to transform an initial object grasp pose to the desired object grasp poses.
The key technique includes a neural stable placement predictor and a regrasp graph based solution through leveraging and changing the surrounding environment.
We introduce a new and challenging synthetic dataset for learning and evaluating the proposed approach.
We demonstrate the effectiveness of our proposed system with both simulator and real-world experiments.
More videos and visualization examples are available on our project
Introduction
“Regrasping must be performed whenever a robot’s grasp of an object is not compatible with the task it must perform.”
Through regrasping an object can be picked up initially, placed down on an intermediate stable state possibly supported by a second object or the environment geometry, and then be grasped in a target pose and position.
Regrasping is a common daily task and plays a crucial role in connecting various manipulation operations to solve complex long-horizon tasks.
For example, when assembling parts to an IKEA chair, we may need to pick-and-place these parts several times to ﬁnd appropriate orientations before mating and putting them together.
When preparing the food, we may need to pick-and-place the knife and spatula several times to ﬁnd the desired grasp poses for tool usages.
The ability of a robot to regrasp a wide variety of objects is of tremendous importance for many application domains such as manufacturing or domestic services.
However, the large diversity of geometry in everyday objects and the high dimensionality of the state and action space make this a challenging manipulation task.
In this paper, we enable a robot to construct a sequence of pick-andplace operations to achieve various desired grasp poses.
Picking objects, or grasping, has attracted great attention in robotics [2, 3, 4, 5, 6, 7].
In contrast, regrasping, which is generating the sequence of pick-and-place operations, has received considerably less attention.
Early work [1, 8] builds a Grasp-Placement (GP) to search for regrasp sequences.
It ﬁrst ﬁlls valid grasp and placement pairs in the GP table and then searches the table to ﬁnd a sequence of pick-and-place motion.
There are many works following this direction [9, 10, 11].
It usually requires a full observability of the environment and takes a signiﬁcantly long time before ﬁnding a solution given the high-dimensionality of the searching space.
Jiang et al.
[12] use Support
Vector Machines with hand-designed features to score the placement suitability of candidate poses and they focus on the object placement.
In this paper, we propose a novel stable object pose prediction model with deep-learned features extracted directly from the raw point clouds and consider both picking and placing operations for regrasping tasks.
partial regrasp poses as the output a learned object placement module able to efﬁciently generate a diverse poses placement a regrasp a
Our primary contributions are: 1) proposing a novel manipulation system that enables robots to regrasp a diverse set of objects through actively leveraging and changing the surrounding environment, 2) introducing a learning-based model to predict stable object placement poses; 3) generating an annotated dataset for object regrasping tasks and conducting extensive experiments to demonstrate the effectiveness of our proposed approach.
Related Work
We review literature related to the two key components in our approach - predicting object placement and planning regrasp graph and describe how we are different from previous works.
2.1 Object Placement
Pick-and-place is one of the most common robotic manipulation tasks.
Picking objects, or grasping, has attracted great attention in robotics.
For a broader review of the ﬁeld on data-driven grasp synthesis, we refer to [13, 7].
In contrast, object placement, which is the process of deciding where and how to place an object, has received considerably less attention.
Harada et al.
[14] presented an object placement planner to ﬁnd object placement poses by matching planar surface patches on the object with planar surface patches in the environment.
Haustein et al.
[15] introduced a motion planning algorithm for robots to place a grasped object in a cluttered environment.
Jiang et al.
[12] employed Support Vector Machines with hand-designed features to score the placement suitability of candidate poses.
Unlike these works either requiring full observability or relying on hand-crafted feature engineering, we propose an end-to-end approach with deep-learned features extracted directly from the raw point clouds.
2.2 Regrasp for Object Reorientation
Early works [1, 8] build Grasp-Placement (GP) to search for regrasp sequences, by solving the inverse kinematics, checking all collisions to invalidate some grasp-placement pairs, ﬁlling them in the
GP table, and ﬁnally searching within the table to ﬁnd a sequence of pick-and-place motion.
There are many works following this direction.
Rohrdanz and Wahl [9] improved the efﬁciency of regrasp planning using an evaluated breadth-ﬁrst-search, rated grasp, and placement quantities.
Stoeter et al.
[10] replaced the GP table with a space of compatible grasp-placement-grasp triplet and searches this space to ﬁnd a sequence of pick-and-place motion.
Regrasp is also studied in the context of task and motion planning (TAMP).
Lozano-P´erez and Kaelbling [16] presented a framework where a symbolic planner plans a sequence of high-level sub-tasks and a constraint satisfaction problem solver plans low-level operations.
A regrasp plan is composed of a symbolic pick-and-place sequence and a set of geometrically feasible placements, grasps, and paths, which requires a precise full observability of the entire system.
Wan et al.
[11] presented a regrasp planning component for object reorientation.
Given the initial and goal poses of an object, the regrasp planning component ﬁnds a sequence of robot postures and grasp conﬁgurations that reorient the object from the initial pose to the goal pose.
Raessa et al.
[17] presented a hierarchical motion planner for planning the manipulation motion to re-pose long and heavy objects.
They developed a graph-based planning system that combines both regrasp and in-hand manipulation motion considering external support surfaces.
Cao et al.
[18] increased the reorientation capability of a pick-and-place regrasp by adding a vertical pin on the working surface and using it as the intermediate location for regrasping.
The work most related to ours is [19] where they developed a regrasp planning algorithm considering intermediate stable states on ﬁxture such as box baskets.
Unlike these planning based approaches, our learning-based approach do not require full observability and learn features directly from raw sensory data.
Our approach also allows robots to actively change other objects in the environment to create extra supports.
Besides regrasping, object reorientation can also be achieved through in-hand manipulation [20, 21] or dualarm manipulation [22, 23].
Problem Deﬁnition
In this section, we mathematically formulate the object regrasping problem and introduce necessary concepts we use.
Note that there may be various other ways to parameterize the same problem.
We consider a single-arm robot equipped with a two-ﬁngered robotic hand.
However, our proposed pipeline can be applied to many other settings, such as multi-arm robots or dexterous robotic grippers, to enhance their manipulation performances.
We assume that the segmentation of all graspable objects within the surrounding environments is provided.
Note that our approach could also be extended to movable objects leveraging non-prehensile manipulation processes such as pushing.
Notations.
Let O =
{O i } denote the set of objects in the scene, including the target object O t , supporting objects
O i (cid:54) = t , and the ground O g .
We use H , R , and C to denote the robotic hand, the robot, and the robot conﬁguration respectively.
Each object O i and the robotic hand H has its own local coordinate frame T i and T H .
Their pairwise relative transformations are also known.
Let T ig and T H
g denote the 6D poses of the object
O i and the robotic hand H in the world frame
T O g .
Grasping Operation.
A grasp pose G ik =
( p ik , q ik , d ik ) of an object O i is parameterized by two contact points on the object surface ( p ik , q ik ) and an approaching direction of the robotic hand d ik .
All three vectors are described in the object local frame T i .
A grasping operation G ik depends on the grasp pose G ik , the object pose in the world frame T ig , the surrounding environment O − {O i } , and the robot’s conﬁguration C .
A grasping operation G ik is valid if the contact points are in forceclosure, reachable by the robot, and occurring no collision in the process.
A valid grasp G ik attaches the object O i to the robotic hand H , and hence its 6D pose in the world frame can change as the robotic hand moves.
Note that all objects, including the supporting objects O i (cid:54)
=
t , can be grasped, since we may rearrange the environment in various ways to support the target object O t ﬁnally.
Placement Operation.
When an object O i is placed stably on a surrounding environment, we use its global world pose T
i k to describe its placement pose.
A placement operation P i k of the object
O i depends on the placement pose T i k , the surrounding environment O − {O i } , and the robot’s conﬁguration C .
The placement operation P ik is valid if 1) the placement pose is reachable by the robot, 2) there is no collision, and 3) the object remains stable after the robotic hand releases the object.
A valid placement P ik detaches the object O i from the robotic hand.
Its 6D pose in the world frame T ig,k remains when the robot moves away.
network pose
The Regrasping Problem.
The problem of regrasping object O t is deﬁned as ﬁnding a sequence of valid grasping operations G and placement operations P over all objects {O i } to reach a target grasp pose G t ∗ for the target object O t .
Technical Approach
In this section, we introduce our regrasping pipeline, consisting of a plan searching algorithm over a regrasp graph (Fig. 2) and a deep-learned model for stable pose placement prediction (Fig. 3).
4.1 Regrasp Graph Construction and Searching
A regrasping problem solves a sequence of valid grasping G and placement P operations to reach a target grasp pose G t ∗ of the object O t .
We also allow robots to change other objects’ poses {T i (cid:54) =
t g } by applying several intermediate grasping-and-placing operations, since changing the poses of other objects rearrange the environment and may provide extra supports that can increase the number of valid grasping operations for the target object O t .
Below, we describe in details how to construct a regrasp graph and run a searching algorithm over it to ﬁgure out a plan.
Graph Construction.
Our system creates and leverages a regrasp graph to solve the regrasping problem.
Each node N in the regrasp graph includes stable poses {T ig } for all objects {O i } .
An edge E uv connects two nodes N u and N v , if there is one and only one object O j whose stable pose is changed between the two nodes, and there exists a valid pair of grasping operation G jk and placement operation P jk that transform the object’s stable pose, suggesting that the robot can ﬁrst grasp the object O j from its stable pose under one node and then place it at the stable pose in the other node.
Initially, all objects {O i } are stably placed in the scene.
All objects’ initial poses deﬁne the ﬁrst node in the regrasp graph.
For each object O j , we use a deep-learned stable object placement pose prediction model (Sec. 4.2) to estimate the object’s alternative stable poses under the current surrounding environment.
Each alternative stable pose of the object O j , along with other objects’ stable poses, leads to a new node in the regrasp graph.
Whenever a new node is created, our system checks whether there are edges connecting it to the rest nodes and add these edges to the graph.
Graph Searching.
Leveraging the constructed regrasp graph, the solution of a regrasp problem is a path from a starting node N 0 to a target node N t that contains a stable object placement pose T tg,k allowing a valid grasping operation G tl to achieve the target grasp pose G t ∗ for the target object O t .
We use depth-ﬁrst-search algorithm to search for a valid path.
[Figure]
4.2 Stable Object Poses Prediction
We train a neural network to predict stable object placement poses under diverse surrounding environments.
Estimating stable placement poses for a wide variety of objects with diverse and complicated geometry is a challenging task.
It requires very accurate pose predictions, since a slight error may easily result in an undesired object falling or penetration between the object and its surrounding environment.
Hence, we design a two-stage pipeline (Fig. 3) that ﬁrst predicts diverse rough poses
and then reﬁnes to obtain more accurate results.
We describe the detailed network designs below.
Stable Poses Proposal.
Given an object O i and a static surrounding environment S
=
O − {O i } , we ﬁrst train a generative model that learns to propose a diverse set of poses {T ik } for the object O i to be stably placed in the environment S .
Fig.
3 (left) illustrates the inputs, outputs, and design overview for this network.
In our implementation, we represent both the partial observations of the object X i ∈
R m × 3 and surrounding environment X S ∈ R m × 3 , captured by the robot’s RGB-D camera, as 3D point clouds consisting of m = 1 , 024 points.
We ﬁrst employ two PointNet++ encoders [25] to extract global point cloud features f i ∈ R 128 and f S ∈ R 128 .
To propose multiple solutions to the stable poses of the object
O i , we sample a random Gaussian noise z ∈ R 3 as an additional input and use an Multilayer perceptron (MLP) to extract a feature f z ∈ R 64 .
We concatenate f i , f S , and f z to produce a single joint feature f i, S ,z and ﬁnally employ another MLP to decode a ﬁnal 6-DoF pose ˆ T
i g,z, S for the object
O i .
For brevity, we omit the subscripts g denoting the global world coordinate, S indicating the environment, and the superscript i representing the shape
O i in the following paragraph.
The 6-DoF pose is composed of a 3D translation and a 3D orientation.
For the orientation part, we adopt the axis-angle representation, which has been shown to be effective for pose prediction task [26].
Sampling different Gaussian noises { z l } rl =1 , we obtain a diverse set of pose predictions { ˆ T z l } rl =1 .
Provided with a ground-truth list of stable poses {T z l } r ∗ l =1 generated by running several random interaction trials in physical simulation, we deﬁne a loss L M to train our pose prediction outputs.
[Equation]
where L measures the distance between two 6-DoF poses.
We implement L as the average Ldistance between the corresponding points of the object point clouds after applying the two poses.
Stable Pose Classiﬁcation and Reﬁnement.
We train a model to classify the stability of the sampled poses.
For each predicted pose ˆ T z l , we ﬁrst concatenate the transformed object point cloud
ˆ T cz l ( X i ) with the surrounding environment point cloud X S .
Then, we augment this combined point cloud with one extra 1D arrays of ones and zeros, to indicate if a point belongs to X i or X S , along the XYZ dimension to form a tensor of shape ( M + M, 3+1) .
With this augmented input, a straightforward method for stable classiﬁcation would be using a PointNet++ [25].
However, we ﬁnd that this naive approach does not provide satisfying results due to the intrinsic difﬁculty of analyzing stability among two separate geometry components.
To facilitate learning, we propose to solve this problem via a multi-task supervised learning approach.
After obtaining the feature from the PointNet++ backbone, we have three task branches that perform stable classiﬁcation, contact classiﬁcation, and contact point regression simultaneously.
For contact point regression, we use a PointNet++ decoder to regress the offset from each point to the nearest ground-truth contact point with a loss denoted as L offset .
We train two MLP classiﬁers for stable and contact classiﬁcation with losses denoted as L stable and L contact respectively.
Our total loss is deﬁned as:
[Equation]
where L stable and L contact are standard binary cross entropy losses and L offset is a smooth L 1 loss.
In our experiments, we set λ 1 = 10 and λ 2 = 1 .
Leveraging the above classiﬁer function denoted as V , we apply CEM [24] to search for a locally optimal goal pose starting from the initial predicted poses.
We ﬁrst apply the predicted transformation ˆ T to the object point cloud X i to get a point cloud
˜ X i .
A point a in the CEM searching action space is a 6D transformation T a which transforms the object point cloud ˜ X i into T a ( ˜ X i ) .
A point s in the CEM state space is the transformed object point cloud along with the surrounding environment point cloud ( T a ( ˜ X i ) , X S ) .
When selecting the action a , we run a derivative-free optimization method CEM [24] to search within the 6D pose space to ﬁnd a 6D transformation T a associated with the highest score in the value model V ( s ) .
[Equation]
The 6D pose associated with a ∗ is the ﬁnal stable placement pose produced by our model.
4.3 Object Grasping
In this subsection, we describe how our system performs valid grasping operations.
Recall that a grasp pose G ik of the object
O i is parameterized by two contact points on the object surface ( p ik , q ik ) and an approaching direction of the robotic hand d ik .
We adopt UniGrasp [6], an efﬁcient data-driven grasp synthesis method, to generate a list of valid grasp poses {G ik } of the object O i .
UniGrasp learns a deep neural network to select a set of contact points ( p ik , q ik ) from the input point cloud of the object O i .
We refer to Shao et al.
[6] for more detailed description of the method.
Given two contact points p ik and q ik , we calculate a plane perpendicular to the line segment of the two contact points intersecting at the middle point of the line segment.
We then consider 36 approaching directions evenly distributed within the plane, which produces 36 potential grasp poses.
For each grasp pose G i k , we solve the inverse kinematics of the robot.
If there is no valid inverse kinematic solution or there are collisions in all possible approaching directions, we drop the contact point pair and continue checking for the next pair of contact points.
We keep running this process until we ﬁnd a valid grasping operation if there exists one.
Experiments
In this work, we develop a learning framework for robots to learn to regrasp objects.
Our experiments focus on evaluating the following questions: (1) How effective is our proposed learning-toplace approach compared to other baselines?
(2) Whether our proposed learning-to-regrasp pipeline can ﬁnd good solutions to the object regrasping problem?
(3) Whether the multi-task learning for contact point predictions we use in Sec.
4.2 is useful or not?
Please refer to supplementary materials for further experiment details, such as ablation study and failure cases analysis.
Dataset.
We construct a dataset containing 50 objects (i.e., spoon, fork, hammer, wrench, etc.) and 30 supporting items (i.e., mug, box, bowl).
We then split these objects and supporting items into training and test sets.
We generate 249 pairs from the training set of objects and supporting items, and 38 pairs from the test.
For each pair of object-supporting item, the initial placement poses are generated by sampling different object poses with respect to the supporting item in PyBullet, and running a forward simulation to see if it remains stable.
We collect the placement data for supporting items by randomly placing the supporting items with respect to the initial environment (i.e., the ground), and running the simulation to check whether the supporting items keep static.
In total, we generate 1,050,884 poses for training items and 15,105 poses for testing items.
We visualize the data in supplementary material.
Settings.
We ﬁrst set up the simulation environment in PyBullet [27] for evaluating our full pipeline on synthetic data.
We load the object at the predicted pose and check if there is collision and whether the object is stable.
We then evaluate the regrasping task on a real robot system.
5.1 Object Stable Placement
We evaluate our proposed framework on learning stable placement poses in this subsection
[Table]
achieves
[Table] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
Evaluation of Pose Proposal Network.
We ﬁrst evaluate whether the pose proposal network can generate a reasonable distribution of the stable poses for placing the object over the supporting items, as a diversifying stable pose distribution is crucial for the success of ﬁnding desirable regrasping solutions.
For each input pair, we draw 128 poses from the proposal network, and we transform the candidate object to the proposed stable poses, the qualitative results are presented in Fig.
Our visualization demonstrates that our proposed stable poses yield a diversifying object placement.
Evaluation of Pose Classiﬁer.
To evaluate how accurate the pose classiﬁer is, we estimate the scores for the sampled and reﬁned 128 poses using the pose classiﬁer.
We use a threshold of 0.8 to ﬁlter out all the unsure poses.
For verifying the remaining poses, we run a forward simulation in PyBullet [27] to check whether the pose keep static, and the accuracy of stable placement for each category are reported in Table 1.
We ﬁrst adopt a baseline which is based on a random sampling strategy that draw the rotation from
SO(3) uniformly, and translation is sampled from [ ± 0 . 05 , ± 0 .
05 , z +0.02] in the supporting item’s local frame, where z is determined by the maximal height of each supporting item.
We also compare our approach with Jiang et al.
[12] in Table 1.
Followed their method, we randomly sample 128 poses in the local frame of the supporting environment, and ﬁlter out all the unstable poses using the learned classiﬁer, the remaining ones are used for evaluating the accuracy in PyBullet [27] through running a forward simulation.
The results suggest that even with random sampled poses, a simple classiﬁer with hand-craft feature can improve the stable placement accuracy.
We also test another baseline that adopt heuristic rules for pose sampling but with the same classiﬁer from [12].
For this baseline, the object translation in XY dimension is sample in the radius of 0 .
9 ∗ r and the translation in Z dimension is determined by z + 0 .
01 , both are in support’s local frame, where r and z are the radius and maximal height of supporting item respectively.
We additionally rotate the object to make the longest axis of the object to be parallel with the XY plane.
While this baseline can achieve some reasonable results in some easier pairs that with a nearly ﬂat surface (i.e., box-wrench, box-spatula), it perform worse with supporting items with complicated geometry (i.e., bowl, mug).
The results demonstrate that a good pose proposal algorithm is also of great importance to the success of stable object placement.
With carefully designed stable pose proposal network and stable pose classiﬁer network, our framework achieves the best results over all the approaches.
Finally, to quantitatively measure the diversity of the remaining stable object placements, we deﬁne two different poses as their translation difference is larger than τ 1 and the rotation angle between them is larger than τ 2 .
In our experiment, τ 1 and τ 2 are set to be 0 .
03 and 30 respectively.
We report the average number of different stable object placements in Table 2.
The results showing that although random uniform sampling combined with heuristic rules can generate diverse stable poses for some easier environment (i.e., boxes with ﬂattened surface), it can not handle complicated geometry.
Our proposed method outperforms the baseline and achieves the best performance on average.
5.2 Object Regrasping Performance
We evaluate the performance of our model in real-world experiments on everyday objects.
We ﬁrst test the pose proposal network on point clouds from real depth sensor, we use Intel RealSense in our experiments.
Utilizing the camera matrix, we can reconstruct a 3D point cloud.
The point clouds are fed into our model to predict stable placement poses.
We then run Unigrasp [6] to generate a list of valid grasp poses and search for the desired target poses based on the pipeline described in Sec.
The generated stable placements and several key frames of the regrasping process are visualized in Fig 5.
For more results on the regrasping task in simulator, please refer to our supplementary material.
Conclusion
We present a system that can regrasp a diverse set of everyday objects to desired grasp poses for various manipulation tasks.
Our system learns to predict the stable placement of objects based on partial point clouds of the object and the surrounding environments.
The system also creates a regrasp graph and allows robots to leverage and actively change other objects’ poses to provide extra supports.
We demonstrate the effectiveness of our system on both challenging synthetic dataset and in real world regrasping tasks.
Currently our system only takes raw vision sensory data as inputs.
For future work, we would like to explore how to incorporate other modalities such as force or tactile signals in our system.