6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting without considering perception feedback, dynamics, and contacts with objects, which makes them sensitive to grasp synthesis errors. In this work, we propose a novel method for learning closed-loop control policies for 6D robotic graspingusing point clouds from an egocentric camera  . We combine imitation learning and reinforcement learning in order to grasp unseen objects and handle the continuous 6D action space, where expert demonstrations are obtained from a joint motion and grasp planner. We introduce a goal-auxiliary actor-critic algorithm , which uses grasping goal prediction as an auxiliary task to facilitate policy learning. The supervision on grasping goals can be obtained from the expert planner for known objects or from hindsight goals for unknown objects. Overall, our learned closed-loop policy achieves over 90\% success rates on grasping various ShapeNet objects and YCB objects in simulation. The policy also transfers well to the real world for grasping unseen objects in both a tabletop setting and a human-robot handover setting in our experiments. Our video can be found at URL .
6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting , which are sensitive to grasp synthesis errors. In this work, we propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object . We combine imitation learning and reinforcement learning and introduce a goal-auxiliary actor-critic algorithm for policy learning. We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects. Our videos and code can be found at URL .
R
without considering perception feedback, dynamics, and contacts with objects, which makes them
, which are
R
novel
new
R
robotic graspingusing point clouds
grasping. Our policy takes a segmented point cloud of an object
A
as input, and outputs continuous 6D control actions of the robot gripper for grasping the object
R
in order to grasp unseen objects and handle the continuous 6D action space, where expert demonstrations are obtained from a joint motion and grasp planner. We
and
R
, which uses grasping goal prediction as an auxiliary task to facilitate
for
R
The supervision on grasping goals can be obtained from the expert planner for known objects or from hindsight goals for unknown objects. Overall, our learned closed-loop policy achieves over 90\% success rates on grasping various ShapeNet objects and YCB objects in simulation. The policy also transfers well to the real world for grasping unseen objects in both a tabletop setting and a
We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a
R
handover setting in our experiments. Our video
handover system to improve the grasping performance of unseen objects. Our videos and code