Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment In this paper, a novel robotic grasping system is established to automatically pick up objects in cluttered scenes. A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably. The suction cup is used for lifting the object from the clutter first and the gripper for grasping the object accordingly. We utilize the affordance map to provide pixelwise lifting point candidates for the suction cup. To obtain a good affordance map, the active exploration mechanism is introduced to the system. An effective metric is designed to calculate the reward for the current affordance map, and a deep QNetwork (DQN) is employed to guide the robotic hand to actively explore the environment until the generated affordance map is suitable for grasping. Experimental results have demonstrated that the proposed robotic grasping system is able to greatly increase the success rate of the robotic grasping in cluttered scenes.

Robot Learning of Shifting Objects for Grasping in Cluttered Environments Robotic grasping in cluttered environments is often infeasible due to obstacles preventing possible grasps. Then, pregrasping manipulation like shifting or pushing an object becomes necessary. We developed an algorithm that can learn, in addition to grasping, to shift objects in such a way that their grasp probability increases. Our research contribution is threefold: First, we present an algorithm for learning the optimal pose of manipulation primitives like clamping or shifting. Second, we learn nonprehensible actions that explicitly increase the grasping probability. Making one skill (shifting) directly dependent on another (grasping) removes the need of sparse rewards, leading to more dataefficient learning. Third, we apply a realworld solution to the industrial task of bin picking, resulting in the ability to empty bins completely. The system is trained in a selfsupervised manner with around 25 000 grasp and 2500 shift actions. Our robot is able to grasp and file objects with 274xc2xb13 picks per hour. Furthermore, we demonstrate the systemxe2x80x99s ability to generalize to novel objects.