# A Practical Approach for Picking Items in an Online Shopping Warehouse
###### Abstract  
Commercially viable automated picking in unstructured environments by a robot arm remains a difficult challenge. The problem of robot grasp planning has long been around but the existing solutions tend to be limited when it comes to deploy them in open-ended realistic scenarios. Practical picking systems are called for that can handle the different properties of the objects to be manipulated, as well as the problems arising from occlusions and constrained accessibility. This paper presents a practical solution to the problem of robot picking in an online shopping warehouse by means of a novel approach that integrates a carefully selected method with a new strategy, the centroid normal approach (CNA), on a cost-effective dual-arm robotic system with two grippers specifically designed for this purpose: a two-finger gripper and a vacuum gripper. Objects identified in the scene point cloud are matched to the grasping techniques and grippers to maximize success. Extensive experimentation provides clues as to what are the reasons for success and failure. We chose as benchmark the scenario proposed by the 2017 Amazon Robotics Challenge, since it represents a realistic description of a retail shopping warehouse case; it includes many challenging constraints, such as a wide variety of different product items with a diversity of properties, which are also presented with restricted visibility and accessibility. robotics; grasping; warehouse automation; manipulation +
Footnote †: journal: Medical  
problem of planning the grasp and its execution, under the conditions mentioned above. _Grasp synthesis_ refers to the problem of finding a grasp configuration that satisfies a set of criteria relevant for the grasping task [1]. The generation, evaluation and selection of grasps can be performed in different ways; in the following paragraphs we review the state-of-the-art on the topic of robot grasping but limiting its scope to approaches that are pertinent to picking objects from a container, such as a tote or a bin. Task-based grasping has been separately studied in the context of Bayesian networks for encoding the probabilistic relations among various task-relevant variables [3]. The synthesis of the category and task has been performed based on 2D and 3D data from low-level features [4; 5]. Instead of relying on sensor data points, another proposal is to synthesize grasps based on semantic content with the hope of yielding more stable grasps that are functionally suitable for specific object manipulation tasks [6; 7]. [PERSON] et al. used shape primitives such as spheres, cones and boxes to approximate object shape in the _GraspIt_ simulation environment [8]. In another approach, supervised learning with local patch-based image and depth features was used for grasping novel objects in cluttered environments [9]. Height maps for the representation of features have been proposed for grasp planning with a reported 92% success rate for single object grasping and taking only 2-3 s [10; 11]; the implementation on the [PERSON] robot is remarkable since it has a precision of only 1 cm, and a simple gripper is used. Using features based on geometry, a combination of analytical and data-driven methods was also proposed [12]. Finally, another approach also uses features but instead of a 3D sensor, it relies on the supervised deep learning of 2D RGB images [13]; as is common for deep learning, this method requires both a large dataset and a long training time. A proposal to alleviate this last drawback has been recently put forward for a related problem in warehouse automation [14]. In recent years, a solid trend has emerged to apply machine and deep learning to deal with object recognition, grasp planning and other components of the _pick-and-place_ pipeline. In some cases, convolutional networks are applied exclusively to image processing [15], whereas in other cases, learning techniques aim to obtain models that link visual perception to grasp planning.