Multistep PickandPlace Tasks Using Objectcentric Dense Correspondences This paper presents an objectcentric method for efficiently performing two types of challenging pickandplace tasks, namely sequential pick and place and object sorting. We propose multiclass dense object nets (MCDONs) for learning objectcentric dense descriptors that maintain not only intraclass variations but also interclass separation. Intraclass consistency is also inherently learned and is useful for our pickandplace tasks. All the tasks only require a single demonstration from users, which can then be generalized to all class instances. A dataset containing eight classes and a total of 52 objects was provided in this study. We obtained a task success rate of 93.33% on a fiveblock stacking task and 97.41% on a threeclass object sorting task.

PoseAware Placement of Objects with Semantic Labels  Brandnamebased Affordance Prediction and Cooperative DualArm Active Manipulation The Amazon Picking Challenge and the Amazon Robotics Challenge have shown significant progress in object picking from a cluttered scene, yet object placement remains challenging. It is useful to have poseaware placement based on human and machine readable pieces on an object. For example, the brandname of an object placed on a shelf should be facing the human customers. The robotic vision challenges in the object placement task: a) the semantics and geometry of the object to be placed need to be analysed jointly; b) and the occlusions among objects in a cluttered scene could make it hard for proper understanding and manipulation. To overcome these challenges, we develop a poseaware placement approach by spotting the semantic labels (e.g., brandnames) of objects in a cluttered tote and then carrying out a sequence of actions to place the objects on a shelf or on a conveyor with desired poses. Our major contributions include 1) providing an open benchmark dataset of objects and brandnames with multiview segmentation for training and evaluations; 2) carrying out comprehensive evaluations for our brandnamebased fully convolutional network (FCN) that can predict the affordance and grasp to achieve poseaware placement, whose success rates decrease along with clutters; 3) showing that active manipulation with two cooperative manipulators and grippers can effectively handle the occlusion of brandnames. We analyzed the success rates and discussed the failure cases to provide insights for future applications. All data and benchmarks are available at https:textpicknplace.github.ioTextPNP