Abstract-This paper explores the idea of manipulation-aided perception and grasping in the context of sorting small objects on a cluttered tabletop. We present a robust pipeline that combines perception and manipulation to accurately sort objects by some property (e.g., color, size, shape, etc.). The pipeline uses two motion primitives to manipulate the scene in ways that help the robot to improve its perception and grasps. This results in the ability to sort cluttered object piles accurately. We also present an implementation on the PR2 robot that applies our algorithm to sort Duplo bricks by color and size, and compare our method to brick sorting without the aid of manipulation. The experimental results demonstrate the benefits of our approach, particularly in environments with a high degree of clutter. Note to Practitioners-Autonomous robots perceive the world using sensors and then typically use the perceived data to manip- This paper has supplementary downloadable multimedia material available at http://ieeexplore.ieee.org provided by the authors. The Supplementary Material consists of a video with four segments: 1) The first segment shows the PR2 accurately sorting Duplo bricks by color or size in uncluttered scenes. The "naive" approach and our method of manipulation-aided sorting behave identically in this scenario and use the pick and drop primitive to simply pick up every object detected and drop it in the appropriate bin. 2) The second segment shows the types of failures encountered when the naive approach is employed in a cluttered scene. We observe that if the clutter is not manipulated prior to pick up, many grasps fail and some bricks are even dropped on the floor. Clearly, attempting to grasp objects directly from clutter is not a good idea. 3) The third segment shows our approach of using manipulation primitives for sorting, particularly the spread primitive in this case. We see a few examples of the spread action that results in isolating one or several bricks for easy pick up. 4) The last segment deals with the tumble primitive for piled up Duplo bricks and we see how this action helps in isolating bricks from the pile. We thus observe that the number of failed grasps for manipulation-aided sorting is much lower than that for the naive approach because the former first manipulates the scene to make it less cluttered and then attempts any grasps, thus increasing the likelihood of success. The clips for the spread and tumble actions in the video are only snapshots of the complete sorting. Note that every time a set of spread and tumble primitives are applied in an iteration, a few bricks get isolated. These bricks are first picked up sorted, and only then a new set of spread and tumble actions are executed in the next iteration. This material is 34.6 MB in size.
Gupta et al. REF present an approach to sort small cluttered objects using a set of motion primitives, but do not show experiments with objects of various size and shapes.
Using Manipulation Primitives for Object Sorting in Cluttered Environments
IEEE Transactions on Automation Science and Engineering
IEEE Transactions on Automation Science and Engineering
Computer Science