The CoSTAR Block Stacking Dataset: Learning with Workspace Constraints A robot can now grasp an object more effectively than ever before, but once it has the object what happens next? We show that a mild relaxation of the task and workspace constraints implicit in existing object grasping datasets can cause neural network based grasping algorithms to fail on even a simple block stacking task when executed under more realistic circumstances. To address this, we introduce the JHU CoSTAR Block Stacking Dataset (BSD), where a robot interacts with 5.1 cm colored blocks to complete an orderfulfillment style block stacking task. It contains dynamic scenes and real timeseries data in a less constrained environment than comparable datasets. There are nearly 12,000 stacking attempts and over 2 million frames of real data. We discuss the ways in which this dataset provides a valuable resource for a broad range of other topics of investigation. We find that handdesigned neural networks that work on prior datasets do not generalize to this task. Thus, to establish a baseline for this dataset, we demonstrate an automated search of neural network based models using a novel multipleinput HyperTree MetaModel, and find a final model which makes reasonable 3D pose predictions for grasping and stacking on our dataset. The CoSTAR BSD, code, and instructions are available at sites.google.comsitecostardataset.

A Cascaded Deep Learning Framework for Realtime and Robust Grasp Planning * Robotic grasping has been regarded as an essential way for robots to interact with the environment, which makes it a research hotspot. To our knowledge, current work on grasp planning could rarely take both robustness and time efficiency into account. To solve this problem, we propose a cascaded deep learning framework, which consists of a regression CNN and a refined CNN. Taking depth image as input, the first network regresses an optimal grasp region. The second network samples and sorts grasp candidates in the region and finally outputs a series of grasp poses with high quality. This architecture ensures realtime performance and acquisition of adequate candidates synchronously, which is especially appropriate for unstructured environment manipulation. We used Gazebo simulation and DART physics engines to build a grasp data generation environment. Comparing with manually annotating or robot collecting, our method makes data generation less expensive, more dynamic, and easier to generalize into the real world. The method proposed is evaluated in both simulation and physical experiments, verifying that our method is able to find multiple robuster grasp poses at a faster rate than the state of the art.