Learning Realtime Closed Loop Robotic Reaching from Monocular Vision by Exploiting A Control Lyapunov Function Structure Visual reaching and grasping is a fundamental problem in robotics research. This paper proposes a novel approach based on deep learning a control Lyapunov function and its derivatives by encouraging a differential constraint in addition to vanilla regression that directly regresses independent joint control inputs. A key advantage of the proposed approach is that an estimate of the value of the control Lyapunov function is available in realtime that can be used to monitor the system performance and provide a level of assurance concerning progress towards the goal. The results we obtain demonstrate that the proposed approach is more robust and more reliable than vanilla regression.

A Cascaded Deep Learning Framework for Realtime and Robust Grasp Planning * Robotic grasping has been regarded as an essential way for robots to interact with the environment, which makes it a research hotspot. To our knowledge, current work on grasp planning could rarely take both robustness and time efficiency into account. To solve this problem, we propose a cascaded deep learning framework, which consists of a regression CNN and a refined CNN. Taking depth image as input, the first network regresses an optimal grasp region. The second network samples and sorts grasp candidates in the region and finally outputs a series of grasp poses with high quality. This architecture ensures realtime performance and acquisition of adequate candidates synchronously, which is especially appropriate for unstructured environment manipulation. We used Gazebo simulation and DART physics engines to build a grasp data generation environment. Comparing with manually annotating or robot collecting, our method makes data generation less expensive, more dynamic, and easier to generalize into the real world. The method proposed is evaluated in both simulation and physical experiments, verifying that our method is able to find multiple robuster grasp poses at a faster rate than the state of the art.