Segmentation and classification of objects from RGB-D imagery for robot grasping

Current research in home and service robotics is largely focused on the development of an intelligent machine that can perform difficult or dangerous tasks in the real-world. Although recent advancements in this research area have resulted in many industrial robots (working in structured environments such as automotive industries and manufacturing plants) to achieve superior speed and performance compared to human workers, robots are not yet able to perform household chores robustly in unstructured real-world environments (e.g., in the presence of unexpected events and uncertainty). Therefore, a considerable research effort has now shifted to the development of perceptual methods which are able to perceive information robustly from cluttered human-living environments. In this context, an essential capability is to understand the scene in terms of its structure, and the recognition of objects in the scene. Once this is achieved, many other high-level tasks (such as grasp planning for human-robot interactions and robotic pick and place of objects) become much easier. However, scene understanding in real-world environments is considered to be a highly challenging task due to various factors such as: the segmentation of the visual information into object-hypotheses, the recognition of these hypotheses, disambiguation between visually/structurally similar objects, the need for robustness to deal with real-world noise (e.g., variable lighting conditions, shadows, clutter, occlusion, and noisy measurements from low cost sensors e.g., Microsoft Kinect), and the capability of generalization to unknown or unseen objects. This thesis describes the development of novel computer vision algorithms for objectlevel scene understanding with high accuracy and real-time performance. The algorithms allow an autonomous robot to understand its environment in terms of the objects in the scene and enable the robot to execute high-level tasks such as robotic grasping of objects without the use of any tactile sensing. To achieve this, we developed novel algorithms for 3D object segmentation, 3D object recognition, grasp detection, and object labeling in 3D reconstructed scenes using RGB-D imagery from low cost sensors such as Microsoft Kinect. We demonstrated the performance of our algorithms on several scene representations (e.g., 2.5D images, 3D point clouds, and densely reconstructed scenes) of different complexity for the tasks of object recognition, grasp detection, and robotic grasping in real-world scenarios. For this, we used challenging datasets and live video streams from Kinect mounted on our in-house robotic platform named “AIPAR” to show the effectiveness and applicability of our algorithms in real-world human living environments. Acknowledgements First and foremost, I would like to pay my sincere gratitude to my parents for their love and support throughout my life. I would also like to thank my wife and my daughters for their patience and support during the course of my PhD. I address special gratitude to my supervisor “Mohammed Bennamoun” for his guidance to do high quality research. His technical reviews, inspiring suggestions, and recommendations exceptionally improved the quality of my work. I would also like to thank my other supervisor “Ferdous Sohel” for his extraordinary support and guidance throughout the course of my PhD. I am grateful to my supervisors for providing me the opportunities to attend high quality research conferences which greatly helped me in broadening my vision about robotics. I am also grateful to my supervisors for providing me the necessary robotics equipment to accomplish my research milestones. I would also like to thank staff, and faculty of the School of Computer Science and Software Engineering for creating a supportive and productive environment. Finally, I would thankfully acknowledge the support of the University of Western Australia for sponsoring this research under the Australian Postgraduate Award (APA). This research was sponsored by The Australian Research Council (Grants: DP110102166, DP150100294 and DE120102960).