Machine Learning for Embodied Agents: From Signals to Symbols and Actions

The aim of this tutorial lecture is to show the role of machine learning and some other AI-related techniques in embodied autonomous agents, and autonomous robots in particular. In this tutorial we bring to the forefront the aspects of robotics that are closely related to computer science. We believe that the progress in algorithms and data processing methods together with the rapid increase in the available computing power were the driving forces behind the successes of modern robotics in the last decade. During this period robots of various classes migrated from university laboratories to commercial companies and then to our everyday life, as now everybody can buy an autonomous vacuum cleaner or lawnmower, while self-driving cars and drones for goods delivery are waiting for proper legal regulations to enter the market. Robotics and Artificial Intelligence already went a long path of mutual inspiration and common development, starting from the symbolic AI (aka Good Old-Fashioned Artificial Intelligence) and its extensive use in early autonomous robots, such as Shakey the robot, created in SRI International by Nils Nilsson, considered one of the "fathers" of modern AI. We briefly characterize the range of the most important applications of typical AI methods in modern robotics, including motion planning algorithms [2,3], interpretation of sensory data leading to creation of a world model [4 ,5], and classical learning methods, such as reinforcement learning [6]. However, what made robotics a part of the new wave of AI applications was the recent "revolution" of machine learning, mostly grounded in the enormous success of the deep learning paradigm and its many variants that proved to outclass classic methods in a broad range of problems related to the processing of images and other types of signals. The quick adoption of the recent advances in Machine Learning (ML) in robotics seems to be motivated by the fact that ML gives the possibility to infer solutions from data, as opposed to the classic model-based paradigm that was for decades used in robotics. Whereas the modelbased solutions are mathematically elegant and theoretically provable (with respect to stability, convergence, etc.) they often fail once confronted with real-world problems and real sensory data, as their underlying mathematical models are only a very rough approximation of the real world. Therefore, a wider adoption of ML in robotics gives a chance to make robots more robust and adaptive. On the other hand, we should try to use the new techniques without discarding the knowledge and expertise we already have machine learning methods can benefit a lot from the prior knowledge and the known structure of the problem that has to be solved by learning. This knowledge and structure can be adopted from the model-based methods that a re already well-established in robotics. In the lecture robots are understood in a broad sense, as all embodied agents that have means to physically interact with the environment. They can be either manipulators, mobile robots, aerial vehicles, self-driving cars, and various "smart" devices and sensors. In the second part of the lecture attention is paid to specific problems that appear in application of machine learning to embodied agents, such as the need to search a for solution in huge, multi-dimensional spaces ("curse of dimensionality"), and the ever-present problem of representation and incorporation of uncertainty in the processing of real-world data. Some examples of applications of autonomous robots are given, which were successful due to the use of AI in particular the probabilistic representation of knowledge and machine learning. The most prominent examples are the DARPA competitions: "Grand Challenge", "Urban Challenge" and "Robotics Challenge" (DRC), and the "Amazon Picking Challenge", which proves the interest of large corporations in the development of AI-based robotics [7]. In the third part of the lecture new research directions offered by machine learning and the increased availability of training