A Survey of Machine Learning Approaches to Robotic Path-Planning

Parameters in robotic systems have traditionally been hand-tuned by human experts through painstaking trail-and-error. In addition to requiring a substantial number of man-hours, hand-tuning usually results in robotic systems that are brittle. That is, they can easily fail in new environments. In the last decade or two, designers have realized that their systems can be made more robust by incorporating concepts developed in the field of machine learning. This paper presents a survey of how machine learning has been applied to robotic path-planning and path-planning related concepts. This survey is concerned with ideas at the juncture of the two otherwise unrelated fields of machine learning and path-planning. Both of these are mature research domains, complete with large bodies of literature. To fully understand the work at their intersection, one must have a grasp of the basic concepts relevant to both. Therefore, this paper is organized into three chapters. One each for robotics, machine learning, and the applications of machine learning to path-planning. Short descriptions of each chapter are provided in the remainder of this introduction. Readers are encouraged to use these outlines to gauge which chapters will be most useful to them, and to skip reviews of familiar work. Chapter 1 provides a high-level overview of robotic systems in general. A complete robotic system is outlined to frame how the various subsystems interact with each other. However, the bulk of chapter 1 is dedicated to the representation and planning robotic subsystems, as these are most relevant to the path-planning problem. The last half of this chapter contains an in-depth discussion on path-planning algorithms, with a particular focus on graph-search techniques. Readers with a sufficient grounding in robotics may be able to skip the first half of chapter 1 without compromising their ability to understand the rest of the paper. However, even seasoned roboticists may find the review of graph-search algorithms interesting (starting with Section-1.5), as it covers what I consider to be the major path-planning breakthroughs of the last decade. Chapter 2 provides general background information on machine learning. Short summaries are provided for the subfields of supervised learning and reinforcement learning, and a handful of specific algorithms are also examined in-depth. Note that the content of this chapter has been chosen based on successful applications to path-planning in the last few years, it does not represent an exhaustive survey of all machine learning techniques. The purpose of chapter 2 is to give readers unfamiliar with machine learning enough background information to understand the rest of this paper. Machine learning experts may opt to skip this review of basic techniques. Chapter 3 is a review of machine learning applications to path-planning. Attention is also given to other machine learning robotics applications that are related to path-planning and/or have a direct effect on path-planning. Machine learning is a multi-purpose tool that has been used in conjunction with robotics in a variety of ways. Therefore, the organization of chapter 3 reflects this fact. Chapter sections are mostly self-contained, with each describing and discussing a unique junction between the two fields. Throughout, an attempt is made to show how each idea fits into the larger context of a robotic system. 1 Robotics Background In order for a robot to operate autonomously, it must be capable of interacting with its environment in an intelligent way [Lee, 1996]. This implies that an autonomous robot must be able to capture information about the environment and then perform actions based on I have chosen to include this review because I am personally interested in graph-search algorithms applied to path-planning, and believe that the advances covered in Section-1.5 have lead to a fundamental shift in the way that field-robotics is approached. that information. A hypothetical robotic system can be dissected into four subsystems: sensing → representation→ planning → actuation Although, the lines between these subsystems are often blurred in practice. A sensor is the term given to any part of the robotic system that provides data about the state of the environment. Although the definition of a sensor is necessarily broad, the type of information provided by sensors can be broken into three main categories: 1. The world (terrain shape, temperature, color, composition) 2. The system and its relationship to the world (battery charge, location, acceleration) 3. Other concepts of interest (collaborator, adversary, goal, reward). Any information available to the robot must be provided a priori or obtained on-line through sensor observations. The representation is the method by which a robot stores and organizes information about the world. Simple representations may consist of a single value—perhaps indicating the output of a particular sensor. Complex representations may include high-level graphical models and/or geometric maps of the environment. The planning subsystem (or planner) is responsible for deciding how the robot should behave, with respect to a predefined task, given the information in the representation. A planner might calculate anything from a desired speed/direction of travel to an entire sequence of actions. Actuation is the method by which the robot acts on the environment. This may involve sending power signals to motors, servos, or other devices that can modify the physical relationship between the robot and the environment. All four system components are interrelated. However, in the context of this paper, the relationship between the representation and planning subsystems is especially important. The structure and content of the representation define what kinds of decisions the planner is capable of making, and ultimately the set of action plans available to the robot. Conversely, a particular planning system may require a specific type of representation in order to function. Most of this chapter is devoted to these two subsystems. However, because the particular sensors available to a robot may influence the type of representation used, a short discussions on sensors is also required.