Sampling-Based Reachability Algorithms for Control and Verification of Complex Systems

In this paper, we survey sampling-based reachability algorithms that may be used for planning, control, and verification of complex systems. These were inspired by two robotics motion planning algorithms: Rapidly-exploring Random Trees (RRTs) and Probabilistic RoadMaps (PRMs). Herein, we review RRTs and PRMs. We review our adaptations and extensions to nonlinear control problems, hybrid syste ms, and completely discrete problems. We also summarize our work on the properties of sampling-based techniques. I. I NTRODUCTION AND OVERVIEW In this paper, we survey a novel approach to global planning, control, and verification that is applicable to complex control systems, including those that are nonlinea r, nonholonomic, hybrid, or purely discrete. In general, complete algorithms for planning, control, and verification of such systems are exponential in the state-space and control dimensions. Attempts to fight this curse of dimensionality, have led to the introduction of randomized (or Monte Carlo or sampling-based ) approaches that are capable of solving many challenging problems efficiently, at the expense of being able to guarantee that a solution will be found in finite time. Two such sampling-based methods that have achieved considerable success in the robotics motion planning literature are Rapidly-exploring Random Trees (RRTs) and Probabilistic RoadMaps (PRMs). The methods have been shown to solve challenging planning problems that involve high state-space dimension (citations given in Section 2; s ee [16], [17] for a review). Over the past several years we have adapted these algorithms for use in solving nonlinear control problems and multi-agent coordination [4], [10]; hybrid-systems plann ing, control, and verification [6], [19]; as well as planning in completely discrete spaces [22], [21]. Finally, we have also been involved in de-randomizing these algorithms [9], in proving their complexity [16], and in examining their coverage and optimality properties [22], [21]. In this paper, we review this work and place the algorithms in a broader context. In particular, we advocate a sampling-based approach to all reachability-based problems, including planning, control, and verification , as outlined in Figure 1. Thus, instead of generating all (local/on estep) successors of the reach set at any iteration, we only add one sample successor—usually at random, but perhaps using other considerations. The sampled (parent and) successor can be chosen as the one that is closest to some goal region, achieves the maximum spread or “coverage,” etc. The successor can be generated by using a search over inputs or by solving a local (perhaps linear) control problem. Search can be terminated if a goal point is reached (as shown), after a fixed number of iterations, etc. Extra information can be stored using a tree/graph/forest struct u e, with edges labeled by the control action taking a parent node to a successor. Search can also proceed backwards from a final set. Throughout the paper, we will examine algorithms which instantiate this general pattern. SAMPLING-BASED REACHABILITY ALGORITHM() 1 SampledReach← InitialSet 2 repeat 3 srand ← random successor of SampledReach 4 SampledReach← SampledReach∪ srand 5 until SampledReach ∩GoalSet 6= ∅ Fig. 1. The template algorithm for sampling-based reachabi lity. It is perhaps obvious to the reader that control or planning from an initial state to a final state (or to one of a set of goal states satisfying some property) can be accomplished with the sampling-based reachability algorithm. In verificatio n problems, one must establish whether or not trajectories exist that satisfy a set of conditions or specifications . By suitable choice of state, such verification problems can be converted into reachability problems, for example, “all trajectories stay within asafe set. ” Our sampling-based approach is due to the fact that classical, completeapproaches to reachability are provably hard [8]. Our hope with sampling-based methods is to trade algorithmic completeness for weaker guarantees such as probabilistic and resolution completeness [17]. We and our colleagues advocated this path some years ago now [8]; it has been advanced since in [1], [5], [6], [7], [19], [21], [22]. Recently, othe r researchers have also begun to pursue this goal [13]. II. SAMPLING-BASED PLANNING BACKGROUND Rapidly-exploring Random Trees (RRTs) are a probabilistic exploration method developed for searching the high-dimensional continuous spaces encountered in motion p anning problems [15], [18]. To construct an RRT from an initial configurationqstart, the algorithm selects a random point in the configuration space, qrand during each iteration (see Figure 2). Next, we findqnear, the vertex in the tree that is closest toqrand. Finally, we find qnew, the new vertex to add to the tree, by taking a finite step from qnear toward qrand. (Note: may not be be possible due to configuration space obstacles , arising, e.g., from workspace objects or joint limits.) If , the step-length, is greater than the distance fromqnear to qrand, qnew is set equal toqrand. The algorithm finishes whenqnew is at, or close enough to, qgoal. At this point, we can say that we have found a free path from the initial configuration qinit to the goal configurationqgoal. The algorithm can continue to grow the tree until a solution of the desired optimality is found (see Figure 3). RRTs have been shown to be probabilistically complete, meaning that if a path exists from start to goal, the algorithm will find it with probability 1 [15]. They have also been show to have good space-filling properties in that their growth is biased toward the largest unexplored region s in the space [15]; see Section VI. This remains true even in the presence of unforeseen obstacles. Variations on the basic algorithm abound. Most notably, growth can be biased toward the goal by choosing it as qrand some percentage of the time. There are also dual-treealgorithms that a second tree from the goal state. A solution path is found when the two trees finally connect. GROW-RRT(qstart, qgoal) 1 INIT(T ) 2 ADD-ERTEX(T, qstart) 3 repeat 4 qrand ← RAND-CONFIGURATION 5 qnear ← NEAREST-NEIGHBOR(qrand, T ) 6 qnew ← EXTEND-RRT(qnear, qrand) 7 ADD-EDGE(T, qnear, qnew) 8 until IS-AT-GOAL(qnew , qgoal) 9 return T Fig. 2. The basic RRT construction algorithm (top) and diagr am (bottom). (Adapted from [18].) Fig. 3. Growth of typical RRT in 2-d over time, from left to rig ht Probabilistic RoadMaps (PRMs) constitute another sampling-based method for solving planning problems in high-dimensional spaces, especially multiple query path planning [12]. The PRM algorithm operates by repeatedly selecting source random nodes qrand and adding them to a set of vertex nodes, if they are obstacle-free. The algorith m then attempts to connect qrand to any other nodes in the set that are within some distance δ of qrand. The connections are made by a local planner; in the simple case of holonomic planning, the local planner is often simply a straight-line generator with obstacle checking. This process is repeated until the PRM reaches a fixed size, or until most or all of the PRM is connected. Path planning queries are then solved by using a local planner to connect qstart and qgoal to the PRM, then finding a path between the connection points through the pre-planned roadmap. Like RRTs, PRMs are probabilistically complete using uniform sampling. Unlik e RRTs, which expand out from one point, the nodes in a PRM follow the sampling distribution exactly, but do not create a connected graph until some threshold point density is reached. III. SAMPLING-BASED NONLINEAR CONTROL For nonlinear control problems, we assume the dynamics