Efficient Parallelization of Relaxation Iterative Methods for Banded Linear Systems ? Pedro Diniz and Tao Yang Department of Computer Science University of California at Santa Barbara Santa Barbara , CA 93106 fpedro,tyangg@cs.ucsb.edu . December 1994 Abstract In this paper we present an efficient parallel implementation of relaxation iterative methods , such as the Gauss-Seidel -LRB- GS -RRB- and Successive-Over-Relaxation -LRB- SOR -RRB- , for solving banded linear systems on distributed memory machines . We introduce a novel partitioning and scheduling scheme in our implementation which allows perfect overlapping of computation with communication , hence minimizing latency effects . We provide analytic and experimental results to verify the effectiveness of our approach and discuss its incorporation in other iterative solvers . Experiments on nCUBE and Intel Paragon machines for several sample banded matrices show that the proposed approach yields good performance on these machines . AMS -LRB- MOS -RRB- subject classifications : 68M20 ; 65Y05 , 65F10 . ? The work of the first author was supported by JNICT - Junta Nacional de Investiga?c ~ ao Cient?ifica e Tecnol?ogica and by the Fulbright program . The second author is supported by NSF-CCR-9409695 and a startup fund from UCSB . 1 Introduction Iterative methods can be used to solve large banded linear systems on concurrent processors . These methods range from the complex Conjugate-Gradient -LRB- CG -RRB- with preconditioning to the simple Successive-Over-Relaxation -LRB- SOR -RRB- . Due to its simplicity SOR methods are still used in solving practical engineering problems -LSB- 2 , 5 -RSB- . In addition , they can also be valuable in obtaining quick approximation of the solution in intermediate steps for more powerful methods -LRB- e.g. preconditioned Conjugate-Gradient -RRB- or even replace direct methods in applications parallelization -LSB- 10 -RSB- . In this paper we concentrate on the implementation of the SOR method on message-passing multiprocessors . The main difficulty in efficiently parallelizing the SOR methods stems from the intra-iteration data dependencies -LRB- cf. Jacobi method -RRB- , as iterates values depend on computed values of the same iteration . A possible approach , as taken in the multi-coloring scheme , is to eliminate some of these dependencies , is to perform variable relabeling -LRB- coloring -RRB- -LSB- 9 -RSB- . This scheme increases the amount of exploitable parallelism since nodes assigned the same color can be computed concurrently . Adams and Jordan -LSB- 1 -RSB- have shown these methods to be numerically equivalent to performing several SOR iterations computed simultaneously in the absence of convergence tests . However , in order to preserve inter-color data dependencies , processors must communicate between each computation phase associated with each color . This constitutes a drawback in the implementation of multicoloring schemes for message-passing machines as communication and computation can not be overlapped in order to hide communication latency . In this paper , we examine the parallelization of SOR iterations , and design a partitioning and scheduling method such that computation can be perfectly overlapped with communication to hide the communication latency . Our work uses the idea of overlapping the computation among several iterations , but also carefully designs its execution ordering and synchronization points for data communication . We provide analytic and experimental results to verify the effectiveness of our approach and compare it with multicoloring schemes . Experimental results show this scheme to deliver good performance and scalability on nCUBE and Paragon distributed-memory multiprocessors . This paper is organized as follows . In Section 2 we formulate the application of the SOR method to solving banded linear systems . Section 3 introduces the difficulties in the implementation of such method on message-passing machines in particular discussing the issues of data mapping and computation/communication scheduling . Section 4 presents the partitioning and scheduling scheme proposed in our algorithm . Section 5 gives an analysis on the performance of this algorithm under synchronous and asynchronous communication models . Section 6 describes the experiments , comparing the performance of our algorithm against a naive implementation of the SOR algorithm as well as a multicoloring approach . In this section we also discuss the applicability of our scheme to other iterative solvers . Conclusions and future work are presented in section 7 . 2 Iterative Relaxation Methods We consider the Successive-Over-Relaxation -LRB- SOR -RRB- iterative method for solving Ax = b when A is n ? n banded with bandwidth w and thus half bandwidth h = -LRB- w ? 1 -RRB- 2 . Figure 1 below illustrates a banded linear system for n = 8 and w = 5 . BBBBBBBBBBBBB @ a11 a12 a13 a21 a22 a23 a24 a31 a32 a33 a34 a35 a42 a43 a44 a45 a46 a53 a54 a55 a56 a57 a64 a65 a66 a67 a68 a75 a76 a77 a78 a86 a87 a88 1 CCCCCCCCCCCCCA BBBBBBBBBBBBB @ x1 x2 x3 x4 x5 x6 x7 x8 1 CCCCCCCCCCCCCA = BBBBBBBBBBBBB @ b1 b2 b3 b4 b5 b6 b7 b8 1 CCCCCCCCCCCCCA . Figure 1 : Example of a banded linear system -LRB- n = 8 ; w = 5 -RRB- . For such systems the SOR iteration below defines the value of a given unknown 's approximation at iteration k +1 -LRB- denoted by xk +1 i -RRB- where we have assumed aii < > and aij = for j < 1 or j > n. xk +1 i = -LRB- 1 ? ! -RRB- xki + ! aii -LRB- bi ? i ? 1 X j = i?h aijxk +1 j ? i + h X j = i +1 aijxkj -RRB- : -LRB- 1 -RRB- Convergence conditions for the SOR as well as for the special case of Gauss-Seidel methods are extensively analyzed by Young -LSB- 12 -RSB- and thus omitted here . From the above formulation several remarks are in order : 1 . The value of given unknown 's iteration approximation depends only on the values of the unknowns at index distance of half bandwidth , one set of such unknowns from the current iteration and another set from the previous iterations . Formally , xk +1 i depends on xki , xk +1 j and xkl for j = i ? h ; ? ? ? ; i ? 1 and l = i + 1 ; ? ? ? ; i + h. 2 . Instead of a single matrix element each non-null aij can be viewed as a matrix subblock -LRB- with some zero elements for blocks near the band boundary -RRB- . Under this scheme , scalar inversion would be replaced by a matrix solver and each xi would represent a set of unknowns by a diagonal matrix . This blocking scheme has been used in previous research -LRB- e.g. -LSB- 10 -RSB- -RRB- as a mean to improve the locality , both temporal and spatial , of memory accesses , in order to take advantage of memory hierarchy organization . Each SOR iteration can be viewed as a set of tasks to be executed with precedence constraints arising from the underlying data dependences . Let the task denoted by Ti ; j -LRB- i < > j -RRB- compute the multiplication of ai ; j and xj . Task T ki ; i performs the computation to derive the new iteration value for xki based on the old value , i.e. xk ? 1 i , bi and the two partial sums from T ki ; 1 ; ? ? ? T ki ; i ? 1 , and from T k ? 1 i ; i +1 ; ? ? ? T k ? 1 i ; i + h. Figure 2 below illustrates the task dependences , in the form of a task dependence graph -LRB- TDG -RRB- for a generic banded matrix system with half bandwidth h and for a generic variable xi . The edges labeled 1 represent data dependences across iterations - dependence distance 1 . Ti , i +1 Ti,i-h Ti,i-1 ... ... Ti , i + h Ti-1 , i Ti +1 , i Ti-h , i Ti + h , i Ti , i ... ... 1 1 1 1 Figure 2 : Task Dependence Graph -LRB- TDG -RRB- for a generic system of bandwidth w = 2h + 1 . A sequential implementation of this method simply loops through the matrix rows computing the new iterate values . The data data dependencies of the algorithm are automatically satisfied as T ki ; j is always executed before T ki ; j +1 and T ki ; j is performed before T k +1 i ; j , i.e. cross-iteration dependencies are respected . Denoting the cost of a floating point operation by c , the total sequential cost per iteration is given by equation 2 . Tseq = 2n -LRB- h + 2 -RRB- c : -LRB- 2 -RRB- 3 Message-Passing Multiprocessor Implementation We now describe a naive implementation of the SOR iterations and a multi-coloring scheme on multiprocessors . For both we provide a simple analysis based on the communication cost model presented below . We then introduce the idea of our approach . 3.1 Communication Model We will consider two communication models , respectively an asynchronous and a synchronous one . Under the first model sending a message causes no delay to the sending processor whereas in the synchronous case the sender blocks until the message has been delivered at its destination . In any of the two models the transmission time is modeled as ff + m where ff is the message startup time , is the transmission and buffering cost per numeric element and m the message length in number of numeric elements . Receiving messages blocks the receiving processor until the message is delivered . Computation cost per element , either a floating point add or multiply is modeled by c. Under any of these two models we do not consider communication cost effects due to long messages , e.g. contention , and non-neighboring communication . This approximation simplifies the analysis and is supported by current multiprocessor implementations as contention-free networks and wormhole routing have become common . 3.2 Naive implementation We assign each processor r = n =p consecutive rows of the system matrix as well the corresponding components of both the solution and the independent solution vectors as discussed above . In the absence of convergence tests , processors may compute simultaneously values of different iterations as depicted in Figure 3 by consecutive parallelogram shaped computations . While processor pi is in iteration k , processor pi +1 is still in iteration k ? 1 . Computations are labeled the iteration they refer to . Processors 1 2 3 4 Time k k k k k-1 k-1 k-1 k +1 k +1 k +1 k +1 k-1 Figure 3 : Overlapping of several SOR iterations . In a naive implementation of the SOR iterations each processor computes the new iteration value of the corresponding component . As soon as the first h rows are processed the corresponding values are sent to the \ upper '' processor in a single message . In the last h rows , however , and after having sent the new values each processor waits for the \ lower '' processor to reply with a message containing updated information about its values . Figure 4 illustrates communication dependence in this naive algorithm for each of the parallelogram sections . Task T1 corresponds to the computation associated with the first h of the r rows assigned to each processor . T2 and T3 correspond to the next r ? 2h and last h rows respectively . T 1 2T h rows h rows -LRB- r-2h -RRB- rows M M MT 3 M Figure 4 : Processor computation partitioning and scheduling for the naive algorithm corresponding to each of the parallelogram section of Figure 3 . For both communication models , and because each processor has to wait for the next one to reply with the updated iterate values , the parallel time per iteration of each computing section is given by equation 3 below . T naive par = 2r -LRB- h + 1 -RRB- c + 4 -LRB- ff + h -RRB- ss wrc + 4ff + 2 w : -LRB- 3 -RRB- Despite the simplicity of the algorithm the synchronization for the last h rows introduces an unnecessary waiting time as the \ lower '' processor computes its T1 section . We expect this to be a limiting factor for large values of the half bandwidth h as the computational cost of T1 is O -LRB- h2 -RRB- . 3.3 Multicoloring implementation The multicoloring technique relies on the ability to label nodes on the physical domain as to eliminate inter-iteration dependencies of the iterative methods that is used to compute them . The labeling of the variables is the key point factor , yielding matrices with diagonal blocks -LSB- 9 -RSB- . The unknowns associated with these diagonal blocks are assigned the same color and can be computed in parallel . After global exchange of the values of variables of a given color , another swap for variable with another color can continue , implicitly overlapping multiple SOR iterations . Adams and Jordan explored this idea for several regular stencils over square domains . Figure 5 below shows a coloring corresponding to the banded matrix of Figure 1 . x1 x3 x5 x7 x8 x6 x4 x2 Figure 5 : Multicoloring of a physical domain corresponding the banded matrix of Figure 1 . Assume that the colored variables are evenly distributed among p processors . Since after computation for the same color , new values for variables with this color are exchanged between processors . The number of such variable per processor is about n -LRB- h +1 -RRB- p . There are -LRB- h + 1 -RRB- communication steps yielding a parallel time per iteration given by equation 4 . T multicoloring par = 2nc -LRB- 2h + 1 -RRB- p + -LRB- h + 1 -RRB- -LRB- ff + n -LRB- h + 1 -RRB- p -RRB- : -LRB- 4 -RRB- Adams and Jordan -LSB- 1 -RSB- have shown multicoloring methods to be equivalent to overlapping multiple SOR iterations . These methods are effective if the communication overhead is negligible . In the presence of non-zero communication costs , as is the case of current message-passing machines -LSB- 3 -RSB- , multicoloring methods exhibit significant communication costs making it less effective . Experimental results presented in section 6 for tested matrix systems do show communication costs to be significant . 3.4 A New Pipelined Approach From the discussion in the previous two sections one can draw the following remarks . First , multicoloring methods implicitly overlap multiple iterations , but fail to exploit more parallelism due to the inter-color dependencies . Second , the naive implementation of the SOR algorithm incurs in unnecessary waiting time at the end of its computation phase . This suggests an improved algorithm by modifying the way computation and communication steps are performed and scheduled . This new approach , the overlapped algorithm , as described in the next section , exploits both the idea of overlapping successive SOR iterations and the overlapping of computation with communication . This is achieved by scheduling the communication and computation differently but still respecting the algorithm data dependencies . 4 The Overlapped Algorithm The data mapping in this approach is the same as in the naive implementation of SOR iterations . A row-wise computation along the full bandwidth of the matrix within each processor , as suggested by a naive interpretation of the SOR iteration definition above , would prolong the overall length of each processor computation due to the data dependencies at the boundary rows . Instead we reorder the way the computation for each iterate is done as to allow overlap of computation with communication . As a result the proposed algorithm is numerically equivalent to the sequential implementation of the SOR algorithm . In our algorithm computation of each iteration at each processor pi is divided into five tasks -LRB- T1 ; T2 ; T3 ; T4 ; T5 -RRB- along the band of the matrix , based on the inter-processors data dependencies , as depicted in Figure 6 . Edges labeled M represent messages sent between processors . Due to the banded structure , and imposing r > = 2h , communication is restricted to the processors assigned to the adjacent sets of matrix rows , named respectively \ lower '' and \ upper '' processors . Furthermore careful data mapping allows communication to be kept in a linear pattern , hence between physically neighboring processors . Notice that if r < 2h the basic scheduling scheme is still applicable but now at the expense of more communication messages , not only between neighboring processors but between logical processors at distance 2 ; ? ? ? ; d2hr e. T1 T2 T3 T4 T5 h rows h rows -LRB- r-2h -RRB- rows MM M M Figure 6 : Processor computation partitioning and scheduling for the overlapped algorithm corresponding to each parallelogram computing section . Each processor repeatedly executes partitioned tasks in the order of T1 ; T2 ; T3 ; T4 and T5 as depicted in Figure 7 . Communication occurs when T1 uses the data from T3 at processor pi ? 1 and T1 produces data used in T5 for processor pi ? 1 . T2 , T3 and T4 only need local data . Computation and communication can be overlapped as T4 in processor pi may be computed while processor pi +1 executes T1 and sends a message back . Figure 8 below outlines an implementation of the overlapped algorithm . The improvement resulting from this scheduling might be significant when the band is large as both the computational cost T1 T2 T3 T4 T5 ... . T1 T2 T3 T4 T5 ... ... T1 T2 T3 T4 T5 ... ... PiPi +1 Pi-1 time Procs Figure 7 : Processor computation scheduling for the overlapped algorithm in terms of -LRB- T1 ? ? ? T5 -RRB- : of T1 and the message size grow with w , respectively quadratically and linearly . 1 : for the first h rows , say j do Execute Tj ; j?h ; Tj ; j?h +1 ; ? ? ? ; Tj ; j 2 : Send the newly computed values of xj to pi ? 1 3 : for all rows j but the first or the last h do Execute Tj ; j?h ; Tj ; j?h +1 ; ? ? ? ; Tj ; j 4 : For the last h rows j do Execute Tj ; j?h ; Tj ; j?h +1 ; ? ? ? ; Tj ; j 5 : Send the newly computed values of xj to pi +1 6 : for every row j do Execute tasks Tj ; j +1 ; ? ? ? ; Tj ; l with l < i nw 7 : Receive values from pi ? 1 and pi +1 8 : for all rows j do Execute Tj ; i nw ; ? ? ? ; Tj ; j + i nw Figure 8 : SOR TDG scheduling for pi . 5 Analysis We provide analysis to show our partitioning and scheduling method to deliver good performance . We derive the Speedup and Efficiency of the proposed method for both an asynchronous and synchronous communication models . We further assume the number of iterations to be very large . First we give the computational cost of the five tasks defined in the overlapped algorithm in Table 1 , which will be used in the rest of this section . Also we consider r = 2h + k with k > as mentioned above . T1 = 2h -LRB- h + 1 -RRB- c T2 = 2 -LRB- r ? 2h -RRB- -LRB- h + 1 -RRB- c T3 = 2h -LRB- h + 1 -RRB- c T4 = 2h -LRB- r ? h + h ? 12 -RRB- c T5 = h -LRB- h + 1 -RRB- c Table 1 : Computational costs of T1 ? ? ? T5 for the overlapped algorithm . 5.1 Asynchronous communication Under this model , as we can observe in Figure 7 , we have inter-processor data dependences between tasks -LRB- T3 ; T1 -RRB- and -LRB- T1 ; T5 -RRB- . By inspection of Figure 7 the condition to have a total overlap of computation with communication is : T4 > = 2 -LRB- ff + h -RRB- + T1 ? 2h -LRB- r ? h + -LRB- h + 1 -RRB- 2 -RRB- c > = 2 -LRB- ff + h -RRB- + 2h -LRB- h + 1 -RRB- c : We can solve this inequality for h and we have h > = 1 2 -LRB- -LRB- 3 + c ? 2k -RRB- -RRB- + s -LRB- -LRB- 3 + c -RRB- ? 2k -RRB- 2 + 4 -LRB- ff c -RRB- -RRB- = hcrit : -LRB- 5 -RRB- Thus for h > = hcrit there is total overlap of computation and communication . In this case the Speedup is simply p , with 100 % Efficiency . Notive again that we have made the assumption of a very large number of iterations . For h < hcrit , however , each processor may have to wait for the next processor to finish his computation of T1 and send the message containing the corresponding updated solution values back . In this case we have Tpar = T1 + T2 + T3 + max -LRB- 2 -LRB- ff + h -RRB- + T3 ; T4 -RRB- + T5 , i.e. Tpar = T1 + T2 + 2T3 + 2 -LRB- ff + h -RRB- + T5 as T4 < 2 -LRB- ff + h -RRB- + T3 . Thus Tpar = 2r -LRB- h + 1 -RRB- c + 2h -LRB- h + 1 -RRB- c + 2 -LRB- ff + h -RRB- = 2 -LRB- h + 1 -RRB- -LRB- r + h -RRB- c + 2ff + 2 h : -LRB- 6 -RRB- Hence Speedup = -LRB- h + 2 -RRB- n -LRB- h + 1 -RRB- -LRB- r + h -RRB- + ffc + c h : -LRB- 7 -RRB- By approximating -LRB- h + 2 -RRB- ss -LRB- h + 1 -RRB- ss h we have Speedup = p 1 + pn -LRB- h + ffhc + c -RRB- : -LRB- 8 -RRB- Thus an overhead and isoefficiency functions -LSB- 6 -RSB- Toverhead -LRB- W ; p -RRB- = p n -LRB- h + ff hc + c -RRB- -LRB- 9 -RRB- where W = n ? w stands for the total amount of work of each SOR iteration . This reveals an isoefficiency of ? -LRB- p -RRB- , for fixed band w , hence showing the algorithm to be optimally scalable , i.e. to keep the same efficiency when we increase the amount of work per processor , we only need a linear increase in the number of processors . If the band is increased , however , the number of processors must be reduced . Theorem 1 Assume the number of iterations to be large and an asynchronous communication model . For h > = hcrit the Speedup and the Efficiency of the overlapped algorithm on p processors are close to p and 100 % respectively . For h < hcrit the Speedup is approximately p 1 + pn -LRB- h + ffhc + c -RRB- and the Efficiency 1 1 + pn -LRB- h + ffhc + c -RRB- . The isoefficiency is ? -LRB- p -RRB- 2 . Several remarks are in order . ffl For large values of r , i.e. either very large matrices or few processors , hcrit may be approxi - mated by hcrit = 12 -LRB- 2k ? c -RRB- + q -LRB- c ? 2k -RRB- 2 + 4 ffc . ffl For values of h < r < 2h a similar analysis is possible as each processor only needs to communicate with adjacent processors . Similar threshold values results from a analogous analysis . For values of r < h , however , a more complex analysis is required as the local data has to be send more than one \ adjacent '' processor . 5.2 Synchronous communication Under this model no overlapping of communication with computation is possible . We derive the Speedup and Efficiency of the overlapped algorithm by a critical path analysis . By inspection of Figure 7 the parallel time for each iteration is given by Tpar = T1 + T2 + T3 + max -LRB- T4 ; T1 -RRB- + T5 + 2 -LRB- ff + h -RRB- ; which under the assumption that r > = 2h yields Tpar = 2r -LRB- h + 1 -RRB- c + 2 -LRB- ff + h -RRB- ss wrc + 2ff + w : Hence the Speedup and Efficiency are Speedup = p 1 + p -LRB- 2ff + w -RRB- nwc and Efficiency = 1 1 + p -LRB- 2ff + w -RRB- nwc : -LRB- 10 -RRB- Theorem 2 Assume the number of iterations to be large and an synchronous communication model . Then the overlapped algorithm on p processors achieves a Speedup and Efficiency given respectively by p 1 + p -LRB- 2ff + w -RRB- nwc and 1 1 + p -LRB- 2ff + w -RRB- nwc . The isoefficiency of this algorithm is ? -LRB- p -RRB- 2 . 6 Experiments and Results We compare the performance in terms of peak Mflops for a naive implementation of SOR iterations against our overlapped method for several matrices on an nCUBE-2 and Intel Paragon multiprocessors . Both codes were written in C and compiler optimized , -LRB- ncc version 3.2 at nCube / icc version R4 .5 at Intel -RRB- without any assembly routines . All calculations are performed in double precision arithmetic . The Table 2 on the left below presents the attained Mflops performance for 64 processors on both machines and varying matrix bandwidth . We verify the scalability of our approach in two settings , namely for a fixed problem size and for constant amount of work per processor , i.e. r ? band = K . This is known as Scaled-Speedup -LSB- 11 -RSB- . Due to memory constraints only data for the Paragon is shown -LRB- Table 2 right -RRB- . 64 procs n :12,800 band : 161 nCUBE-2 Paragon Paragon band naive overlap naive overlap procs n :12,800 r : 200 21 39.5 37.3 346 309 4 41 38 41 37.8 41.4 387 430 8 81 76 81 32.0 43.9 361 537 16 162 152 161 23.8 45.3 280 608 32 314 304 Table 2 : Overlapped algorithm performance results -LSB- Mflops -RSB- . 64 procs n :12,800 band : 161 band nCUBE-2 Paragon procs n :12,800 r :200 21 24.5 149 4 22.4 11.1 41 26.0 165 8 41.3 22.1 81 26.1 167 16 73.9 44.3 161 26.5 170 32 121.4 88.9 Table 3 : Multicoloring algorithm performance results -LSB- Mflops -RSB- . 6.1 Discussion The performance of the overlapped algorithm is impressive for high values of bandwidth . An excellent scalability is also revealed making it attractive for very large systems . The results show this overlapped algorithm to clearly outperform the naive implementation in all but the sample cases where the matrix bandwidth is small . We attribute this to the added code complexity as well as to a worse memory hierarchy tailoring . Multicoloring methods reveal poor performance . It is clear from the results that communication and computation can not be overlapped in this method . This is particularly serious in distributedmemory multiprocessors . The \ pure '' multicoloring method can also benefit from the idea of overlapping several iterations among the same colored variables yielding a significant improvement -LRB- typically 2 and 5 times better on the nCUBE-2 and the Paragon respectively -RRB- . Even with this enhancement multicoloring methods are by far outperformed by the overlapped algorithm . Although the incorporation of the overlapped scheme proposed in our algorithm in the multicoloring methods is possible we believe this would not lead to a significant performance improvement as the ratio of the number of messages between the two approaches is still of O -LRB- n -LRB- h +1 -RRB- p -RRB- . 6.2 Other Iterative Methods Incorporation the overlapping scheme proposed in our algorithm in other methods is not advantageous in situations where there is not an inter-iteration dependence between processors . As an example , iterative methods like the CG can not benefit from this approach . However it is possible to take advantage of overlapping several CG iterations as described by Leland -LSB- 7 , 8 -RSB- . The main obstacle to parallelization in this method is due to the synchronization caused by the dot products . By substituting the recurrences for r -LRB- the residual -RRB- and p -LRB- the search direction -RRB- into the expressions for the dot products one can obtain expressions for the dot products in terms of the earlier r and p values , hence avoiding some communication and therefore minimizing the synchronization delay . 7 Conclusions and future work The scheduling algorithm for SOR iterations on banded linear systems presented here reveals good performance over a naive scheduling for large values of the matrix bandwidth as well as over multicoloring methods . A perfect scalability was observed for fixed bandwidth and increasing dimension of the linear system . We believe that this result makes our approach attractive in obtaining fast solution approximations as an intermediate step in other more powerful methods . Future work will concentrate in assessing the performance of the technique presented here on shared-memory multiprocessors as well as extending it to other linear solvers . Acknowledgements We thank Robert Leland at Sandia national Laboratories and Gerasoulis Apostolos at Rutgers University for several insightful discussions . This research was performed in part using parallel computing resources located at Sandia National Laboratories in Albuquerque , New Mexico . References -LSB- 1 -RSB- L. Adams and H. Jordan , Is SOR color-blind ? , SIAM J. Sci . Stat . Comp. , 7 -LRB- 1986 -RRB- , pp. 490 -LCB- 506 . -LSB- 2 -RSB- R. Barrett et . al , Templates for the Solution of Linear Systems : Building Blocks for Iterative Methods , SIAM Publications . Philadelphia , PA , 1994 -LSB- 3 -RSB- T. H. Dunigan , Performance of the INTEL iPSC/860 and nCUBE 6400 Hypercube , ORNL/TM -11790 , Oak Ridge National Lab. , TN , 1991 . -LSB- 4 -RSB- G. Fox , M. Johnson , G. Lyzenga , S. Otto , J. Salmon and D. Walker , Solving Problems on Concurrent Processors , Prentice-Hall , New Jersey , 1988 . -LSB- 5 -RSB- G. Huang and W. Ongsakol , An Efficient Task Allocation Algorithm and its use to Parallelize Irregular Gauss-Seidel Type Algorithms , In Proc . of the Eighth International Parallel Processing Symposium , Cancun , Mexico , -LRB- 1994 -RRB- , pp. 497 -LCB- 501 . -LSB- 6 -RSB- V. Kumar , A. Grama , A. Gupta , G. Karypis , Introduction to Parallel Computing : Design and Analysis of Algorithms , Benjamin-Cummings , 1994 . -LSB- 7 -RSB- R. Leland and J. Rollett , Evaluation of a Parallel Conjugate Gradient Algorithm , In Numerical Methods in Fluid Dynamics III , Ed . K. Morton and M. Baines , Oxford Univ. . Press , 1988 . -LSB- 8 -RSB- Robert W. Leland , The Effectiveness of Parallel Iterative Algorithms for Solution of Large Sparse Linear Systems , PhD . Thesis , Oxford University 1989 . -LSB- 9 -RSB- James M. Ortega , Introduction to Parallel and Vector Solution of Linear Systems , Plenum Press , 1988 . -LSB- 10 -RSB- Jaswinder Singh and John Hennessy , Finding and Exploiting Parallelism in an Ocean Simulation Program : Experience , Results and Implications , In Journal of Parallel and Distributed Computing , vol . 15 , no. 1 , -LRB- 1992 -RRB- , pp. 27 -LCB- 58 . -LSB- 11 -RSB- Xian-He Sun and Diane T. Rover , Scalability of Parallel Algorithm-Machine Combinations , In IEEE Transactions on Parallel and Distributed Systems , vol . 5 , no. 6 , -LRB- 1994 -RRB- , pp. 599 -LCB- 613 . -LSB- 12 -RSB- David Young , Iterative Solution of Large Linear Systems , Academic Press , 1971 .

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B I I O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O B I I O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

banded linear systems
partitioning
scheduling
sor


Executing

Reactive

Behavior

for

Autonomous

Navigation

Benny

Rochwerger

Claude

L.

Fennema

Bruce

Draper

Allen

R.

Hanson

and

Edward

M.

Riseman

CMPSCI

February

This

work

was

supported

by

the

Advanced

Research

Projects

Agency

-LRB-

via

TACOM

-RRB-

under

contract

number

and

by

the

National

Science

Foundation

under

grant

number

Executing

Reactive

Behavior

for

Autonomous

Navigation

Benny

Rochwerger

Claude

L.

Fennemay

Bruce

Draper

Allen

R.

Hanson

Edward

M.

Riseman

Computer

Vision

Laboratory

Dept.

of

Computer

Science

University

of

Massachusetts

Amherst

MA

Abstract

The

aim

of

the

Unmanned

Ground

Vehicle

-LRB-

UGV

-RRB-

project

at

the

University

of

Massachusetts

is

to

develop

a

system

capable

of

navigating

both

on-road

and

cross

country

avoiding

obstacles

and

determining

its

position

using

landmarks

Complex

problems

such

as

driving

can

be

solved

more

easily

by

decomposing

them

into

smaller

sub-problems

solving

each

sub-problem

and

then

integrating

the

solutions

In

the

case

of

an

autonomous

vehicle

the

integrated

system

should

be

able

to

react

in

real

time

to

a

changing

environment

and

to

reason

about

ways

to

achieve

its

goals

This

paper

describes

the

approach

taken

on

the

UMass

Mobile

Perception

Laboratory

-LRB-

MPL

-RRB-

to

integrate

independent

processes

-LRB-

each

solving

a

particular

aspect

of

the

navigation

problem

-RRB-

into

a

fully

capable

autonomous

vehicle

This

research

was

supported

by

DARPA

via

TACOM

under

contract

number

and

by

the

National

Science

Foundation

under

grant

number

CDA

yComputer

Science

Program

Mount

Holyoke

College

South

Hadley

MA

Introduction

One

of

the

goals

of

the

autonomous

vehicle

effort

at

the

University

of

Massachusetts

is

the

development

of

a

landmark-based

navigation

system

capable

of

robust

navigation

both

on-road

and

cross-country

While

driving

the

vehicle

must

exhibit

classic

driving

behaviors

such

as

staying

on

the

road

or

prescribed

path

avoiding

obstacles

etc.

while

using

landmarks

for

determining

vehicle

pose

that

is

the

position

and

orientation

of

the

vehicle

with

respect

to

a

world

model

The

emphasis

of

this

effort

is

model-based

navigation

The

experimental

laboratory

vehicle

-LRB-

Figure

-RRB-

for

this

effort

is

the

Mobile

Perception

Lab

-LRB-

MPL

-RRB-

a

heavily

modified

Army

HMMWV

ambulance

that

is

equipped

with

actuators

and

encoders

for

the

throttle

steering

and

brake

The

interface

to

the

on-board

computer

is

through

a

controller

On

board

diesel

generators

provide

power

The

vehicle

's

vision

sensor

package

includes

a

Staget

which

is

a

rotating

stabilized

platform

mounted

on

the

roof

of

the

cab

with

a

CCD

color

camera

and

a

FLIR

sensor

Two

forward-looking

stereo

cameras

and

a

forward

looking

color

CCD

camera

are

mounted

in

a

rectangular

enclosure

at

the

front

edge

of

the

cab

's

roof

The

primary

computing

engine

for

vision

processing

goal-oriented

reasoning

and

path

planning

is

a

Silicon

Graphics

four-node

multiprocessor

The

multiprocessor

is

interfaced

to

the

sensor

suite

through

a

Datacube

image

processor

which

provides

frame

rate

image

processing

for

certain

type

of

operators

Space

and

power

is

also

provided

for

the

Image

Understanding

Architecture

-LRB-

IUA

-RRB-

-LSB-

-RSB-

a

massively

parallel

heterogeneous

processor

The

MPL

is

an

experimental

laboratory

for

testing

and

integrating

different

approaches

to

problems

in

autonomous

navigation

including

but

not

limited

to

landmark-based

navigation

obstacle

detection

and

avoidance

model

acquisition

and

extension

road

following

and

path

planning

It

is

therefore

important

that

MPL

have

a

software

environment

where

multiple

visual

modules

addressing

different

subtasks

can

be

easily

integrated

to

provide

autonomous

driving

functions

It

is

also

important

to

provide

an

environment

in

which

researchers

can

quickly

experiment

with

different

combinations

and

parameterizations

of

those

modules

At

the

same

time

MPL

's

software

environment

must

be

efficient

enough

to

meet

the

demands

of

real-time

navigation

research

In

addition

control

of

the

system

must

be

both

planned

as

well

as

driven

by

external

events

To

illustrate

the

problem

consider

the

task

of

driving

to

your

friend

's

new

house

You

were

given

instructions

such

as

at

the

second

light

on

Main

street

turn

right

then

stop

at

the

third

house

on

your

left

Obviously

this

is

not

enough

information

to

get

you

there

and

you

will

most

likely

consult

Figure

The

Mobile

Perception

Laboratory

built

on

a

modified

HMMWV

chassis

has

computer

controlled

steering

throttle

and

brakes

a

complete

research

laboratory

on

the

back

a

map

to

find

out

how

to

get

from

where

you

are

to

Main

street

Once

you

have

a

general

idea

of

where

are

you

heading

you

will

get

into

your

car

and

start

driving

towards

your

friend

's

house

On

the

way

you

may

have

to

take

actions

that

were

not

in

your

original

plan

like

stopping

suddenly

to

let

a

child

cross

the

street

In

some

cases

you

may

have

to

completely

rethink

your

plan

when

one

of

the

streets

you

wanted

to

take

is

closed

for

repairs

For

an

autonomous

vehicle

to

mimic

human

driving

it

is

not

sufficient

for

it

to

have

the

ability

to

drive

down

the

road

or

recognize

traffic

lights

It

must

also

know

how

to

combine

all

of

its

capabilities

-LRB-

count

traffic

lights

while

driving

-RRB-

as

well

as

how

to

react

to

unexpected

events

-LRB-

stop

there

is

a

child

crossing

the

street

-RRB-

how

to

keep

going

when

completely

new

situations

are

encountered

-LRB-

a

road

is

closed

-RRB-

and

how

to

complete

the

missing

information

-LRB-

how

to

get

to

Main

street

-RRB-

The

task

of

autonomous

navigation

as

with

many

other

complex

tasks

can

be

decomposed

into

more

specific

subproblems

like

road

following

obstacle

avoidance

and

landmark

recognition

In

order

to

achieve

a

fully

autonomous

vehicle

the

solutions

to

all

these

subproblems

must

be

integrated

into

a

coherent

system

This

paper

focuses

on

the

preliminary

work

done

on

the

problem

of

integration

for

the

UMass

Mobile

Perception

Laboratory

-LRB-

MPL

-RRB-

Since

each

of

the

subproblems

is

still

a

field

of

active

research

in

which

approaches

and

solutions

may

change

rapidly

over

time

the

integrated

system

should

be

flexible

enough

to

allow

testing

different

sub-systems

To

achieve

the

desired

flexibility

we

have

implemented

the

control

system

as

a

programable

finite

state

machine

-LRB-

FSM

-RRB-

where

the

states

are

the

different

modes

of

operation

of

the

system

-LRB-

behaviors

-RRB-

and

the

transitions

are

the

system

's

reactions

to

events

-LRB-

either

external

or

internal

-RRB-

The

composition

of

the

states

and

the

transitions

is

not

fixed

i.e.

the

FSM

can

be

tailored

to

the

particular

task

the

system

is

trying

to

achieve

The

paper

is

structured

as

follow

the

first

section

presents

a

brief

review

of

behavior

based

systems

and

the

terminology

used

throughout

this

paper

is

introduced

Then

a

formal

definition

of

a

script

-LRB-

section

-RRB-

and

the

script

monitor

-LRB-

section

-RRB-

are

presented

Experiments

on

the

MPL

are

described

in

section

Section

discusses

our

thoughts

on

how

to

incorporate

a

planner

into

the

FSM

model

and

section

presents

some

of

the

implementation

open

issues

Finally

section

summarizes

the

work

What

is

a

behavior

behavior

the

manner

of

conducting

oneself

anything

that

an

organism

does

involving

action

and

response

to

stimulation

the

response

of

an

individual

group

or

species

to

its

environment

the

way

in

which

something

-LRB-

as

a

machine

-RRB-

behaves

Webster

's

dictionary

In

the

robotics

literature

behavior

has

generally

been

used

to

describe

processes

that

connect

perception

to

action

i.e.

a

behavior

senses

the

environment

and

does

something

based

on

what

was

perceived

A

combination

of

behaviors

is

also

called

a

behavior

thus

a

complex

behavior

can

be

achieved

by

combining

simpler

behaviors

In

Brooks

subsumption

architecture

-LSB-

-RSB-

the

task

of

robot

control

is

decomposed

into

levels

of

competence

each

level

in

combination

with

lower

levels

defines

a

behavior

Payton

Rosenblatt

and

Keirsey

-LSB-

-RSB-

in

their

Distributed

Architecture

for

Mobile

Navigation

system

-LRB-

DAMN

-RRB-

refer

to

behaviors

as

very

low

level

decision-making

processes

which

are

guided

by

high

level

plans

and

combined

through

arbitration

In

their

DEDS

work

Ramadge

and

Wonham

-LSB-

-RSB-

as

well

as

Rivlin

-LSB-

-RSB-

events

are

considered

the

alphabet

of

a

formal

language

a

behavior

is

a

sequence

of

events

or

a

string

over

Note

that

in

this

terminology

every

prefix

of

a

string

is

also

a

behavior

i.e.

the

sequential

combination

of

behaviors

is

a

behavior

These

definitions

although

consistent

can

be

confusing

the

same

term

is

used

for

individual

processes

and

for

the

composition

of

these

processes

To

make

the

distinction

clear

we

have

chosen

to

think

of

a

behavior

as

a

mode

of

operation

-LSB-

-RSB-

in

which

several

perception-action

processes

-LSB-

-RSB-

are

executed

concurrently

Each

process

converts

sensory

data

into

some

kind

of

action

-LRB-

either

physical

or

cognitive

-RRB-

and

at

any

time

may

generate

an

event

a

signal

to

let

the

system

know

that

something

significant

has

occurred

All

inter-process

communication

is

achieved

through

a

global

blackboard

a

section

of

shared

memory

accessible

to

all

The

system

reacts

to

events

by

changing

its

behavior

hence

the

sequence

of

behaviors

actually

executed

depends

upon

the

sequence

of

events

Since

the

latter

is

unpredictable

so

is

the

former

To

program

such

a

system

one

must

specify

which

processes

constitute

a

behavior

and

for

each

possible

event

describe

the

system

's

reaction

We

employ

a

finite

state

machine

-LRB-

FSM

-RRB-

with

states

representing

behaviors

and

transitions

representing

reactions

to

events

to

specify

the

system

For

example

the

following

set

of

statements

describe

a

system

that

will

drive

on

the

road

while

obeying

traffic

lights

until

a

given

distance

is

travelled

its

current

incarnation

the

blackboard

is

built

on

top

of

the

a

symbolic

realtime

database

for

vision

-LSB-

-RSB-

Although

the

was

mainly

designed

as

an

in-memory

database

it

provides

a

very

efficient

memory

management

mechanism

-LRB-

on

top

of

UNIX

-RRB-

and

a

set

of

primitives

necessary

for

shared

memory

based

communication

While

driving

down

the

road

if

the

traffic

light

turns

red

wait

if

goal

is

reached

stop

While

waiting

at

the

traffic

light

if

it

turns

green

start

driving

These

statements

correspond

to

the

FSM

in

Figure

Note

that

a

state

represents

a

behavior

or

mode

of

operation

i.e

a

set

of

concurrent

perceptionaction

processes

Based

on

the

notion

of

behaviors

represented

as

states

of

a

finite

state

machines

a

Behavior

Description

Language

-LRB-

BDL

-RRB-

was

designed

and

implemented

Behaviors

are

described

as

two

sets

of

perception-action

processes

and

a

transition

table

The

run

set

specifies

the

minimum

set

of

processes

that

form

the

behavior

the

kill

set

specifies

those

processes

that

should

not

be

running

for

the

correct

execution

of

the

behavior

The

transition

table

specifies

what

to

do

for

each

of

the

valid

events

-LRB-

events

not

specified

in

the

state

description

are

not

valid

-RRB-

A

simple

BDL

example

is

shown

in

Figure

for

a

more

complete

example

see

section

choice

of

two

sets

implies

that

processes

that

were

running

when

the

behavior

started

will

continue

to

run

unless

explicitly

killed

DoneGreenRed

Drive

-LSB-

rf

tl

dm

-RSB-

-LRB-

a

-RRB-

-LRB-

b

-RRB-

Stop

-LSB-

vs

rf

tl

-RSB-

Wait

-LSB-

vs

tl

rf

-RSB-

PROCS

rf

driveOnRoad

tl

checkTrafficLight

vs

vehicleStop

dm

distanceMonitor

-RCB-

STATES

-LCB-

drive

wait

stop

-RCB-

EVENTS

-LCB-

red

green

done

-RCB-

MSGS

-LCB-

distance

-RCB-

WHILE

drive

-LRB-

d

-RRB-

-LCB-

SET

distance

d

RUN

rf

tl

dm

EVENT

red

GOTO

wait

EVENT

done

GOTO

stop

-RCB-

WHILE

wait

-LRB-

-RRB-

-LCB-

KILL

rf

RUN

vs

tl

EVENT

green

GOTO

drive

-RCB-

Figure

Script

of

a

simple

driving

system

-LRB-

a

-RRB-

Finite

state

machine

-LRB-

FSM

-RRB-

representation

Listed

below

the

state

name

are

the

perception-action

processes

that

should

be

run

and

killed

-LRB-

marked

with

a

-RRB-

in

that

state

This

example

is

composed

of

four

perception-action

processes

follow

the

road

-LRB-

rf

-RRB-

check

for

traffic

lights

-LRB-

tl

-RRB-

monitor

the

distance

traveled

-LRB-

dm

-RRB-

and

stop

the

vehicle

-LRB-

vs

-RRB-

-LRB-

b

-RRB-

BDL

representation

First

the

perception-action

processes

available

are

listed

Then

the

set

of

states

-LRB-

or

behaviors

-RRB-

and

the

set

of

events

are

declared

Finally

a

description

of

each

state

is

provided

Scripts

Augmented

finite

state

machines

Returning

to

the

simple

driving

system

consider

the

following

task

Drive

for

meters

-LRB-

while

obeying

traffic

lights

-RRB-

Turn

left

Drive

for

meters

-LRB-

while

obeying

traffic

lights

-RRB-

In

the

FSM

model

presented

in

the

previous

section

this

task

will

be

represented

as

a

six

state

FSM

-LRB-

Figure

-RRB-

As

the

complexity

of

the

vehicle

's

task

grows

this

approach

will

lead

to

long

and

repetitive

representations

To

alleviate

this

problem

the

FSM

is

augmented

with

a

fetch-goal

state

and

the

behavior

description

is

parameterized

A

particular

behavior

-LRB-

like

drive

for

meters

or

drive

for

meters

-RRB-

is

instantiated

from

this

generic

description

through

the

use

of

blackboard

messages

at

run

time

The

system

starts

in

the

fetch-goal

state

where

it

reads

goals

-LRB-

in

terms

of

behaviors

-RRB-

from

a

precompiled

plan

When

a

goal

is

retrieved

the

script

monitor

-LRB-

section

-RRB-

writes

relevant

blackboard

messages

into

the

blackboard

before

creating

the

perception-action

processes

Once

such

a

process

is

running

it

looks

for

its

parameters

in

the

blackboard

-LRB-

Figure

-RRB-

With

this

approach

the

same

task

is

represented

in

four

states

-LRB-

Figure

-RRB-

-LRB-

b

-RRB-

-LRB-

a

-RRB-

DoneDoneGreenRedGreenRedDrive

m.

-LSB-

rf

tl

dm

-RSB-

Turn

right

-LSB-

dt

dm

rf

-RSB-

Done

Drive

m.

-LSB-

rf

tl

dm

dt

-RSB-

Wait

-LSB-

vs

tl

rf

-RSB-

Wait

-LSB-

vs

tl

rf

-RSB-

Stop

-LSB-

vs

rf

tl

-RSB-

DoneDoneRedGreenTurn

direction

-LSB-

dt

dm

rf

-RSB-

Drive

distance

-LSB-

rf

tl

dm

dt

-RSB-

Wait

-LSB-

vs

tl

rf

-RSB-

Fetch

goal

-LSB-

vs

ALL

-RSB-

Figure

Script

as

an

augmented

finite

state

machine

-LRB-

a

-RRB-

Specifying

a

task

in

terms

of

a

sequence

of

behaviors

-LRB-

dt

is

a

dead

reckoning

turning

process

-RRB-

-LRB-

b

-RRB-

Parameterizing

the

states

and

adding

the

fetch-goal

state

can

greatly

simplify

the

FSM

Blackboard

messages

-LRB-

in

quotes

-RRB-

are

used

to

refine

the

behavior

at

run

time

The

fetch-goal

state

differs

from

all

states

in

two

ways

transitions

out

of

the

state

are

unlabelled

since

the

next

state

is

explicitly

specified

in

the

goal

retrieved

from

the

script

Unless

the

plan

is

empty

the

kill

and

run

sets

are

ignored

-LRB-

ALL

is

a

shortcut

to

all

running

processes

-RRB-

Distance

Monitor

ScriptMonitorScript

MSGS

-LCB-

distance

-RCB-

WHILE

drive

-LRB-

d

-RRB-

-LCB-

SET

distance

d

drive

-LRB-

-RRB-

distancedata

Blackboardname

data

name

data

name

data

Figure

Passing

parameters

thru

the

blackboard

The

script

monitor

interprets

the

script

When

the

drive

state

is

entered

the

script

monitor

writes

a

blackboard

message

for

the

distance

monitor

The

distance

monitor

reads

the

distance

message

from

the

blackboard

Formally

a

script

S

is

defined

as

the

eight-tuple

-LRB-

P

Q

E

M

ffi

ae

G

-RRB-

where

ffl

P

is

the

set

of

available

perception-action

processes

ffl

Q

is

the

set

of

states

-LRB-

Q

B

-LSB-

fetch-goal

where

B

is

the

set

of

behaviors

-RRB-

ffl

E

is

the

set

of

possible

discrete

events

-LRB-

transitions

in

the

FSM

-RRB-

ffl

M

is

the

set

of

valid

blackboard

messages

ffl

ffi

is

the

transition

table

ffi

B

E

Q

ffl

is

the

kill

table

B

P

ffl

ae

is

the

run

table

ae

B

P

ffl

G

is

the

plan

expressed

in

terms

of

subgoals

Each

subgoal

is

of

the

form

bg

Mg

for

bg

B

and

Mg

M

Scripts

can

be

generated

by

an

automated

planner

or

by

hand

-LRB-

using

BDL

-RRB-

The

script

monitor

The

script

monitor

in

our

system

is

in

charge

of

high

level

control

reading

interpreting

and

executing

BDL

scripts

Essentially

the

script

monitor

is

a

plan

executer

-LSB-

-RSB-

very

similar

to

PRS

-LSB-

-RSB-

The

monitor

does

not

perform

any

action

on

the

vehicle

by

itself

but

controls

the

set

of

running

processes

at

any

time

The

script

monitor

consists

of

two

modules

an

interpreter

and

an

executer

The

interpreter

takes

a

BDL

script

S

and

builds

the

transition

-LRB-

ffi

-RRB-

kill

-LRB-<=-RRB-

and

run

-LRB-

ae

-RRB-

tables

then

the

subgoals

in

S

's

plan

are

stacked

into

the

execution

stack

G

The

executer

simulates

a

finite

state

machine

as

follows

bg

fetch-goal

-LRB-

Start

at

the

fetching

state

-RRB-

if

-LRB-

bg

fetch-goal

-RRB-

then

-LRB-

a

-RRB-

if

G

is

empty

then

P

i.

kill

-LRB-

p

-RRB-

-LRB-

Terminate

ALL

processes

-RRB-

ii

if

-LRB-

ae

-LRB-

bg

p

-RRB-

-RRB-

then

run

-LRB-

p

-RRB-

-LRB-

Run

cleanup

processes

-RRB-

iii

stop

-LRB-

S

was

successfully

executed

-RRB-

-LRB-

b

-RRB-

bg

Mg

pop

-LRB-

G

-RRB-

-LRB-

Fetch

next

goal

-RRB-

this

context

high

level

control

is

used

to

differentiate

the

control

of

processes

from

the

low

level

control

of

the

vehicle

actuators

-LRB-

c

-RRB-

blackboard

Mg

-LRB-

Write

blackboard

messages

-RRB-

P

if

-LRB-

-LRB-

bg

p

-RRB-

-RRB-

then

kill

-LRB-

p

-RRB-

-LRB-

Terminate

processes

-RRB-

P

if

-LRB-

ae

-LRB-

bg

p

-RRB-

-RRB-

then

run

-LRB-

p

-RRB-

-LRB-

Create

processes

-RRB-

Wait

for

an

event

e

E

-LRB-

Wait

-RRB-

bg

ffi

-LRB-

bg

e

-RRB-

-LRB-

React

to

event

follow

transition

table

-RRB-

goto

-LRB-

Repeat

until

all

goals

are

achieved

-RRB-

Clearly

for

the

system

to

complete

the

task

in

G

the

following

must

hold

Q

E

s

t

ffi

-LRB-

b

sb

-RRB-

fetch

goal

where

ffi

is

the

transition

function

applied

to

a

sequence

of

events

-LSB-

-RSB-

An

example

The

following

perception-action

processes

have

been

successfully

tested

on

the

MPL

ffl

Vehicle

pose

determination

based

on

landmark

model

matching

-LSB-

-RSB-

ffl

Neural-network

road

following

-LRB-

ALVINN

-RRB-

-LSB-

-RSB-

ffl

Servo-based

steering

-LSB-

-RSB-

ffl

Obstacle

detection

via

stereo

ffl

Reflexive

obstacle

avoidance

ffl

A

distance

monitor

ffl

Turning

via

dead

reckoning

An

experiment

was

designed

in

order

to

demonstrate

the

capabilities

of

the

vehicle

and

the

performance

of

the

independent

perception-action

processes

The

following

script

was

tested

Drive

on

the

road

while

avoiding

obstacles

for

x

meters

Estimate

vehicle

position

using

landmarks

Drive

on

the

road

while

avoiding

obstacles

for

y

meters

Estimate

vehicle

position

using

landmarks

Turn

left

-LRB-

at

the

experimental

site

this

command

is

a

transition

to

off-road

navigation

-RRB-

Drive

off

road

while

avoiding

obstacles

for

z

meters

The

full

BDL

script

for

this

coordinated

action

with

x

y

and

z

is

shown

in

Figure

the

FSM

representation

is

shown

in

Figure

Note

that

the

same

result

can

be

achieved

with

a

different

script

if

in

the

driving

states

the

success

condition

takes

the

system

back

to

the

fetch-goal

state

and

a

compute-pose

statement

is

added

after

each

driveonroad/drive-offroad

statement

in

the

plan

the

result

will

be

similar

As

in

other

programming

languages

in

BDL

some

things

can

be

expressed

in

more

than

one

way

Planning

The

script

mechanism

attempts

to

decompose

the

navigation

problem

into

three

sub-problems

goal

planning

reaction

planning

and

behavior

planning

Goal

planning

Given

a

particular

finite

state

machine

and

a

task

-LRB-

e.g.

drive

to

Main

street

-RRB-

what

is

the

sequence

of

behaviors

that

the

system

must

execute

to

achieve

this

task

In

a

BDL

script

this

sequence

is

specified

in

the

goal

list

G.

Reaction

planning

Given

a

set

of

behaviors

a

set

of

possible

events

and

an

abstract

task

-LRB-

drive

-RRB-

what

should

be

the

reactions

to

detected

events

i.e.

how

should

the

behaviors

be

connected

into

a

finite

state

machine

This

is

given

in

the

transition

table

ffi

PROCS

pe

poseEstimation

rf

roadFollow

od

obstacleDetect

oa

obstacleAvoid

se

servo

dm

distanceMonitor

dt

deadReckoningTurn

vs

vehicleStop

-RCB-

STATES

-LCB-

drive-onroad

drive-offroad

turn

compute-pose

avoid-obstacles

-RCB-

EVENTS

-LCB-

success

obstacles

clear

-RCB-

MSGS

-LCB-

distance

direction

-RCB-

WHILE

drive-onroad

-LRB-

dist

-RRB-

-LCB-

SET

distance

dist

RUN

rf

od

dm

EVENT

success

GOTO

compute-pose

EVENT

obstacle

GOTO

avoid-obstacles

-RCB-

WHILE

drive-offroad

-LRB-

dist

-RRB-

-LCB-

SET

distance

dist

RUN

se

od

dm

EVENT

success

GOTO

compute-pose

EVENT

obstacle

GOTO

avoid-obstacles

-RCB-

WHILE

turn

-LRB-

dir

dist

-RRB-

-LCB-

SET

direction

dir

SET

distance

dist

RUN

dt

dm

EVENT

success

GOTO

FETCH

-RCB-

WHILE

avoid-obstacles

-LRB-

-RRB-

-LCB-

KILL

rf

se

RUN

oa

od

EVENT

clear

GOTO

BACK

-RCB-

WHILE

compute-pose

-LRB-

-RRB-

-LCB-

KILL

rf

se

RUN

pe

EVENT

success

GOTO

FETCH

-RCB-

GOALS

-LCB-

drive-onroad

-LRB-

-RRB-

drive-onroad

-LRB-

-RRB-

turn

-LRB-

left

-RRB-

drive-offroad

-LRB-

-RRB-

-RCB-

Figure

BDL

script

to

achieve

the

following

goal

drive

on

the

road

for

meters

then

for

meters

take

a

left

turn

and

drive

off

road

for

meters

clearclearobstaclesuccesssuccesssuccessobstaclesuccessdrive-onroad

-LSB-

rf

od

dm

-RSB-

compute-pose

-LSB-

pe

rf

se

-RSB-

avoid-obstacles

-LSB-

oa

od

rf

se

-RSB-

drive-off

road

-LSB-

se

od

dm

-RSB-

turn

-LSB-

dt

dm

-RSB-

fetch-goal

-LSB-

vs

ALL

-RSB-

Figure

An

augmented

FSM

for

on/off

road

navigation

drive

while

avoiding

obstacles

for

a

specified

distance

then

check

position

It

can

also

take

turns

The

perception-action

processes

involved

include

pose

estimation

-LRB-

pe

-RRB-

road

following

-LRB-

rf

-RRB-

obstacle

detection

-LRB-

od

-RRB-

obstacle

avoidance

-LRB-

oa

-RRB-

servoing

-LRB-

se

-RRB-

distance

monitor

-LRB-

dm

-RRB-

and

dead

reckoning

turning

-LRB-

dt

-RRB-

Behavior

planning

How

to

construct

behaviors

from

a

set

of

perceptionaction

processes

i.e.

how

should

processes

be

combined

to

form

intelligent

behaviors

This

is

achieved

in

the

script

thru

the

run

-LRB-

ae

-RRB-

and

kill

-LRB-<=-RRB-

tables

Conceptually

a

solution

to

each

of

these

sub-problems

-LRB-

a

script

-RRB-

could

be

generated

automatically

The

first

problem

can

be

viewed

as

a

traditional

planning

problem

where

each

of

the

behaviors

can

be

considered

to

be

primitive

actions

At

this

level

a

hierarchical

planner

such

as

the

one

implemented

in

the

RML

system

-LSB-

-RSB-

can

be

easily

integrated

with

the

script

monitor

The

monitor

fetches

a

subgoal

from

the

stack

G

if

the

specified

action

is

not

primitive

the

planner

is

invoked

to

refine

the

subgoal

Note

that

planning

is

another

behavior

and

hence

planner

invocation

is

done

through

the

state

transition

mechanism

If

a

plan

failure

occurs

the

planner

may

also

be

invoked

by

the

behaviors

to

dynamically

create

a

new

plan

or

at

least

part

of

a

plan

Adding

a

planner

in

this

way

to

the

example

of

the

previous

section

will

produce

the

FSM

in

Figure

In

this

particular

example

the

planning

state

may

be

entered

when

the

compute-pose

behavior

can

not

estimate

the

current

position

-LRB-

i.e.

the

system

is

lost

-RRB-

Note

that

at

this

level

only

the

subgoals

stack

G

is

modified

dynamically

successnot

primitivelostclearclearobstaclesuccesssuccesssuccessobstaclesuccessdri

ve-onroad

-LSB-

rf

od

dm

-RSB-

compute-pose

-LSB-

pe

rf

se

-RSB-

avoid-obstacles

-LSB-

oa

od

rf

se

-RSB-

drive-off

road

-LSB-

se

od

dm

-RSB-

turn

-LSB-

dt

dm

-RSB-

Plan

-LSB-

vs

-RSB-

fetch-goal

-LSB-

vs

ALL

-RSB-

Figure

Integrating

the

monitor

and

the

planner

If

the

monitor

finds

a

non-primitive

action

the

planner

is

called

to

decompose

it

into

primitives

If

the

compute-pose

behavior

can

not

determine

the

position

of

the

vehicle

the

planner

is

called

to

generate

a

plan

to

get

to

a

known

point

The

second

problem

raises

the

issue

of

dynamic

modification

of

reactions

-LRB-

FSM

transitions

ffi

-RRB-

There

are

several

reasons

we

may

want

to

have

this

capability

ffl

Partial

planning

Planning

reactions

to

all

possible

events

can

take

a

long

time

when

in

most

cases

only

a

small

subset

of

events

typically

occur

in

any

particular

state

The

system

should

be

able

to

pre-plan

only

for

those

events

that

are

likely

to

occur

or

even

those

which

are

unlikely

but

require

an

immediate

reaction

thereby

saving

considerable

amounts

of

pre-planning

time

This

can

be

useful

in

situations

that

rarely

occur

and

if

they

do

occur

then

the

need

to

stop

the

vehicle

to

deliberate

is

not

a

bad

tradeoff

An

example

could

be

discovering

a

mechanical

failure

in

the

vehicle

if

this

happens

the

best

thing

to

do

is

to

stop

and

think

ffl

Modifying

reactions

With

only

partial

information

about

the

environment

and

possibly

inaccurate

heuristics

guiding

the

planning

some

reactions

may

be

either

wrong

or

not

optimal

As

more

information

becomes

available

the

system

may

improve

its

reactive

capabilities

For

example

consider

driving

on

a

rainy

day

under

normal

-LRB-

dry

-RRB-

conditions

the

best

reaction

to

an

obstacle

which

suddenly

appears

in

front

of

the

vehicle

is

to

brake

sharply

On

a

wet

road

this

may

be

a

very

bad

de

cision

as

soon

as

the

planner

realizes

that

the

road

is

wet

the

reaction

to

a

sudden

obstacle

should

be

reconsidered

The

composition

of

behaviors

-LRB-

sub-problem

-RRB-

raises

many

questions

Each

state

in

the

FSM

-LRB-

a

behavior

in

our

terminology

-RRB-

can

be

viewed

as

an

instance

of

Brooks

layered

control

-LSB-

-RSB-

as

such

arbitration

of

commands

to

the

actuators

and

allocation

of

sensors

-LRB-

and

other

resources

-RRB-

become

problems

-LRB-

see

section

-RRB-

The

solution

of

these

problems

is

fundamental

for

a

planner

that

composes

behaviors

Open

Issues

Although

limited

the

script

monitor

system

in

its

current

form

has

given

us

an

idea

of

the

complexity

involved

in

building

intelligent

controllers

particularly

in

a

real-time

application

domain

where

safety

must

be

ensured

Encoding

system

reactions

as

a

finite

state

machine

seems

a

reasonable

approach

but

it

is

not

clear

how

to

optimally

construct

the

individual

states

i.e.

which

perception-action

processes

constitute

each

state

and

how

these

processes

interact

with

one

another

These

are

difficult

problems

and

an

efficient

solution

to

them

is

essential

Resource

Sharing

Concurrent

processes

accessing

sensors

and

sending

commands

to

actuators

will

inevitably

cause

problems

if

resource

allocation

is

not

handled

correctly

For

example

consider

the

case

where

the

vehicle

is

to

travel

down

the

road

looking

for

a

landmark

while

avoiding

obstacles

This

behavior

can

be

achieved

by

three

perception-action

processes

road-follow

obstacleavoidance

and

find-landmark

running

concurrently

The

problem

is

that

the

resources

needed

by

these

processes

are

limited

and

conflicts

will

surely

arise

-LRB-

see

Figure

-RRB-

Even

in

this

simple

example

each

resource

poses

some

difficult

problems

ffl

The

vehicle

controller

Commands

from

different

processes

have

to

be

combined

-LRB-

arbitration

-RRB-

The

way

this

combination

is

achieved

is

crucial

if

the

vehicle

is

to

execute

rational

behaviors

ffl

The

image

processor

Interleaving

processes

in

this

unit

may

considerably

slow

down

the

system

On

the

other

hand

it

is

clear

that

while

moving

some

processes

-LRB-

such

as

avoid

obstacles

-RRB-

can

not

be

suspended

for

long

periods

of

time

while

the

image

processor

is

being

used

by

some

other

process

AvoidObstacles

Follow

Road

FindLandmark

Movable

camera

Vehiclecontroller

Imageprocessor

Figure

Resource

allocation

graph

for

a

driving

state

Perception-action

processes

are

represented

as

ovals

and

resources

as

rectangles

A

directed

edge

from

a

process

P

to

a

resource

R

indicates

that

P

uses

R.

Assuming

there

is

only

one

instance

of

each

resource

the

potential

for

deadlocks

is

large

ffl

The

movable

camera

It

is

likely

that

different

processes

will

try

to

point

the

camera

in

different

directions

or

require

its

use

at

the

same

time

Generally

the

time

to

move

a

mechanical

device

such

as

this

is

much

longer

than

computation

time

Again

we

encounter

the

problem

of

probable

long

waits

Inter-process

Communication

In

the

current

implementation

all

communication

between

processes

is

done

thru

the

blackboard

In

this

particular

domain

where

potentially

large

amounts

of

data

are

shared

-LRB-

images

maps

etc.

-RRB-

the

shared

memory

paradigm

seems

the

most

efficient

method

of

communication

for

processes

running

on

the

same

machine

But

if

perception-action

processes

were

to

run

in

a

distributed

architecture

other

means

of

communication

will

be

necessary

-LRB-

UNIX

sockets

TCX

-LSB-

-RSB-

etc.

-RRB-

Ideally

the

blackboard

should

support

several

mechanisms

for

communication

Processes

running

on

the

same

machine

can

still

communicate

through

shared

memory

processes

on

different

machines

use

some

other

mechanism

Of

course

whenever

timing

constraints

allow

it

the

actual

mechanism

should

be

transparent

to

the

perception-action

processes

i.e.

access

routines

should

look

the

same

regardless

of

where

is

the

data

or

how

it

is

copied

from

one

process

to

another

In

hard

real-time

situations

this

will

not

be

possible

To

try

to

alleviate

possible

bottlenecks

the

communication

needs

of

each

process

should

be

considered

when

it

is

assigned

a

machine

The

scheduler

can

try

to

run

those

processes

with

large

communication

needs

-LRB-

those

sharing

big

chunks

of

data

-RRB-

in

the

same

machine

All

of

this

becomes

part

of

the

planning

process

Conclusions

An

autonomous

vehicle

should

be

able

to

react

in

real-time

to

a

changing

environment

but

it

should

also

be

able

to

reason

about

its

goals

In

our

system

these

seemingly

contradictory

capabilities

are

achieved

through

the

use

of

a

programable

finite

state

machine

In

this

model

reactions

to

events

-LRB-

either

internal

or

external

-RRB-

are

represented

as

state

transitions

The

system

can

react

rapidly

by

following

a

transition

table

Goal-directed

reasoning

is

supported

by

the

programability

of

the

FSM

the

system

does

not

enforce

what

goes

into

a

state

or

a

particular

transition

table

instead

it

executes

scripts

which

eventually

will

be

written

by

an

automated

planner

Acknowledgments

Many

people

were

involved

in

the

design

and

implementation

of

the

system

described

in

this

paper

Special

thanks

go

to

G?okhan

Kutlu

for

his

work

on

the

Srinivas

Ravela

for

his

work

on

the

reflexive

obstacle

avoidance

process

Sumit

Badal

for

his

work

on

the

stereo

based

obstacle

detection

process

Jonathan

Lim

Shashi

Buluswar

and

Katja

Daumeller

for

their

work

in

the

landmark

based

pose

estimation

process

Carnegie-Mellon

University

and

Dean

Pomerleau

for

making

ALVINN

available

Alan

Boulanger

for

bringing

up

the

ALVINN

code

to

Bob

Heller

for

his

general

help

and

to

Gila

Kamhi

for

her

careful

review

of

this

paper

References

-LSB-

-RSB-

R.

Beveridge

Local

search

algorithms

for

geometric

object

recognition

Optimal

correspondence

and

pose

Ph.D.

Thesis

CMPSCI

University

of

Massachusetts

at

Amherst

-LSB-

-RSB-

J.

Brolio

B.

Draper

R.

Beveridge

and

A.

Hanson

The

ISR

an

intermediate-level

database

for

computer

vision

Computer

-LRB-

-RRB-

-LCB-

-LSB-

-RSB-

R.

A.

Brooks

A

robust

layered

control

system

for

a

mobile

robot

IEEE

Journal

of

Robotics

and

Automation

-LRB-

-RRB-

March

-LSB-

-RSB-

R.

A.

Brooks

Intelligence

without

representation

In

Proceedings

of

the

Workshop

on

the

Foundations

of

Artificial

Intelligence

Cambridge

MA

MIT

Press

-LSB-

-RSB-

B.

Draper

S.

Buluswar

A.

Hanson

and

E.

Riseman

Information

acquisition

and

fusion

in

the

mobile

perception

laboratory

In

Proc

of

Sensor

Fusion

VI

pages

-LCB-

Boston

MA

Sept.

-LSB-

-RSB-

B

Draper

A.

Hanson

and

E.

Riseman

Integrating

visual

procedures

for

mobile

perception

In

H.

Christensen

editor

Experimental

Environments

World

Scientific

Press

to

appear

also

in

CVGIP

IU

March

-LSB-

-RSB-

B.

Draper

G.

Kutlu

and

J.

Wong

The

user

's

manual

University

of

Massachusetts

Amherst

-LSB-

-RSB-

C.

Fedor

TCX

task

communications

-LRB-

version

-RRB-

Robotics

Institute

Carnegie

Mellon

University

Jan

-LSB-

-RSB-

C.

L.

Fennema

Interweaving

reason

action

and

perception

Ph.D.

Thesis

COINS

University

of

Massachusetts

-LSB-

-RSB-

C.

L.

Fennema

and

A.

R.

Hanson

Experiments

in

autonomous

navigation

In

Proceedings

of

the

Tenth

International

Conference

on

Pattern

Recognition

pages

-LCB-

-LSB-

-RSB-

M.

P.

Georgeff

Planning

In

Readings

in

Planning

chapter

Introduction

pages

-LCB-

Morgan

Kauffmann

-LSB-

-RSB-

J.

E.

Hopcroft

and

J.

U.

Ullman

Introduction

to

Automata

Theory

Languages

and

Computation

Addison-Wesley

-LSB-

-RSB-

F.

F.

Ingrand

M.

P.

Georgeff

and

A.

S.

Rao

An

architecture

for

realtime

reasoning

and

system

control

IEEE

Expert

-LSB-

-RSB-

K.

Kluge

Multisensor

system

integration

for

autonomous

navigation

tasks

In

I.

Masaki

editor

Intellegent

Vehicles

IEEE

Press

to

appear

in

-LSB-

-RSB-

R.

Kumar

Model

dependent

inference

of

information

from

a

sequence

of

images

Ph.D.

Thesis

COINS

University

of

Massachusetts

at

Amherst

-LSB-

-RSB-

J.

Lee

M.

J.

Huber

E.

H.

Durfee

and

P.G.

Kenny

UM-PRS

An

implementation

of

the

procedural

reasoning

system

Artificial

Intelligence

Laboratory

University

of

Michigan

-LSB-

-RSB-

D.

W.

Payton

An

architecture

for

reflexive

autonomous

vehicle

control

In

Proceedings

IEEE

Robotics

Automation

Conference

pages

-LCB-

-LSB-

-RSB-

D.

W.

Payton

K.

Rosenblatt

and

D.

M.

Keirsey

Plan

guided

reaction

IEEE

Transactions

on

Systems

Man

and

Cybernetics

pages

-LCB-

-LSB-

-RSB-

D.

A.

Pomerleau

Neural

network

based

autonomous

navigation

In

Charles

Thorpe

editor

Vision

and

Navigation

The

CMU

Navlab

Kluwer

Academic

Publishers

-LSB-

-RSB-

P.

J.

Ramadge

and

W.

M.

Wonham

The

control

of

discrete

event

systems

Proceedings

of

the

IEEE

-LRB-

-RRB-

-LCB-

January

-LSB-

-RSB-

E.

Rich

and

K.

Knight

Artificial

Intelligence

McGraw-Hill

Inc.

second

edition

-LSB-

-RSB-

E.

Rivlin

DEDS

formalism

for

systems

with

vision

University

of

Maryland

-LSB-

-RSB-

C.

Weems

et

al.

Status

and

current

research

in

the

image

understanding

architecture

program

In

Proc

Image

Understandig

Workshop

pages

-LCB-

DARPA

April

O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

control
finite state machines
integration

real-time systems and applications
vision-guided robotics