Motion Planning Transformers: One Model to Plan them All

1 INTRODUCTION . Motion planning is an integral component of any 2D navigation stack . It is an age-old problem with more than a few decades of innovative solutions . By far the most popular recent methods involve constructing trees or graphs in the planning space by randomly sampling points ( LaValle & James J. Kuffner , 2001 ; Kavraki et al. , 1996 ) or discretizing the space into grids ( Hart et al. , 1968 ) . Although these methods provide theoretical guarantees , they do not scale well to problems with larger planning spaces and crowded environments . This is a frequent problem for robots in large warehouses and household cleaning robots . Recently , learning-based methods that leverage environment structure from the given data have been proposed for these kind of problems . They combine convolution neural networks ( CNN ) and multi-layer perceptrons ( MLP ) to predict where samples should be generated to construct trees ( Qureshi et al. , 2019 ; Johnson et al. , 2020 ; Kumar et al. , 2019 ) , but do not extend to maps of larger sizes . Methods such as Value Iteration Network ( VIN ) ( Tamar et al. , 2016 ) , which do extend to maps of different sizes , perform poorly with larger maps ( Nardelli et al. , 2018 ) . Hence , in our ever-expanding pursuit of automation , we need planning algorithms that can scale up while effectively keeping the planning problem ’ s complexity and computation time in check . The trajectory of a local plan is influenced by the orientation of far-away obstacles , similar to language models where the semantics of a sentence is inferred only by reading the entire sentence . The ability of transformers to understand the syntactic and semantic structure of sentences was shown in Vaswani et al . ( 2017 ) . Thus the capacity of transformers to learn such connections efficiently is the motivation of this work . Our model plans in a two-stage approach ( Fig . 1 ) , first a transformer-based region proposal network recommends a region where a potential path can exist . We refer to this module as the Motion Planning Transformer ( MPT ) . Next , a sampling-based motion planner ( SMP ) tries to plan a path given the proposed region . Since the inputs to the transformers are sets rather than vectors , maps of different sizes can be used , and by reducing the search space , SMP can generate solutions much faster with fewer nodes . We assess the performance of MPT augmented SMPs ( Karaman & Frazzoli , 2011 ; Gammell et al. , 2014 ) on a synthetic and real-world dataset comprising different map sizes in cluttered environments for point-mass and non-holonomic car models . Our results indicate that MPT augmented planners achieve 7-28 % improvement in accuracy over recent learning-based planners , while matching the accuracy to traditional planners . MPT planners also reduce both planning time by 7-25 times and the vertices on the planning tree by 2-12 times compared to traditional planners . We also propose a new training routine to allow for MPT to generalize to maps of different sizes while maintaining accuracy and reducing both planning time and vertices in the planning tree . We also show that MPT maintains the same qualities to real-world environments without fine-tuning or additional training . The diversity of environments represented , from block obstacles to narrow passage mazes speak to the potential general application of the MPT approach across multiple environment types and domains . 2 RELATED WORK . The most relevant work to our transformers-based region proposal network for motion planning is perhaps the guided sampling-based motion planning methods . They analytically or through learned heuristics determine a subset in robot space that probably contains a path solution . For instance , ( Qureshi & Ayaz , 2016 ; Tahir et al. , 2018 ) employ Artificial Potential Fields ( APF ) within samplingbased methods such as RRT * ( Karaman & Frazzoli , 2011 ) and Bidirectional RRT * ( Qureshi & Ayaz , 2015 ) to guide a subset of random samples towards promising regions that possibly contain an optimal path solution . In contrast , Informed-RRT * ( IRRT * ) ( Gammell et al. , 2014 ) depends on an initial path from an RRT * algorithm to compute an ellipsoidal region probably containing an optimal path solution . However , in most planning problems finding an initial path solution is itself challenging . In a similar vein , Batch Informed Trees ( BIT * ) ( Gammell et al. , 2015 ) begins from an elliptical region formed by a straight line path ignoring all obstacles and incrementally expand it until an initial path solution is found . Once an initial path is determined , it is further optimized by adapting the precomputed ellipsoid and generating new samples within that space . Despite all advancements , these methods only consider geometric planning , i.e. , path planning under collision-avoidance constraints . They are yet to be evaluated in practical problems with complex constraints such as kinodynamic or non-holonomic constraints for autonomous car navigation tasks . Many have also used learning-based methods to reduce search spaces . Williams ( 1992 ) used the REINFORCE algorithm to guide the underlying SMP planner on a discretized workspace , others use a learned value function to guide the graph search ( Choudhury et al. , 2018 ; Chen et al. , 2020 ) , while Kumar et al . ( 2019 ) learned a generative model to sample points along bottleneck regions . Similarly , Value Iteration Networks ( VIN ) ( Tamar et al. , 2016 ) also discretizes the space and learns a value map to guide path planning . Universal Planning Networks ( UPN ) ( Srinivas et al. , 2018 ) extends VIN to continuous control spaces . Chaplot et al . ( 2021 ) proposed one of the few works that use transformers to learn a value function . Other works employ latent representation of state-spaces to generate a plan ( Banino et al. , 2018 ; Emmons et al. , 2020 ; Ichter & Pavone , 2019 ) . These methods are often difficult to train and interpret , and many of them are yet to be evaluated in real world navigation tasks . Although there exist random sampling-based approaches such as RRT * ( Karaman & Frazzoli , 2011 ; Arslan et al. , 2017 ) and SST ( Li et al. , 2016 ) that explore the robot state space and satisfy advance constraints , they suffer from the curse of dimensionality and take a considerable computational time in cluttered spaces . Neural Motion Planning ( Qureshi et al. , 2020 ; Ichter et al. , 2018 ) has recently emerged as a promising tool for solving a wide range of planning problems under various task constraints , ranging from non-holonomic ( Johnson et al. , 2020 ; Li et al. , 2021 ) to advanced manifold kinematic constraints ( Qureshi et al. , 2020 ) , with high computational speed . These methods learn sampling distributions from expert demonstrations and , on execution , generate samples for an underlying planner forming a subset that potentially contains a path solution . However , these approaches assume a fixed size input environment map and often require redefining network architectures and retraining for different map sizes . However , recent developments in deep learning , primarily through Transformers ( Dosovitskiy et al. , 2021 ; Liu et al. , 2021 ) , have provided us with ways to relax such assumptions . Our proposed approach leverages these developments and introduces a region proposal framework that can work with variable map sizes and enhances underlying motion planners to solve complex problems in cluttered environments . 3 METHOD . The set of motion planning problems that MPT aims to solve is of 2D navigation . Given a start ( xs ) and goal ( xg ) state , the objective is to propose a sequence of states xi ∈ X for i ∈ { 0 , 1 , . . . n } such that x0 = xs , xn ∈ Xgoal and the trajectories joining these states should be both kinematically feasible and collision free . Here , X represents the robot state space and Xg , { x : ‖x− xg‖ ≤ , x ∈ X } for a user defined threshold ( ) . Since we focus on solving the motion planning problem for 2D mobile systems , X ∈ { R2 , SE ( 2 ) } . MPT proposes promising regions in X where the underlying planners can search for path solutions . In the following sections we describe our method in detail . 3.1 MOTION PLANNING TRANSFORMER ( MPT ) . The MPT module is a region proposal network , similar to Faster R-CNN ( Ren et al. , 2015 ) , but uses a transformer network to identify regions of interest . An overview of the model is shown in Figure 2 . The input to the model is a representation of the planning scene , and an encoding of the start and goal points . The planning scene is represented using an occupancy matrix , mc ∈ RH×W , where an element with 1 indicates an occupied space and 0 denotes free space . In some navigation problems , these representations are also called costmap and have additional cost terms associated safety and robot constraints . The start-goal encoding of the planning problem is formed by highlighting patches of size p× p on a tensor of size H ×W with values -1 and 1 for the start and goal point respectively . These two matrices are concatenated to form a tensor m and passed to the feature extractor . Feature Extractor : The feature extractor is a Fully Convolution Network ( FCN ) that encodes the environment and the given planning problem into a latent space . As shown in prior works ( Dosovitskiy et al. , 2021 ; Ren et al. , 2015 ) , the feature extractor reduces the dimensionality of the input space by using a series of convolution , ReLU , and MaxPool layers . The FCN passes a sliding window of size p× p over m to generate an output of size Hl ×Wl × d , where Hl and Wl is determined by the size of the costmap and the FCN , and d is the latent dimension of the transformer encoder . Each patch that the sliding window moves over is parameterized by an anchor point similar to Faster R-CNN ( Ren et al. , 2015 ) . We choose the 2D co-ordinate corresponding to the center pixel of the patch as the anchor point . The output of the FCN is reshaped to size ( Hl ·Wl ) × d and fed to the position encoder . Each row vector of this matrix corresponds to an anchor point on the input map . Position Encoding : Transformer and convolutional models are agnostic to the spatial location of their inputs Dosovitskiy et al . ( 2021 ) . A common solution is to add learned or fixed vectors to encode the position of each input ( Gehring et al. , 2017 ) . For testing our model we used the same position encoding as Vaswani et al . ( 2017 ) , but for training , we use the following position encoder : PE ( j , 2i ) = sin ( k + j 100002i/d ) PE ( j , 2i+ 1 ) = cos ( k + j 1000 ( 2i+1 ) /d ) where k is uniformly sampled from a discrete set of integer numbers Z , j is the index of the anchor point , and i is the index of latent vector . This modification enables us to generalize to maps of different sizes . For maps larger than the training data , we observed that the position encoder from Vaswani et al . ( 2017 ) created a bias that prevented MPT from selecting regions of the state space outside the training region . To overcome this bias , we leverage the fact that a proposed plan is invariant to linear translation of the state space , and train our model by randomly shifting the position encoder . This ensures that the model is not biased by the position encoder . For further details refer to Section C in the Appendix . The modified latent vector is then passed to the transformer encoder . Transformer Encoder : The transformer module is responsible for learning the connections between the different local regions on the map for a given planning problem . It infers these connections by passing the latent vectors through a series of multi-headed self-attention ( MSA ) and multi-layer perceptron ( MLP ) blocks . Between every MSA and MLP block we apply Dropout , Layer Norm and residual connections similar to Vaswani et al . ( 2017 ) and Dosovitskiy et al . ( 2021 ) . We also add gradient checkpoints ( Chen et al. , 2016 ) after the MSA blocks to be more memory efficient . Hence by attending to all patches , the network encodes the importance of each patch to the given planning problem in its output . Classifier : Using the encoded importance , the classifier predicts if each of the anchor point is of interest to the current planning problem . This is efficiently implemented using a 1× 1 convolution layer , similar to Faster R-CNN ( Ren et al. , 2015 ) . A mask is generated by setting p×p patch , centered around selected anchor points , to 1 on a matrix of size H ×W .

Summary: The paper presents a Transformer-based planning approach that attempts to use an attention mechanism to reduce the search-space of a traditional planner such as RRT*. Approaches such as Informed RRT* constrain the search space of the planner, which allows much faster convergence time. In a similar way, the authors present an approach that uses attention on a 2D map to create a "mask" for a standard planner to draw samples from. The proposed Motion Planning Transformer (MPT) works by using sliding window on the original map, creating features that are then positionally encoded (which helps with generalisation) and then put through a transformer encoder and classified. The result is a prediction, for each patch, as to whether a planner should draw samples from it. When combined with a planner it is able to achieve state-of-the-art results in a reduced time.