Title: GraphMP: Graph Neural Network-based Motion Planning with Efficient Graph Search
Venue: NeurIPS 2023
Accepted: yes
Keywords: graph neural network

deep learning
Abstract: Motion planning, which aims to find a high-quality collision-free path in the configuration space, is a fundamental task in robotic systems. Recently, learning-based motion planners, especially the graph neural network-powered, have shown promising planning performance. However, though the state-of-the-art GNN planner can efficiently extract and learn graph information, its inherent mechanism is not well suited for graph search process, hindering its further performance improvement. To address this challenge and fully unleash the potential of GNN in motion planning, this paper proposes GraphMP, a neural motion planner for both low and high-dimensional planning tasks. With the customized model architecture and training mechanism design, GraphMP can simultaneously perform efficient graph pattern extraction and graph search processing, leading to strong planning performance. Experiments on a variety of environments, ranging from 2D Maze to 14D dual KUKA robotic arm, show that our proposed GraphMP achieves significant improvement on path quality and planning speed over the state-of-the-art learning-based and classical planners; while preserving the competitive success rate.

GraphMP: Graph Neural Network-based Motion Planning with Efficient Graph Search

Abstract

Motion planning, which aims to find a high-quality collision-free path in the configuration space, is a fundamental task in robotic systems. Recently, learningbased motion planners, especially the graph neural network-powered, have shown promising planning performance. However, though the state-of-the-art GNN planner can efficiently extract and learn graph information, its inherent mechanism is not well suited for graph search process, hindering its further performance improvement. To address this challenge and fully unleash the potential of GNN in motion planning, this paper proposes GraphMP, a neural motion planner for both low and high-dimensional planning tasks. With the customized model architecture and training mechanism design, GraphMP can simultaneously perform efficient graph pattern extraction and graph search processing, leading to strong planning performance. Experiments on a variety of environments, ranging from 2D Maze to 14D dual KUKA robotic arm, show that our proposed GraphMP achieves significant improvement on path quality and planning speed over state-of-the-art learning-based and classical planners; while preserving competitive success rate.

1 Introduction

Motion planning aims to find a high-quality collision-free path connecting the start and goal states in the configuration space of a robot. As a fundamental cognitive task in robotic systems, motion planning plays a critical role in many practical applications, such as autonomous driving, in-warehouse package handling and assisted surgery, etc. A motion planning problem can be solved from different perspectives. Sampling-based solutions, e.g., RRT and its variants [1, 2] randomly sample the configuration space to build a space-filling tree, which grows towards connecting the start and goal configurations. Search-based approaches, such as A* and Dijkstra [3, 4], interpret the planning as a graph search problem, and they then find the feasible path via traversing the graph.

Recently, learning-based planners have obtained substantial attention because of their comparable or superior performance to classical planners. In general, such data-driven strategy can efficiently learn the patterns in the configuration space and/or the behaviors of the oracle planners, optimizing the important operation (e.g., sampling mechanism) in the planning process, and thus reducing the unnecessary collision check with the improved path quality. To date, various neural networkpowered motion planners, including multilayer perceptron (MLP)-based [5], convolutional neural networks (CNN)-based [6], recurrent neural network (RNN)-based [7] and graph neural network (GNN)-based [8, 9], have been proposed in the literature. Among these different choices, the GNNbased neural planner is the most promising approach because of its unique advantages in processing

InputtoTheRight

PredictedEdgePriority

GNN-Explorer

n:Edge Priority

Path

Pathcost=2+3+1+1=7

OStart○Goal

Obstacle

10

2

2

4

Explore

3

3

2

3

2

3

2

3

Q

n: Edge Weight

8

7

Q

3

RawRGG

PredictedCollision-freeRGG

PredictedHeuristicValues

GraphMP

Pathcost=3+3=6

3

3

3

2

3

3

2

S

2

31

3

3

2

A*Search

0

1-

5

2

3

3

2

0

1

5

a

2

31

3

2

D

0

1

5

2

3

3

2

0

1

3

3

1

5

Q

3

3

1

02

5

3

3

1

5

O

3

3

1

62

5

3

3

1

62

challenging high-dimensional planning tasks. By sampling the continuous configuration space and generating the random geometric graph (RGG), the equipped GNN, which is naturally powerful for learning graph patterns, can be applied to predict the priority of nodes (configurations) [9] or edges (robot movement) [8], improving the efficiency and quality of path exploration. In particular, GNN- Explorer [8], as the most recent and representative GNN planner, has demonstrated superior performance in a series of low and high-dimensional planning tasks than various learning-based and classical planners.

However, we believe the potential of GNN-based motion planners is not fully unleashed and the state-of-the-art solution can be further improved. Our key observation is that the edge priority-based path exploration, by its nature, is not the optimal choice for solving graph search problems. For instance, as illustrated in Figure 1, neither the edge priority generation nor the path construction process fully considers the impact on the total path cost when selecting a new edge to the current path. In other words, the critical information of the accumulated cost is not properly represented, leveraged or maintained in the existing GNN-based planners, thereby limiting the quality of the finally searched path. On the other hand, the classical graph search approaches, such as A*, can provide the near-optimal solution for finding the shortest path over the weighted graph, if the suitable heuristic function can be identified. Essentially, the key factors for the success of A*-like approaches in the graph search task are 1) they fully explore the graph structure via routinely visiting the neighboring nodes; and 2) they formally consider the impact of the accumulated cost during path exploration; and these two features are exactly what the existing GNN and most of other learning-based planners lack.

Motivated by these insights, in this paper we propose to simultaneously leverage the advantages of GNN and graph search algorithm, leading to an efficient neural motion planner, namely GraphMP. GraphMP first effectively extracts and learns the important patterns in the configuration space via its GNN modules, and then identifies the near-optimal path over the processed RGG using its low-cost graph search component. More specifically, to enable this processing pipeline, a GNN-based neural collision checker and a neural heuristic estimator are proposed to extract key graph information from input RGG and provide it to A* module, which is also reformulated in the differentiable way to facilitate end-to-end training. More details of GraphMP is shown in Figure 2 and described in Section 3. We evaluate GraphMP and other learning-based and classical planners in a variety of environments, ranging from 2D Maze to 14D dual KUKA robotic arm. Experimental results show that our proposed GraphMP achieves significant improvement on various planning performance metrics (i.e., path quality and planning speed) over the state-of-the-art planning methods; while preserving the competitive success rate.

2 Related Work

Classical Planners. Today's most popular motion planners are either search-based or sampling-based. Search-based planners , including BFS [10], Dijkstra [4], and A* [3], are typically used for finding optimal paths in low-dimensional environments. Different from BFS and Dijkstra that are significantly slow in large maps, A* is an informed search algorithm that uses a heuristic function to guide the search towards the goal, significantly reducing the search space and improving the search efficiency.

Neural Collision Checker(NCC)

Training Raw RGG

OStart

○Goal

Predicted RGG

Oracle Collision-freeRGG

Edges

Obstacles

Nodes

O

NCC

NCC

Loss

n: Edge Weight

←Back-propagation

Attention

Attention

n:Predicted Value

1

↑

Edge

Embedding

MLP

Embedding

Node

NHE

Collision-free RGG

NHE

PredictedHeuristicValues

5

3

Differentiable A*

Explored Nodes

Loss

Nodes on Optimal Path

Predicted CollisionProbability

5

3

Neural HeuristicEstimator (NHE)

Inferencea.NewTask

c.RawRGG

d.PredictedRGG

Edges

Goal

Nodes

Sample

o

o

K-NN

3

NCC

3

o

MessagePassing

Node

GraphMP

e. Predicted Heuristic Values

f.Searched Path

g. Final Path

MLP

Embedding

NHE

5

3

31010

Classical A*

3

o

LazyNodeRemoval

Lazynode

2

PredictedHeuristicValues

Motivated by this benefit, many variants of A* have been developed, such as D* [11], WA* [12] and Hybrid-A* [13]. The heuristic functions adopted in these methods are typically admissible hand-craft versions, such as Euclidean distance and Manhattan distance.

Unlike search-based planners, sampling-based planners probe the high-dimensional configuration space with random sampling, alleviating the exponential complexity growth incurred by the increasing number of dimensions. In general, this type of planner first constructs a randomly sampled graph, and then explores the feasible path over the sampled graph. One important sub-category of samplingbased planner is tree-based solution, including RRT [1], RRT* [14] and Informed-RRT* [15], etc. Some other popularly used sampling-based planners include PRM [16], BIT* [2] and LazySP [17].

Learning-based Planners. Recently, a series of learning-based methods further extend the searchbased planners by producing better heuristics to guide the search. For instance, SAIL [18] trains heuristic policies by imitating clairvoyant oracles and demonstrates the capability of reducing search effort. [19] and [20] learn the heuristic functions using the U-Net [21] and transformer [22] architecture, respectively. On the other hand, VIN [23] and SPT [24] learn the planning policy directly, and [25] develops the differentiable solvers for integer linear optimization problems by treating them as black boxes. Furthermore, Neural Weighted A* [26] learns the image-format graph costs and heuristics simultaneously via being supervised under the planning examples. Neural A* [27] proposes to reformulate the entire procedure of the canonical A* search algorithm to be differentiable, enabling the end-to-ending training of a guidance map for 2-D path planning tasks. However, this method still relies on a hand-crafted heuristic function and can only be applied to 2D images, hindering its applications.

In addition, a variety of deep learning-powered sampling-based planners have also been proposed in the literature. Fastron [28] and ClearanceNet [29] learn the function approximators to facilitate collision detection. On the other hand, LEGO [30] and [31] predict the high-quality sampling distribution using conditional variational auto-encoder (CVAE), reducing the exploration effort. MPNet [5] and STP-Net [7] predict the next sample directly by encoding the environments together with the configurations via the contractive auto-encoder (CAE) and convolutional layers, respectively. In particular, motivated by their powerful capability of learning the topology of environments of any dimension, the application of GNNs in motion planning has gained popularity in recent years. [9] introduces a GNN-based sampler to predict the critical node in a randomly generated graph, showing promising performance for identifying the optimal samples in high-dimensional tasks. [8] proposes GNN-Explorer, which prioritizes the exploration of graph edges in the planning process. Since the planner can now focus on the most promising paths first, the planning efficiency of GNN-Explorer is significantly improved, outperforming various classical and learning-based planners.

3 Our Approach: GraphMP

3.1 Preliminaries

Consider a motion planning problem defined in the d -dimensional continuous configuration space X , where X obs is the obstacle space and X free = X \ X obs is the free space. Given an RGG G = ( V , E ) V , where V is the set of nodes sampled from X free and E is the set of weighted edges constructed by K-nearest neighbors (KNN). A motion planner aims to find a collision-free path that connects the source node v s and goal node v g by a collision-free path. The prior work GNN-Explorer [8] initializes an exploration tree T rooted from v s , and visits edges e ∈ E sorted by their predicted priorities. Each visit performs the accurate collision check on the edge and tries to append it into T , until v g is reached. Here the search of GNN-Explorer relies on the predicted edge priorities without fully exploring graph structure and formally consider the impact of accumulated path cost during the path exploration, thereby hampering the path quality. On the other hand, graph-based A* computes the path from v s by iteratively selecting the best candidate v sel = arg min v ∈O ( g ( v ) + f ( v )) from the open list O . Here, g ( v ) accumulates the actual cost from v s to v and f ( v ) estimates the heuristic value of the cost from v to v g . Once v sel is selected, each of ites reachable neighboring nodes v nbr ∈ V sel is checked and updated. By using this way, A* can better explore graph stcuture and perform cost-aware search, but it requires manual design of high-quality heuristic function h ( v ) , which is a challenging task for high-dimensional tasks. Also, visiting the neighboring nodes can be time-consuming because the accurate collision check must be performed on each edge e v sel ,v nbr . To overcome these limitations of prior works, GraphMP proposes to use GNN to extract and learn the important patterns of RGG, and then identifies the near-optimal path using learnable graph search component. More specifically, a neural collision checker and a neural heuristic estimator are proposed to extract key graph information from input RGG and provide it to the proposed reformulated differentiable A* module for end-to-end training. Therefore, the path planning is now a graph structure-aware and cost-aware process with powerful graph pattern extraction capability and learnable heuristics function, making it can achieve high planning performance and suitable for both low and high-dimensional tasks.

3.2 Overall Framework

Fig. 2 shows the overall architecture of the proposed GraphMP. In the training phase , a neural collision checker takes the raw RGG and obstacle information as input and predicts the collision status of all edges. The difference between the estimated and ground-truth collision-free RGGs serves as the training loss to improve the prediction quality of the neural collision checker. Meanwhile, a neural heuristic estimator is also individually trained to assign the proper heuristic value to each node in the ground-truth collision-free RGG, making the graph search-based planning become possible. Here in order to make the neural heuristic estimator learnable, its predicted heuristic values are sent to A* module to generate a list of nodes that should be visited. The difference between this predicted node list and the optimal path is then calculated as the training loss, guiding the update of neural heuristic estimator. Notice that the A* module in the training phase is designed in a differentiable way to enable the backward propagation of gradients. More details will be described in Section 3.5.

After individually training neural collision checker and neural heuristic estimator, these two modules are then concatenated in the inference phase . More specifically, neural collision checker first predicts the potential collisions existed in the RGG and generates a predicted collision-free RGG. The neural heuristic estimator then assigns the estimated heuristic value to each node of this roughly collision-free RGG, enabling the graph search-based planning via a A* module. Notice that because in the inference phase the input RRG of A* is only approximately collision-free without guarantee, a non-differentiable classical A* is adopted here to incorporate the potentially additional collision check during the search procedure. After the nodes of a collision-free path are finally identified, a lazy node removal (details in Section 3.5) is performed to further improve path quality.

3.3 Neural Collision Checker

Because detecting the potential collision of an edge, by it nature, can be interpreted as a binary classification problem, we build a neural network-based collision checker to roughly predict the collision status of the edge e ij in the input RGG G = ( V , E ) with obstacle information X obs :

01100

Step 1

Step 2

Step 3

Step 4

10111

5

5

5

5

A=

11010

01101

6

2

2

0

6

2

2

0

6

2

2

0

6

2

1

2

0

02200

01010

0

3

41

4

3

41

4

0

31

41

4

3

4

4

20342

2

2l

2

2

2

2

W=

23020

04202

3

2

2

6

3

2

3

6

3

2

2

6

3

2

2

3

6

02020

0

bheu = [6536 0]

0

0

1

1

0

SelectedNode&

Vsel

=

0

Vnbr

Vsel

1

Vnbr

=

0

Vsel

=

0

Vnbr

0

0

0

0

1

0

ExploreNeighbors

0

0

0

0

0

0

OT

0

0

２２０

1

0

２２

1

0

２２

0

00

00

Update bacc.

bacc

一

0

1

0

bacc

0

0

1

bacc

0

0

C

1

Vsel

=

0

=

0,&c

0

0

4

4

0

0

0

0

Start node:

0

4

OSelectednode

ONodeintheclosed list

Connection toexplorednode

where I ( e ij ) is the function that gives the index of e ij in the edge set E , and p = ( p 0 , p 1 , ...p |E|1 ) is the collision-free probability vector that can be calculated as: p = f prob ( f oe ( V , E , X obs )) . Here f oe is the attention-based iterative obstacle encoder [8] that encodes the information of input RGG and obstacles to a concatenated embedding vector u = ( x ( l ) i , x ( l ) j , x ( l ) i -x ( l ) j , y ( l ) ij ) for each edge e ij , where x ( l ) i and y ( l ) ij denote the node embedding of v i and edge embedding of e ij at the l -th iteration of obstacle encoding, respectively. u is then processed by a three-layer MLP f prob to calculate the collision-free probability for each edge e ij . More details of f oe and f prob are in the Appendix.

Training Procedure. In the training phase the neural collision checker is individually trained on batches of collision check problem instances { ( V ( i ) , E ( i ) ) , X ( i ) obs } , where the collision-free probabilities of all the collision-free edges in the i -th problem instance is set as 1, otherwise 0. The accurate labeling on the training data is provided by the oracle collision checker. The difference between the estimated collision-free probability vector p and the binary representation of the ground-truth label, is measured via calculating binary cross-entropy (BCE) loss and minimized during the training procedure to improve the prediction quality of neural collision checker.

3.4 Neural Heuristic Estimator

Given the input collision-free RGG G free = ( V , E free ) and goal node v g , the neural heuristic estimator is constructed as a graph neural network (GNN)-based module to predict the heuristic value b heu,i for each v i ∈ V . To that end, two MLPs (detailed in the Appendix) are first used to embed the information of v i and e ij to latent space as q (0) i ∈ R d h and r (0) ij ∈ R d h , respectively, where d h is the embedding size, and then the GNN iteratively updates the node and edge embeddings via aggregating the local information of each node from its neighbors N ( v i ) = { v j | e ij ∈ E free } as follows:

Here, g , f q and f r are two-layer MLPs with output dimension as d h . After L iterations, the heuristic value of node i is calculated via encoding the node embedding q ( L ) i as b heu,i = f val ( q ( L ) i ) , where f val is a three-layer MLP.

3.5 Differentiable Graph-based A*

In order to ensure that the neural heuristic estimator can provide good estimation of heuristic value for graph search-based planning, in the training phase it is concatenated to the A* module, allowing the final planning result to directly guide the training procedure. However, a challenging issue for enabling such end-to-end learning is the non-differentiability of A*, incurred by the discrete nature of its incremental search procedure. Notice that though [27] proposes a differentiable grid-based A*, this existing solution can only work for 2-D grid search and planning; while addressing the non-differentiability of the more general graph-based A*, which is used for our proposed GraphMP towards high-dimensional planning, is non-trivial and not explored yet.

Next we describe the key reformulation for differentiable graph-based A*, which is illustrated in Fig. 3. Recall that the essence of A* on G free = ( V , E free ) is to iteratively select and move the node associated with the lowest path cost from open list O , which contains all the currently candidate nodes, to the closed list C that stores the visited nodes, and explore the neighbors of the selected nodes to expand O (More details of A* are described in the Appendix). To make this procedure differentiable, we first use two binary vectors o ∈ [0 , 1] |V| and c ∈ [0 , 1] |V| to represent the nodes information in O and C , respectively, where the 1 entries indicate the contained nodes. Also, the accumulated costs and the estimated heuristic values of all the nodes in V are denoted as real-valued b acc = ( b acc, 0 , b acc, 1 , ..., b acc, |V|1 ) and b heu = ( b heu, 0 , b heu, 1 , ..., b heu, |V|1 ) , respectively, where b heu,i is predicted by the neural heuristic estimator. Then, the operations of selecting the lowest-cost node in O and moving it from O to C can be vectorized via using a one-hot vector v sel ∈ [0 , 1] |V| as follows:

Here similar to [27], element-wise product ⊙ is used to mask the nodes in C , and λ and I max ( . ) are the pre-set parameter and the function returns a one-hot vector with the '1" entry associated with the lowest path cost ( b acc,i + b heu,i ). v sel is then further used to vectorize the neighborhood exploration process. To be specific, let A ∈ { 0 , 1 } |V|×|V| and W ∈ ❘ |V|×|V| denote the unweighted and the weighted adjancency matrix of E free , respectively. Accordingly, the neighboring nodes that should be explored can be represented as

where v nbr ∈ [0 , 1] |V| is the binary vector marking the entries corresponding to the neighboring nodes as ones, and ✶ is the all-one vector. Notice that here the indices of the neighboring nodes that are already in the closed list C are masked out via using ( ✶ -c ) . Upon the identification of these newly explored neighboring nodes, the accumulated costs g and the vectorized open list o are further updated as:

where b ′ acc denotes the accumulated costs of the nodes that are associated the path containing the selected node. The operation b acc > b ′ acc compares the node-wise accumulated costs and yields a binary vector of length |V| . More specifically, b acc ⊙ v sel extracts the accumulated cost of the currently selected node, and Wv sel represents the distance from the selected node to each of its neighbors. Notice that as shown in Eq. 7, the accumulated cost of neighboring nodes are updated in two cases: (1) the nodes do not exist in O ; and (2) the nodes are in O but its current accumulated cost is already smaller than the updated one. The identification of such two types of nodes are realized by a binary vector Φ . After updating the accumulated costs, the open list O is expanded by adding newly explored neighboring nodes (Eq. 8).

End-to-End Training. With the differentiability of graph-based A*, neural heuristic estimator and graph search planning can be now jointly trained in an end-to-end way, providing better guidance of learning heuristic values. Algorithm 1 describes the overall training procedure.

Algorithm 1 End-to-End Training Framework

Input: Full training set D heu , neural heuristic estimator f heu ( V , E , v g , Θ heu ) with weights Θ heu , the max iteration T , learning rate γ

(a) Key idea of LNR.

(b) 2D maze.

3

3

0

0

1

2

Q

Lazy node

2

Path cost=4.37

Path cost=4.10

Path cost=13.54

Path cost=8.90

3.6 Inference Procedure for Online Planning

As illustrated in Fig. 2, when GraphMP performs online planning (detailed in the Appendix) for the new task, a non-differentiable classical A* module is adopted to perform graph search on the approximated collision-free RGG prepared by the neural collision checker. Here the heuristic values desired in the search procedure is predicted by the neural heuristic estimator. Notice that in order to reduce the computational cost and path cost, the following two optimization operations are adopted in the inference procedure.

In-Search Collision Check. Due to the approximation property of neural networks, some edges in the approximated collision-free RGG may collide with the obstacles. We integrate the accurate collision check into A* search that only occurs on necessary edges with low prediction confidence. specifically, we perform the time-consuming accurate collision check only when (1) the neighbor of one selected node is explored, and (2) the collision-free probability of the corresponding edge is lower than a threshold θ . Such operation balances the trade-off between the planning efficiency and the path safety.

Lazy Node Removal. After A* finishes the path search, lazy node removal is performed to further reduce path cost. As illustrated in Fig. 4, within the planned path , the pairs of the nodes that are not directly connected are iteratively checked to see whether if the potential direct connection (as new edge) is collision-free and brings shorter path. If so, the two nodes can be connected and the nodes between them are removed as 'lazy nodes". In general, the worst time complexity of lazy node removal is O ( T 2 ) , where T is the number of nodes in the path found by A*. Since the searched path typically contains only few nodes ( T is small), this technique effectively reduces the path cost with minor computing overhead.

Prediction accuracy and confidence(%)

Meantime costpergraph(ms)

Path cost

100

Confidence

6

20

A* with NHE

Vanilla A*

15

50

10

2

5

Maze2

UR5

Snake7KUKA7KUKA13KUKA14

0

Maze2

UR5

Maze2

UR5

Snake7KUKA7KUKA13KUKA14

(a) Performance of NCC.

(b) Performance of NHE.

4 Results

4.1 Dataset and Experimental Setup

We demonstrate the effectiveness of our algorithm in six types of planning tasks as the same in [8]: (1) Maze2: a 2 DoF point-robot in 2D maze workspace of dense obstacles, (2) Ur5: a 6 DoF UR5 robot in 3D workspace, (3) Snake7: a 7 DoF snake robot in 2D workspace, (4) Kuka7: a 7DoF KUKA arm in 3D workspace, (5) Kuka13: a 13 DoF Kuka arm in 3D workspace, and (6) Kuka14: a pair of 7DoF KUKA arms in 3D workspace.

For each environment, we prepare two different training datasets each of which consists of 2000 different workspaces, to train the neural collision checker and heuristic estimator separately. For each workspace, we randomly construct 20 RGGs by sampling a random number of nodes ([100, 200, 300, 400]) and a random value of KNN ([5, 10, 15, 20]). Specifically, for the neural collision checker, each problem contains the raw RGG built on a set of randomly sampled free states and the different set of obstacles, while the ground-truth is the collision-free RGG obtained by the oracle collision checker. For the neural heuristic estimator, each problem contains a feasible pair of v s and v g , the exactly collision-free graph, and the ground-truth is the optimal path computed by Dijkstra. We also use 500 problem instances as the validation set. We keep the weights of heurisitic estimator which yields the smallest average path cost on the validation set. After the training, GraphMP is evaluated in an end-to-end manner on 1000 problem instances with unseen workspaces. Each input of the testing problems contains a pair of v s and v g , and a set of obstacles.

Our neural collision checker adopts 3 iterations of obstacle encoding with an output dimension of 64, and the neural heuristic estimator has 5 loops of message passing in Eq. 2 with the output dimension of 32. For both training of these two models, we select ADAM [32] as the optimizer and set the learning rate as 1 e -3 . The training epoch is 400 and the batch size is set as 8. We set the threshold θ in the in-search collision check as 80%. The number of graph nodes per sampling is 100 and the K value of K-NN is 10. The experiments are conducted on a computere quipped with an AMDEPYC 74202P24-Core Processor and an NVIDIA RTXA6000 GPU.

Baselines. We evaluate the performance of GraphMP by comparing it with three classical planners (BIT*, RRT* and LazySP) and one state-of-the-art learning-based planner (GNN-Explorer). In addition, the Smoother proposed in [8] serves as an additional step to reduce the path cost of the

computed path. Therefore, we also compare the performance of GraphMP with Smoother and GNN-Explorer with Smoother.

Ablation study. Details are reported in the Appendix.

4.2 Comparison With Baselines

Experimental results show that our GraphMP achieves 98.6% - 100% success rate across all the environments, being competitive to the state-of-the-art planners (Table 1). Table 2 shows that GraphMP achieves 16.79%, 35.84%, 12.62%, 23.39%, 18.99% and 25.94% shorter paths than GNNExplorer, on Maze2, UR5, Snake7, KUKA7, KUKA13 and KUKA14, respectively. Furthermore, the GraphMP with the path smoother has the lowest path cost compared to all baselines with a competitive success rate. Table 3 compares the time cost of all the motion planners. Compared to GNN-Explorer, GraphMP finds valid solutions with 19.94%, 42.14%, 5.07%, 36.44% and 20.62% lower time cost on Maze2, UR5, Snake7, KUKA7 and KUKA14, respectively. Overall, the results indicate that GraphMP outperforms all baseline planners by producing high-quality paths with the fastest planning speed and nearly 100% success rate in most of the planning tasks from 2D to 14D.

4.3 Study of Individual Modules of GraphMP

The Performance of Neural Collision Checker. We first evaluate the neural collision checker by reporting the mean prediction accuracy, confidence score and time cost. Specifically, the mean prediction accuracy and confidence are measured by averaging over all the edges from the test graphs, and the mean time cost represents the latency of predicting all edges per testing graph. Fig. 5 shows that our neural collision checker can perform super fast and accurate prediction on the graph-level collision check with high prediction confidence, in all the environments.

The Performance of Neural Heuristic Estimator. We then evaluate the performance of the neural heuristic estimator by comparing the A* with different heuristic functions. To be specific, we equip A* with our neural heuristic estimator and the Euclidean function which serves as the commonly used heuristic function to solve planning tasks, respectively. From Fig. 5, it is shown that the A* with our neural heuristic predictor significantly outperforms the vanilla A*, by producing solutions of lower path cost. The advantage of our neural heuristic predictor also grows quickly with the increasing dimensions of the planning tasks. The results illustrate that the neural heuristic estimator learns and performs better search guidance, especially in the configuration space of higher dimensions.

The Impact of In-Search Collision Check (ICC) and Lazy Node Removal (LNR). We further analyze the impact of the ICC and LNR modules on GraphMP, to justify their necessity in improving

the planning performance. Table 4 compares GraphMP with the version without ICC and the one without LNR, with respect to the path cost and time cost, respectively. While all these planners achieve 98.6% - 100% success rate across different environments, GraphMP outperforms the version without LNR by yielding 17.38%, 17.11%, 4.38%, 21.92%, 14.22% and 14.53% shorter paths with a limited increase of the time cost, on Maze2, UR5, Snake7, KUKA7, KUKA13 and KUKA14, respectively. Besides, compared to the one that uses accurate collision check (GraphMP w/o ICC), GraphMP achieves the same path quality with a significantly smaller time cost. Overall, it is seen that ICC and LNR are effective in improving planning speed and path optimality, respectively.

5 Limitations

Despite its good empirical performance across different tasks, GraphMP still has some limitations. First, it does not provide probabilistic completeness when the collision check threshold θ < 100% . That means, if the collision status of some edges is determined by the neural collision checker (NCC), even if the prediction accuracy of NCC is high and a collision-free path exists in the input RGG, GraphMP still cannot guarantee to find the feasible solution asymptotically. Notice that though the probabilistic completeness can be achieved when setting θ = 100% , the planning time will accordingly increase due to the extra costs incurred by performing accurate collision check on all the explored edges. Second, it does not offer asymptotical optimality. GraphMP performs the graph search on an implicit RGG which is incrementally expanded with more batches of nodes. Once a path is found, GraphMP validates its legality and returns the solution. Because 1) this mechanism naturally leads that the quality of the sampled RGGs has a heavy impact on the path cost - the waypoints along the paths are restricted to be a subset of the existing nodes of RGG, but RGG itself cannot be guaranteed to contain the optimal path; and 2) the search process will be terminated once the path is found, without further seeking better solutions, GraphMP cannot theoretically guarantee to find the optimal path asymptotically. Third, its efficiency is still limited by inefficient RGG construction. More specifically, 1) the uniform sampling of nodes disregards the environmental topology, causing some unnecessary node exploration; and 2) the construction of raw edges is also involved with unnecessary edge generation, thereby limiting the further runtime speedup provided by the proposed approach.

7 Conclusion

This paper proposes GraphMP, a neural motion planner for both low and high-dimensional planning tasks. With the customzied model architecture and training mechanism design, GraphMP can efficiently learn the graph pattern and process graph search, enabling strong planning performance. Experiments on a variety of environments show that GraphMP achieves significant planning speed-up and higher path quality than the state-of-the-art learning-based and classical planners.

References

Steven M LaValle et al. Rapidly-exploring random trees: A new tool for path planning. 1998.

Jonathan D Gammell, Timothy D Barfoot, and Siddhartha S Srinivasa. Batch informed trees (bit*): Informed asymptotically optimal anytime search. The International Journal of Robotics Research , 39(5):543-567, 2020.

Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics , 4(2):100-107, 1968.

Edsger W Dijkstra. A note on two problems in connexion with graphs. In Edsger Wybe Dijkstra: His Life, Work, and Legacy , pages 287-290. 2022.

Ahmed H Qureshi, Anthony Simeonov, Mayur J Bency, and Michael C Yip. Motion planning networks. In 2019 International Conference on Robotics and Automation (ICRA) , pages 2118-2124. IEEE, 2019.

Tomas Kulvicius, Sebastian Herzog, Timo Lüddecke, Minija Tamosiunaite, and Florentin Wörgötter. One-shot multi-path planning for robotic applications using fully convolutional networks. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 1460-1466. IEEE, 2020.

Xiao Zang, Miao Yin, Lingyi Huang, Jingjin Yu, Saman Zonouz, and Bo Yuan. Robot motion planning as video prediction: A spatio-temporal neural network-based motion planner. arXiv preprint arXiv:2208.11287 , 2022.

Chenning Yu and Sicun Gao. Reducing collision checking for sampling-based motion planning using graph neural networks. Advances in Neural Information Processing Systems , 34:42744289, 2021.

Arbaaz Khan, Alejandro Ribeiro, Vijay Kumar, and Anthony G Francis. Graph neural networks for motion planning. arXiv preprint arXiv:2006.06248 , 2020.

Charles E Leiserson and Tao B Schardl. A work-efficient parallel breadth-first search algorithm (or how to cope with the nondeterminism of reducers). In Proceedings of the twenty-second annual ACM symposium on Parallelism in algorithms and architectures , pages 303-314, 2010.

Anthony Stentz et al. The focussed dˆ* algorithm for real-time replanning. In IJCAI , volume 95, pages 1652-1659, 1995.

Ira Pohl. Heuristic search viewed as path finding in a graph. Artificial intelligence , 1(3-4):193204, 1970.

Dmitri Dolgov, Sebastian Thrun, Michael Montemerlo, and James Diebel. Practical search techniques in path planning for autonomous driving. Ann Arbor , 1001(48105):18-80, 2008.

Sertac Karaman and Emilio Frazzoli. Sampling-based algorithms for optimal motion planning. The international journal of robotics research , 30(7):846-894, 2011.

Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Informed rrt*: Optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 29973004. IEEE, 2014.

Lydia E Kavraki, Petr Svestka, J-C Latombe, and Mark H Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE transactions on Robotics and Automation , 12(4):566-580, 1996.

Nika Haghtalab, Simon Mackenzie, Ariel Procaccia, Oren Salzman, and Siddhartha Srinivasa. The provable virtue of laziness in motion planning. In Proceedings of the International Conference on Automated Planning and Scheduling , volume 28, pages 106-113, 2018.

Mohak Bhardwaj, Sanjiban Choudhury, and Sebastian Scherer. Learning heuristic search via imitation. In Conference on Robot Learning , pages 271-280. PMLR, 2017.

Takeshi Takahashi, He Sun, Dong Tian, and Yebin Wang. Learning heuristic functions for mobile robot path planning using deep neural networks. In Proceedings of the International Conference on Automated Planning and Scheduling , volume 29, pages 764-772, 2019.

Daniil Kirilenko, Anton Andreychuk, Aleksandr Panov, and Konstantin Yakovlev. Transpath: Learning heuristics for grid-based pathfinding via transformers. arXiv preprint arXiv:2212.11730 , 2022.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention , pages 234-241. Springer, 2015.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. Advances in neural information processing systems , 29, 2016.

Devendra Singh Chaplot, Deepak Pathak, and Jitendra Malik. Differentiable spatial planning using transformers. In International Conference on Machine Learning , pages 1484-1495. PMLR, 2021.

Marin Vlastelica Poganˇ ci´ c, Anselm Paulus, Vit Musil, Georg Martius, and Michal Rolinek. Differentiation of blackbox combinatorial solvers. In International Conference on Learning Representations , 2019.

Alberto Archetti, Marco Cannici, and Matteo Matteucci. Neural weighted a*: Learning graph costs and heuristics with differentiable anytime a. In International Conference on Machine Learning, Optimization, and Data Science , pages 596-610. Springer, 2021.

Ryo Yonetani, Tatsunori Taniai, Mohammadamin Barekatain, Mai Nishimura, and Asako Kanezaki. Path planning using neural a* search. In International conference on machine learning , pages 12029-12039. PMLR, 2021.

Nikhil Das and Michael Yip. Learning-based proxy collision detection for robot motion planning applications. IEEE Transactions on Robotics , 36(4):1096-1114, 2020.

J Chase Kew, Brian Ichter, Maryam Bandari, Tsang-Wei Edward Lee, and Aleksandra Faust. Neural collision clearance estimator for batched motion planning. In International Workshop on the Algorithmic Foundations of Robotics , pages 73-89. Springer, 2021.

Rahul Kumar, Aditya Mandalika, Sanjiban Choudhury, and Siddhartha Srinivasa. Lego: Leveraging experience in roadmap generation for sampling-based planning. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 1488-1495. IEEE, 2019.

Brian Ichter, James Harrison, and Marco Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA) , pages 7087-7094. IEEE, 2018.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies learning-based motion planners with GNNs. To improve the performance of GNN-based previous work, this paper proposes the GraphMP, possibly applied to low and high-dimensional planning tasks. The important idea is using (1) using predicted heuristic values, (2) NCC (Neural collision checker), and (3) differentiable A* modules (encoded with matrix operations) for efficient motion planning. The experimental results in Sec.4 showed that the proposed GraphMP performs better with limited computational times.\n\nSTRENGTHS:\n- The proposed concept using multiple components (e.g., learning modules, NCCs, differentiable A*) is clearly explained.  Figure 3 and Algorithm 1 are good for following the concept.\n- The solid experimental results in various motion planning domains.\n\nWEAKNESSES:\n- I have no ideas except some assumptions (the input RGC is given and how to set the upper bound Tmax for unfamiliar environments).\n\nQUESTIONS:\nI can follow the concept and descriptions in the submitted version, and feel that the contributions are solid. Here are some comments and questions to clarify the paper.\n\n- (Related to the curse of dimensionality?): In the results in Table 1, KUKA7 seems to have a bit worse success rate than those KUMA14. Do you have any reasons for them? I feel that the problems having a larger DoF are harder than those with a lower DoF.\n    - Regarding this point, Table 2 and Table 3 are reasonable for me because they have more DOFs. However, only for the success rates in Table 1, the results should be explained in my opinion (maybe, they are some random effects? If so, it is better to mention them, e.g., random seeds, trials, etc.).\n- (About the environment $G$): In the setting, RGG $G$ seems to be given and updated lazily. Can we learn G itself together with other learnable modules?\n    - In the reported results, LNR seems to be effective in refining paths after A* is finished. Since the current LNR seems independent from the learning part, I guess that the performance depends on the given RGG; therefore, I conjecture that learning representations on graphs is an interesting direction (e.g., learning a mask matrix to prune nodes).\n- (About the learning setting): How can we set the max transition time $T_\\mathrm{max}$ in the end-to-end learning procedure? What happens if $T_\\mathrm{max}$ is not set adequately?\n    - I feel that putting the larger $T_\\mathrm{max}$ is enough to find feasible solutions (i.e., a path that reaches $v_g$), though it may cause some inefficient phenomena in practice. Do you have any evaluations and discussions?\n- (Neural heuristic estimators): The comparison is a bit trivial because vanilla A* adopts the fundamental heuristic functions. Do you have any discussions on Maze2 (where Valina A* showed better path costs)? Do you have any findings of learning heuristic values in your domains?\n- Other minor comments:\n    - I prefer using explicitly an indicator vector $\\mathbb{I}[\\mathbf{b}_\\mathbf{acc} > \\mathbf{b}’_\\mathbf{acc}]$ to $\\mathbf{b}_\\mathbf{acc} > \\mathbf{b}’_\\mathbf{acc}$ for clarity and following other literature in the ML.\n    - $\\mathbb{1}$ should be with its dimension (e.g., $\\mathbb{1}_{|V|}$) for clarity.\n\nLIMITATIONS:\nI have no explicit concerns about the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an improved graph searching technique, called GraphMP. The algorithm consists of two modules, the neural collision checker and the neural heuristic estimator, which are utilised by a differentable graph-based A* module for path planning.\n\nSTRENGTHS:\n1. The algorithm is presented in detail, supplemented by toy-case studies shown in Fig.1 and Fig.2. They are illustrative. \n2. The structure of Secion 3 is good. Each module is explained. \n3. The proposed algorithm is compared to multiple existing algorithms, including non-learning-based algorithms.\n\nWEAKNESSES:\n1. I think Section 3.1 needs be further polished. Currently, it only states the problem to be solved. Necessary knowledge about how prior works finished the task, and which parts in the existing framework are to be improved are missing. (I can see some of the information in the supplementary file. But I suggest put them in the main text. )\n\n2. Table 1 is not illustrative. The algorithms such as BIT* and RRT* have probabilistic completeness, hence the success rate is high if the computational time is sufficiently long. Currently we cannot appreciate the goodness of the proposed algorithm regarding the completeness. I suggest the authors present the set time limit for the experiments in Table 1 (I didn't find it in Section 4.2). Then, as such, the authors should reduce the time limit to its 75% and show the algorithms' performance. Some of the existing algorithms will not have such good results as shown in Table 1 now. \n\n3. Just a minor comments: It looks like in Section 3.1 the problem is stated in the similar form to what RRT* did, but the algorithm is claimed as a searching-based algorithm. This is a little bit strange, because if the RGG is pre-constructed before running the path searching algorithm, then the presentation can safely start from \"give a graph, ..\". \n \n4. I didn't understand the \"lazy node removal\" module. Why is it required? The performance of the algorithm without LNR is not explained well in Section 4.3. \n\n5. The meaning of the terms \"w/\" and \"w/o\" are not intuitive. I suggest change them.\n\nQUESTIONS:\nI hope the authors can answer my comments above, and tell me how they will improve the manuscript.\n\nLIMITATIONS:\nI don't think the paper has a potential negative societal impact. I hope the authors can summarise the limitation and briefly talk about the future of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends the previous GNN-Explorer work and replaces its greedy search strategy with A* search using a neural heuristic. Furthermore, it utilizes the neural collision checker and lazy node removal to improve the success rate and path cost. The result is evaluated based on a benchmark from 2D maze to 14D dual arms and shows that it outperforms the baselines.\n\nSTRENGTHS:\n1. The approach is clear and easy to understand.\n2. The extensive results on 6 environments seem thorough and show the soundness of the approach. Especially for UR5, the planning time is fast.\n3. The ablation studies show the effectiveness of each key component.\n\nWEAKNESSES:\n1. The novelty is incremental. All the key components can be found in previous works, including graph-based search (from GNN-Explorer), neural heuristic and differential training (from neural A*), neural collision checker (from Fastron and ClearanceNet), and lazy node removal (from Motion Planning Networks). \nHowever, combining them and showing validness is also important, therefore, I recommend borderline acceptance.\n\nQUESTIONS:\n1. I wonder how sensitive the algorithm is to the hyperparameter θ. As claimed in the paper, this parameter balances the trade-off between the planning efficiency and the path safety. So could you please give an ablation study on how robust the algorithm is to the choice of θ?\n\n2. Do you mind providing the TP, TF, NP, NF scores for Figure 5 (Left)? I would love to know more about whether this neural collision checker is overestimating or underestimating the collision risk.\n\n3. During training, does the NHE take the graph input that is predicted and filtered by the NCC, or does it completely take a collision-free RGG? If it is the later one, then there might be some distribution shift happening during inference.\n\nLIMITATIONS:\nPlease discuss the limitations, since I do not see the texts mentioning the limitations. An obvious limitation is that the algorithm is not complete. For example, it is possible that the neural network predicts an edge to be collision-free with very high confidence (say 90%), and the algorithm could find a path that is actually in collision because of it.\n\nI do not see any noticeable negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents GraphMP, a neural motion planner that uses GNNs and graph search techniques to do motion planning in various scenarios. GraphMP has two components: a neural collision checker that estimates the collision status of edges in a randomly sampled graph, and a neural heuristic estimator that assigns heuristic values to nodes for directing the graph search. This paper also develops a graph-based A* algorithm that allows end-to-end learning of the neural heuristic estimator. Thi paper evaluates GraphMP on six planning tasks with dimensions from 2D to 14D and demonstrates that it performs better than several classical and learning-based planners in terms of path quality, planning speed and success rate.\n\nSTRENGTHS:\n- This paper provides extensive experiments and comparisons with baselines on various environments and metrics. It shows strong experimental results in many planning settings. \n- The paper leverages the advantages of both GNNs and graph search algorithms to improve performance over existing methods like GNN-Explorer. \n- Effective techniques like in-search differentiable graph-based A*, collision check and lazy nod removal are added to improve the performance of GraphMP.\n\nWEAKNESSES:\n- This paper does not study or show how different factors and design choices affect the performance of the proposed GraphMP algorithm, including\n  - number of neighbors (K) in K-NN (was set to 10 and 20 in different experiments)\n  - the graph size\n  - the in-search collision check threshold\n- This paper does not mention the challenges or drawbacks of GraphMP, such as how it deals with dynamic or uncertain settings, or how it adapts to more complicated tasks.\n- It will be helpful to provide more information about how the neural heuristic estimator and the neural collision checker are trained.\n\nQUESTIONS:\n- For A* to work provably, how to ensure that the neural heuristic estimator produces admissible or consistent heuristic values?\n- How to handle non-reachable goal states?\n- Can a discussion about different design choices in the algorithm be included?\n\nLIMITATIONS:\nPlease include a limitation section", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}