SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xiao Zang $^{1}$  Miao Yin $^{2*}$  Jinqi Xiao $^{1}$  Saman Zonouz $^{3}$  Bo Yuan $^{1}$
$^{1}$ Department of Electrical and Computer Engineering, Rutgers University  $^{2}$ Department of Computer Science and Engineering, The University of Texas at Arlington
<sup>3</sup>School of Cybersecurity and Privacy, Georgia Institute of Technology
$^{1}\{xz514, jx257, bo.yuan.ece\} @rutgers.edu$
$^{2}$ miao.yin@uta.edu  $^{3}$ szonouz6@gatech.edu
Motion planning, which aims to find a high-quality collision-free path in the configuration space, is a fundamental task in robotic systems. Recently, learning-based motion planners, especially the graph neural network-powered, have shown promising planning performance. However, though the state-of-the-art GNN planner can efficiently extract and learn graph information, its inherent mechanism is not well suited for graph search process, hindering its further performance improvement. To address this challenge and fully unleash the potential of GNN in motion planning, this paper proposes GraphMP, a neural motion planner for both low and high-dimensional planning tasks. With the customized model architecture and training mechanism design, GraphMP can simultaneously perform efficient graph pattern extraction and graph search processing, leading to strong planning performance. Experiments on a variety of environments, ranging from 2D Maze to 14D dual KUKA robotic arm, show that our proposed GraphMP achieves significant improvement on path quality and planning speed over state-of-the-art learning-based and classical planners; while preserving competitive success rate.
Motion planning aims to find a high-quality collision-free path connecting the start and goal states in the configuration space of a robot. As a fundamental cognitive task in robotic systems, motion planning plays a critical role in many practical applications, such as autonomous driving, in-household package handling and assisted surgery, etc. A motion planning problem can be solved from different perspectives. Sampling-based solutions, e.g., RRT and its variants [1, 2] randomly sample the configuration space to build a space-filling tree, which grows towards connecting the start and goal configurations. Search-based approaches, such as  $\mathrm{A}^*$  and Dijkstra [3, 4], interpret the planning as a graph search problem, and they then find the feasible path via traversing the graph.
Recently, learning-based planners have obtained substantial attention because of their comparable or superior performance to classical planners. In general, such data-driven strategy can efficiently learn the patterns in the configuration space and/or the behaviors of the oracle planners, optimizing the important operation (e.g., sampling mechanism) in the planning process, and thus reducing the unnecessary collision check with the improved path quality. To date, various neural network-powered motion planners, including multilayer perceptron (MLP)-based [5], convolutional neural networks (CNN)-based [6], recurrent neural network (RNN)-based [7] and graph neural network (GNN)-based [8, 9], have been proposed in the literature. Among these different choices, the GNN-based neural planner is the most promising approach because of its unique advantages in processing
*This work was done when the author was with Rutgers University.
Figure 1: The comparison between the key ideas of GNN-Explorer and our GraphMP. The finally identified paths are marked as green lines. The GNN-Explorer first predicts the exploration priority of all edges and then incrementally expands the exploration tree with collision-free edges in sorted order. On the other hand, our GraphMP predicts both the collision status of edges and the heuristic values of all nodes using two neural networks, and utilizes the edge weights to perform the  $\mathbf{A}^*$  search. For each step of the  $\mathbf{A}^*$  search, we mark the connection to the candidate nodes with red lines. It is seen that our method fully leverages the information of real accumulated costs during the search procedure, leading to a higher quality of the searched path.
challenging high-dimensional planning tasks. By sampling the continuous configuration space and generating the random geometric graph (RGG), the equipped GNN, which is naturally powerful for learning graph patterns, can be applied to predict the priority of nodes (configurations) [9] or edges (robot movement) [8], improving the efficiency and quality of path exploration. In particular, GNN-Explorer [8], as the most recent and representative GNN planner, has demonstrated superior performance in a series of low and high-dimensional planning tasks than various learning-based and classical planners.
However, we believe the potential of GNN-based motion planners is not fully unleashed and the state-of-the-art solution can be further improved. Our key observation is that the edge priority-based path exploration, by its nature, is not the optimal choice for solving graph search problems. For instance, as illustrated in Figure 1, neither the edge priority generation nor the path construction process fully considers the impact on the total path cost when selecting a new edge to the current path. In other words, the critical information of the accumulated cost is not properly represented, leveraged or maintained in the existing GNN-based planners, thereby limiting the quality of the finally searched path. On the other hand, the classical graph search approaches, such as  $\mathbf{A}^*$ , can provide the near-optimal solution for finding the shortest path over the weighted graph, if the suitable heuristic function can be identified. Essentially, the key factors for the success of  $\mathbf{A}^*$ -like approaches in the graph search task are 1) they fully explore the graph structure via routinely visiting the neighboring nodes; and 2) they formally consider the impact of the accumulated cost during path exploration; and these two features are exactly what the existing GNN and most of other learning-based planners lack.
Motivated by these insights, in this paper we propose to simultaneously leverage the advantages of GNN and graph search algorithm, leading to an efficient neural motion planner, namely GraphMP. GraphMP first effectively extracts and learns the important patterns in the configuration space via its GNN modules, and then identifies the near-optimal path over the processed RGG using its low-cost graph search component. More specifically, to enable this processing pipeline, a GNN-based neural collision checker and a neural heuristic estimator are proposed to extract key graph information from input RGG and provide it to A* module, which is also reformulated in the differentiable way to facilitate end-to-end training. More details of GraphMP is shown in Figure 2 and described in Section 3. We evaluate GraphMP and other learning-based and classical planners in a variety of environments, ranging from 2D Maze to 14D dual KUKA robotic arm. Experimental results show that our proposed GraphMP achieves significant improvement on various planning performance metrics (i.e., path quality and planning speed) over the state-of-the-art planning methods; while preserving the competitive success rate.
Classical Planners. Today's most popular motion planners are either search-based or sampling-based. Search-based planners, including BFS [10], Dijkstra [4], and  $\mathrm{A}^*$  [3], are typically used for finding optimal paths in low-dimensional environments. Different from BFS and Dijkstra that are significantly slow in large maps,  $\mathrm{A}^*$  is an informed search algorithm that uses a heuristic function to guide the search towards the goal, significantly reducing the search space and improving the search efficiency.
Figure 2: (Left): The architectures of neural collision checker (NCC) and neural heuristic estimator (NHE). The neural collision checker takes a raw RGG and obstacles as input and predicts the collision status of all edges. The neural heuristic estimator takes the collision-free RGG and the goal as input and predicts the heuristic values of all nodes. (Top right): The independent training phases of NCC and NHE. Notice that here we design and utilize a differentiable graph-based  $\mathrm{A}^*$  concatenated to NHE, enabling their joint training in an end-to-end manner. (Bottom right): The main steps of the inference phase to solve the planning task. Once a valid solution is found, we perform the lazy node removal to further reduce the path cost.
Motivated by this benefit, many variants of A* have been developed, such as D* [11], WA* [12] and Hybrid-A* [13]. The heuristic functions adopted in these methods are typically admissible hand-craft versions, such as Euclidean distance and Manhattan distance.
Unlike search-based planners, sampling-based planners probe the high-dimensional configuration space with random sampling, alleviating the exponential complexity growth incurred by the increasing number of dimensions. In general, this type of planner first constructs a randomly sampled graph, and then explores the feasible path over the sampled graph. One important sub-category of sampling-based planner is tree-based solution, including RRT [1], RRT* [14] and Informed-RRT* [15], etc. Some other popularly used sampling-based planners include PRM [16], BIT* [2] and LazySP [17].
Learning-based Planners. Recently, a series of learning-based methods further extend the search-based planners by producing better heuristics to guide the search. For instance, SAIL [18] trains heuristic policies by imitating clairvoyant oracles and demonstrates the capability of reducing search effort. [19] and [20] learn the heuristic functions using the U-Net [21] and transformer [22] architecture, respectively. On the other hand, VIN [23] and SPT [24] learn the planning policy directly, and [25] develops the differentiable solvers for integer linear optimization problems by treating them as black boxes. Furthermore, Neural Weighted A* [26] learns the image-format graph costs and heuristics simultaneously via being supervised under the planning examples. Neural A* [27] proposes to reformulate the entire procedure of the canonical A* search algorithm to be differentiable, enabling the end-to-ending training of a guidance map for 2-D path planning tasks. However, this method still relies on a hand-crafted heuristic function and can only be applied to 2D images, hindering its applications.
In addition, a variety of deep learning-powered sampling-based planners have also been proposed in the literature. Fastron [28] and ClearanceNet [29] learn the function approximators to facilitate collision detection. On the other hand, LEGO [30] and [31] predict the high-quality sampling distribution using conditional variational auto-encoder (CVAE), reducing the exploration effort. MPNet [5] and STP-Net [7] predict the next sample directly by encoding the environments together with the configurations via the contractive auto-encoder (CAE) and convolutional layers, respectively. In particular, motivated by their powerful capability of learning the topology of environments of any dimension, the application of GNNs in motion planning has gained popularity in recent years. [9] introduces a GNN-based sampler to predict the critical node in a randomly generated graph, showing promising performance for identifying the optimal samples in high-dimensional tasks. [8] proposes GNN-Explorer, which prioritizes the exploration of graph edges in the planning process. Since the planner can now focus on the most promising paths first, the planning efficiency of GNN-Explorer is significantly improved, outperforming various classical and learning-based planners.
3 Our Approach: GraphMP
Consider a motion planning problem defined in the  $d$ -dimensional continuous configuration space  $\mathcal{X}$ , where  $\mathcal{X}_{obs}$  is the obstacle space and  $\mathcal{X}_{free} = \mathcal{X} \setminus \mathcal{X}_{obs}$  is the free space. Given an RGG  $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ $\mathcal{V}$ , where  $\mathcal{V}$  is the set of nodes sampled from  $\mathcal{X}_{free}$  and  $\mathcal{E}$  is the set of weighted edges constructed by K-nearest neighbors (KNN). A motion planner aims to find a collision-free path that connects the source node  $v_{s}$  and goal node  $v_{g}$  by a collision-free path. The prior work GNN-Explorer [8] initializes an exploration tree  $\mathcal{T}$  rooted from  $v_{s}$ , and visits edges  $e \in \mathcal{E}$  sorted by their predicted priorities. Each visit performs the accurate collision check on the edge and tries to append it into  $\mathcal{T}$ , until  $v_{g}$  is reached. Here the search of GNN-Explorer relies on the predicted edge priorities without fully exploring graph structure and formally consider the impact of accumulated path cost during the path exploration, thereby hampering the path quality. On the other hand, graph-based  $\mathrm{A}^*$  computes the path from  $v_{s}$  by iteratively selecting the best candidate  $v_{sel} = \arg \min_{v \in \mathcal{O}} (g(v) + f(v))$  from the open list  $\mathcal{O}$ . Here,  $g(v)$  accumulates the actual cost from  $v_{s}$  to  $v$  and  $f(v)$  estimates the heuristic value of the cost from  $v$  to  $v_{g}$ . Once  $v_{sel}$  is selected, each of its reachable neighboring nodes  $v_{nbr} \in \mathcal{V}_{sel}$  is checked and updated. By using this way,  $\mathrm{A}^*$  can better explore graph scuture and perform cost-aware search, but it requires manual design of high-quality heuristic function  $h(v)$ , which is a challenging task for high-dimensional tasks. Also, visiting the neighboring nodes can be time-consuming because the accurate collision check must be performed on each edge  $e_{v_{sel}, v_{nbr}}$ . To overcome these limitations of prior works, GraphMP proposes to use GNN to extract and learn the important patterns of RGG, and then identifies the near-optimal path using learnable graph search component. More specifically, a neural collision checker and a neural heuristic estimator are proposed to extract key graph information from input RGG and provide it to the proposed reformulated differentiable  $\mathrm{A}^*$  module for end-to-end training. Therefore, the path planning is now a graph structure-aware and cost-aware process with powerful graph pattern extraction capability and learnable heuristics function, making it can achieve high planning performance and suitable for both low and high-dimensional tasks.
3.2 Overall Framework
Fig. 2 shows the overall architecture of the proposed GraphMP. In the training phase, a neural collision checker takes the raw RGG and obstacle information as input and predicts the collision status of all edges. The difference between the estimated and ground-truth collision-free RGGs serves as the training loss to improve the prediction quality of the neural collision checker. Meanwhile, a neural heuristic estimator is also individually trained to assign the proper heuristic value to each node in the ground-truth collision-free RGG, making the graph search-based planning become possible. Here in order to make the neural heuristic estimator learnable, its predicted heuristic values are sent to  $A^*$  module to generate a list of nodes that should be visited. The difference between this predicted node list and the optimal path is then calculated as the training loss, guiding the update of neural heuristic estimator. Notice that the  $A^*$  module in the training phase is designed in a differentiable way to enable the backward propagation of gradients. More details will be described in Section 3.5.
After individually training neural collision checker and neural heuristic estimator, these two modules are then concatenated in the inference phase. More specifically, neural collision checker first predicts the potential collisions existed in the RGG and generates a predicted collision-free RGG. The neural heuristic estimator then assigns the estimated heuristic value to each node of this roughly collision-free RGG, enabling the graph search-based planning via a  $\mathbf{A}^*$  module. Notice that because in the inference phase the input RRG of  $\mathbf{A}^*$  is only approximately collision-free without guarantee, a non-differentiable classical  $\mathbf{A}^*$  is adopted here to incorporate the potentially additional collision check during the search procedure. After the nodes of a collision-free path are finally identified, a lazy node removal (details in Section 3.5) is performed to further improve path quality.
3.3 Neural Collision Checker
Because detecting the potential collision of an edge, by its nature, can be interpreted as a binary classification problem, we build a neural network-based collision checker to roughly predict the collision status of the edge  $e_{ij}$  in the input RGG  $G = (\mathcal{V},\mathcal{E})$  with obstacle information  $\mathcal{X}_{obs}$ :
$$
e _ {i j} \text {i s c o l l i s i o n - f r e e} = \left\{ \begin{array}{l l} \text {T r u e} & \text {i f} p _ {\mathcal {I} \left(e _ {i j}\right)} > 0. 5 \\ \text {F a l s e} & \text {o t h e r w i s e ,} \end{array} \right. \tag {1}
$$
Figure 3: An example of performing the differentiable graph-based  $\mathbf{A}^*$  to find a path from node 0 to node 4. Given a weighted graph,  $\mathbf{A}$ ,  $\mathbf{W}$  and  $\mathbf{b}_{\mathrm{heu}}$  are the corresponding binary adjacency matrix, weighted adjacency matrix and the heuristic values, respectively. At each step, we first compute the lowest-cost candidate  $\mathbf{v}_{\mathrm{sel}}$  according to Eq. 3 and explore its neighboring nodes  $\mathbf{v}_{\mathrm{nbr}}$  via Eq. 5. The accumulated cost  $\mathbf{b}_{\mathrm{acc}}$  is updated via Eq. 7. Besides, we keep track of the nodes to be visited and have been visited by updating  $\mathbf{o}$  and  $\mathbf{c}$  via Eq. 4 and Eq. 8. Once the goal is reached, all the visited nodes are recorded in the closed list vector  $\mathbf{c}$ .
where  $\mathcal{I}(e_{ij})$  is the function that gives the index of  $e_{ij}$  in the edge set  $\mathcal{E}$ , and  $\mathbf{p} = (p_0, p_1, \dots, p_{|\mathcal{E}| - 1})$  is the collision-free probability vector that can be calculated as:  $\mathbf{p} = f_{prob}(f_{oe}(\mathcal{V}, \mathcal{E}, \mathcal{X}_{obs}))$ . Here  $f_{oe}$  is the attention-based iterative obstacle encoder [8] that encodes the information of input RGG and obstacles to a concatenated embedding vector  $\mathbf{u} = (x_i^{(l)}, x_j^{(l)}, x_i^{(l)} - x_j^{(l)}, y_i^{(l)})$  for each edge  $e_{ij}$ , where  $x_i^{(l)}$  and  $y_{ij}^{(l)}$  denote the node embedding of  $v_i$  and edge embedding of  $e_{ij}$  at the  $l$ -th iteration of obstacle encoding, respectively.  $\mathbf{u}$  is then processed by a three-layer MLP  $f_{prob}$  to calculate the collision-free probability for each edge  $e_{ij}$ . More details of  $f_{oe}$  and  $f_{prob}$  are in the Appendix.
Training Procedure. In the training phase the neural collision checker is individually trained on batches of collision check problem instances  $\{(\mathcal{V}^{(i)},\mathcal{E}^{(i)}),\mathcal{X}_{obs}^{(i)}\}$ , where the collision-free probabilities of all the collision-free edges in the  $i$ -th problem instance is set as 1, otherwise 0. The accurate labeling on the training data is provided by the oracle collision checker. The difference between the estimated collision-free probability vector  $\mathbf{p}$  and the binary representation of the ground-truth label, is measured via calculating binary cross-entropy (BCE) loss and minimized during the training procedure to improve the prediction quality of neural collision checker.
3.4 Neural Heuristic Estimator
Given the input collision-free RGG  $\mathcal{G}_{free} = (\mathcal{V},\mathcal{E}_{free})$  and goal node  $v_{g}$ , the neural heuristic estimator is constructed as a graph neural network (GNN)-based module to predict the heuristic value  $b_{heu,i}$  for each  $v_{i}\in \mathcal{V}$ . To that end, two MLPs (detailed in the Appendix) are first used to embed the information of  $v_{i}$  and  $e_{ij}$  to latent space as  $q_i^{(0)}\in \mathbb{R}^{d_h}$  and  $r_{ij}^{(0)}\in \mathbb{R}^{d_h}$ , respectively, where  $d_h$  is the embedding size, and then the GNN iteratively updates the node and edge embeddings via aggregating the local information of each node from its neighbors  $\mathcal{N}(v_i) = \{v_j|e_{ij}\in \mathcal{E}_{free}\}$  as follows:
$$
m _ {i} ^ {(l)} = \max (\{f _ {q} (q _ {i} ^ {(l)}, q _ {j} ^ {(l)}, q _ {j} ^ {(l)} - q _ {i} ^ {(l)}, r _ {i j} ^ {(l)}) | v _ {j} \in \mathcal {N} (v _ {i}) \}), q _ {i} ^ {(l + 1)} = g (q _ {i} ^ {(l)}, m _ {i} ^ {(l)}), \forall v _ {i} \in \mathcal {V},
$$
$$
r _ {i j} ^ {(l + 1)} = \max  \left(r _ {i j} ^ {(l)}, f _ {r} \left(q _ {i} ^ {(l)}, q _ {j} ^ {(l)}, q _ {j} ^ {(l)} - q _ {i} ^ {(l)}\right)\right), \forall e _ {i j} \in \mathcal {E} _ {\text {f r e e}}. \tag {2}
$$
Here,  $g$ ,  $f_{q}$  and  $f_{r}$  are two-layer MLPs with output dimension as  $d_{h}$ . After  $L$  iterations, the heuristic value of node  $i$  is calculated via encoding the node embedding  $q_{i}^{(L)}$  as  $b_{heu,i} = f_{val}(q_{i}^{(L)})$ , where  $f_{val}$  is a three-layer MLP.
3.5 Differentiable Graph-based A*
In order to ensure that the neural heuristic estimator can provide good estimation of heuristic value for graph search-based planning, in the training phase it is concatenated to the  $\mathrm{A}^*$  module, allowing the final planning result to directly guide the training procedure. However, a challenging issue for enabling such end-to-end learning is the non-differentiability of  $\mathrm{A}^*$ , incurred by the discrete nature of its incremental search procedure. Notice that though [27] proposes a differentiable grid-based  $\mathrm{A}^*$ , this existing solution can only work for 2-D grid search and planning; while addressing the non-differentiability of the more general graph-based  $\mathrm{A}^*$ , which is used for our proposed GraphMP towards high-dimensional planning, is non-trivial and not explored yet.
Next we describe the key reformulation for differentiable graph-based  $\mathbf{A}^*$ , which is illustrated in Fig. 3. Recall that the essence of  $\mathbf{A}^*$  on  $\mathcal{G}_{free} = (\mathcal{V},\mathcal{E}_{free})$  is to iteratively select and move the node associated with the lowest path cost from open list  $\mathcal{O}$ , which contains all the currently candidate nodes, to the closed list  $\mathcal{C}$  that stores the visited nodes, and explore the neighbors of the selected nodes to expand  $\mathcal{O}$  (More details of  $\mathbf{A}^*$  are described in the Appendix). To make this procedure differentiable, we first use two binary vectors  $\mathbf{o}\in [0,1]^{|\mathcal{V}|}$  and  $\mathbf{c}\in [0,1]^{|\mathcal{V}|}$  to represent the nodes information in  $\mathcal{O}$  and  $\mathcal{C}$ , respectively, where the 1 entries indicate the contained nodes. Also, the accumulated costs and the estimated heuristic values of all the nodes in  $\mathcal{V}$  are denoted as real-valued  $\mathbf{b_{acc}} = (b_{acc,0},b_{acc,1},\dots,b_{acc,|\mathcal{V}| - 1})$  and  $\mathbf{b_{heu}} = (b_{heu,0},b_{heu,1},\dots,b_{heu,|\mathcal{V}| - 1})$ , respectively, where  $b_{heu,i}$  is predicted by the neural heuristic estimator. Then, the operations of selecting the lowest-cost node in  $\mathcal{O}$  and moving it from  $\mathcal{O}$  to  $\mathcal{C}$  can be vectorized via using a one-hot vector  $\mathbf{v_{sel}}\in [0,1]^{|\mathcal{V}|}$  as follows:
$$
\mathbf {v} _ {\text {s e l}} = \mathcal {I} _ {\max } \left(\frac {\exp \left(- \left(\mathbf {b} _ {\text {a c c}} + \mathbf {b} _ {\text {h e u}}\right) / \lambda\right) \odot \mathbf {o}}{\exp \left(- \left(\mathbf {b} _ {\text {a c c}} + \mathbf {b} _ {\text {h e u}}\right) / \lambda\right) \mathbf {o}}\right), \tag {3}
$$
$$
\mathbf {o} = \mathbf {o} - \mathbf {v} _ {\text {s e l}}, \mathbf {c} = \mathbf {c} + \mathbf {v} _ {\text {s e l}}. \tag {4}
$$