SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here similar to [27], element-wise product  $\odot$  is used to mask the nodes in  $\mathcal{C}$ , and  $\lambda$  and  $\mathcal{I}_{\max}(.)$  are the pre-set parameter and the function returns a one-hot vector with the "1" entry associated with the lowest path cost  $(b_{acc,i} + b_{heu,i})$ .  $\mathbf{v}_{\mathbf{sel}}$  is then further used to vectorize the neighborhood exploration process. To be specific, let  $\mathbf{A} \in \{0,1\}^{|\mathcal{V}| \times |\mathcal{V}|}$  and  $\mathbf{W} \in \mathbb{R}^{|\mathcal{V}| \times |\mathcal{V}|}$  denote the unweighted and the weighted adjacency matrix of  $\mathcal{E}_{free}$ , respectively. Accordingly, the neighboring nodes that should be explored can be represented as
$$
\mathbf {v} _ {\mathbf {n b r}} = \mathbf {A} \mathbf {v} _ {\mathbf {s e l}} \odot (\mathbb {1} - \mathbf {c}), \tag {5}
$$
where  $\mathbf{v}_{\mathbf{nbr}} \in [0,1]^{|\mathcal{V}|}$  is the binary vector marking the entries corresponding to the neighboring nodes as ones, and  $\mathbb{1}$  is the all-one vector. Notice that here the indices of the neighboring nodes that are already in the closed list  $\mathcal{C}$  are masked out via using  $(\mathbb{1} - \mathbf{c})$ . Upon the identification of these newly explored neighboring nodes, the accumulated costs  $\mathbf{g}$  and the vectorized open list  $\mathbf{o}$  are further updated as:
$$
\mathbf {b} _ {\mathbf {a c c}} ^ {\prime} = \mathbf {b} _ {\mathbf {a c c}} \odot \mathbf {v} _ {\mathbf {s e l}} + \mathbf {W} \mathbf {v} _ {\mathbf {s e l}}, \tag {6}
$$
$$
\Phi = \left(\left(\mathbb {1} - \mathbf {o}\right) + \mathbf {o} \odot \left(\mathbf {b} _ {\mathbf {a c c}} > \mathbf {b} _ {\mathbf {a c c}} ^ {\prime}\right)\right) \odot \mathbf {v} _ {\mathbf {n b r}}, \quad \mathbf {b} _ {\mathbf {a c c}} = \mathbf {b} _ {\mathbf {a c c}} \odot \left(\mathbb {1} - \Phi\right) + \mathbf {b} _ {\mathbf {a c c}} ^ {\prime} \odot \Phi , \tag {7}
$$
$$
\mathbf {o} = \mathbf {o} + (1 - \mathbf {o}) \mathbf {v} _ {\mathbf {n b r}}, \tag {8}
$$
where  $\mathbf{b}_{\mathrm{acc}}^{\prime}$  denotes the accumulated costs of the nodes that are associated with the path containing the selected node. The operation  $\mathbf{b}_{\mathrm{acc}} > \mathbf{b}_{\mathrm{acc}}^{\prime}$  compares the node-wise accumulated costs and yields a binary vector of length  $|\mathcal{V}|$ . More specifically,  $\mathbf{b}_{\mathrm{acc}} \odot \mathbf{v}_{\mathrm{sel}}$  extracts the accumulated cost of the currently selected node, and  $\mathbf{W}\mathbf{v}_{\mathrm{sel}}$  represents the distance from the selected node to each of its neighbors. Notice that as shown in Eq. 7, the accumulated cost of neighboring nodes are updated in two cases: (1) the nodes do not exist in  $\mathcal{O}$ ; and (2) the nodes are in  $\mathcal{O}$  but its current accumulated cost is already smaller than the updated one. The identification of such two types of nodes are realized by a binary vector  $\Phi$ . After updating the accumulated costs, the open list  $\mathcal{O}$  is expanded by adding newly explored neighboring nodes (Eq. 8).
End-to-End Training. With the differentiability of graph-based  $\mathrm{A}^*$ , neural heuristic estimator and graph search planning can be now jointly trained in an end-to-end way, providing better guidance of learning heuristic values. Algorithm 1 describes the overall training procedure.
Algorithm 1 End-to-End Training Framework
Input: Full training set  $\mathcal{D}_{heu}$ , neural heuristic estimator  $f_{heu}(\mathcal{V},\mathcal{E},v_g,\Theta_{heu})$  with weights  $\Theta_{heu}$ , the max iteration  $T_{max}$ , learning rate  $\gamma$   
1: for  $(\mathcal{V}^{(i)},\mathcal{E}_{free}^{(i)},v_s^{(i)},v_g^{(i)},\hat{\mathbf{c}}^{(i)})$  in  $\mathcal{D}_{heu}$  do  
2: Predict  $\mathbf{b}_{heu} \gets f_{heu}(\mathcal{V}^{(i)},\mathcal{E}_{free}^{(i)},v_g^{(i)},\Theta_{heu})$ . # Predict the heuristic values.  
3:  $\mathbf{b}_{acc} \gets 0$ ,  $\mathbf{o} \gets \mathbf{v}_s$ ,  $\mathbf{c} \gets 0$ .  
4: for  $t = 1,2,\dots,T_{max}$  do  
5: Select  $\mathbf{v}_{\mathrm{sel}}$  via Eq. 3.  
6: if  $\mathbf{v}_{\mathrm{sel}} == \mathbf{v}_g$  then Break #  $\mathbf{v}_g$  is the one-hot vector representing the goal  $v_g^{(i)}$ .  
7: if sum(o) == 0 then Break # Early stopping when the open list is empty.  
8:  $\mathbf{o} \gets \mathbf{o} - \mathbf{v}_{\mathrm{sel}}, \mathbf{c} \gets \mathbf{c} + \mathbf{v}_{\mathrm{sel}}$ .  
9:  $\mathbf{v}_{\mathrm{nbr}} \gets \mathbf{A}\mathbf{v}_{\mathrm{sel}} \odot (\mathbb{1} - \mathbf{c})$ .  
10:  $\mathbf{b}_{acc} \gets \mathbf{b}_{acc} \odot (\mathbb{1} - \Phi) + \mathbf{b}_{acc}' \odot \Phi$  (Eq. 6-7).  
11:  $\mathbf{o} \gets \mathbf{o} + (1 - \mathbf{o})\mathbf{v}_{\mathrm{nbr}}$ .  
12: end for  
13: #  $\hat{\mathbf{c}}^{(i)}$  is a length- $|\mathcal{V}|$  binary vector that marks all nodes along the optimal path as ones.  
14: Compute the loss  $\mathcal{L}_{heu} \gets ||\hat{\mathbf{c}}^{(i)},\mathbf{c}||_1$ .  
15: Update weights  $\Theta_{heu} \gets \Theta_{heu} - \gamma\nabla_{\Theta_{heu}}\mathcal{L}_{heu}$ .  
16: end for
(a) Key idea of LNR.
Path cost  $= 4.37$
(b) 2D maze.
Figure 4: The key idea of the lazy node removal (LNR) and the examples of applying LNR on 2D maze and 7D KUKA environments. We compare the paths before LNR (left) and after LNR (right).
Path cost  $= 4.10$
Path cost  $= 13.54$
Path cost  $= 8.90$
(c) 7D KUKA.
3.6 Inference Procedure for Online Planning
As illustrated in Fig. 2, when GraphMP performs online planning (detailed in the Appendix) for the new task, a non-differentiable classical A* module is adopted to perform graph search on the approximated collision-free RGG prepared by the neural collision checker. Here the heuristic values desired in the search procedure is predicted by the neural heuristic estimator. Notice that in order to reduce the computational cost and path cost, the following two optimization operations are adopted in the inference procedure.
In-Search Collision Check. Due to the approximation property of neural networks, some edges in the approximated collision-free RGG may collide with the obstacles. We integrate the accurate collision check into  $\mathbf{A}^*$  search that only occurs on necessary edges with low prediction confidence. Specifically, we perform the time-consuming accurate collision check only when (1) the neighbor of one selected node is explored, and (2) the collision-free probability of the corresponding edge is lower than a threshold  $\theta$ . Such operation balances the trade-off between the planning efficiency and the path safety.
Lazy Node Removal. After  $\mathrm{A}^*$  finishes the path search, lazy node removal is performed to further reduce path cost. As illustrated in Fig. 4, within the planned path, the pairs of the nodes that are not directly connected are iteratively checked to see whether the potential direct connection (as new edge) is collision-free and brings shorter path. If so, the two nodes can be connected and the nodes between them are removed as "lazy nodes". In general, the worst time complexity of lazy node removal is  $O(T^{2})$ , where  $T$  is the number of nodes in the path found by  $\mathrm{A}^*$ . Since the searched path typically contains only few nodes ( $T$  is small), this technique effectively reduces the path cost with minor computing overhead.
(a) Performance of NCC.
Figure 5: (Left): The performance of neural collision checker (NCC) on 1000 raw RGGs with 300 nodes and  $K$ -value of 10 (K-NN). (Right): The comparison between A* with the heuristic function of neural heuristic estimator (NHE) and vanilla A*, with respect to the mean path cost. More numerical results are in the Appendix.
(b) Performance of NHE.
4.1 Dataset and Experimental Setup
We demonstrate the effectiveness of our algorithm in six types of planning tasks as the same in [8]: (1) Maze2: a 2 DoF point-robot in 2D maze workspace of dense obstacles, (2) Ur5: a 6 DoF UR5 robot in 3D workspace, (3) Snake7: a 7 DoF snake robot in 2D workspace, (4) Kuka7: a 7DoF KUKA arm in 3D workspace, (5) Kuka13: a 13 DoF Kuka arm in 3D workspace, and (6) Kuka14: a pair of 7DoF KUKA arms in 3D workspace.
For each environment, we prepare two different training datasets each of which consists of 2000 different workspaces, to train the neural collision checker and heuristic estimator separately. For each workspace, we randomly construct 20 RGGs by sampling a random number of nodes ([100, 200, 300, 400]) and a random value of KNN ([5, 10, 15, 20]). Specifically, for the neural collision checker, each problem contains the raw RGG built on a set of randomly sampled free states and the different set of obstacles, while the ground-truth is the collision-free RGG obtained by the oracle collision checker. For the neural heuristic estimator, each problem contains a feasible pair of  $v_{s}$  and  $v_{g}$ , the exactly collision-free graph, and the ground-truth is the optimal path computed by Dijkstra. We also use 500 problem instances as the validation set. We keep the weights of heuristic estimator which yields the smallest average path cost on the validation set. After the training, GraphMP is evaluated in an end-to-end manner on 1000 problem instances with unseen workspaces. Each input of the testing problems contains a pair of  $v_{s}$  and  $v_{g}$ , and a set of obstacles.
Our neural collision checker adopts 3 iterations of obstacle encoding with an output dimension of 64, and the neural heuristic estimator has 5 loops of message passing in Eq. 2 with the output dimension of 32. For both training of these two models, we select ADAM [32] as the optimizer and set the learning rate as  $1e^{-3}$ . The training epoch is 400 and the batch size is set as 8. We set the threshold  $\theta$  in the in-search collision check as  $80\%$ . The number of graph nodes per sampling is 100 and the  $K$  value of K-NN is 10. The experiments are conducted on a computere quipped with an AMDEPYC 74202P24-Core Processor and an NVIDIA RTXA6000 GPU.
Baselines. We evaluate the performance of GraphMP by comparing it with three classical planners (BIT*, RRT* and LazySP) and one state-of-the-art learning-based planner (GNN-Explorer). In addition, the Smoother proposed in [8] serves as an additional step to reduce the path cost of the
Table 1: The overall success rate on 1000 testing problems for each environment. GraphMP archives nearly  $100\%$  success rate across all environments, being competitive with other baseline planners.
Maze2	UR5	Snake7	KUKA7	KUKA13	KUKA14
BIT*	1.00	1.00	1.00	1.00	1.00	1.00
RRT*	0.54	0.39	0.69	0.83	0.67	0.70
LazySP	1.00	0.99	1.00	1.00	0.99	0.99
GraphMP	1.00	0.99	1.00	0.99	1.00	0.99
GNN-Explorer	1.00	0.98	1.00	0.99	1.00	0.99
GraphMP with Smoother	1.00	0.99	1.00	0.99	1.00	0.99
GNN-Explorer with Smoother	1.00	0.98	1.00	0.99	1.00	0.99
Table 2: Mean path cost on 1000 testing problems for each environment. GraphMP with Smoother achieves the lowest path cost across all environments, compared to the baselines with a similar success rate. (The success rate of RRT* on Maze2 is  $54\%$ .)
Maze2	UR5	Snake7	KUKA7	KUKA13	KUKA14
BIT*	2.52	11.20	5.96	7.59	12.07	12.05
RRT*	1.82	10.35	5.06	7.07	10.03	10.65
LazySP	2.63	11.81	6.52	9.49	16.72	16.82
GraphMP	2.33	8.09	5.68	7.01	13.57	12.22
GNN-Explorer	2.80	12.61	6.50	9.15	16.75	16.50
GraphMP with Smoother	1.96	7.73	5.02	6.27	9.26	9.89
GNN-Explorer with Smoother	2.36	8.87	5.31	6.55	9.92	10.01
Table 3: The mean time cost (ms). GraphMP demonstrates the fastest planning speed, among 4 of 6 environments.
Maze2	UR5	Snake7	KUKA7	KUKA13	KUKA14
BIT*	125.2	541.2	161.1	407.0	352.6	205.2
RRT*	200.4	432.0	436.5	175.3	506.4	423.7
LazySP	334.0	921.8	144.7	401.8	224.5	404.9
GraphMP	124.5	192.6	211.6	59.3	116.3	95.5
GNN-Explorer	155.5	332.9	222.9	93.3	101.9	120.3
GraphMP with Smoother	140.9	363.0	268.1	65.6	142.7	120.4
GNN-Explorer with Smoother	175.7	529.3	293.0	107.3	136.2	150.7
computed path. Therefore, we also compare the performance of GraphMP with Smoother and GNN-Explorer with Smoother.
Ablation study. Details are reported in the Appendix.
4.2 Comparison With Baselines
Experimental results show that our GraphMP achieves  $98.6\%$  -  $100\%$  success rate across all the environments, being competitive to the state-of-the-art planners (Table 1). Table 2 shows that GraphMP achieves  $16.79\%$ ,  $35.84\%$ ,  $12.62\%$ ,  $23.39\%$ ,  $18.99\%$  and  $25.94\%$  shorter paths than GNN-Explorer, on Maze2, UR5, Snake7, KUKA7, KUKA13 and KUKA14, respectively. Furthermore, the GraphMP with the path smoother has the lowest path cost compared to all baselines with a competitive success rate. Table 3 compares the time cost of all the motion planners. Compared to GNN-Explorer, GraphMP finds valid solutions with  $19.94\%$ ,  $42.14\%$ ,  $5.07\%$ ,  $36.44\%$  and  $20.62\%$  lower time cost on Maze2, UR5, Snake7, KUKA7 and KUKA14, respectively. Overall, the results indicate that GraphMP outperforms all baseline planners by producing high-quality paths with the fastest planning speed and nearly  $100\%$  success rate in most of the planning tasks from 2D to 14D.
4.3 Study of Individual Modules of GraphMP
The Performance of Neural Collision Checker. We first evaluate the neural collision checker by reporting the mean prediction accuracy, confidence score and time cost. Specifically, the mean prediction accuracy and confidence are measured by averaging over all the edges from the test graphs, and the mean time cost represents the latency of predicting all edges per testing graph. Fig. 5 shows that our neural collision checker can perform super fast and accurate prediction on the graph-level collision check with high prediction confidence, in all the environments.
The Performance of Neural Heuristic Estimator. We then evaluate the performance of the neural heuristic estimator by comparing the A* with different heuristic functions. To be specific, we equip A* with our neural heuristic estimator and the Euclidean function which serves as the commonly used heuristic function to solve planning tasks, respectively. From Fig. 5, it is shown that the A* with our neural heuristic predictor significantly outperforms the vanilla A*, by producing solutions of lower path cost. The advantage of our neural heuristic predictor also grows quickly with the increasing dimensions of the planning tasks. The results illustrate that the neural heuristic estimator learns and performs better search guidance, especially in the configuration space of higher dimensions.
The Impact of In-Search Collision Check (ICC) and Lazy Node Removal (LNR). We further analyze the impact of the ICC and LNR modules on GraphMP, to justify their necessity in improving
Table 4: The comparison of the path cost and time cost (ms) between different versions of GraphMP. To be specific, the GraphMP w/o ICC uses accurate collision check only. GraphMP achieves the significantly lower path cost compared to the one without LNR with a limited time increase, and achieves reduced time cost compared to the one without ICC.
Maze2	UR5	Snake7	KUKA7	KUKA13	KUKA14
Path	Time	Path	Time	Path	Time	Path	Time	Path	Time	Path	Time
GraphMP w/o LNR	2.82	106.75	9.76	183.84	5.94	208.27	8.83	54.57	15.82	104.29	14.32	88.18
GraphMP w/o ICC	2.33	167.85	8.09	258.27	5.68	286.54	7.01	104.94	13.57	162.44	12.24	137.70
GraphMP	2.33	124.47	8.09	192.55	5.68	211.66	7.01	59.34	13.57	116.29	12.24	95.47
the planning performance. Table 4 compares GraphMP with the version without ICC and the one without LNR, with respect to the path cost and time cost, respectively. While all these planners achieve  $98.6\%$  -  $100\%$  success rate across different environments, GraphMP outperforms the version without LNR by yielding  $17.38\%$ ,  $17.11\%$ ,  $4.38\%$ ,  $21.92\%$ ,  $14.22\%$  and  $14.53\%$  shorter paths with a limited increase of the time cost, on Maze2, UR5, Snake7, KUKA7, KUKA13 and KUKA14, respectively. Besides, compared to the one that uses accurate collision check (GraphMP w/o ICC), GraphMP achieves the same path quality with a significantly smaller time cost. Overall, it is seen that ICC and LNR are effective in improving planning speed and path optimality, respectively.
Despite its good empirical performance across different tasks, GraphMP still has some limitations. First, it does not provide probabilistic completeness when the collision check threshold  $\theta < 100\%$ . That means, if the collision status of some edges is determined by the neural collision checker (NCC), even if the prediction accuracy of NCC is high and a collision-free path exists in the input RGG, GraphMP still cannot guarantee to find the feasible solution asymptotically. Notice that though the probabilistic completeness can be achieved when setting  $\theta = 100\%$ , the planning time will accordingly increase due to the extra costs incurred by performing accurate collision check on all the explored edges. Second, it does not offer asymptotical optimality. GraphMP performs the graph search on an implicit RGG which is incrementally expanded with more batches of nodes. Once a path is found, GraphMP validates its legality and returns the solution. Because 1) this mechanism naturally leads that the quality of the sampled RGGs has a heavy impact on the path cost – the waypoints along the paths are restricted to be a subset of the existing nodes of RGG, but RGG itself cannot be guaranteed to contain the optimal path; and 2) the search process will be terminated once the path is found, without further seeking better solutions, GraphMP cannot theoretically guarantee to find the optimal path asymptotically. Third, its efficiency is still limited by inefficient RGG construction. More specifically, 1) the uniform sampling of nodes disregards the environmental topology, causing some unnecessary node exploration; and 2) the construction of raw edges is also involved with unnecessary edge generation, thereby limiting the further runtime speedup provided by the proposed approach.
This work is partially supported by National Science Foundation under Grant CCF-2239945, National Science Foundation CPS and SaTC programs.
This paper proposes GraphMP, a neural motion planner for both low and high-dimensional planning tasks. With the customzied model architecture and training mechanism design, GraphMP can efficiently learn the graph pattern and process graph search, enabling strong planning performance. Experiments on a variety of environments show that GraphMP achieves significant planning speed-up and higher path quality than the state-of-the-art learning-based and classical planners.
[18] Mohak Bhardwaj, Sanjiban Choudhury, and Sebastian Scherer. Learning heuristic search via imitation. In Conference on Robot Learning, pages 271-280. PMLR, 2017.
[19] Takeshi Takahashi, He Sun, Dong Tian, and Yebin Wang. Learning heuristic functions for mobile robot path planning using deep neural networks. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 29, pages 764-772, 2019.
[20] Daniil Kirilenko, Anton Andreychuk, Aleksandr Panov, and Konstantin Yakovlev. Transpath: Learning heuristics for grid-based pathfinding via transformers. arXiv preprint arXiv:2212.11730, 2022.
[21] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.
[22] Ashish Vavwani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[23] Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. Advances in neural information processing systems, 29, 2016.
[24] Devendra Singh Chaplot, Deepak Pathak, and Jitendra Malik. Differentiable spatial planning using transformers. In International Conference on Machine Learning, pages 1484-1495. PMLR, 2021.
[25] Marin Vlastelica Pogancic, Anselm Paulus, Vit Musil, Georg Martius, and Michal Rolinek. Differentiation of blackbox combinatorial solvers. In International Conference on Learning Representations, 2019.
[26] Alberto Archetti, Marco Cannici, and Matteo Matteucci. Neural weighted a*: Learning graph costs and heuristics with differentiable anytime a. In International Conference on Machine Learning, Optimization, and Data Science, pages 596-610. Springer, 2021.
[27] Ryo Yonetani, Tatsunori Taniai, Mohammadamin Barekatain, Mai Nishimura, and Asako Kanezaki. Path planning using neural a* search. In International conference on machine learning, pages 12029-12039. PMLR, 2021.
[28] Nikhil Das and Michael Yip. Learning-based proxy collision detection for robot motion planning applications. IEEE Transactions on Robotics, 36(4):1096-1114, 2020.
[29] J Chase Kew, Brian Ichter, Maryam Bandari, Tsang-Wei Edward Lee, and Aleksandra Faust. Neural collision clearance estimator for batched motion planning. In International Workshop on the Algorithmic Foundations of Robotics, pages 73–89. Springer, 2021.
[30] Rahul Kumar, Aditya Mandalika, Sanjiban Choudhury, and Siddhartha Srinivasa. Lego: Leveraging experience in roadmap generation for sampling-based planning. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1488-1495. IEEE, 2019.
[31] Brian Ichter, James Harrison, and Marco Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7087-7094. IEEE, 2018.
[32] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.