Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding
Authors: Ruipeng Zhang
Chenning Yu
Jingkai Chen
Chuchu Fan
Sicun Gao
Venue: NeurIPS 2022 Accept (2022)

Abstract: Learning-based methods have shown promising performance for accelerating motion planning, but mostly in the setting of static environments. For the more challenging problem of planning in dynamic environments, such as multi-arm assembly tasks and human-robot interaction, motion planners need to consider the trajectories of the dynamic obstacles and reason about temporal-spatial interactions in very large state spaces. We propose a GNN-based approach that uses temporal encoding and imitation learning with data aggregation for learning both the embeddings and the edge prioritization policies. Experiments show that the proposed methods can significantly accelerate online planning over state-of-the-art complete dynamic planning algorithms. The learned models can often reduce costly collision checking operations by more than 1000x, and thus accelerating planning by up to 95%, while achieving high success rates on hard instances as well. 

Learning-based Motion Planning in Dynamic Environments Using GNNs and Temporal Encoding
Ruipeng Zhang UCSD
Chenning Yu UCSD
Jingkai Chen MIT
Chuchu Fan MIT
Sicun Gao UCSD
Abstract
Learning-bar based methods have shown promising performance for accelerating mo- tion planning, but mostly in the setting of static environments. For the more challenging problem of planning in dynamic environments, such as multi-arm assembly tasks and human-robot interaction, motion planners need consider the trajectories of the dynamic obstacles and reason about temporal-spatial interactions very large state spaces. We propose GNN-base approach that uses temporal encoding and imitation learning with data aggregation for learning both the em- beddings and the edge prioritization policies. Experiments show that the proposed methods can significantly accelerate online planning over state-of-the-an complete dynamic planning algorithms. The learned models can often reduce costly collision checking operations by more than 1000x, and thus accelerating planning by up 95%, while achieving high success rates on hard instances as well.
Introduction
Learning- based approaches can exploit patterns in the configuration space to accelerate planning with promising approaches typically combine reinforcement learning (RL) and imitation learning (IL) to learn policies for sampling ranking the options at each step of the planning process 33 5. Graph Neural Networks (GNNs) are a popular choice of representation for motion planning problems, because of their capability to capture geometric information and
Motion planning for manipulation has been longstanding challenge in robotics 22] 12
performance [|
6 4 Existing learning-basee
invariant to the permutations the sampled graph [1 24 25
Motion planning in dynamic environments, such as for multi-arm assembly and human-robot inter- action, significantly more challenging than in static environments. Dynamic obstacles produce trajectories in the temporal-spatial space, so the motion planner needs to consider global geometric constraints in the configuration space at each time step (Figure D This dynamic nature of the environment generates the much larger space of sequences of graphs for sampling and learning, and is also very sensitive to the changes one single dimension: time. A small change in the timing of the ego-robot the obstacles in two spatially similar patterns may result in completely different planning problems. For instance, the dynamic obstacle may create a small time window the ego-robot to pass through, and if that window is missed, then the topology of configuration space completely change. Consequently, we need to design special architectures that can not only encode the graph structures well, but also infer temporal information robustly. Indeed, complete search algorithms for dynamic motion planning, such as the leading method of Safe Interval Path Planning 30 focus reasoning about the temporal intervals that safe for the ego-robot. These complete algorithms typically require significantly more computation and collision checking operations compared the static setting. As it proved in 377, the computational complexity of planning with the moving obstacles is NP-hard even when the ego-robot has only a small and fixed number of degrees of freedom of movement.
(SIPP) and its variations 34 26
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Obstacle
1
Start
Goal
L=T
Figure 1: Left: A sampled graph from the configuration space. A dynamic obstacle, colored in yellow, moves over time from to / T. The goal of approach search for path on the graph connecting the start to the goal, without collision with the obstacle at any timestep. Right: A successful plan where the ego-robot (grey arm) avoids collision with the dynamic obstacle (blue arm) and reaches the goal.
We propose novel Graph Neural Network (GNN) architecture and the corresponding training algorithms for motion planning in dynamic environments. We follow the framework of sampling- based motion planning [20 19 where path planning is performed
random graphs sampled from the configuration space. The GNN takes in the following inputs: the sampled graph in the configuration space, the obstacle's trajectory in the workspace, and the current state of the ego-robot. The output is a vector of priority values on the candidate edges at the current state of the ego-robot. The encoding performed in two stages. In the first stage, we encode the graph structure using attention mechanisms 43. and also design a temporal encoding approach for the obstacle trajectories. The temporal encoding uses the idea of positional encoding from the Transformer and NeRF 53:
which encourages the neural network to capture temporal patterns from figh-frrquency input signals. In the second stage of encoding, we incorporate the ego-robot's current vertex in the configuration space, the local graph structure, and the current time-shifted trajectories of the obstacles. This two-stage structure extends the previous use of GNNs in static environments 22
25 46 and
is important for making high-quality predictions on the priority values. The entire GNN of both stages will be trained simultaneously in an end-to-end fashion. Due to the complexity of the GNN architecture, we observe that RL-based approaches can hardly train generalizable models based on the architecture, and using imitation learning with data aggregation (DAgger) the key to good performance. We utilize SIPP as the expert, first perform behavior cloning as warm-up, and then allow the ego-robot to self-explore and learn from the expert following the DAgger approach [3
We evaluate the proposed approach in various challenging dynamic motion planning environments ranging from 2-DoF 7-DoF KUKA arms. Experiments show that our method can significantly reduce collision checking, often by more than 1000x compared to the complete algorithms, which leads to reducing the online computation time by over 95%. The proposed methods also achieve high success rates in hard instances and consistently outperform other learning and heuristic baselines.
2 Related Work
Motion Planning in Dynamic Environments. Planning in dynamic environment is fundamentally difficult [37] Safe Interval Path Planning (SIPP) [4 and its variations [ 30 26 are the leading framework for motion planning dynamic environments. The method significantly reduces the
temporal-spatial search space by grouping safe configurations over period of time safe intervals, Based on this data structure, can find the optimal paths on a configuration graph. Further improvements on SIPP include leveraging state dominance [[]] and achieving anytime search (30). Recently, 26 shows SIPP also powerful in planning the paths of high-dimensional manipulators on configuration roadmaps
20 with the presence of dynamic obstacles, which is the problem of

---

interests in this paper. Other methods have been proposed for path planning in dynamic environments for mobile robots specifically
Such problems are typically more efficiently solvable because of the low-dimensional configuration space and the use of more conservative planning.
14 33
Learning-based Motion Planning. Learning-based approaches typically consider motion planning as sequential recision-making problem that can be tackled with reinforcement learning or imitation learning. With model-based reinforcement learning, DDPG-MP 18 integrates the known dynamic of robots and trains a policy network. [00 improves obstacle encoding with the position and normal vectors. OracleNet [ learns via oracle imitation and encodes the trajectory history by an LSTM [IS] Other than greedily predicting nodes that sequentially form trajectory. various approaches have been designed to first learn to sample vertices, and then apply search algorithms on the sampled learn sampling distributions to generate critical vertices from configuration design neural networks that can better handle structured inputs. searning-based approaches have also been proposed to improve collision detection [ Add and for exploration of edges on fixed graphs 41 23 For non-static environments, most works focus on multi-agent scenarios 32 that do not involve non-cooperative dynamic obstacles.
graphs [I 17 36 177
space.
Graph Neural Networks for Motion Planning. Graph neural networks are permutationally invariant to node ordering, which becomes a natural choice for learning patterns on graphs. For motion planning. 21 utilizes GNN to identify critical samples.
predicts the action for each agent in the grid-world environment using graph neural networks. 42 learns the control policy for each robot in a large-scale swarm. 4X learns the submodular action selection for continuous space. 46 learns GNN-based heuristic function for search in static environment with completeness guarantee.
24 25
3 Preliminaries
Sampling- Motion Planning with Dynamic Obstacles. We focus on the sampling-basee motion planning, in which random graph is formed over samples from the configuration space C' C I where n is the number of degree-of-firedoo for the ego-robot. The sampled vertex set V always contains the start vertex v, and goal vertex Eg. The edges in the graph G (V, E} are We assume global knowledge of the trajectories of the dynamic obstacles. We represent the trajectories of dynamic obstacles in the workspace as the vector of all the joint positions in the time window of length T > 0. The goal of the motion planning problem is to find path from v, to V/ in the sampled graph that is free of collision with the dynamic obstacles at all time steps in [0, T)
determined by r-disc or enearest-neighhee (k-NN) rules [IO 45
Graph Neural Networks (GNNs). GNNs learn representations over of vertices and edges on graphs
by message passing. With MLP networks f and g, GNN encodes the representation h (*++11 of vertex 2/2 after k aggregation steps defined as
h ( = _(h(t) (H), @l ((F(0)))) (00l()).))))))) (0),,;)) EE))
(1)
where 1,(3) F, can be some arbitrary vector of initial data for the vertex uy + is typically some permutation-invariaal aggregation function on sets, such as mean, max, or sum. We use the attention mechanism to encode the obstacle features. In the general form of the attention mechanism, there are n keys, each with dimension de: K E R=xd, each key has a value V E Rxd_ Given m query vectors Q Rmxd,,, we typical attention function Att (K Q V) for each query as (Kt Q V)= softmax(QK" (VKT +
Imitation Learning. Imitation learning aims to provide guidance to train policy without explicitly designing reward functions. Given distribution of the oracle actions Toracle. it tries to learn new policy distribution π that minimize the deviation from the oracle, i.c. x¹ argmin, D(T, Rorade) where D is the difference function between the two distributions, which can be represented as p-norm or -idivergeecee We use imitation learning to train our GNN models from the demonstration of the oracle planner. Specifically, in the task of sampling- based motion planning, the oracle predicts the priority values of subsequent edges and prioritize the one given by the oracle. However, imitation learning based on behavior cloning often suffers from distribution drift, which can be mitigated by imitation with data aggregation (DAgger) [X] With DAgger, the learner can actively query the
oracle states that are not typically provided in the expert trajectories to robustify the learned policy.
3

---

Stage
Agent
Encoder
Attention
Norm
Poreart
GNN
Enender
Mecsage Passing
Attention
Add Norm
Food Poewart
Asil Nacn
Obstacle
Encoder
Global GNN Encoder Ng
Stage
Priority
Policy
MLP
171
shired
1]2
max
shared
MLP
7]3
training
Time:
Local Planner Np
Time window
Oracle
Figure 2: The overall two-stage architecture the proposed GNN-TE. In Stage 1, we encode global information of the ego-arm and the obstacles, using attention mechanisms, and output the encoding of each edge. In Stage 2, the local planner will take in the output from Stage along with the obstacle encoding within the relevant time window, to predict the priority value of each outgoing edge. The planner will propose the edge with the highest priority value to take as the output policy.
GNN-TE: Dynamic Motion Planning with GNNs and Temporal Encoding
4.1 Overall Architecture
We design the dynamic motion planning network GNN-TE capture the spatial-temporal nature of dynamic motion planning. The forward pass the network consists of two stages. The first stage is the global GNN encoder NG that encodes the global information of the ego-robot and obstacles in the environment. The second stage is the local planner N₂ that assigns priority on edges, utilizing the encoding output of the first stage. Fig shows the overall two-stage architecture.
First-stage global GNN encoder NG. The GNN encoder NG takes a sampled random geometric graph G EV, V Vg. For n-dimensional configuration space, each vertex E - contains n-dimensional configuration component and 1-dimensional one-hot label indicating if it the special goal vertex.
The vertices and the edges first encoded into latent space with RIVixd, y R Fl where dy is the size of the encoding. Specifically, to get the feature x; for the i-th node U, E V, we æ, gar (Vs, Ug. Vi Ugo |lu 12 To get the feature yı for the 1-th edge e (vi, (g), E E, use = By (v, Uj: - 0²). The and Jy are two different two-layer MIPs. The 1.2 distance to the goal | - u serves as the heuristic information for NG.
The dynamic obstacles O form barriers on the top of the graph G, and we incorporate their trajectories to Ng to leverage the global environment and assist the planning of Np. Additionally, inform the networks about the relative time over the horizon, we incorporate temporal encoding with the obstacles. Given the obstacle position O, at time step and two-layer MLP goi obstacle encoded as O, (O,) + TE((). which adds (O,) and TE(() element-wisely, TE(() is the temporal encoding time step I, which we will discuss at Section 4.2 With the sequential property of the trajectory, we use the attention mechanism to model the temporal- spatial interactions between

---

the ego-arm and the obstacles. Concretely, the obstacles are encoded into the vertex and edge of G as:
I == x + Att (0), y ー y Att(f (0)
(2)
(3)
Taking the vertex and edge encoding z, y. the GNN Na aggregates the local information for each vertex and edge from the neighbors with the following operation with 2 two-layer MLPs fa and fy:
3; max (0,, max
maxx(f22s (fz(1z Iz, Fj Fa, M) a (Ma, 11) E)), Ve, 3/1 = max(u,, (a - Iz, Ry x:)), Ve (us, 11) E E
V
(4)
Note that here we use max as the aggregation operator to gather the local geometric information, due to its empirical robustness to achieve the order invariance [35]. The edge information is also incorporated into the vertex by adding y as the input to fa- Also, because Equation 4 is a homoge-
neous function that updates the and y in self-iterating way, can update without introducing redundant layers over multiple loops. After several iterations, the first-stage NG outputs the encoding of each vertex 2;; and edge yi-
Second-stage local planner Ny. After NG encodes the information of the configuration graph and obstacles, the second-stage local planner Np utilizes the encoding and performs motion planning. Specifically, when arriving at a vertex v; at time t, NN predicts the priority value Tl, of all the connected edges E E; with the expression 1/e = fy Ot, - O₁, - Ot, + O O1++w), where fp is an MLP. Note that in addition to the encoding of the connected edges, we also input the local obstacle encoding within time window of the current arrival time 4.. This provides local information for Np to plan towards the goal vertex, while considering the barriers of dynamic obstacles to avoid collisions. At inference time, use Np to choose the edge with the highest priority value while keeping track of the current time.
4.2 Temporal Encoding
Positional encoding is a crucial design in the Transformer architecture 44 for making use of the order of the sequence. Dynamic motion planning requires the models to infer the relative position of obstacles and how they interact with the ego-arm at each time step. So along with the positions of the obstacles in the workspace, we add temporal encoding TE(t) E ddd at each time step t [0, (0...... T). where its 2k-th and 2k 1-th dimensions are computed
Tt 2k) = sin(w 22/drEt) and TE 2k + 1) cos(w 2kk--2///mm 2k/dT=))
(5)
where w E = is a fixed frequency. We select the temporal encoding to have the same dimension as the obstacle input, and add them as the obstacle encoding before inputting into the networks NG and Np. We illustrate the overall encoding procedure on the left of Figure S
Temporal Encoding
Training Procedures
Time
Dataset
Oracle Dataset Aggregated
Obstade Trajectory
M2
24
Obstaele Encoding
Training Plane 2
Tempural
N
Encoding
Training Phose
Figure 3: Left: Temporal encoding is incorporated when representing the dynamic obstacle sequence. Right: Training procedures with DAgger. The proposed GNN is first trained to imitate an optimal oracle, then improves itself by self-exploring the data with feedback from the oracle.
5

---

4.3 Training and Inference Procedures
In each training problem, along with the dynamic obstacles O start vertex U. and goal vertex Vgg we sample a k-NN graph G (V, F), V {va, "gg "g>"), where is the vertices sampled from the configuration space of the ego-arm. In the first stage, we use the global GNN encoder NG to encode the graph and dynamic obstacles, and the local planner N₁, in the second stage uses the encoding as the input to predict the priority value of the subsequent edges.
Imitation from SIPP with Data Aggregation. We train our two-stage network NN and Mip in an end-to-end manner by imitating an oracle. Specifically, we use Safe Interval Path Planning (SIPP) to compute the shortest non-collision motion path and use it as the oracle.
In the first stage, NG will process the graph and the obstacle trajectories, then output the encoded features of each vertex and edge. Then in the second stage, we train the networks to imitate the oracle SIPP along the optimal path. Concretely, starting from the vs. SIPP provides the optimal path n² = [(v3, ft) hefo..) with the vertex Eद and the corresponding arrival time t,. When arriving the vertex of at time f,, the local planner will take in the edge feature Ye, along with the obstacle encoding in the time window [t: +i to predict the priority value of all the subsequent edges E. Then it prioritizes the next edge on the optimal path oj : ((07, ()) (of4 1)) among E,. We maximize the priority value :e; of e over all other e, E, \ (et) } with the standard cross entropy loss L
(og(e(((ne))/CC
exp(
Since SIPP only provides the guidance on the optimal path, when the planned path given by Nv deviates from the optimal path, our network cannot sufficiently imitate the oracle. To this end, we use DAgger 63 to encourage the network to learn from these sub-optimal paths. We first train our network for k iterations with pure demonstrations from SIPP. Then we explore a path n² on the graph using the priority value predicted by the current network, which may not reach the goal vertex v nor be optimal. We randomly stop at the vertex of at time 1/ and query the oracle. SIPP treats of and th as the start vertex and the start time respectively, along with the obstacles trajectory starting at t calculates the optimal path. The new demonstrations are aggregated to the previous dataset to keep training the network. The training procedures are showed on the right of Figure B
Inference with Time Tracking. Given graph G = =V, V = {uy, Ug v}. with the trajectories of obstacles O, Ne will first encode the graph and obstacles. Next, NNp executes motion planning by predicting the priority values NV, V E,, O,NG, 44) when arriving at vertex vi at time 4, and follow the edge with the maximum one, i.e. ex; argmax E,E, lle-- After the edge ex; is proposed by the network, we check the collision on ea, on which the ego-arm starts moving at t4. If there is no collision, we add (ex, t.) into the current path æ. Otherwise, we query Np for another edge with the next greater priority value. The planning will end if we succeed in finding a path π from to Ug or fail when stopping at a vertex with all the connected edges with collisions. Optionally, when the latter one happens. we can backtrack to the last step and query for the top-k greatest priorities in turn. Further discussions are covered in the experiments section.
Experiments
Experiment Setup. We evaluate the proposed methods on various multi-arm assembly tasks where manipulate the ego-arm and avoid collisions with other obstacles, including moving arms and static obstacles. Specifically, the environments cover arms of 2 to 7 Degree of Freedom and 1 to 3 moving obstacle arms, including (i) 2Arms: 1 obstacle arm with 2 DoF. (ii) Kuka-4DoF: I obstacle arm with 4 DoF. (iii) Kuka-5DoF: 1 obstacle arm with 5 DoF. (iv) Kuka-7DoF: 1 obstacle arm with 7 DoF. (v) 3Arms: 2 obstacle arms with 2 DoF. (vi) Kuka3Arms: obstacle arms with DoF. Without loss of generality, we assume all the robot arms in an environment are of the same DoF and move at the same and constant speed. The static obstacles are represented as cuboids in the experiments.
We compare our method GNN-TE with the lazy sampling- dynamic motion planning (Dijkstra-H). It prioritizes the edges based on the shortest distance to the goal on the configuration graph at each time step. We also compare our method with the oracle SIPP 3d which is search algorithm that generates optimal paths using safe intervals.
Baselines.
method
Datasets. We randomly generate 2000 problems for training and 1000 problems for testing. In each problem, we randomly generate fixed-length trajectories of the moving obstacle arms and sample

---

randomly generated problems
Collision Check
Success Rate
Path Time Ratio
GNNeTE w's Dugge
Diumnol SIPP
200
18
0.2
JAzm
KdalArm
Kuta-Eaa KateSDer Kata-Toor
More
Kaka 4oo Sro.
Mam
Kelakkem
hard problems
#
Collision Check
Success Rate
Poth Time Ratio
Degar
Degar
210
36
1800
Shres
Kate-Dbe Katelike Kake
Mon
Figure 4: Overall Performance baselines on collision checking, success rate, and path time ratio. Our approach significantly reduces collision checking more than 1000x compared to the complete algorithm SIPP, improves the overall planning efficiency, and achieves high success rates.
1000 vertices in the configuration space of the ego-arm that are collision-free with the static obstacles, then provide them to the oracle generate demonstration trajectories. In order to further investigate the performance of algorithms in challenging environments, we also generate 1000 hard problems, where Dijkstra-H fails find feasible paths.
Training and Testing Details. We first train the GNN-TE on the training problems for 200 epochs. Afterward, epochs. We test all the algorithms on the provided graphs. We randomly split the test problems into groups, and calculate the performance variance.
generate 1000 new training data with DAgger, and trained for another 100
Evaluation Metrics. We measure the average number of collision checking of the common success cases among all the algorithms to evaluate the effectiveness in avoiding obstacles. Collision checking the more expensive computation in motion planning, and reducing can lead to a significant acceleration of online computation. We also report the average success rate over all the testing problems for all the algorithms. A trajectory successful, only the ego-arm does not collide with any obstacles, and eventually reaches the goal given limited time horizon. We also measure the optimality of the methods by comparing the path time ratio to the optimal paths found by SIPP on each success case. Note that the path time metric only measures the cost of the planned path, and does not include the online computation time needed.
5.1 Overall Performance
Figure shows the overall performance of the algorithms in various environments, including all the baselines and GNN-TE without DAgger, i.e., pure imitation learning from the oracle.
In all the environments, SIPP gives the optimal complete non-collision path, but it suffers from the excessive amount of collision checking. GNN-TE significantly reduces the collision checking by more than 1000x, which corresponds reducing online computation time by over 95%. At the same time, the methods have high success rates and a better path time ratio compared to simpler heuristics.
Fig. shows the performance snapshots of the algorithms 2Arms and Kuka-7DoF test case. In both cases, our method successes in planning a near-optimal path compared to the oracle SIPP whereas Dijkstra-H fails.
Performance on randomly generated test problems. As shown Fig our methods signif- icantly reduces the collision checking by over 1000x compared to SIPP, i.e., 60k, 97k, 171k, 197k, 269k, 269k to 17.24, 33.08, 37.36, 56.89, 61.99, 72.63 on 2Arms, 3Arms, Kuka-4DoF, Kuka-5Dol Kuka-7DoF, Kuka3Arma respectively. Because SIPP needs to search in both space

---

and time dimensions, it suffers from large number of collision checking, Our approach ben- efits from learning and achieves much less amount of collision checking even in the dy- namic setting while not compromising much of the success rate, in which case our method outperforms Dijkstra-H in all the environments with higher success rates. Note that as a cru- cial part of the training procedures, DAgger also improves performance in all environments.
Performance on hard test problems. On the hard test problems, Dijkstra-H fails to find fea- sible paths in the dynamic environment. Com- paratively, our GNN-TE can successfully find solutions to these problems with considerable success rates and acceptable path time ratios. is also worth noting that DAgger can better as- sist in improving the performance in the more challenging scenarios compared to the randomly generated problems.
GNN-TE
Dijkstra-H
Optional backtracking search. In the Ap- pendix we also report the result of GNN- TE and Dijkstra-H and them with backtracking GNN-TE w. BT and Dijkstra-H w. BT on 2Arms. On the same algorithm, the optional backtracking search will only result in higher success rate while not affecting the path time ratio and collision checking on the common suc- cess cases. The result of Dijkstra-H w. BT shows that although it improves the success rate significantly but sacrificing tremendous num- ber of collision checking. Nevertheless, our method outperforms the heuristic method both with or without the backtracking search.
Figure 5: Snapshots of the trajectories of the output path on test cases. The environments are 2Arms (first row) and Kuka-7DoF (second row). The ego- arm is black and white with a red end-effector. The obstacle arm is blue and orange with yellow end-effector. In both environments, Dijkstra-H fails to find a path, while method can yield near-optimal path compared to the oracle SIPP.
Comparison with end-to-end RI. We also compare our approach with RL approaches, including
DQN 2 and PPO K3 The neural network architectures for these two baselines are implemented with the same GNN architectures as ours. We observe that even 2Arms, the average success rate of DQN on the training set is only around 54%, while PPO only has success rate of around 20%. They fail to find plans test cases. We provide more details in the Appendix C.4
Comparison with OracleNet-D. We compare GNN-TE with learning- based approach OracleNet- D. by modifing OracleNet[[] the dynamic version. We observe that the performance of OracleNet- D falls behind GNN-TE largely on all the metrics both in random and hard problems. Details are provided in the Appendix
5.2 Ablation Studies
We perform ablation studies on 2Arms of the different encoding in model, including global and local obstacle encoding and temporal encoding. The results are shown in Fig.
Global Obstacle Encoding. In stage 1, we encode both the configuration graph and the global obstacle trajectories with the GNN-based global encoder. To investigate the effectiveness of leveraging global obstacle information, we conduct experiment in which we input the sampled configurations in stage and only introduce the obstacle information stage As we can observe in the figure that although on random problems there slight degrades of collision checking and success rate, the performance path time ratio, success rate, and collision checking improves greatly in the hard environment. This results from the model receiving the overall trajectories of the obstacles, and the encoding helps to reduce collision checking and improves success rate, especially on complicated problems.
Local Obstacle Encoding. We compare our model with the one omitting the local obstacle encoding in stage 2. We only input the temporal encoding corresponding to the arrival time to the local planner as the time indicator for planning. The result has shown that the local obstacle encoding within time window directly helps the local planner perform better.
8

---

Global Obstacli Encoding
Local Obstacle Eacoding
Temporal Encoding
win
16.6
D- Collaiin Chock
SteceH Rabe
Pah Time Sada
Colliciin Cleck
Succoa Jale
Park Tamo Ratto
Collatio Check
Succes
PAb Tixa Rato
Figure 6: Ablation studies on 2Arms of (1) global obstacle encoding, (2) local obstacle encoding, and (3) temporal encoding. We have demonstrated that all these three components improve the effectiveness of the proposed approach. See Section 5.2 for more details.
Temporal Encoding, We analyze the importance of temporal encoding where we remove it from both two stages and only input the trajectory of obstacles in the models. The results also show that temporal information helps GNN-TE make use of the order of the trajectory sequence on both random and hard problems.
6 Discussion and Conclusion
We proposed a GNN-based neural architecture GNN-TE for motion planning in dynamic environ- ments, where we formulate the spatial-temporal structure of dynamic planning problem with GNN and temporal encoding. We also use imitation learning with DAgger for learning both the embedding and edge prioritization policies. We evaluate the proposed approach in various environments, ranging from 2-DoF arms to 7-DoF KUKA arms. Experiments show that the proposed approach can reduce costly collision checking operations by more than 1000x and reduces online computation time by over 95%. Future steps in this direction can involve using ideas from the complete planning algorithms, such as incorporating safe intervals, to improve the success rate on hard instances, as well as more compact architectures for further reducing online computation.
7
Acknowledgement
This material is based on work supported by DARPA Contract No. FA8750-18-C-0092, AFOSR YIP FA9550-19-1-0041, NSF Career CCF 2047034, NSF CCF DASS 2217723, and Amazon Research Award. We appreciate the valuable feedback from Ya-Chien Chang, Milan Ganai, Chiaki Hirayama, Zhizhen Qin, Eric Yu, Hongzhan Yu, Yaoguang Zhai, and the anonymous reviewers.
References
[1] M. J. Bency, A. H. Qureshi, and M. C. Yip. Neural path planning: Fixed time, near-optimal path generation via oracle imitation. In 2019 JEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019, Macau, SAR, China, November 3-8, 2019, pages 3965 -9722. IEEE, 2019. doi: 11L.1//RRO0007720099900000 IROS40897 2019. 8968089
URL https:///oi. org/ /00 1109/
[2] D. Berenson, S. S. Srinivasa, and J. J. Kuffine. Task space regions: A framework for pose- constrained manipulation planning. Inf. J. Robotics Res., 30(12):1435-1461 2011. doi: 10. 10.1177/02783644110666688 URL https: org/ 10. 1177 1177/022733649911886633
[3] L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig. Safe learning roboties: From learning- based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5:411-444 2022.

---

[4] J. Chase Kew, B. Ichter, M. Bandari, T.-W. E. Lee, and A. Faust. Neural collision clearance estimator for batched motion planning. In S. M. LaValle, M. Lin, T. Ojala, D. Shell, and J. Yu, editors, Algorithmic Foundations of Robotics XIV, pages 73-89, Cham, 2021. Springer International Publishing. ISBN 978-330030-66723-88
[5] B. Chen, B. Dai, Q. Lin, G. Ye, H. Liu, and L. Song. Learning to plan in high dimensions via neural iitooration-eploitaatin trees. In 8th International Conference Learning Represen- tations, ICIR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https:://ppenreveev..t wtt..a.ete..oooo///dd..........
[6] N. Das and M. Yip. Learning-bii based proxy collision detection for robot motion planning applications. IEEE Transactions on Robotics, (044):)999-1144,, 2020.
[7] J. Dong, M. Mukadam, F. Dellaert, and B. Boots. Motion planning as probabilistic inference using gaussian processes and factor graphs. In Robotics: Science and Systems, volume 12, 2016.
[8] Gama, Q Li, E. I. Tolstaya, A. Prorok, and A. Ribeiro. Synthesizing decentralized controllers with graph neural networks and imitation learning. IEEE Trans. Signal Process. 70: 1932- 1946, 2022. doi: 10. 10.1T/99/9.. 366401. URL https:://oi..rrg/ 10. 1109/TSP.PP.. 2022 3166401
[9] V. Garg, S. Jegelka, and Jaakkola. Generalization and representational limits of graph neural networks. International Conference on Machine Learning, pages 3419-3430. PMLR, 2020.
[10] E. N. Gilbert. Random plane networks. Journal of the society for industrial and applied mathematics, 9(4):533-544 1961.
[11] Gonzalez, A. Dornbush, and M. Likhachev. Using state dominance for path planning in dynamic environments with moving obstacles. In 2012 IEEE International Conference on Robotics and Automation, pages 4009-4011 IEEE, 2012.
[12] C. Guan, W. Bgga-Brown, and N. Roy. Efficient planning for near-optimal compliant manipula- tion leveraging environmental contact. In 2018 IEEE International Conference Robotics and Automation, ICRA 2018, Brishane, Australia, May 21-25, 2018 pages 215-222 IEEE, 2018 doi: 10.1109/ICRA.200 TTT.:/AAA...1..222222..22
URL httpss//::/// org/ 10. 1109/ ICRA 2018. 8462696
[13] H. Ha, J. Xu, and Song. Learning decentralized multi-amn motion planner. In J. Kober, F. Ramos, and C. J. Tomlin, editors, 4th Conference on Robot Learning, CoRL 2020, 16-18 November 2020, Virtual Event Cambridge, MA, USA, volume 155 of Proceedings of Machine Learning Research, pages 103-114. PMLR, 2020. URL httpttttts//:/eeeeeeeeeee
mlr, press/
v155/ha21a html
[14] K. Hauser. On responsiveness, safety, and completeness in real-time motion planning. Au- tonomous Robots, 32(1):35-48, 2012.
[15] S. Hochreiter and J. Schmidhuter. Long Short-Term Memory. Neural Computation, 9()): 1735-1780. 11 1997. ISSN 0899-7667. doi: 10.1162//coo.1999 1997.9...9999...77.9 1735. URL https //doi.org/10 10000..16666 1162/necoo..oe 1997.9.8....... 9.8.1735.
[16] B. Ichter and M. Pavone. Robot motion planning in learned latent spaces. IEEE Robotics Autom. Leur, 443)2444--44 2019. doi: 10.1 2.99L....0019.9909 898. URL htllhttts:///d org/10
1109/LRA. 2019. 2901898
[17] B. Ichter, J. Harrison, and M. Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 7087-7094. IEEF, 2018. doi: 22.1109/CRA.2218.84455 URL httph:tt///oooo orgo 10. 1109/ ICRA. 2018. 8460730
[18) T. Jurgenson and A. Tamar. Harnessing reinforcement learning for neural motion planning. In A. Bicchi, H. Kress-Gazit, and S. Hutchinson, editors, Robotics: Science and Systems XV, University of Freiburg, Freiburg im Breisgau, Germany, June 22-26, 2019, 2019. doi: 02.15607/RS$.2011 XV.06. URL htttps://dot http///iiiog/////////// org/ 15607 RSS 2019 XV. .26
10

---

[19] S. Karaman and E. Frazzoli. Sampling- based algorithms for optimal motion planning. Int.
J. Robotics Res., 30(7):846-89) 894, 2011. doi: 10.17770273366911400761.1 URL https //doi oroi..og//00.. 1177 1177702783349114667755
[20] L. E. Kavraki, P. Svestka, J. Latombe, and M. H. Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robotics Autom, 12(4): 566-580, 1996. doi: 10. 10977000009 URL httph: org/10 1109/70.
[21] A. Khan, A. Ribeiro, V. Kumar, and A. G. Francis. Graph neural networks for motion planning. CoRR, abs/2006.06245 2020. URL https::/aaxxv..rrrs http://rrxi....rga//2/20000 06248
[22] A. Khan, E. Tolstaya, A. Ribeiro, and V. Kumar. Graph policy gradients for large scale robot control. In Conference on robot learning, pages 823-834. PMLR, 2020.
[23] L. Lee, E. Parisotto, D. S. Chaplot, E. P. Xing, and R. Salakhutdinov. Gated path planning networks. In J. G. Dy and A. Krause, editors. Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmüssuun Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2953-2961. PMLR, 2018 8.
URL http::tpproceeeinnggee mlr.
html
[24] Q. Li, F. Gama, A. Ribeiro, and A. Prorok. Graph neural networks for decentralized multi- robot path planning. CoRR, abs/191 1206099, 2019. URL http://arxiii.eeee/e// org/ abs 19112.1911119195
[25] Q Li, W. Lin, Z. Liu, and A. Prorok. Message-aware graph attention networks for large- scale multi-robot path planning. IEEE Robotics Autom. Lett., 6(3):5533-55400, 2021. doi:
10.1109/LRA.1 2021.3077866 http:t:::::////t http::////oo..o///.o.// www.oo.ggo..o.....o 1109 LRA . 3077863
[26] S. Li and J. A. Shah. Safe and efficient high dimensional motion planning in space-time with time parameterized prediction. In 2019 international conference on robotics and automation (ICRA), pages 5012-5018. IEEE, 2019.
[27] Z. S. O. B. S. S. Madaan, R. Learning adaptive sampling distributions for motion planning by oefffimitation. Workshop on Machine Learning in Robot Motion Planning, IEEE IROS, 2018.
[28] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference computer vision, pages 405-421. Springer, 2020.
[29] V. Mnih, K. Kavukcuoglu, Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv: 1312.502, 2013.
[30] V. Narayanan, M. Phillips, and M. Likhachev. Anytime safe interval path planning for dynamic environments. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4708-477 -715. IEEE, 2012.
[31] S. Niu, S. Chen, H. Guo, C. Targonski, M. C. Smith, and J. Kovacevic. Generalized value iteration networks: Life beyond lattices. In S. A. Mcllraith and K. Q. Weinberger, editors, Proceedings of the -ecooy AAAI Conference on Artificial Intelligence. (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (FAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 6246-6253. AAAI Press, 2018. http::ttt://// aaai orgg/
ocs/ uos/index..r php/ AAAI/AAAIIE http///AAAA///ppp///ee///e/ paper/view/ 16552
[32] K. Okumura, R. Yonetani, M. Nishimura, and A. Kanezaki. Ctrms: Learning to construct cooperative timed roadmaps for multi-agent path planning continuous spaces. In Fal- iszewski, V. Mascardi, C. Pelachaud, and M. F. Taylor, editors, 21 International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022, Auckland, New Zealand, May 9-13, 2022, pages 972-981. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2022. URL //tp::///www... /wwp//wwwwoooooo ifa.aaaaaaa.....a..aaa .rg. wtw/Proceedings/////oooooooos aamas pdff/p972.ppa
[33] C. C.Park, J. Pan, and D. Manocha. Itomp: Incremental trajectory optimization for real-time replanning in dynamic environments. Twenty-Second International Conference on Automated Planning and Scheduling, 2012.
11

---

[34] M. Phillips and M. Likhachev. Sipp: Safe interval path planning for dynamic environments, In 2011 IEEE International Conference on Roboties and Automation, pages 5628-5635 IEEE, 2011.
[35] C. R. Qi H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 77-85. IEEE Computer Society, 2017. doi: 10.1109/CVR.2017...6 URL https::///doo org/ 1109//0.0111110111 1109/ 2017
16
[36] A. H. Qureshi, Y. Miao, A. Simeonov, and M. C. Yip. Motion planning networks: Bridging the gap between learning- based and classical motion planners. IEEE Transactions on Robotics, 37 (1):48-66 66, 2021. doi: 10.1 109/TRO.2000.300000 2020.3006716..
[37] J. Reif and M. Sharir. Motion planning in the presence of moving obstacles. Journal of the ACM (JACM), 41(4):764-7) 790,,,9991111 1994.
[38] S. Ross, G. J. Gordon, and D. Bagnell. A reduction of imitation learning and structured prediction no-regret online learning. In G. J. Gordon, D. B. Dunson, and M. Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011, Fort Lauderdale, USA, April 11-13, 2011, volume 15 of JMLR Proceedings, mlr wwt.ppees//////osss1l/
pages 627-635. JMLR.org, 2011. URL WRRLhtttp//p/rceediiiii 1oss1la.pdf
[39] Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063: 2017.
[40] R. A. M. Strudel, R. Garcia, J. Carpentier, J. Laumond, I. Laptev, and C. Schmid. Learning obstacle representations for neural motion planning. CoRR, abs/2008.11174, 2020. URL Sttps://arxxr
org/ 2008.. 11174
[41] A. Tamar, S. Levine, P. Abbeel, Y. Wu, and G. Thomas. Value iteration networks. In D. D. Lee, M. Sugiyama, U. von Luxburg, Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neu- ral Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 2146-2154, 2016. URL httpps:/pproceedinngg neur www..pp.cc.pappr////.. http/papp/222///eepeeee 22100146645f555eee33883333333333
ibbstract.htm
[42] E. Tolstaya, F. Gama, J. Paulos, J. Pappas, V. Kumar, and A. Ribeiro. Learning decentralized controllers for robot swarms with graph neural networks. In L. P. Kaclbling, D. Kragic, and K. Sugiura, editors, 3rd Annual Conference on Robot Learning. CoRL 2019, Osaka, Japan, October 30 November I, 2019, Proceedings, volume 100 of Proceedings of Machine Learning Research, pages 671 682. PMLR, 2019. URL httt::/ tolstaya20a. html
mlr. press/v100/
[43] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polo- sukhin. Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fer- gus, V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Anmual Conference on Neural Information P Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008, 2017. URL http::tttt////oceediiiii hewp:oeeeddnnng.uu.i.p.p neu cc cc/ cc/paper/2ee 2017 hhtp:/aahhaaaaaa /ttpr//022/aaaaa//66244443333222
[44] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. Advances neural information processing systems, 30, 2017.
[45] F. Xue and P. R. Kumar. The number of neighbors needed for connectivity of wireless networks. Wireless networks, 10(2):169-181, 2004.
[46] C. Yu and S. Gao. Reducing collision checking for sampling-based motion planning using graph neural networks. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6- 14, 2021, virtual,
12

---

pages 4274 2021. URL Utt.hppp///prooodngnnoooooaaa wew.eeedddinnnn.n............... httpa///s../ppppp///////////
[47] C. Zhang. Huh, and D. Lee. Learning implicit sampling distributions for motion planning.
In 2018 IEEE/RSJ International Conference Intelligent Robots and Systems, IROS 2018, Madrid, Spain, October 1-5, 2018, pages 3634 3661. IEEE, 2018. doi: 10.1109/IROS.011 2011. 8594028. URL httpp:://oo org 10. 000009/IRRSS 2018. 8594028
[48] L. Zhou, D. Sharma, Q. Li, A. Prorok, A. Ribeiro, and V. Kumar. Graph neural networks for decentralized multi-robot submodular action selection. arXiv preprint 10Xiv:210500860 2021.
13


Reviews:
[{"review_id": "eCKkci9wMmI", "reviewer": "Reviewer_oTjD", "summary": "This paper proposes an imitation learning approach which mimics the behavior of a search-based configuration space motion planner with dynamic obstacles (SIPP [32]) using DAgger. Planning is performed on a sampled road map graph which is encoded with the motion of a dynamic obstacle and the state of static obstacles in a GNN. The method seems to assume a fixed shape and kinematic structure of dynamic (robot arms) and static (boxes) obstacles. The method is evaluated in simulation and compared in planning performance metrics with ablations, SIPP and a simpler Dijkstra-based baseline. The approach well improves the required computation in terms of collision checks, while keeping a high level of success rate and path efficiency.", "questions": "* l. 226 it's unclear how the Dijksta-H baseline works from the description. How does it handle arrival time and does it use collision checking ? Please revise and provide more implementation details in the supplementary material.\n* l. 246 is SIPP guaranteed to find optimal paths and for which planning problem ?\n* The ablation study in 5.2 only reveals little improvements by the individual components relative to the performance of the baselines. What is the overall performance gain by all the components over the basic GNN ?\n\n\n", "limitations": "* The paper does not address limitations and assumptions properly. For instance, it should be clearly stated that the model needs to be trained for the specific actor arm, the obstacle arms and the static obstacles which it is tested on, i.e. that specific kinematic structures and obstacle shapes are assumed. How can the approach be generalized to arbitrary obstacle shapes and arm kinematics?", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The approach seems novel and effective in cloning the planner behavior for the trained environment settings.\n* The paper is well structured and can be easily followed. \n* Improvements over ablations, the search-based planning baseline SIPP, and a simpler Dijkstra based planner are demonstrated in several planning problems with various robot kinematics and difficulties.\n\nWeaknesses:\n* The writing requires significant proof reading to correct spelling and grammar mistakes.\n* The title mentions \"Manipulation Planning\" but actually there is no object manipulation performed. The approach performs motion planning. Title should be changed accordingly.\n* The supplementary material should provide the network details to allow for reproducibility of the work. Ideally, also code and datasets should be provided publicly.\n* l.115 the explanation of where the attention mechanism will be used is confusing, because the concepts are unclear at that point of reading. Just introduce the attention mechansim abstractly.\n* l. 143, please introduce the subscript notation v_i,v_j, x_i, x_j and that the subscript links nodes and respective components of the features. the y needs to be separated with a comma.\n* The notation O = O + TE is unclear. first of all, this is not a proper equation but an assignment. Rather use a new symbol for the left O. Also, O and TE as well as the + operation are not defined.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YUkI6rNpXjF", "reviewer": "Reviewer_4Jb4", "summary": "The paper aims to improve the efficiency of motion planning problems in dynamic environments, such as with moving obstacles or multiple manipulators. The approach taken is graph neural networks and temporal encoding framework. Trained with imitation learning and data aggregation procedures, the proposed methods significantly reduce collision checks and planning time, at the cost of reducing planning success rate.\n\nConcretely, the core building blocks are graph neural networks, attention mechanism, temporal encoding of moving obstacles, and imitation learning with DAGGER style of data aggregation. Extensive experiments are provided to compare the proposed method with search-based baselines, ablation baselines, and end-to-end RL methods.", "questions": "1. How is the speed (saving number of collision checks) in trade-off with quality (success rate of finding paths), and how does this trade-off vary with different amount of training? E.g., currently 2000 problems are used for training. Is the performance increasing with more training problems? Is it in a linear or log trend? Does it saturate?\n\n2. The separation between random and hard cases is determined by Dijkstra-H. Is there a justification of this choice or because there are no better ways? It's also helpful to provide a direct measure on the complexity of the planning problems.\n\n3. Using the attention mechanism to encode obstacles is not analyzed separately, i.e., why is attention mechanism the most proper to model the obstacles?", "limitations": "No specific analysis or discussion is provided on the limitations by the authors. Though, from the result section, it can be observed the proposed GNN based motion planning has a reduced success rate compared to the baseline SIPP. More comprehensive analysis on this would be greatly appreciated by the community. Refer to Questions.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality\n\nThis paper takes a graph neural network (GNN) approach and temporal encoding to tackle motion planning in dynamic environments. It is a novel way of applying existing approaches such as GNN and attention mechanisms to an existing problem. Many details were reasonably designed, e.g., the temporal encoding of moving obstacles.\n\nQuality and Clarity\n\nThe algorithm is reasonably designed, justified by the provided ablation studies of module and data flow choices. The approach is clearly presented, backed with diagrams and example drawings.\n\nThere are quite a few typos throughout the paper. Please do more proofread for future revisions. Just to name a few, \"(\" in L112, why (k) is needed for the aggregation function in eq(1)? L118 \"has to\". 165 \"E_j\" -> \"E_i\".\n\nSignificance\n\nThe problems of motion planning in dynamic environments is an important problem, relevant for a wide group in the community. Leveraging GNNs (likely the first time) to solve this problem is interesting. If the pros and cons are thoroughly analyzed and presented, it can be of greater significance.\n\nThis is an important issue to address. It is not a surprise using a GNN and temporal embedding, trained on the same set of static and dynamic objects, can achieve with more efficient search heuristics. However, this efficiency benefits comes at a price, sacrificing the success rate, i.e., paths cannot be found in many cases. The authors did an attempt using backtracing, but only showed the result on the most simple setting, two-DOF arms. Yet it is not strong result (98% for random and 89% for hard). From a reader's point of view, understanding this trade-off in a more systematic way would be a key contribution. See also Questions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Se2eLywhK4I", "reviewer": "Reviewer_sJvQ", "summary": "This paper proposes to use a learning-based approach for motion planning with dynamic obstacles. Each possible robot configuration (after discretization) is represented as a node. This method assumes it knows the full trajectory of the obstacles. It first generate expert trajectories using some sampling-based path planing method (SIPP) and use that as the ground-truth and try to imitation that policy. The performance is further enhanced by using DAGGER. The advantage of using learning-based approach to imitation the tradition approach is that the learning-based approach is much faster.", "questions": "The following questions are essentially what I said in the above section:\n\n1. How is the obstacle trajectory represented?\n\n2. How to deal with cases where the obstacles' trajectories are unknown?\n\n3. Is there any possible comparison with other learning-based methods?", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[originality] This work is inspired by the transformer literature and successfully modify it to the motion planning community. I believe this paper has original contribution for this adapatation.\n\n[quality] I believe overall this paper gives detailed and comprehensive evaluation of the proposed method. However, I found the comparison between the proposed method and other learning-based method is missing. For example, one simple baseline I could come up is using [1]’s network, with an additional input of all the possible obstacle trajectory (since the full trajectory is known), and output the next optimal configuration.\n\n[clairty] I think the technical part is clear except for one point: this paper claims previous method considers fixed graphs ([39, 21]). Then I’m wondering how the obstacle trajectory is represented? My understanding is the obstacle’s full trajectory is represented as a node in the graph. However, in that case the graph is fixed and some previous methods need to be considered in the experiments (such as [21, 39]). The authors may want to clarify this.\n\n[significance] The proposed method is much master than the traditional optimization-based method. This can be a great advantage for many applications. However, the performance drop seems to be huge in the hard case (Figure 4 and Table B2). Since motion planning would be safety-critical in many cases. This may be a major limitation for the proposed method. I think it would be better to provide some illustrations and convincing improvement direction in the paper. Another possiblity to make the contribution more significant is to consider cases where the object trajectories are not known thus prediction and rapid re-planning is needed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept