NTFields: Neural Time Fields for Physics-Informed Robot Motion Planning
Authors: Ruiqi Ni
Ahmed H Qureshi
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Robotics
Motion Planning
Neural Fields
Implicit Neural Representation
Physics Informed Deep Learning

Abstract: Neural Motion Planners (NMPs) have emerged as a promising tool for solving robot navigation tasks in complex environments. However, these methods often require expert data for learning, which limits their application to scenarios where data generation is time-consuming. Recent developments have also led to physics-informed deep neural models capable of representing complex dynamical Partial Differential Equations (PDEs). Inspired by these developments, we propose Neural Time Fields (NTFields) for robot motion planning in cluttered scenarios. Our framework represents a wave propagation model generating continuous arrival time to find path solutions informed by a nonlinear first-order PDE called Eikonal Equation. We evaluate our method in various cluttered 3D environments, including the Gibson dataset, and demonstrate its ability to solve motion planning problems for 4-DOF and 6-DOF robot manipulators where the traditional grid-based Eikonal planners often face the curse of dimensionality. Furthermore, the results show that our method exhibits high success rates and significantly lower computational times than the state-of-the-art methods, including NMPs that require training data from classical planners.

Published as a conference paper at ICLR 2023
NTFIELDS: NEURAL TIME FIELDS FOR PHYSICS- INFORMED ROBOT MOTION PLANNING
Ruiqi Ni
Ahmed H. Qureshi
Department of Computer Science
Department of Computer Science
Purdue University nil 70purdudee edu
Purdue University
ahquresh! epurdue.ueeeeee edu
ABSTRACT
Neural Motion Planners (NMPs) have emerged as a promising tool for solving robot navigation tasks in complex environments. However, these methods often require expert data for learning, which limits their application to scenarios where data generation is time-consuming Recent developments have also led to physics- informed deep neural models capable of representing complex dynamical Partial Differential Equations (PDEs). Inspired by these developments, we propose Neural Time Fields (NTFields) for robot motion planning in cluttered scenarios. Our framework represents wave propagation model generating continuous arrival time to find path solutions informed by nonlinear first-orde PDE called the Eikonal equation. We evaluate our method in various cluttered 3D environments, including the Gibson dataset, and demonstrate its ability to solve motion planning problems for 4-DOF and 6-DOF robot manipulators where the traditional grid-based Eikonal planners often face the curse of dimensionality. Furthermore, the results show that our method exhibits high success rates and significantly lower computational times than the state-of state-of-the-art the-art methods, including NMPs that require training data from classical planners. Our code released:
com ruiiiiiig
NIFields
INTRODUCTION
Motion Planning (MP) is one of the core components of an autonomous robot system that aims to interact physically with surrounding environments. MP algorithms find path solutions from the robot's start state to the goal state while respecting all constraints, such collision avoidance. The quest for fast, scalable MP methods has led from traditional approaches such as RRT (LaVel al. 2001), Informed-RRT* (Gammell et al. 2014), and FMT (Janson et al. 2015) to NMPs that exhibit promising performance in sigh--imensionall spaces. However, a significant bottleneck in state-of-the-ate art NMPs is their need for expert trajectories from traditional MP methods, limiting their application to high-dimensional scenarios where large-scale data generation is time-consuming.
Recent developments have provided us with ways have physics-informed deep learning models (Raissi et al. 2019 Li et al. 2020) (Smith et al.] (020)) that can directly solve complex PDEs such as Navier-Stoker Burgers, and Schrodinger equations. Inspired by these neural PDE solvers and to overcome the expert data needs of NMPs, this paper introduces NTFields for robot motion planning in cluttered environments. Our NTFields are generated by physics-informed neural model driven by first-order, non-linear PDE called the Eikonal equation, whose solution represents the shortest arrival time from a source to a destination location under a pre-defined speed model, and leads to the continuous mortest--ath between two locations (Sethian 1996, Clawson et al. (2014). Our model generates continuous time field between the robot's given start and goal configurations while respecting collision-avoidance constraints, leading path solution in sub-scconds.
Our method's s salient features and contributions are summarized as follows: 1) novel physics- informed PDE solving the Eikonal equation formulation for robot motion planning under collision- avoidance constraints in high-dimensional spaces. 2) A novel neural architecture design that en- capsulates various properties of our PDE, resulting in scalable, fast NMP method. 3) A novel bidirectional algorithm that quickly finds path solutions by iteratively following the gradient neural time fields, marching towards each other from the start and goal configurations. 4) Unlike prior,

---

Published as a conference paper at ICLR 2023
state-of-the-ar NMP methods requiring extensive expert motion trajectories, NTFields only require randomly sampled robot start and goal configurations and learn robot motion time fields by directly solving the PDE formulation to find path solutions in sub-sccond times. Our data generation takes less than 3 minutes, even in complex high-dimensional scenarios, and yet, our method computes paths significantly faster than traditional planners while retaining high success rates. 5) We demonstrate our method in various 3D environments, including the Gibson dataset, and also solve motion plan- ning problems for 4-DOF and 6-DOF robot manipulators where traditional Eikonal equation based planners such as the Fast Marching Method (FMM) (Sethian 1996) struggle due to computational time complexity. 6) Finally, we compare our method against state-of of-the--rtt NMP methods, which require expert motion trajectories, and against existing Neural PDE solvers for the Eikonal equation to highlight their limitations in solving complex motion planning problems.
2 RELATED WORK
Perhaps the most relevant work is the FMM (Sethian 1996 Valero- -orrez et al., 2013: Treister Haber 2016), which numerically solves the Eikonal equation to determine the time field for robot motion planning. However, numerical approaches require discretization of configuration spaces, thus failing to provide the continuous-time fields and suffering from the curse of dimensionality in terms of computational tractability. There also exist other traditional methods for solving MP problems without relying on the Eikonal equation. These methods range from sampling- -ased MP (Karaman & Janson et al. 2015 Gammell et al. 2015) to gradient- based optimization techniques 2009 Kalakrishnan et al. 2011). The former techniques relatively scale better than FMM but still exhibit large computation times in high-dimensional space. In contrast, the latter methods do not provide global solutions and often lead to local minima.
Frazzoli 2011
(Ratliff et al.
Recent developments have introduced NMPs, which learn from expert data via imitation and improve the performance of classical techniques in higher-dimensional problems. For instance, (Ichter et al.) 2018: Kummr et al.) (2019; Qureshi & Yip 2018) generate informed samples in the regions containing the path solution guide the underlying classical MP methods. (Qureshi et al., 2019 2020) generate end-to-end path solutions and revert to classical planners to inherit their completeness guarantees. (Ichter& & Pavone 2019) perform MP in latent spaces find path solutions but lacks interpretability. In general, NMPs, despite their computational gains, usually rely on classical MP methods to provide the training data, which hinders their application scenarios where data generation is challenging
Recent approaches (Ortiz et al. 2022 Liu et al. 2022 Saulnier et al.) 2020; Fincan et al. 2021 use gradient information from Neural Distance Field (NDF) (Chibane et al. 22200] Xie et al. 2021 for collision avoidance resulting in real-time robot motion generation methods. In similar vein, (Adamkiewicz.c et al.) (202)) uses gradient information from Neural Radiance Field (NeRF) (Mildenhall et 20.00 for visual navigation. Other relevant and recent methods to our NTFields are Implicit Environment Functions (IEF) (Li et al. 2022) and cost-to-go (c2g) function (Hub et al. 2021)) which also generate time fields or c2g function, then compute gradients for robot motion planning. However, IEF and c2g are not physics-drive drivvn methods; instead, they are supervised learning approach that requires training data from FMM or PRM. Note that, unlike prior NMP methods. our approach does not require motion path trajectories for imitation learning and instead directly solves the PDE formulated by the Eikonal equation to generate continuous-tine field for motion planning.
Recent deep learning advances encapsulate various physics- based continuous functions solving PDEs without relying on explicit numerical derivatives. These methods exploit the neural network's 's back- propagation for gradient computation of output concerning related inputs. Similarly, EikoNet (Smith et al. 2020) solves the Eikonal equation for modeling the time field concerning seismology. However, Eiko we show in our experiments, does not generalize to motion planning tasks requiring speed constraints in obstacle regions. Instead, we introduce novel formulation and architecture design enabling continuous time field modeling through solving the Eikonal equation for complex robot. motion planning tasks.
NTFIELDS: PROPOSED FRAMEWORK
This section presents our physics-infoni informed NTFields framework (Fig. and discusses its formulation, architecture design, training details, and overall bidirectional planning execution.
2

---

Published as a conference paper at ICLR 2023
3.1 ROBOT MOTION PLANNING AND EIKONAL EQUATION FORMULATION
Let robot configuration-ppace (c-space) be denoted as Q C Ted with dimensions d E N, and its surrounding environment, often known workspace, be denoted as X' C C with dimensions m E N. The workspace usually contains obstacles, forming formidable Xohe C X and feasible iree Free X', Nots workspace, and their corresponding obstacle Qubs C Q and obstacle-free Qreee Q Lobs configuration space. The standard robot motion planning objective is to find a path in an obstacle-free c-space ffree connecting a given start q. E Qrree and goal qg free configurations. In addition, it is often desirable for these paths to fulfill optimality conditions, such as having the shortest Euclidean distance minimum arrival time from given start to goal configurations. In practice, planning using time fields is preferred as it allows constraints on robot speed near the obstacles, leading safe navigation (Herbert et al. 2017). However, since modeling the time field in configuration space is computationally expensive under speed constraints, especially in higher- dimensional problems, most existing methods impose distance-based optimality and rely on advanced path tracking techniques for robot safety around obstacles.
The Eikonal equation a first-order, nonlinear PDE approximating wave propagation. solution is the shortest arrival time from a source to destination location under pre-defined speed model, which corresponds to the continuous shortest path problem (Sethian 1996 Clawson et al. 2014). Let ((z) and [2x y) represent the speed model on point and the corresponding wave arrival time from the given source æ to the target y. Then, the Eikonal equation relates the speed and arrival time as S(y) , y)ll, where S(y) is the speed at target point y. and V T y) is partial derivative of the arrival time model with respect to the target point y. However, the solutions to the Eikonal equation have singularity near the source point when the arrival time is almost zero. This limitation is often resolved by factored formulation represented as T(x,, y) = || yll f(x,, y) (Fomel et al., 2009; Treister & Haber 2016 Smith et al. 2020), where / represents factorized time field from time field. However, such factored representation is not suitable for motion planning problems as the speed inside obstacles, i.e., formidable space Ooner needs to be zero, making arrival time infinity. In addition, we also observe that the arrival time model needs to obey the following symmetry property for motion planning between given start and goal configurations T(g.. 9g )=) 9.). This implies that the arrival time from start qs to destination gg and vice versa are equal under the assumption that only one optimal path solution can exist connecting the given configurations. The symmetry property also applies to partial derivatives of the time field as V 9. T(gs, 0g) = V, T(1g- 9s). Therefore, we factorize the arrival time T in the following form and introduce a new factorized time field 7(g., (g) as:
T(qe, 9g) = 11.. qa
(1)
Tgg. 9n)
In the above equation, the model time field T(q.; 9g) when 9. = 9g- As we require speed in formidable obstacle space to be almost zero, we can explicitly make T(g., 9g) + 0 for any arbitrary configurations obstacle space, i.e., {gs- 9g) E Qats- Therefore, T(2s, 9g) E [0, oo) and by our factorization in Equation 1, [(g,- 9gy's value range from O to 1. Furthermore, the expansion of the Eikonal equation using the chain rule, with T'gs, 9g) as defined in Equation 1, results in the following formulation (see Appendix A for full derivation).
72 (g 9g)
5(gg) V (go. 9g) +
99. 9g 2.
[g.. 90)l 27 (g(90.0 9g) (9g qo) Va, (g.. Ag)
4o
3.2 NEURAL TIME FIELDS
We model the physics governed by Equation (2) using a deep neural network. Our neural architecture outputs the factorized time field 7 for the given robot configurations' representation concerning the given enviromment. We also introduce novel neural architecture to respect the symmetric property of time field w.r.t motion planning, as described above. In addition, avoid the numerical issue of the time field being so caused by zero speed in obstacle space, we introduce a speed model respecting collision constraints.
3

---

Published as a conference paper at ICLR 2023
Speed Field
Bidirectional Plan
S
Path Solution
VT
IVTT
Workspace Encoder g())
4
C-Space Encoder
Time Field Generator
Q
f()
7()
VT
Time Field
Figure 1: Our method consists of a C-Space encoder, workspace Encoder and time field generator. The C-space encoder encodes the start (g,) and goal (gg) configurations, whereas the workspace encoder provides environment-centry robot configuration representations. These c-space and workspace features are passed through the symmetric operator X forming the input for our time field generator. Finally, the gradient of the time field results in the speed field, and the final path computed by following those gradients in bidirectional manner. The black arrows show the forward propagation of our network, and the red arrows show the backpropagation to compute the loss.
Speed Model 5 (q)): We define our ground truth speed model, denoted as (*), at any robot configuration q E Q as
Sconat
(g) dmaz
(p))(d dmnin; dmax))
where p C IR' represents robot surface obtained via forward kinematics for the given configuration E freee and d.m computes distance between robot surface and obstacles Xots in the workspace, The dmii and dmaz are minimum and maximum distance thresholds, and Sconet is user-defined speed constant. The clip functions bound the distance function with range [dimin; dmaz). We assume constant speed if robot's distance from obstacles exceeds dmax-
Robot configuration and workspace encoding: For given robot configuration E Q. our encoders generate the robot-centric workspace embedding g(4) and direct configuration embedding f(g). The former enables reasoning about the relative position of obstacles and the robot, including collisions, for decision-man makinn.. Whereas the latter allows direct computation of the gradient time for input configurations needed for solving the Eikonal equation.
Workspace encoding g(q): We extend the procedure described in (Chibanc et al, 2020) to obtain the robot-centric workspace embedding gg)). First, sample the sparse workspace obstacle point-cloud X C X' and convert it into a voxel grid W, of dimension N x N x x 1, where the last dimension indicates each voxel's occupancy. Then, the Wo is passed through 3D CNN layer to get the output W, of size N X N x N x K, where K is the number of feature maps. Note that the input W and output W, have the same first 3D dimensions, which retain via padding. Next, combine W, and output W, such that the resulting workspace feature representation W becomes of the size N N N (K + 1). Note that our W is multi-scale feature representation comprising both input and output layers of the 3D CNN module. To compute g(4) based on W and we generate several samples (P1, P2, ... ... , Pn. E p(g) on the robot surface p 3 using forward kinematics (FK) at the given configuration q. FK allows computing robot joint positions in the workspace using the robot joint angles. Therefore, given all joints' positions and robot geometry, i.e., links' lengths and widths, the robot workspace representation robot surface points are the workspace, as is the obstacle point cloud. Therefore, using both robot surface point cloud and obstacle point cloud to obtain workspace encoding enables reasoning about the relative position obstacles and the robot itself, including collisions, for decision-making To do so, compute the nearest grid cell in W for each point p;- The grid cell is 3DD cube with
point cloud, p(g). is obtained. Note that the
4

---

The loss Published as a conference paper at ICLR 2023
eight corners. Therefore the feature size for each grid cell is 8 x (K + 1). Next, the feature vectors 8 x (K + 1) are further combined via trilinear interpolation to form a representation for point p. of size (K + 1). A full robot-centric workspace representation g(q) becomes of size x (K 1).
C-space encoding f(q): Finally, to compute a direct latent embedding of q. our e-space encoder uses ResNet-style (He et al., 2016) ferw-forward neural network, denoted as f. Since we need to compute the gradient of arrival time with respect to input configurations, the direct encoding of with MLP-based ResNet blocks allows gradient computation. In summary, the function f takes the robot configuration as an input, passes them through ResNet blocks, and outputs latent embedding.
Hence, given a robot configuration, our encoders provide robot configuration embedding [(g) and their robot-centric workspace representation g(9), as shown in Fig.
Nonlinear symmetric operator:
To address the symmetry property of the Eikonal Equation, in this section, we introduce a nonlinear symmetric operator, denoted as X to combine the features (g.), f(g), 9(qx), ). and g(qg) to form an input to our time fields generator predicting т. We choose a nonlinear symmetric operator min and max to combine the given feature vectors. However, since each of these operators loses some feature information, we observed that concatenating multiple operators' output leads to better performance. Therefore, we combine the robot configuration feature vectors in our setup by concatenating both min and max operations. Let the concatenation of two arbitrary vectors a and b be denoted as [a, b]. Then, our nonlinear operator X combines the arbitrary feature vectors a and as a X b = [max(a a, b), min(a, a. b)]. Therefore, our configuration feature vectors are combined as [f()) X f(gy), ((q=) X g(g))] In Appendix. B, we further discuss the details to motivate our choice of nonlinear symmetric operator.
Time field generation: Our time field generation function is a ResNet-style neural architecture that takes the robot's start and target configuration embeddings. i.e., [f(gs) ) X [gg)) X s(w))] and outputs the factorized time field r. Our method learns to output 7 and its partial derivative w.r.t inputs via backpropagation using our Eikonal- based physics model, i.c., Equation The training procedure and the execution of our NTFields framework are described in the following.
Training details: We randomly sample N E N different start and goal configura- tion pairs in the given environment and compute their speed field using our model (g) described in Equation 3 The resulting dataset to train NTFields is of form {(g.. Sy. (gs), S* (qg)) (9.. 4g. S"
(g., 9g S* (qx), (())).
function to train NTFields is computed as follows. For given start (gs) and goal (gg), NTFields outputs T (As: 9g). Next, we use the predicted r(..; Qg) to compute the corresponding speed values S(ga) and S(g,) using Equation 2 Finally, the NTFiclds' training loss between the ground truth speed values (gs), (q,)) and the predicted speed values [S(q=), S(g,)l is defined as:
1 1- (-VS
((g.)/5* (4)
Our loss function is isotropic to prevent predicted speed from non-uniform errors due to different training sample pairs Furthermore, we use square root to smooth the loss function's 's gradient. We train our models end-to-end with their objective functions using the AdamW (Loshchilov & Hutter 2017) optimizer. Additional training data generation details are available in Appendix Section D.
Execution of bidirectional motion planning: Once trained, we leverage our NTFields model to parametrize Equation 1, i.e.. T'(a.: 9g) ||4. 114--9ul1 //tlll//(ga/ 4.), to compute the time field T'q.. 4g) and its partial derivatives V, Vu,T(4e, 9,) and V 9, T(4,, 9,). The norms of these partial derivatives relate to the speed model as V1 g. (q,, 41 = 1/5(g,) (q.) and T( 9g)ll 1/ 1/S(gg). as described in Section 1. Since the speed model governs the magnitude of the gradient, the gradient step will be high when speed low, especially near obstacles, leading to unsafe robot navigation. To mitigate such unsafe maneuvers, we multiple gradients with S (y), leading to || (g.) VV T(2., = S S(%) and || (g,)), V'., ql)l = S(q,). Since our speed model is small near obstacles, the term 5² dynamically scales down the step size near obstacles. Furthermore, our model's symmetry behavior allows us to perform gradient steps bidirectionally from start to goal and from goal to start. Hence, we compute the final path solution bidirectionally using iterative gradient descent by updating the start and target configurations as follows, where a E R is step size hyperparameter.
qs + 2s cas5(gg. (q=)) Vg. T(p g)); Og 4g + aas a Va, T(gs
(5)
5

---

Published as a conference paper at ICLR 2023
4 EXPERIMENTS
This section evaluates approach through the fol- lowing experiment analysis in solving a wide range of motion planning tasks.
(1) We analyze our architecture design by comparing method with EikoNet (Smith al., (2200) and the traditional FMM on simple task. We also eval- uate our choice of non-linear symmetry operator for feature embedding.
(2) We perform a comparative study and compare our approach with the state-of-the-art MP methods, Baselines: Our baselines comprise NMP and clas- sical MP methods, including MPNet (Qureshi al.)
FNN
E kahWe
2019 2020 IEF (Li et al., 2022).. FMM (Sethian
Figure 2: We compare our approach against FMM and EikoNet box and bunny environ- ment for time field generation. Our method is able generate correct time fields for both scenes, whereas EikoNet fails under complex environment geometry.
1996.. RRT Karaman & Frazzoli 201T and lazy-
PRM (Bohlin & Kavraki| (2000). Our IHF3D stems from the IEF paper, both of which learn via super- vising learning using the training data from FMM. However, IEF3D introduces the following modifica- tions and upgrades to IEF: First, unlike IEF, which used the grid points as goal configurations, IEF3D uses FMM's s nearest numerical solutions on fine-grained grid for randomly sampled start and goal pairs. Second, IEF3D uses 3DCNN for 3D environment encoding. Third, IEF3D uses update steps, as in Equation 5, to prevent large gradi- ents leading to tangling paths. In contrast, IEF uses multi-directional gradient search to select the direction with a lower distance to the goal, which prevents tangling paths but consumes larger computational times than IEF3D. Evaluation Metries: Our evaluation metrics include planning time, path lengths, safety margin, and success rate. The planning time indicates the time took for planner to find path solution, whereas the path length measures the sum of Euclidean distance between path waypoints. The safety margin indicates the closest distance to obstacles along the trajectory. Finally, the success given start and goal found by given planner.
represent the percentage of collision-free paths connecting the
(3) We conduct scalability analysis of our method to high- -ish-diimenional manipulators' configuration spaces where methods like FMM and IEF3D are computationally expensive to execute and train, respectively.
4.1 ARCHITECTURE DESIGN ANALYSIS
This section analyzes formulation and architec- ture design enabling physics-drive driven neural motion planning under compare the generated time field by NTFields (ours), EikoNet (Smith et al. 2020), and FMM, where FMM represents the optimal time field for benchmark- ing. The EikoNet, similar ours, also parametrizes the Eikonal equation. However, unlike our frame- work (see Equation 1), EikoNet overcomes singu- larity using the standard factored formulation, i.e., T(q., 9g) lg. 9g f(q.: 9g) (Smith et al.) 2020), and does not consider c-space and workspace symmetric embeddings and speed ranges. Therefore, EikoNet represents ablation of our framework to highlight the importance of our formulation and architectural designs.
avoidance constraints. We
PAN
MTP dolii
NI M
Figure We compare NTFields with and without nonlinear symmetric operator against FMM on box environment for time field gen- eration. NTFields with nonlinear symmetric operator recover better quality field.
We consider two 3D environments, named Box and Bunny, to represent simple and complex geome- tries for time field modeling. Fig. shows all methods' speed and time field results, depicted for a single source point. The speed field is shown using the color intensity scale, where yellow and blue colors indicate the maximum and minimum speed, respectively. The contours represent the time field. From Fig. contours,
that NTFields perform similarly to FMM, successfully approximate

---

Published as a conference paper at ICLR 2023
gradients for pathfinding, and ensure low speed near obstacle space. For EikoNet. they can predict the speed field in the box environment, but their time field appears as concentric circles in both scenarios. In summary, Fig. 2 higghlights that the formulation presented in Section 3.1 for NTFields enables solving motion planning problems under-speed constraints. Furthermore, we also observe that our architecture can represent complex geometries, which are necessary for generalization to various start and goal configurations for generalizable MP.
We further analyze our NTFields framework's structure by evaluating the role of non-linear symmetry operators in recovering time fields. Fig. 3 shows the fields generated by FMM and NTFields with and without our non-linear symmetric operator. In NTFields, without non non-linear symmetry operator, denoted as NTFields(-S2 ) we concatenate all feature vectors, i.e., [f(g), f(9g), g(g)).g(())) ggg)) ]- can be seen that our proposed operator NTFields ( recovers a better quality field, closer to expert FMM, whereas excluding it leads to distortion. Furthermore, the loss of NTFields ( and NTFields(- - X at the same epoch were 2.3e-2 and 2.8e-2, respectively. Hence, we can conclude that proposed non-linear symmetric operator improves performance in recovering time fields from source to destination.
4.2 COMPARATIVE ANALYSIS
This section presents a comparative analysis of our method and other baselines on cluttered 3D and Gibson environments.
Cluttered 3D environment: Our cluttered 3D environment (Fig. 4 comprises randomly placed ten obstacles of variable sizes and 2000 unseen start and goal pairs for solving motion planning tasks. We compare NTFields' performance with IEF3D, FMM, MPNet, RRT*, and LazyPRM*. The training data generation for each C3D environment took 3.74 seconds for NTFields, about 2.6 hours for IEF3D using FMM, and about 5 hours for MPNet (Qureshi et al. 2020) using RRT to get comparable performances. For FMM, since it requires discretization, and goal pairs for finding solutions. In the case of MPNet, we follow their standard training procedure using demonstration data from RRT* algorithm. Furthermore, we run RRT and LazyPRM on our test set until they find a path solution of length similar to MPNet's solution in the given time limit of 5 seconds. Fig. 4 shows the paths for a test start and goal pair where NTFields, IEF3D, FMM, MPNet, RRT and LazyPRM* path solutions are illustrated in orange, purple, green, blue, cyan, and yellow colors. respectively. It can be seen that the paths by NTFields (ours), IEF3D, and FMM are more smooth with sufficient safety margins compared to other methods. Additionally, the path lengths of our method are comparable to FMM since the Eikonal model governs both. However, the MPNet, LazyPRM* and RRT* have shorter paths as both do not consider speed constraints near obstacles and often have sharp turns with lower safety margins. Note that the lower safety margins of LazyPRM*, RRT* and MPNet's paths are considered unsafe for robot navigation (Herbert al. 2017 The table in Fig. 4 shows the statistical result of six methods on our test dataset. The computation speed of NTFields almost 6 times faster than MPNet and FMM, and over 10 times faster than RRT* and LazyPRM* in these cluttered scenarios. Although NTFields' computational speed is similar to IEF3D, the latter requires a large amount of training data for supervised learning. Furthermore, the success rates of all methods are high and similar, i.c.. NTFields (ours) 99.3% IEF3D 96.9% MPNet 98.5%, FMM 99,8% LazyPRM* 100%, and RRT* 100%. Finally, Appendix provides an additional comparison analysis across eight different cluttered 3D environments.
select the nearest grid cells to our start
Large Complex Environments (Gibson):
This section compares our method with IEF3D, FMM, RRT*, and LazyPRM* in cluttered, large Gib- son environments (Fig. o to highlight our approach's scalability to home-like complex environments. We excluded MPNet as it performed slower than our method and IEF3D in our earlier comparison in cluttered 3D scenarios. Our test set contains randomly sampled 500 unseen start and goal pairs. Fig. 5 presents the speed field of our method and FMM in two environments. Our evaluation metrics for all methods are presented in Fig. Galong with paths' depiction on a test start and goal pair. In the case of RRT* and LazyPRM* R, we set the time limit of 5 seconds. Finally, the time to generate complete data was 2.2 minutes for NTFields and 2.6 hours for IEF3D.
The results demonstrate that NTFields recover similar, oear-ottimal speed fields as FMM in complex Gibson environments Fig. [] leading to smooth path solutions while not requiring supervised training data. The paths shown in Fig. are between start and goal pairs selected on two distant corners of the
7

---

Published as a conference paper at ICLR 2023
(sec)
length
safe margin
sx(
NTFields
0.07
15.06
35.58
1.77 + 1.20
TEF3D
0.09 = 0.00
15.31
41.66
1.84 + 1.22
96.9
FMM
0.61 = 0.03
15.01
34.77
1.80 - 1.20
99.7
MPNNet
0.44 - 0.10
13.78
30.44
0.79 + 1.63
98.5
RRT
1.61 1 0.00
13.79
28.88
0.62
1: 1.63
100
LayYPRM
0.84 0.00
13.77
27.65
0.84
+: 1.77
100
Figure 4: Comparison cluttered 3D environments. The left figure shows six paths generated by our method (orange). IEF3D (purple), FMM (green), MPNet (blue), RRT* (cyan), and LazyPRM* (yellow). The gray blocks are obstacles. We make them slightly transparent for visualization. The right table shows statistical results on 2000 different starts and goals. Our method generates smooth paths with computational time about 6X faster than other methods besides IEF3D.
depicted environment. In this case, all methods except IEF3D find collision-fru frree paths. According to our test set evaluation, NTFields and IEF3D perform similarly. The FMM outperforms all methods in success rate and safety margin, but computational times almost 10 times slower than our method and IEF3D. The traditional techniques LazyPRM* and RRT* have the lowest safety margins and higher computational times. The success rate of RRT* this case, is the lowest computational time to perform collision checking for every path, resulting in failures concerning the time bound of 5 seconds. LazyPRM* performed faster with higher success rate than RRT* because we used the precomputed graph in the former case. Overall, it can be seen that our method scales well to large complex environments, retains its low computational times, high safety margins, and success rates, and do not require supervised training data for learning.
takes significant
Environment 1
Environment 2
1.00
0.75
Speed
0.50
0.25
FMM
NTFields
FMM
NTFields
0.00
Figure 5: Speed Field and Time Field visualization, the colors show speed, and contour lines show arrival time from source point, comparing NTFields (our) and FMM in two different, Gibson environments. Our method recovers expert-like fields required for path planning.
(sec)
length
margin
sE (%)
NTFields
0.06 + 0.00
6.91
12.93
1.10 + U.51
90.9
TEF3D
0.08 + 0.00
7.37
15.93
1.00
0.51
90.3
FMM
0.67 + 0.01
7.43 4: 14.85
1.33
- 0.46
97.8
5.38
6.69
7.68
0.02
69.9
LaryPRM
1.15 4 0.30
6.07
4 7.60
0.21 4 0.07
98.7
Figure 6: (Left) Demonstration of paths computed by all methods large Gibson environment from selected start and goal pair. The paths' colors follows: method (orange), IEF31) (purple), FMM (green), RRT* (cyan), and I PRM* (yellow). (Right) Evaluation metrics depicting the performance of all methods on our test
8

---

Published as a conference paper at ICLR 2023
Demonstration of our method in solving motion planning problems of a 6- G-D (left figure) and 4-DOF (right figure) manipulator in 3D environments. The orange curve shows the final path with black arrows indicating robot transversal from start to goal configurations.
Figure 7:
4.3 SCALABILITY ANALYSIS: HIGH-DIMENSIONAL ROBOT MANIPULATOR C-SPACE
This section demonstrates the NTFields' generalization 4 and 6 DOF robot configuration spaces with 3D workspace obstacles (Fig. 7 The training data generation for these scenarios took around 60 seconds on our machine. The execution pipeline of this scenario is also highlighted in Fig. [ We generate 100 random start and goal configuration pairs in each scenario for testing, where the training of our neural model is guided by Equation 2 computational times, especially FMM and IEF3D. The FMM is very computationally expensive to execute in high- nigh-ddmensional spaces due to a very large number of grid cells. Consequently, the IEF3D is bottlenecked by its need for training data from FMM, which also limits its applicability to high-dimensional C-spaces (see Appendix E for more details). However, since our method directly solves the Eikonal equation without requiring the supervised training data from FMM, it generalizes
We exclude other baselines due to their large
and scales to higher-dimensional robot c-spaces. In the 4-DOF scenario (Fig. 7 Left). NiFields take about 0.08 + 0.00 seconds with 96% success rate; whereas in the 6-DOF scenario (Fig. 7 Right) takes about 0.11 + 0.00 seconds with 91% % success rate. Hence, our results show that our method successfully generates a continuous time field leading to path solutions in high-dimensional spaces. In contrast, it is computationally infeasible for grid-based methods like FMM and their data-driven methods like IEF3D or IEF to solve motion planning problems using the Eikonal equation.
CONCLUSIONS, LIMITATIONS, AND FUTURE WORK
We introduce a physics- informed neural motion planner that requires no expert demonstration data from classical planners and finds path solutions with significantly low computation times and high success rates in complex environments. Additionally, we formulate physics-driven objective function and reflect it in architecture design to directly parameterize the Eikonal equation and generate time fields for different scenarios, including 6-DOF manipulator space, under constraints. Finally, compare method with traditional algorithms and state-of-the-art learning- based motion planners (requiring expert data) and demonstrate up to 10X computational speed gains while retaining high success rates, Furthermore, our statistics also show that the data generation for our method takes at max 3 minutes in complex scenarios, whereas for other searning-based methods, it can take from hours several days.
Our future work revolves around solving the limitations of our proposed work. First, currently, our method considers only the collision and speed constraints. Therefore, possible avenue of research will be to explore other PDE formulations that have similar property to the Eikonal equation and can encode more complex dynamical constraints such acceleration. Second, our method only generalizes to the new start and goal configurations in a given environment. Hence, we also aim to extend our neural field method to generalize across different environments by exploring novel environment encoding architectures. Third, we also aim to validate our approach real robot systems in our future work. Finally, since neural networks (NNs) are an approximation to PDE, there are few failure cases due errors in learning the time-fiell model using the Eikonal equation. Therefore, another possible research direction could be to leverage a bag of neural networks to bootstrap the learning for improved performance.

---

Published as a conference paper at ICLR 2023
REFERENCES
Michal Adamkiewicz. Timothy Chen, Adam Caccavale, Rachel Gardner, Preston Culbertson, Jean- nette Bohg, and Mac Schwager. Vision-only robot navigation a neural radiance world. IEEE Robotics and Automation Letters, 7(2):46064611,, 2022.
R. Bohlin and L.E. Kavraki. Path planning using lazy prm. Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065), volume 1, pp- 521-528 vol.1, 2000. doi: 10. 109/00000T.2000044411 8441007.
Julian Chibane, Gerard Pons-Moll, et al. Neural unsigned distance fields for implicit function learning. Advances in Neural Information Processing Systems,
2020.
Zachary Clawson, Adam Chacon, and Alexander Vladimirsky. Causal domain restriction for eikonal equations. SIAM Journal on Scientific Computing, 36(5):A2478-AM) A2505, 2014.
Mark Nicholas Finean, Wolfgang Merkt, and Ioannis Havoutis. Predicted composite signed-distance fields for real-time motion planning in dynamic environments. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 31, pp. 616-624, 2021.
Sergey Fomel, Songting Luo, and Hongkai Zhao. Fast sweeping method for the factored eikonal equation. Journal of Computational Physics, 228(17):6440 645554454456555, 2009.
Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Informed rrt*; Optimal sampling based path planning focused via direct sampling of an admissible ellipsoidal heuristic. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2997-3004. IEFE, 2014.
Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Batch informed trees (bit*): Sampling-ber optimal planning via the heuristically guided search implicit random geometric graphs. In 2015 IEEE international conference robotics and automation (ICRA), pp. pp. 3067-3074. IEEE, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Sylvia L. Herbert, Mo Chen, SooJean Han, Somil Bansal, Jaime Fisac, and Claire Tomlin. Fastrack: A modular framework for fast and guaranteed safe motion planning. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 1517-1522. IEEE, 2017.
Jinwook Huh, Volkan Isler, and Daniel D Lee. Cost-to-go function generating networks for high dimensional motion planning. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 8480- 8486. IEEE, 2021.
Brian Ichter and Marco Pavone. Robot motion planning in learned latent spaces. IEEE Robotics and Automation Letters, 243)224072241, 2019.
Brian Ichter, James Harrison, and Marco Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Roboties and Automation (ICRA), pp- 7087-7094. IEEE, 2018.
Lucas Janson, Edward Schmerling, Ashley Clark, and Marco Pavone. Fast marching tree: A fast marching sampling-basee based method for optimal motion planning in many dimensions. The International journal of robotics research, 34(7):883-9)) 2015.
Mrinal Kalakrishnan, Sachin Chitta, Evangelos Theodorou, Peter Pastor, and Stefan Schaal. Stomp: Stochastic trajectory optimization for motion planning. In 2011 IEEE international conference on robotics and automation, pp. 4569 4574. IEFF, 2011.
Sertac Karaman and Emilio Frazzoli. Sampling based algorithms for optimal motion planning. The international journal of robotics research, 30(7):846-88) 2011.
10

---

Published as a conference paper at ICLR 2023
Rahul Kumar, Aditya Mandalika, Sanjiban Choudhury, and Siddhartha Srinivasa. Lego: Leveraging experience roadmap generation for sampling-bate planning. In 2019 JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 1488-1495. IEEE, 2019.
Steven M Valale, James J Kuffner, BR Donald, et al. Rapidly-explorinn random trees: Progress and prospects. Algorithmic and compulational robotics: new directions, 5:293-308, 2001.
Xueting Li, Sifei Liu, Shalini De Mello, Xiaolong Wang, Ming-Hsuan Yang, and Jan Kautz. Learning continuous environment fields via implicit functions. In International Conference on Learning Representations, 2022. URL htttttsppp/pppeeeeieeeeee net httpeennrreeeeeeeee/orrrrrrr m
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08899, 2020.
Puze Liu, Kuo Zhang, Davide Tateo, Snehal Jauhri, Jan Peters, and Georgia Chalvatzaki. Regularized deep signed distance fields for reactive motion generation. arXiv preprint arXiv:2203.00 044339 2022.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 2017.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision, pp. 405-421. Springer, 2020.
Joseph Ortiz, Alexander Clegg, Jing Dong, Edgar Sucar, David Novotny, Michael Zollhoefer, and Mustafa Mukadam. isdf: Real-time neural signed distance fields for robot perception. arXiv preprint TrXiv:2204.02296, 2022.
Ahmed H Qureshi and Michael C Yip. Deeply informed neural sampling for robot motion planning. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6582-6588. IEEE, 2018.
Ahmed Qureshi, Anthony Simeonov, Mayur Bency, and Michael C Yip. Motion planning networks. In 20/9 International Conference on Robotics and Automation (ICRA), pp. 2118-2124.
IEEE, 2019.
Ahmed Hussain Qureshi, Yinglong Miao, Anthony Simeonov, and Michael C Yip. Motion plan- ning networks: Bridging the gap between learning-based and classical motion planners. IEEE Transactions on Robotics, 37(1):48-66 2020.
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-infonee neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Nathan Ratiff, Matt Zucker, Andrew Bagnell, and Siddhartha Srinivasa. Chomp: Gradient
optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation, pp. 489. IEEE, 2009.
Kelsey Saulnier, Nikolay Atanasov, George J Pappas, and Vijay Kumar. Information theoretic active exploration in signed distance fields. In 2020 IEEE International Conference Robotics and Automation (ICRA), pp. 4080-4085. IEEE, 2020.
James A Sethian. A fast marching level set method for monotonically advancing fronts. Proceedings of the National Academy of Sciences, 93(4):1591-1555, 1996.
Jonathan D Smith, Kamyar Azizzadenesheli, and Zachary E Ross. Eikonet: Solving the eikonal equation with deep neural networks. IEEE Transactions on Geoscience and Remote Sensing. 59 (12):10655-1066, 2020.
Ioan A. Şucan, Mark Moll, and Lydia E. Kavraki. The Open Motion Planning Library. IEEE Robotics & Automation Magazine, 19(4)72--2. December 2012. doi: 10.1009MAA220222056661..
https: /topps://mmpl kavraki lab.org
11

---

Published as a conference paper at ICLR 2023
Eran Treister and Eldad Haber. A fast marching algorithm for the factored eikonal equation. Journal of Computational physics, 324:210-221 225, 2016.
Alberto Valero-Gomez, Javier Gomez, Santiago Garrido, and Luis Moreno. The path to efficiency: Fast marching method for safer, more efficient mobile robot trajectories. IEEE Robotics & Automation Magazine, 20(4): 111-120, 2013.
Malcolm CA White, Hongjian Fang, Nori Nakata, and Yehuda Ben-Zion. Pykonal: a python package for solving the eikonal equation in spherical and cartesian coordinates using the fast marching method. Seismological Research Letters, 91(4):2378-235 2389, 2020.
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields visual computing and beyond. arXiv preprint arXiv22111.11226, 2021.
12

---

Published as a conference paper at ICLR 2023
APPENDIX
This appendix provides details on our model architectures, training and testing parameters, imple- mentation of benchmark methods, and our computational device specifications.
A FACTORED EIKONAL EQUATION
Let x and y be the source and destination configurations. We have the Eikonal equation , T(x v)ll 1/5(y) and factored time yxx ||r yl1/T(x,,)) y)
Sty) = V, T(r, y)
(6)
V | y/ )||
Syy)
T(I, y)
V | y/
| 3/ TT
Sty)
72
y
|
y
VIT
Sty)
T|K vl
72
|
,
2(y - x)) VNT
(10)
S
72
7
33
72
S(y)
(11)
V
+ 1
V
2 2T(y - I) VNT
S(x)
(12)
V +
V
2T(x 8) VAT
B MODEL ARCHITECTURE
Our model has four parts: configuration space (c-space) encoder f(.), workspace (w-space) encoder g(-), nonlinear symmetric operator 8, and time field generator. The input to our framework includes the workspace point cloud, which we convert into voxel grid X, and the robots start 9, and goal qy configurations in M-dimensional c-space. We use the forward kinematic function to compute the robot surface points associated with the given configuration. Furthermore, all environments' network architecture layer sizes are available in the supplementary source code. The network architecture flow and motivations are described as follows.
C-space encoder: It takes a robot configuration and passes them through fully connected (FC), non-linearity ELU, and several ResNet ELU blocks (He et al. 2016) generate the embedding (q).
W-space Encoder: To generate the configuration-depenneen workspace encoding, we use the structure shown in Fig (b). First, it uses 3D CNN (Conv3d) extract two-level obstacle features Wo. W₁ from obstacle grid X. Second, trilinear interpolation function is used to extract the local, multi-scale features from Wo W, for robot surface points p(g). Finally, these local features are passed through an FC+ELU layer to compute the configuration-depeneen workspace embedding g(g).
Nonlinear symmetric operator: We highlight the two properties of the arrival time field and its gradient w.r.t source to motivate our choice of a nonlinear operator instead of a linear operator for combining features. First, assuming there is only one optimal path solution to motion planning problem in a static environment, the time field between source (q.) and destination (gg) and its partial derivatives for the source needs to exhibit the symmetry property i.e., T(g.- 9y T(1 9s). Second, the time field T T(g.: 9,) and its gradient w.r.t source changes when 9, changes. Therefore, explicitly respect both properties, introduce nonlinear operator. For brevity, introduce
13

---

Published as a conference paper at ICLR 2023
few extra notations. Let u(ge) and u(gg) be arbitrary robot start and goal feature vectors. Let h be neural network that takes the combined features u(q,) X u(gg) as an input and outputs the time field, ie., 7 T(9.. 9g) h hblugg)) X u(gg)]. Then, their gradient w.n. to start configuration u(q,) becomes 74s, = V u(qx) @ u(9.)" d x Vg. 9. [u(q.) X u(g,)). Therefore, if operator X is linear (such as summation or average), the target term u(gg) will be independent of the start u(4g) and disappear in the gradient equation. Hence, if the target configuration changes, the Vg. (qo Qg) will not change. Thus linear operator violates the second property described earlier. In contrast, the nonlinear operator leads to v, (gs: (g) that depends on both source and target features, thus satisfying both symmetry and gradient properties of the arrival time field.
Time field generator: takes the robot start and goal embedding f())). flg.), ((g:)(g.)(()) and passes them through the symmetric operator and several FC+ELU and ResNet+ELU layers to generate the time field T (Fig (c)). For symmetric operator, we concatenate max(f(g.). f(g)). min(f (q=), (g))), max(g(g.), g(gg),, min(g(q.).
c-space point
w-space points p(g) obstacle grid X
pla.),
Symmetric Operator FC + ELU
(FC + ELU) x 2
Conv3d + ReLU
(ResNet + ELU) 4
Trilinear Interpolation
(ResNet ELU)
FC+ ELU FC
FC + ELU FC Sigmoid
FC
c-space feature (q)
w-space feature g(2)
time field
(a) configuration space encoder
(b) workspace encoder
(c) time field generator
Figure 8: NTFields neural architecture: FC is a fully connected layer, ResNet is Residual Network block, and Conv3d is 3D CNN layer. (a): is configuration space point, and f(-) is a c-space sample embedding; (b): p(4) denotes robot surface points, X is voxel representation of obstacle point cloud, and 9(-) is a configuration depended w-space encoding: (c): T is factorized time field from time field.
C DATA PRE-PROCESSING
Our training data contains sampled start and goal pairs robot c-space randomly. We first normalize robot c-space to [-0.5, 0.5) on each dimension. Then we define the c-space sample range hypercube, i.c., all dimensions have the same range scale. Furthermore, we compute the speed for those configurations using our speed model. Our workspace point cloud includes 20000 points on the workspace obstacle surface, which we convert into 128 x 128 x 128 occupied voxel grid following the procedure described in (Chibane et al. 2020). Finally, our pre- pre-processing and training hyper-parameters are shown in our released code.
D TRAINING DETAILS
During training, we use weighted random sampling to form batches for our model training during each epoch. The traditional way of forming training batches is randomly sampling small set of training data for each epoch. However, we observed that weighted random sampling of data with sample weights inversely proportional to Fuclidean distance between start and goal pairs leads to
14

---

Published as a conference paper at ICLR 2023
better performance than traditional random sampling. To train NTFields framework end-to to-tttoedd, we gradually introduce our workspace encoder as the training epochs increase using a scalar weight X, i.e., (gx) X f(g)). Ag(q=) X )g The À is set to Ó for the first ln epochs, then increased linearly to between epoch In to I, and then remains at for the rest the epochs. We observed that gradually increasing À stabilizes the training process, preventing from converging local minima.
We choose 6o 500,4 1000 for Gibson scenes.
E EXPERIMENT DETAILS
For benchmark methods, we use OMPL's RRT* LazyPRM* implementation (Sucan et al. 201 2). MPNet's open source code (Qureshi al., 2019) and the pykonal library (White et al. 2020) for FMM on 100 x 100 x 100 resolution grids. All experiments were performed a device with 3.50GHz x Intel Core i9 processor, 32 GB RAM, and GeForce RTX 3090 GPU. We use million start and goal pairs to train models in environments besides Gibson scenes, and we use million start and goal pairs in Gibson scenes. Furthermore, on our system, the training took around 5-16 hours for each model.
Env
Model
Data Generation Time Training Time
Box
NTFields
3.90s
1500 epochs 4.9h
Bunny
NTFields
14.15s
1500 epochs 4.9h
NTFields
3.74s
2000 epochs 7.2h
Cluttered 3D
IEF3D
2.6h
2000 epochs 7.8h
NTFields
87.49s
2000 epochs 15.9h
Gibson
IEF3D
2.6h
2000 epochs 16.7h
NTFields
130.36s
2000 epochs 15.9h
Gibson 2
IEF3D
2.6h
2000 epochs 16.7h
4-DOF Manipulator
NTFields
54.86s
1200 epochs 8.5h
6-DOF Manipulator
NTFields
41.33s
1200 epochs 10.5h
Table I: Data Generation and Training Time
We run FMM on the grid with resolutions 11, 21, and 51 on 4, 5, and DOF robot manipulators to
show its computational time and corresponding approximate IEF3D data preparation times in Table
Resolution
4-DOF
5-DOF
6-DOF
11
0.008s
2.2h
0.15s
41.7h
3.74s
43.3d
21
0.12s
33.3h
8.19s
94.8d
382.5s
4427d
10.74s
124.3d
Table 2: Resolution shows the number of divisions along each axis. The higher number, the better. Under each robot system, the left column shows the FMM running time from a source point, and right column shows approximate data generation times for IEF3D. The notations h, and d denote seconds, hours, and days, respectively. The empty parts in the table mean running out of memory.
E.1 CLUTTERED 3D EXPERIMENTS
We run our method on multiple cluttered 3D environments. The speed and time fields' visualization is shown in Fig. In all eight environments, method recovers similar fields as FMM, thus validating the correctness of our time-field learning approach. Table shows the statistical results of method and traditional planning techniques in these eight environments.
15

---

Published as a conference paper at ICLR 2023
(xec)
length
sale margin
sr(%)
NIFields
0.05
15.48
13.75
2.09
IEF3
0.04 1: 0.00
15.48
1 15.75
2.09
1: 1.06
96.35
15.59
2.10
RRT*
1.51 + 0.00
14.43
+ 12.16
0.61
= 2.45
100
LazyPRM
0.82 0.00
14.10
11:48
0.85 = 2.52
Table 3: The table shows statistical results on 1000 different starts and goals for eight different scenes.
Env 1
Env 2
Env
Env 4
FMM
NTFields
Env
Env 6
Env
Env
FMM
NTFields
Figure 9: Speed Field and Time Field visualization, the colors show speed, and contour lines show arrival time from source point, comparing NTFields (our) and FMM eight different cluttered 3D environments
E.2 GIBSON EXPERIMENTS
We use Gibson scene (Fig. 10 discuss the design of our method and our choice of w-space encoder training parameters by and 4, The value of lo is selected based on time-field results. We let the time-field net and c-space encoder train end-to-end without the w-space encoder until they learn to recover time fields to some extent. We observed that we usually get an approximate time field in about 500 epochs in all cases. Therefore In set to 500. The i is set larger than lg. In
16

---

Published as a conference paper at ICLR 2023
experiments, it is set to 1000. Thus, the w-space encoder begins training after l. Fig. 10 indicates that directly adding w-encoder (NTFields w/). lu, and 1 = 0, will lead to the local minimum. Without w-encoder ((NTFields w/o). ie., to =inf, it can lead to the correct result but takes longer converge. Since we want to make our model converge faster, we add a w-encoder after lo 500 (NTFields), which converges to recover the time field quickly and efficiently.
1.00
0.75
Speed
0.50
0.25
FMM
NTFields
NTFields w/
NTFields w/o
0.00
Figure 10: Speed Field and Time Field visualization. The colors show speed, and contour lines show arrival time from a source point, comparing NTFields, NTFields with w-encoder (NTFields w/), NTFields without w-encoder (NTFields w/o), and FMM in Gibson environment
17


Reviews:
[{"review_id": "uGB_jcBOUr", "reviewer": "Reviewer_8txW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a novel method of neural motion planning called neural time fields (NTFields) for robot navigation tasks. The key idea is to use an eikonal equation for modeling a wave propgation of arrival time, which probably corresponds to the shortest path from the origin (and the target) to each arbitrary continuous points using a wave propgation. The model is composed of three subnetworks: (1) C-space encoder, (2) workspace encoder, and (2) time field generator. The C-space and workspace encoders encode the general goal configuration and robot configuration. Using these representations, the time field generator conducts biddirectional (both direction from the start and the target) motion planning and updates the configurations via propagation of each step, without expert data.", "summary_of_the_review": "Before the authors’ response, I have noted some concerns regarding motivation, methodology, experiments, and novelty. I will refine this assessment based on the response and discussion.", "strength_and_weaknesses": "## Strengths\n\n- As far as I understand, the latent embedding of the configuration-space, that is processed with a nerual network, is novel. I believe this work contibutes to general stuides of motion planning and representation learning.\n- The overall methodology is sound, and there are some experiments that support the authors’ claim.\n\n## Weaknesses\n\n- Even though I think using a physics-insipred differential equation in a model is a good idea, but I think current version of manuscript the paper does not fully cover the reasoning behind introducing an eikonal equation in the motion planing domain. The authors are encouage to elaborate on the motivation of the equation for solving the navigation problem and fundamental benefits of using them over other planning methods for continous spaces.\n- In the article, it is not clear whether introducing a space encoder f(q) enables a finer planning results theoretically; likewise, I have an impression that the encoding method for the C-space seems to be not carefully designed to solve this specific problem.\n- In experiments, the performance of NTFileds (Figs. 4 & 6)  is not stellar. The authors are encouraged to put effort on validating their methodology qualitatively in a real-world problem.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "dZrgyQtTQKG", "reviewer": "Reviewer_t8K7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel neural motion planning algorithm based on Eikonal equations. It requires no trajectory data generation in advance for training, and has great performance in both cluttered environments and high-dof manipulation cases.", "summary_of_the_review": "I think this paper proposes a novel and interesting framework for motion planning with physics-informed framework.\nThough I do think there are some places of the paper can be polished, the overall is sound.", "strength_and_weaknesses": "Strength:\n\n1. The paper is well-written and well-motivated.\n\n2. The algorithm is novel to me. Incorporating physics prior to alleviate the costly computation is interesting.\n\n3. The techniques are sound and performance on the benchmark tasks are good.\n\nWeakness:\n\n1. The deduction from Equ. 1 to Equ. 2 can be clearer or supplemented in appendix.\n\n2. Will the choice of ``s_const’’ in speed model influence the training? I think change of s_const will also change the numeric distribution of  corresponding speed field and time field.\n\n3. Adopting Eikonal equation as physics prior is actually a little arbitrary. I do not see the physics connection between a wave propogation model to a planning problem. Does the trajectory of wave propogation guarantee an optimal trajectory path? Apparently no. It is also confirmed in the result tables. It does not generate the shortest path with highest success rate. \n\n4. Analysis on failure cases would be nice.\n\n5. Typos:\n\n    a) In many places, “expensive” is written as “expansive”", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GMHT9z0sGH", "reviewer": "Reviewer_9pD7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Neural Time Fields (NTFields), A novel physics-informed formulation of learning the continuous arrival time field for robot motion planning. NTFields generate accurate time fields and plans faster than existing baselines in cluttered 3D environments.", "summary_of_the_review": "This paper proposes a novel perspective and technical framework for learning physics-informed neural time fields for motion planning. However, I believe there is much room for improvement in the experimental comparison, and it lacks enough discussions on limitations.\n\n--------------------------------------\n### Post-rebuttal update:\nThe authors improved the experimental comparison and included discussions on limitations. Given the novelty, the sound technical approach, and adequate experimental comparison, I recommend acceptance. In my opinion, the major limitations are computational overhead and generalization to different scenes, which deserve to be future work.", "strength_and_weaknesses": "### Strengths:\n- A novel technical approach formulation based on physics-informed learning for motion planning. This perspective is interesting.\n- Empirically better field generation quality and faster planning time in execution given random start and goal configurations.\n\n### Weaknesses:\n- Experimental comparisons are weak. In my opinion, it can be much improved in several ways. See more details below. \n- Expensive training time as shown in Table 3 of the Appendix. Though the data generation is fast, training takes on the magnitude of hours. In this case, given the budget of hours, FMM or IEF3D might not be computationally intractable, so more comparisons are needed.\n- Not clear if the learned time field can generalize to different scenes (as IEF does). Do we need to retrain from scratch (for hours) when the underlying scene/layout changes?\n- No explicit discussions on limitations and failure cases - this is very important for a fair understanding of the paper.\n\n### Experimental comparison issues:\n- It would be better to show qualitative comparisons on field generation on more examples (only 2 currently) or present quantitative metrics to support the claim that NTFields generate correct time fields.\n- Lack of environment variations in the cluttered 3D environments - what would happen for different layouts? Also, the performance metrics “length” and “safe margin” seem somewhat contradicting to me, so not sure what the preference is here and if the proposed approach can control the trade-off.\n- As mentioned above, FMM and IEF3D should be included in the comparison on the 4-DOF and 6-DOF manipulation examples, as the training time for NTFields takes 15+ hours. It’s necessary to see under the same generation+training time budget, how good FMM and IEF3D can achieve (the grid resolution does not need to be very high if they are indeed intractable, say it takes hundreds or more hours to generate the data).\n\n### Questions:\n- Does the bi-directional gradient-following approach always converge to a single path? Have you seen any cases where following the gradient will get stuck in a local plateau?\n- Regarding the training details in Appendix D, how are the hyper-parameters of l_0 and l_1 chosen for different examples, or how to adjust them in a generalizable way?\n- The nonlinear symmetric operator seems to help a little when looking at Figure 3. However, it is not clear how necessary it is for more examples and if it introduces extra training overheads.\n\n--------------------------------------\n### Post-rebuttal update:\nThe authors addressed most of my concerns described above, except for some major limitations of the method.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3y3_jRGe9N", "reviewer": "Reviewer_E33u", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The proposed paper introduces a novel method for robot motion planing based on a wave propagation model. The introduced model generates path solutions based on a non-linear first-order PDE (Eikonal Equation). The method is validated for motion planning tasks in cluttered 3D environments (Gibson). The discussed results indicate high success rates and lower computation times compared to SOTA methods. ", "summary_of_the_review": "Overall, this is an interesting paper that can be accepted to ICLR 2023. The paper is very well written, the experiments are sound, and the problem the proposed method addressees is important. The proposed NTField framework is a novel contributions that also has the potential to stimulate future work in the field of motion planing for robotics. Furthermore, it seems that physics-informed approaches are an important and novel direction in the field of robotics. An interesting direction (for future work) would be to employ the proposed framework for real world robotics experiments. Finally, I would like to encourage the authors to carefully discuss the limitations of the propose method. ", "strength_and_weaknesses": "Strengths: \n\n- The paper addresses an important problem with a sound technical solution. \n- The paper is very well written and easy to read. \n- The paper discusses a set of meaningful experiments that show the benefits of the introduced NMP based on the NTField framework.\n\nWeaknesses: \n\n- The paper does not discuss any limitations. \n- The method is only validated on synthetic data and not on real robots. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%