Learning Continuous Environment Fields via Implicit Functions
Authors: Xueting Li
Shalini De Mello
Xiaolong Wang
Ming-Hsuan Yang
Jan Kautz
Sifei Liu
Venue: ICLR 2022 Poster (2022)
Keywords: Continuous Scene Representation
Implicit Neural Networks

Abstract:    We propose a novel scene representation that encodes reaching distance -- the distance between any position in the scene to a goal along a feasible trajectory. We demonstrate that this environment field representation can directly guide the dynamic behaviors of agents in 2D mazes or 3D indoor scenes. Our environment field is a continuous representation and learned via a neural implicit function using discretely sampled training data. We showcase its application for agent navigation in 2D mazes, and human trajectory prediction in 3D indoor environments. To produce physically plausible and natural trajectories for humans, we additionally learn a generative model that predicts regions where humans commonly appear, and enforce the environment field to be defined within such regions. Extensive experiments demonstrate that the proposed method can generate both feasible and plausible trajectories efficiently and accurately.

Published as a conference paper at ICLR 2022
LEARNING CONTINUOUS ENVIRONMENT FIELDS VIA IMPLICIT FUNCTIONS
Xueting Shalini De Mello², Xiaolong Wang', Ming-Hsual Yang', Jan Kautz²; and Sifei Liu²
UC Merced
NVIDII
3 UC San Diego
(a) 20 Maze
(b) SDF level set
[c] SDF visualized
Environment field level set
[e] Environment field visuelized in 3D
Searched trajectories environments
Figure 1: Implicit environment fields in 2D and 3D space. With implicit functions, learn continuous environment field encodes the reaching distance between any pair of points and space. present utilization of the environment field for 2D maze navigation (d)-(c) and human scene interaction modeling in 3D scenes in (f). Note that we flip the environment field in (c) upside down for visualization purposes.
ABSTRACT
We propose novel scene representation that encodes reaching distance the distance between any position in the scene to goal along a feasible trajectory. directly guide the
We demonstrate that this environment field representation dynamic behaviors of agents in 2D mazes or 3D indoor scenes. Our environment field continuous representation and learned via neural implicit function using discretely sampled training data. We showcase its application for agent navigation in 2D mazes, and human trajectory prediction in 3D indoor environments. To produce physically plausible and natural trajectories for humans, we additionally learn a generative model that predicts regions where humans commonly appear, and enforce the environment field to be defined within such regions. Extensive experiments demonstrate that the proposed method can generate both feasible and plausible trajectories efficiently and accurately.
INTRODUCTION
Scene understanding aims to analyze and interpret given environment. The past few years have witnessed tremendous success in scene representation learning for semantic segmentation (Long et al., 2015; Li et al., 2017), 3D scene reconstruction (Sitzmann et al., 2020; 2019), and depth estimation (Godard et al., 2019). The learned representations can be used as inputs for training embodied tasks including visual navigation (Sax et al., 2018; Zhou et al., 2019) and robotic manipu- lation (Chen et al., 2020; Yen-Cher al., 2020). This overall process falls into two-step paradigm, which first leams a static scene representation, and then trains an agent to interact with the scene. While this progress is exciting, ask for exploring alternative research direction: Can the scene

---

Published as a conference paper at ICLR 2022
representation itself be dynamicalll interactive? That is, instead of training Reinforcement Learn- ing agent on top of the scene, we encode the dynamic functioning (i.e., the ability to control and guide the agent's interactions) in the scene representation in one-step training.
Specifically, we study representation that guides an embodied agent to navigate inside the scene. We propose scene representation dubbed the environment field that encodes the reaching dis- tance distance from any position to a given goal location along a feasible and plausible trajectory. Such an environment field needs to be continuous and densely covering the entire scene. With this continuous scene representation, given an agent in any locations in the scene, we can perform itera- tive optimization (e.g. gradient descent) over the continuous field move the agent goal. The optimization objective is to minimize the reaching distance, and the agent reaches a given goal when the reaching distance equals to zero.
Compared to search-based algorithms (Sethian, 1996; Valale 1998; Kavraki et al., 1996), the pro- posed environment field enjoys two key advantages: a) Continuity. Search-based methods (Sethian, 1996) assume discrete searching space, while the proposed environment field is continuous and can easily generalize to continuous scenes such as a 3D indoor environment. b) Efficiency. Instead of applying the time-consuming searching algorithm to every new environment or goal position, a single neural network is learned to efficiently produce different environment fields for different scenes or goal positions, with a forward pass.
How to learn this continuous environment field? Naively learning such a dense field in a super- vised matter requires ground truth reaching distance annotation at all possible locations in the scene, which are infinitely many and thus impossible to collect. In this paper, we leverage neural implicit functions (Park et al., 2019; Xu al., 2019; Sitzmann al., 2020; Xu et al., 2019; Mescheder al., 2019) that can learn a continuous field using discretely sampled training data. Specifically, we de- sign an implicit neural network that takes the coordinates of a position in the scene input and outputs its reaching distance to a given goal location. Even though this implicit neural network is trained using discretely sampled position and reaching distance pairs, yet it produces a dense envi- ronment field of a scene. During inference, the reaching distance between any position in the scene and the goal can be efficiently queried via fast network forward pass and used to navigate an agent towards the goal.
We showcase the application of the proposed environment field both human trajectory modeling in 3D indoor environments well agent navigation in 2D mazes and observe comparable if not better results than state-of-the-art methods. Specially, when applying the environment field to human trajectory modeling, additionally propose and learn accessible region plausible regions for humans to appear in the scene via a variational auto-encoder (VAE). This accessible region can be implicitly encoded into the environment field so that it can guide humans to avoid locations that do not conform to common human behaviors (e.g., navigate a human to travel around rather than under a desk).
The main contributions of this work are:
We propose the environment field-a novel scene representation that facilitates direct guid- ance of dynamic agent behaviors in a scene.
We learn dense environment fields via implicit functions using discretely sampled training pairs.
We apply the environment fields human trajectory prediction in 3D indoor environments and agent navigation in 2D mazes, where both feasible and plausible trajectories can be produced efficiently and accurately.
RELATED WORK
Scene representations. Scene understanding aims at parsing and interpreting an environment. Prior approaches learn various scene representations comprising of semantic segmentation (Long et al., 2015; Li et al., 2017), depth (Godard et al., 2019) and meshes (Sitzmann et al., 2020). Nev- ertheless, these representations only capture static scene attributes while the proposed environment field aims to encode the dynamic functioning of scene.
Path planning. Classical path planning algorithms including Breath First Searching (BFS), Di- jkstra's 's algorithm (Dijkstra. 1959) and the Fast Marching Method (FMM) (Sethian, 1996; Valero- Gomez et al., 2015) require applying the searching algorithm to every new environment or goal position, and hence are not suitable for dynamically changing environments. On the contrary, our
2

---

Published as a conference paper at ICLR 2022
environment field generalizes to changing goals and environments during inference and is thus com- putationally efficient. Instead of solving path planning numerically, sampling- based methods such as Rapidly Exploring Random Tree (RRT) (LaValle, 1998; Valll et al., 2001; Bry & Roy, 2011) or Probabilistic Roadmaps (PRM) (Kavraki et al., 1996; 1998) grow and search a tree by sampling feasible locations in the scene. However, the efficiency and performance of such methods depend on the sampling density and algorithm. In contrast, we learn a continuous environment field that predicts the next optimal move instantly for any location in the scene. The potential field algorithm manually computes and combines an attractive and repulsive field and is prone to local minimas, while our algorithm learns a single field in a data-driven way.
The proposed environment field can also be considered as a value function used in learning- -based path planning methods (Tamar et al., 2016; Campero et al., 2020; Al-Shedivat et al., 2018; Chaplot et al., 2021). The reaching distance at each location is essentially the reward the agent gets while try- ing to reach the goal. However, compared to other learning-basse et al., 2016; Campero et al., 2020; Al-Shedivat et al., 2018) 8), the environment field enjoys two major benefits. First, it produces the environment field for all locations in the scene with a single network forward pass (in discrete cases such as 2D mazes), instead of predicting optimal steps sequentially, which requires multiple network forward passes. Second, it utilizes implicit functions instead of convolution neural networks and is thus more flexible to generalize to continuous scenarios such as human navigation in 3D scenes.
path planning algorithms (Tamar
Implicit neural representations. Implicit neural representations have recently attracted much at- tention in computer vision due to their continuity, memory-efficiency and capacity to capture com- plex object details. Numerous implicit neural representations have been proposed to reconstruct 3D objects (Park et al., 2019; Xu al., 2019; Michalkiewicz et al., 2019; Tulsiani et al., 2020; Mescheder et al., 2019), scenes (Sitzmann et al., 2019; Jiang et al., 2020; Peng et al., 2020; Milden- hall et al., 2020), humans (Atzmon & Lipman, 2020; Saito et al., 2019 2020) or even audio and video signals (Sitzmann et al., 2020). Instead of reconstruction, in this work, we use implicit neural function to encode a continuous environment field and target at agent navigation. A more detailed overview of implicit neural networks can be found in Section 3.1.
Affordance prediction. Affordance prediction models the interactions between humans and ob- jects or scenes (Koppula& & Saxena, 2014; Chuang et al., 2018; 8; Zhu et al., 2014; Koppula et al., 2013; Li al., 2019; Karunratanakul et al., 2020). One line of work describes static human-scene interac- tions by generating human poses ( (Wang et al.. 2017; Li et al., 2019) or human body meshes (Zhang al., 2020b;a) given scene images or point clouds. Another line of work (Cao et al., 2020; Mon- szpart et al., 2019) models dynamic human-scene interactions by predicting a sequence of future human poses given pose history. The proposed method falls into the second category, where we pre- dict plausible human motion sequences containing different actions (e.g., walking and sitting down) while avoiding collisions in 3D scenes. The proposed approach enjoys two major advantages com- pared to existing work (Cao et al., 2020; Monszpart et al., 2019): a) Our work does not require any past human motion history as a reference. Once the environment field of a scene is encoded by an implicit function, an agent can be navigated from any position to the goal position. b) Our work can generate much longer navigation paths than existing work thanks to the learned environment field that covers the entire scene.
ENVIRONMENT FIELD
We first introduce the environment field using agent navigation in 2D mazes as a toy example. A real-world application in human trajectory modeling is discussed in Section 4.
3.1 OVERVIEW OF IMPLICIT NEURAL NETWORKS
Implicit neural networks have been widely used to represent 3D object or scene as signed distance function (SDF). The input to an implicit neural network is the coordinates of a 3D point (a E R²) and the output is the signed distance (s R) from this point to the nearest object surface: where 8 >0 0 point is inside the object surface and s < 0 when the point is outside the object surface. A desirable property of an implicit neural network is that it learns continuous surface of an object given discretely sampled training pairs X := (I, 8), i.e., it is able to predict the SDF value for points not in the training set X once trained. During inference, a continuous object surface can be explicitly recovered by extracting an arbitrarily number of points with zero SDF values, i.c., points on the object surface (Lorensen & Cline, 1987).
SDF(x),
3

---

Published as a conference paper at ICLR 2022
3.2 ENVIRONMENT FIELD LEARNING VIA NEURAL IMPLICIT NETWORKS
In this work, we take advantage of the continuity property of implicit neural networks described above and learn continuous enviromment field from discretely sampled location and reaching dis- tance pairs. Specifically, we represent the environment field as a mapping from location coordinates
a E R² to its reaching distance u(x)) E towards a given goal and learn an implicit neural network fe (see Fig. 3 (a)) such that:
(I) To collect a set of training pairs X == far, =(x)), we resort to an analytical method, e.g., fast
a(z) = fo(x).
marching method (FMM) (Sethian, 1996), that solves u(x) for each cell in discrete grid, We then train the implicit function So to regress the reaching distance u(x) computed by the FMM for
each cell X using the L1 loss function:
[(fe(z), (())) = [f(()) =(x)
(2)
Thanks to the continuity of implicit functions, the learned implicit neural network is able to predict the reaching distance even for points outside the training set X, enabling us to query the reaching distance for any location in the scene during inference.
3.3 TRAJECTORY SEARCH USING ENVIRONMENT FIELD
Given the learned implicit function in Sec- tion 3.2, we now deseribe how to search a tra- jectory from a starting position 2, to the goal position Fe- Since the learned implicit neural network is continuous, one intuitive solution for navigation is through gradient descent method (see Fig. 2(c)). We first feed the current location Te of an agent into the learned implicit func- tion. Then, by minimizing the output reaching distance, the agent can be nudged by the gra- dients Ar. proppprop from the implicit function to a new position z,-- - Ax., where the reaching distance is smaller, i.e., closer to the goal. Starting at Tg. we repeat the above pro- cess and gradually decrease the reaching distance from the current position to the goal until the agent reaches the goal at Ie. The gradient descent method computes the direction at each move based the function values within infinitesimal area surrounding the current location, and thus the agent can only take an infinitesimally small step at each move.
Figure 2: Trajectory searching. (a) The 8 possible directions that an agent can take 2D maze. (b) Searched trajectory by querying the environment field values of positions the agent can reach within one step. (c) Scarched trajectory by gradient descent. The red tri- angle and the green square represent the starting and goal positions, respectively.
However, in 2D maze, agent usually takes fixed, much larger step, e.g., one grid cell each move. Therefore, we constrain that an agent moves by one grid cell along one of 2 8 possible directions at each time, as shown in Fig. 2(a). We then query the learned implicit function for the reaching distance values at all possible positions that the agent can reach in one time step and move the agent to the position with the smallest reaching distance. We keep updating the agent's 's position until it reaches the goal (see Fig. 2(b)).
3.4 CONDITIONAL ENVIRONMENT FIELD
We now present how to generalize the environment field to arbitrary goal positions and scenes. Environment field for arbitrary goal positions. To learn an implicit function that predicts dif- ferent environment fields for different goal positions the same scene, we utilize a conditional implicit function (Park et al., 2019) as shown in Fig. 3(b). The input to the implicit function is the concatenation of both the goal coordinates and the query position coordinates. The output is the reaching distance from the query position to the goal, i.e., environment field value. During training, we randomly sample a pair of empty grid cells and set one of them as the goal. We train the implicit function with the reaching distance computed by FMM. Environment field for arbitrary goal and environment. to an arbitrary environment, we propose a context-aligned implicit function as shown in Fig. 3(c). Given scene context (i.e., 2D maze map), we first extract scene context features by fully convo- lutional environment encoder. Then, we forward the concatenation of (a) goal coordinates, (b) query position coordinates, and (c) scene context features aligned to the query position into the implicit
To further extend the implicit function
4

---

Published as a conference paper at ICLR 2022
every: (4
Ob
goal
implict enwiromment
Implict cnvironment
different goals
PAODO moges
IEF
environment
encoder
Implict environment field conditioned envronment
Figure 3: Variants of implicit environment functions (IEFs). The IEF in (a) assumes fixed environment and a fixed goal position; (h) generalizes to different goal positions in the same environment; (c) further gen- eralizes different goal positions in different environments. Goal positions this figure, we use the 2D maze as an example, models (a) and (b) can be generalized to scenes by simply changing the inputs from 2D coordinates to 3D coordinates.
represented by blue eircles. In
IB D
50 TiTheee- 50
Figure 4: Bird's eye view navigation results. We show the searched trajectories for two people in (a). In (b)-(d), we illustrate the positions of the two people at different time steps and how the position of one person (green) affects the reaching distance level-set of the other person (yellow) dynamically to avoid collision. Brighter color in (b), (c) and (d) indicates locations that closer the goal.
function and regress the reaching distance value for the query position. The fully convolutional environment encoder and scene environment feature alignment are two key ingredients in this de- sign. The former enlarges the receptive field the query position so that the implicit function has sufficient contextual information to predict the reaching distance to the goal. The latter encourages the implicit function to focus only on the query position while ignoring other scene context infor- mation. We note that hypernetworks (Sitzmann et al., 2020) another option to encode arbitrary environments and goals. However, the hypemetwork solution is empirically less accurate than the proposed context-an aantext-aligned implicit function, especially for mazes with larger sizes. Experimental comparisons between these two designs are presented in Section 5.2 and details of the hypernetwork in Appendix C.I.
For the maze navigation task, we learn the context-aligned implicit function (Fig. 3(c) using differ- mazes, along with the reaching distance computed by FMM training data. During inference, we predict an environment field for an unseen maze with given goal position and search for feasible trajectory from any starting position to the goal using the strategy discussed in Section 3.3. Fig. 6 shows learned environment fields along with searched trajectories for unseen maze maps.
4 HUMAN NAVIGATION IN INDOOR ENVIRONMENTS
Going beyond the toy example of agent navigation in 2D mazes, discuss how to apply the envi- ronment field to real-world 3I) scenes in this section. Given the point cloud of each scene, we aim generate physically feasible and semantically plausible trajectory for human from starting position to the goal. We then align a random sequence of human poses onto the generated trajectory (see Fig. 7). While the accessible and obstacle locations in 2D mazes well-defined, the acces- sible region for humans in 3D scenes is difficult to identify and usually requires prior knowledge of human behavior. We propose two solutions determine the accessible region 3D scenes: (a) A simple solution that utilizes the bird's eye view rendering of the 3D scene (see Section 4.1) and only models simple human motion, i.e., walking. (b) A learning- basedd solution that generates the accessible region in 3D scenes and models more complex human actions, e.g., sitting down, etc. (see Section 4.2).
5

---

Published as a conference paper at ICLR 2022
4.1 BIRD'S s EYE VIEW ENVIRONMENT FIELD
To model human walking, we define the floor area as the accessible region for humans from the bird's eye view of the 3D scene. We first produce a bird's eye view image and depth through rendering. Then we check the depth of each pixel in the bird's eye view image and label pixels with the maximum depth as the floor. All pixels belonging to the floor area are marked as accessible while other pixels are marked as obstacles. We then this map the scene environment condition. Similar to 2D maze navigation discussed in Section 3.4, we assume arbitrary goal positions in a dynamically changing indoor environment and learn a context-aligned implicit function shown in Fig. 3(c) to encode the environment field of the bird's eye view image. During inference, given the starting and goal positions on the floor, we search for a feasible trajectory that leads the human(s) to travel from the starting position to the goal while avoiding collision with obstacles in the room. We target both single and multiple human navigation.
Single human navigation. From starting position, we assume an average human step size and move the human one step along one of the eight directions (see Fig. 2(a)) that yields the largest reduction in reaching distance. We repeat this process until the human reaches the goal. Given the searched trajectory in the bird's eye view, we align an existing human walking sequence onto it by rotating the human pose towards the tangent direction of the trajectory at each time step. More details of the alignment can be found in Appendix C.3. Fig. I(f) and Fig. 4(a) show searched trajectories for humans from the bird's eye view together with the aligned human walking pose sequence.
Multiple human navigation. In addition to single human navigation, we show that our method can be easily applied to multiple people navigation a challenging and computationally expensive task to be solved by any existing approach. Taking the two people in Fig. 4 as an example, we denote the yellow and green ones as person P1 and P2. respectively. To navigate P1 from the left bottom corner to the center of the room while avoiding collisions with the furniture and Pg. at each time step, we modify the scene environment by marking the location of Pa as an obstacle. Similarly, we update the scene environment for P2 by marking the current position of P1 as an obstacle and move P2 accordingly. For this task, the scene environment is dynamically changing based on the motion of the other human and so is the predicted environment field. This is why we adopt the context- aligned implicit function in Fig. 3(C) rather than the model in Fig. 3(b). We also randomly mark empty spaces as obstacles during training to mimic the arbitrary scene context encountered during inference. Fig. 4(b)-(d) illustrate how the environment field of P1 changes w.r.t. the motion of Pa, where the current position of P2 is predicted as inaccessible to P1 by the learned implicit function.
4.2 3D SCENE ENVIRONMENT FIELD
The solution discussed above simplifies human navigation in the 3D space to the 2D image space but is restricted to modeling the motion that mostly happens in the horizontal floor plane, i.c., walking. Next, we demonstrate how to generalize the environment field to 3D scenes and model more complex motions such as sitting and lying down, getting up, etc..
Modeling data-driven accessible region in 3D scenes. Unlike 2D mazes or the bird's eye view where the unoccupied floor region is explicitly given as the accessible region, in the 3D indoor scene, we define accessible region as plausible regions for human torsos to be placed. Given human pose sequences from the training set, we learn a VAE (Kingma & Welling, 2013) to model the distribution of human torso locations conditioned layout. As shown in Fig. 5, given a scene point cloud, we first take the global feature pro- duced by the pooling layer of PointNet (Qi al., 2017) as the scene context feature. We map the context feature together with human torso location observations to a normal distri- bution using an encoder. A decoder then reconstructs the human torso location given the concatenation of a sampled noise from the normal distribution and the scene context feature. We learn this conditional VAE using the reconstruction objective on hu- man torso locations together with the Kullback-Leibler (KL.) divergence objective (Kingma & Welling, 2013). During inference, we sample random noise from a standard normal distri- bution to generate feasible locations for human torsos to formulate the accessible region in 3D scene. To further suppress the noisy generation by the VAE model and filter out locations that po- tentially lead to collisions with furniture in the room, we project all generated locations to the bird's
the scene
point doud
D
Porehees
reconsnnsca
torso (4.3.2)
Figure 5: Conditional VAE for accessible space generation.

---

Published as a conference paper at ICLR 2022
eye view and remove locations that fall furniture or walls. Examples of generated accessible regions are shown in Fig. 7, where the VAE successfully predicts plausible locations suitable for humans sit and walk on, etc. while avoiding collisions with furniture in the room.
Environment field learning for 3D scenes. Given the learned accessible region above, we define the environment field value of locations outside the accessible region as negative (i.e., inaccessible). We then model the field using the network in Fig. 3 (b), where the input to the implicit function is the concatenation of both the goal coordinates and the query position's 's coordinates. The output is the distance from the query position to the goal.
Pose-dependent trajectory from a continuous field. Given the starting and goal positions as well as random sequence of human poses, we use the learned environment field search a path and then align the human pose sequence onto the predicted path. Note that we focus on trajectory instead of human pose modeling, therefore we assume the action of the given pose sequence is consistent with the given starting and goal position. For instance, if given a random sequence of a human sitting down, the goal position should be on an object (e.g., soft, chair, etc.) that can afford the sitting action. Such a setting is commonly observed in applications such as motion matching (Holden et al., 2020). The key to navigating the human at each step is deciding its step length and direction. For direction computation,
take one of the 3 = 27 directions (denoted as 2 = 1, 2, ..., 27) in 3D space, similarly to the 2D maze scenario shown in Fig. 2, For step length, compute the distance of adjacent torso locations in the given pose sequence as the step length denoted by 8. Thus each possible location 1 human can reach within one step can be computed as 12 = ++++0 U, x s, where / is the human's current location. Although the possible locations can be anywhere in the scene, the continuity of the learned environment field allows us to efficiently query the reaching distance for each possible location by feeding each I/ together with the goal's coordinates into the learned environment field. The human is then moved towards the location with the minimum reaching distance. This process is repeated until the human reaches the goal position. We emphasize that different from the existing human motion prediction method (Cao et al., 2020), the above trajectory prediction process can be carried out given arbitrarily far away starting and goal positions since the environment field densely covers the entire scene.
assume the human
Path optimization by affordance. The path searching process discussed above can be further op- timized by taking affordance into consideration at each step. For each possible location the human can reach within one step size, we check if the human is well supported and does not collide with other objects at these locations. We move the human to the location with the smallest reaching distance value while obeying these two constraints. Similar to (Li et al., 2019), to determine if pose is well supported, we check if the torso, the left lap, and the right lap joints of sitting poses, as well as the feet joints of standing poses have non-positive signed distances to the scene surface. To determine if a pose collides with other objects, we check if all joints of the pose except the afore- mentioned support joints have non-negative signed distances. Finally, we align the pose sequences onto the searched path, details of this alignment process are discussed in Appendix C.3. Fig. 7(b), (d) and (e) illustrate the searched trajectory along with the aligned human poses.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETTINGS
We train the conditional VAE described in Section 4.2 and all implicit functions using the Adam solver (Kingma& & Ba, 2014) with learning rate of 5 x 10-5, To balance the KL-divergence and the reconstruction objectives in the VAE, we adopt the Cyclical Anncaling Schedule introduced in (Fu et al., 2019). We evaluate the proposed environment field on agent navigation in 21) mazes (Section 5.2) and compare with the VIN (Tamar et al., 2016). For dynamic human motion modeling in 3D scenes (Section 5.3), we compare with the HMP (Cao et al., 2020) and continuous sampling- based methods RRT (Bry & Roy, 2011) and PRM (Kavraki et al., 1996). We note that VIN is based on Convolutional Networks and cannot be easily extended 3D scenes.
5.2 AGENT NAVIGATION IN 2D MAZES
We evaluate the proposed environment field on the Minigrid (Chevalier-Boisver et al., 2018) and Gridworld maze datasets (Tamar et al., 2016). which include 2D mazes with randomly placed ob- stacles (see Fig. 6(a)). Fig. 6(b) and (c) show the learned environment fields and the level sets (each level set consists of locations of the same reaching distance to the goal) respectively. Each envi- ronment field accurately encodes the reaching distance from any accessible point to the goal while
7

---

Published as a conference paper at ICLR 2022
Table 1: Ablation study on network architecture. The x mazes are from Minigrid (Chevalier- Boisvert et al., 2018) dataset while all other mazes are from the Gridworld (Tamar et al., 2016) dataset. IF stands for implicit function.
Maze Size
8x8
11x11
16x 16
x8
Hypernetworks
98.08
82.05
20.77
3.13
Context-aligned IF
99.69
80.14
98.29
97.00
Table 2:
Success rate and efficiency for maze navigation.
Success ratef
Efficiency (milliseconds))
Size
8x8
16x 16
28x28
8x8
16x 16
28
VIN
99.60
99.30
96.97
3.88
22.72
82.07
Ours
99.69
98.29
97.00
13.26
14.81
20.41
successfully detecting all obstacles the maze. Figure 6(a) demonstrates the effectiveness of using
the learned environment field in navigating an agent to the goal while avoiding collision with obsta- cles. Quantitatively, we use the success rate the percentage of paths that successfully lead an agent to the goal among all searched paths, as the evaluation metric.
Hypernetworks v.s. context-aligned implicit function. Table shows the ablation study on the network architectures discussed in Sec- tion 3.4. The context-aligned implicit func- tion performs comparably to hypernetworks smaller-sized mazes (8 x 8 and 11 x 11 mazes). However, hypernetworks fail on larger-sized mazes (16 x 16 and 28 x 28 mazes), demon- strating the superiority of context-aligned im- plicit functions over hypernetworks.
Comparison with VIN. We compare the pro- posed method with VIN (Tamar et al., 2016) on the Minigrid dataset (Chevalier-Boisver) et al., 2018). We use the same train and test split as VIN, and learn different implicit functions for mazes of different sizes. Table 2 shows the suc- cess rate and average time taken to search path by different methods. The proposed method achieves a comparable success rate with much less time in larger-sized mazes compared to VIN. This is because the VIN requires network forward pass each location to decide the next opti- mal step, while the proposed method only needs one network forward pass to obtain the reaching distance values of all locations in the maze and can guide the agent at any location reach the goal.
[a]
Envrromment field
Level
Figure 6: Grid-world maze navigation. Blue paths in (a) are planned FMM while red paths scarched by the learned environment field. Green circles and orange squares represent starting positions and goal positions. In (b)(c), a brighter area or contour is closer to the goal.
5.3 INDOOR HUMAN NAVIGATION MODELING
Bird's eye view navigation. In Fig. 4(b)-(d), we illustrate how the current position of one person affects the environment field of the other person. First, the learned level set effectively encodes the reaching distance between any point to the goal position. Areas that are closer to the goal have lower reaching distance values and vice versa. Second, the learned environment field dynamically changes based on the position of the other person. For instance, in Fig. 4(b), the current location of the person colored green is successfully predicted as inaccessible by the other person, i.e., it has a large reaching distance value. These results show that the learned implicit function can effectively model the environment field in scene and generalize to dynamically changing environment.
Data-driven human accessible region learning. We show qualitative results of the generated accessible region (see Section 4.2) in Fig. 7(b). The generated accessible region uniformly spreads out in the 3D room and is plausible for all kinds of human actions. For instance, the points on the chair and sofa are feasible locations for sitting human torsos, while the points in midair are plausible locations for walking human torsos. Besides generated accessible region, we show the learned environment field in Fig. 7(a), where the closer a point is to the goal, the smaller its reaching distance value is (brighter point).
Human trajectory prediction. We search a trajectory for random human pose sequence (see Section 4.2) in Fig. 7(c) and (d). The trajectory successfully avoids colliding with furniture in the room while leading the human towards the goal. We emphasize that the starting and goal positions are 120 frames (8 seconds) apart, showing the capacity of the proposed method in predicting long- term dynamic human motion (see the video in the supplementary material). Our model outperforms
8

---

Published as a conference paper at ICLR 2022
(4) fextromment fiedd
Ir Trsjettrr with a sitting dewn pose seguenze
Ib) Accessible region
Idi Tragestory with a walking p050 secuence
Figure 7: Dynamic human motion modeling on PROX (Hassan et al., 2019) The accessible region in (b)
includes generated torso locations by the VAE model discussed in Section 4.2. In the environment field in (a), the brighter point is, the closer it is to the goal.
Table 3: Quantitative comparison with RRT Valle, 1998) and PRM (Kavraki et al., 1996) on the PROX dataset. "AFF" indicates the pose dependent search discussed in Section 4.2.
Method
RRT
PRM
Ours (w/o AFF)
Ours
distance &
0.127
0.139
0.015
0,015
time (s) 0.153
1.310
0.028
0.038
the state-of-art method (Cao et al., 2020) that can only effectively capture human motion within 2 seconds. Furthermore, the human poses are plausible at each location on the trajectory, demon- strating the effectiveness of the pose-dependent trajectory search process discussed in Section 4.2.
Comparison with RRT (La 1998) and PRM (Kavraki et al., 1996). We present quantita- tive comparisons with sampling-bas based path planning methods in Table 3. We evaluate two metries: (a) distance between the end points of a searched path and the goal. (b) average time cost for single trajectory search. The number of sampling points and nearest neighbors in PRM are set to 500 and 5. For RRT, the maximum iteration for trajectory search is 500. More details can be found in Appendix C.6. Overall, our method is more efficient thanks to the continuous environment field, from which the next step can be predicted by single fast network forward pass. In addition, our method yields better trajectories that successfully reach the goal as shown in Table 3.
Comparison with HMP (Cao et al., 2020). Since we focus on trajectory prediction, we quanti- tatively evaluate with the PathNet in the HMP using the distance metric discussed above as well as the ratio of valid poses to all poses on the trajectory. As described in Section 4.2, we define a pose as valid it is both well supported and collision-free in the scene. We use the same train and test trajectories as the HMP method, with each trajectory including 30 frames. We note that the PathNet in the HMP requires torso locations in the first 10 frames of each trajectory as input and predicts the torso locations for the following 20 frames. While our method directly predicts 30 locations that lead the human to the goal for the entire 30 frames. Quantitative results in Table 4 show that method is effective for navigating humans towards goal positions. Furthermore, after taking affordance into consideration (see Section 4.2), our model able to better fit the human poses to the scene, shown in the last column of Table 4.
Table 4: Quantitative evaluations of human motion trajectory prediction the PROX dataset. "AFF" is the same as Table "Support" and "free" indicate the ratio poses that are either well supported or collision- free, respectively. "Valid" indicates the ratio of poses that obey both constraints.
Method
distance |
support 1
free
valid 1
HMP (Cao et al., 2020)
0.019
81.40
72.34
65.00
Ours (w/o AFF)
0.015
84.47
75.43
71.36
Ours
0.015
86.22
78.56
75.32
CONCLUSION
In this paper, we propose the environment field that encodes the reaching distance between a pair of points in either 2D or 3D space, with implicit functions. The learned environment field is continu- ous energy surface that can navigate agents in 2D mazes in dynamically changing environments. We further extend the environment field to 3D scenes model dynamic human motion in indoor envi- ronments. Extensive experiments demonstrate the effectiveness of the proposed method in solving 2D mazes and modeling human motion in 3D scenes.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
Ming-Hsuan Yang is supported in part by the NSF CAREER grant 1149783.
REPRODUCIBILITY STATEMENT
We describe the proposed framework in Fig. 3, with training and implementation details described in Section C.5 the Appendix. Our method evaluated on publicly avilable dataset (Chevalier- Boisvert et al., 2018; Tamar et al., 2016; Hassan et al., 2019). We plan to release the code.
REFERENCES
Maruan Al-Shedivat, Lisa Lee, Ruslan Salakhutdinov, and Eric Xing. On the complexity of explo- ration in goal-driven navigation. arXiv preprint S.ki::8811.08880 2018. 3
Matan Atzmon and Yaron Lipman. SAL: Sign agnostic learning of shapes from data. In CVPR, 2020. 3
Adam Bry and Nicholas Roy. Rapidly--xploring random belief trees for motion planning under uncertainty. In ICRA, 2011. 3,7, 15
Andres Campero, Roberta Raileanu, Heinrich Küttler, Joshua B Tenenbaum, Tim Rocktäschel, and Edward Grefenstette. Learning with amigo: Adversarially motivated intrinsic goals. arXiv preprint arXiv:2006.12122 2020. 3
Zhe Cao, Hang Gao, Karttikeya Mangalam, Qizhi Cai, Minh Vo, and Jitendra Malik. Long-term human motion prediction with scene context. In ECCV. 2020. 3, 7,9, 14, 15, 16, 18
Devendra Singh Chaplot, Deepak Pathak, and Jitendra Malik. Differentiable spatial planning using transformers. In ICML, 2021.
Bryan Chen, Alexander Sax, Gene Lewis, Iro Armeni, Silvio Savarese, Amir Zamir, Jitendra Malik, and Lerrel Pinto. Robust policies via mid-level visual representations: An experimental study in manipulation and navigation. arXiv preprini MiXiv:2011006660 2020. 1
Maxime Chevalier- -hevalier-Boisvert, Lucas Willems, and Suman Pal. Minimalistic gridworld environment for openai gym. hctps: /tt//ttttttt /ttptttt.bb//ggg////////gg...h.t www hwtmm//maaiieee//aeee http://mmeem/mmmmmmmmmeeeememeee gym-minigrid, 2018. 7, 8, 10
Ching-Yao Yao Chuang, Jiaman Li, Antonio Torralba, and Sanja Fidler. Learning to act properly: Pre- dicting and explaining affordances from images. In CVPR, 2018. 3
Edsger W Dijkstra. note on two problems in connexion with graphs. Numerische Mathematik, 1959. 2, 15
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cyclical annealing schedule: A simple approach to mitigating KL vanishing. In NAACL, 2019. 7
Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J. Brostow. Digging into self- supervised monocular depth prediction. In ICCV, 2019. 1,2
David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint 1rXiv:1609.09 2016. 13
Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J. Black. Resolving 3d human pose ambiguities with 3d scene constraints. In ICCV, 2019. 9, 10, 17
Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, and Michael Black. Stochastic scene-aware motion prediction. In ICCV, 2021. 14
Daniel Holden, Oussama Kanoun, Maksym Perepichka, and Tiberiu Popa. Learned motion match- ing. ACM Transactions Graphies (TOG), 2020. 7
10

---

Published as a conference paper at ICLR 2022
Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, and Thomas Funkhouser. Local implicit grid representations for 3d scenes. In CVPR, 2020. 3
Korrawe Karunratanakul, Jinlong Yang, Yan Zhang, Michael Black, Krikamol Muandet, and Siyu Tang. Grasping field: Learning implicit representations for human grasps. arXiv preprint arXiv:2008. 04451, 2020.
Lydia E. Kavraki, Petr Svestka, J.-C. Latombe, and Mark H. Overmars. Probabilistic roadmaps for path planning in nigh-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 1996. 2, 3, 7, 15
Lydia Kavraki, Mihail N Kolountzakis, and J-C Latombe. Analysis of probabilistic roadmaps for path planning. IEEE Transactions on Robotics and Automation, 1998. 3
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14Xiv:44/2.6900, 2014. 7, 15
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Hema Koppula and Ashutosh Saxena. Physically grounded spatio-temporal object affordances. In ECCV, 2014.
Hema Swetha Koppula, Rudhir Gupta, and Ashutosh Saxena. Learning human activities and object affordances from rgb-d videos. IJRR, 2013. 3
Steven M. LaValle. Rapidly-exploring random trees: A new tool for path planning. Technical Report TR98-11, Computer Science Department, lowa State University, 1998. 2, 3,
Steven M. Valalee, James J. Kuffner, BR. Donald, et al. Rapidly-exploring random trees: Progress and prospects. Algorithmic and Computational Robotics: New Directions, 2001. 3
Xueting Li, Sifei Liu, Kihwan Kim, Xiaolong Wang, Ming- Yang, and Jan Kautz. Putting humans in scene: Learning affordance in 3d indoor environments. In CVPR, 2019. 3.7
Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, and Yichen Wei. Fully convolutional instance-aware semantic segmentation. In CVPR, 2017. 1,2
Jonathan Long, Evan Shelhamer, and Trevor Darrel.. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 1
William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3d surface construc- tion algorithm. SIGGRAPH,, 1987. 3
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, 2019. 2,3
Mateusz Michalkiewicz, Jhony K. Pontes, Dominic Jack, Mahsa Baktashmotlagh, and Anders Eriks- son. Implicit surface representations as layers neural networks. In ICCV, 2019. 3
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Aron Monszpart, Paul Guerrero, Duygu Ceylan, Ersin Yumer, and Niloy J. Mitra. iMapper: Interaction-guided scene mapping from monocular videos. SIGGRAPH, 2019.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In CVPR, 2019. 2, 3, 4
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In NeurIPS, 2019. 15

---

Published as a conference paper at ICLR 2022
Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolu- tional occupancy networks. In ECCV, 2020.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In ICCV, 2019. 3
Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In CVPR, 2020.
Alexander Sax, Bradley Emi, Amir R Zamir, Leonidas Guibas, Silvio Savarese, and Jitendra Ma- lik. Mid-level visual representations improve generalization and sample efficiency for learning visuomotor policies. arXiv preprint 1rXiv: 2018.
James Sethian. fast marching level set method for monotonically advancing fronts. PNAS, 1996. 2, 4
Vincent Sitzmann, Michael Zollhöfer, and Gordon Wetzstein. Scene representation networks: Con- tinuous ad-structure-award neural scene representations. NewrlPS, 2019. 1,
Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wet- zstein. Implicit neural representations with periodic activation functions. NeurIPS, 2020. 1, 2. 5, 13
Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel. Value iteration networks. In NeurIPS, 2016. 3, 7, 8, 10, 13
Shubham Tulsiani, Nilesh Kulkarni, and Abhinay Gupta. Implicit mesh reconstruction from unan- notated image collections. arXiv preprint arXiv:2007.08504, 2020. 3
Alberto Valero-Gomez, Javier Gómez, Santiago Garrido, and Luis Moreno. Fast marching methods in path planning. IEEE Robotics Automation Magazine, 2015. 2
Gül Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid. Learning from synthetic humans. CVPR, 2017.
Xiaolong Wang, Rohit Girdhar, and Abhinav Gupta. Binge watching: Scaling affordance learning from sitcoms. In CVPR, 2017.
Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. DISN: Deep implicit surface network for high-quality single-view 3d reconstruction. In NeurlPS, 2019. 2, 3
Lin Yen- ChenC Andy Zeng, Shuran Song, Phillip Isola, and Tsung- Yi Lin. Learning to see before learning to act: Visual pre-training for manipulation. In ICRA, 2020. 1
Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, and Siyu Tang. PLACE: Proximity learning of articulation and contact in 3D environments. In 3DV, 2020a. 3
Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J. Black, and Siyu Tang. Generating 3d people in scenes without people. In CVPR, 2020b.
Brady Zhou, Philipp Krähenbühl, and Vladlen Koltun. Does computer vision matter for action? arXiv preprint arXiv: /2887, 2019.
Yuke Zhu, Alireza Fathi, and Fei-Fei. Reasoning about object affordances in a knowledge base representation. In ECCV, 2014. 3
12

---

Published as a conference paper at ICLR 2022
A OVERVIEW
In this appendix, provide more details and experimental results of the proposed implicit environ- ment field approach. First, we differentiate the proposed method and VIN (Tamar et al., 2016) in Section B. Then, discuss the implementation details of method and baselines in Section C. Finally, more experimental results, including failure cases are shown in Section D.
B COMPARISON WITH VIN
Our method is fundamentally different from VIN (Tamar et al., 2016) in the following aspects: a) Goal. We aim to utilize scene understanding to model dynamic agent behaviors, especially human behaviors in indoor environments. However, VIN aims at learning neural networks to approximate the value-iteration algorithm and focuses on learning reward, transition and value functions to obtain the optimal policy for agent navigation. b) Capacity. Our method is more powerful than VIN and other reinforcement learning methods. For instance, naturally enables navigating multiple humans in the same scene, which is hard to achieve using VIN if corresponding supervision is not provided. c) Flexibility. We utilize implicit functions - representation of a 3D scene to model the inherent property (reaching distance) of the scene, which captures how long it takes travel from a point to given goal plausibly and feasibly. Thanks to such a representation, our method can be flexibly generalized complex continuous scenes like 3D indoor enviromments. While VIN depends on convolution operations, which cannot be directly applied to continuous 3D scenes.
Comparisons with other path planning methods can be found in Section
C IMPLEMENTATION DETAILS
C.I HYPERNETWORKS
To encode different maze environments as well as goal positions, we use hypernetwork (Ha al., 2016) as shown in Fig. 8. For each maze map, we first use hypernetwork (blue trapezoid in Fig. 8) to predict the parameters of a hyponetwork (yellow trapezoid in Fig. 8). The hyponetwork takes the concatenation of the goal coordinates as well as the query position coordinates and outputs the reaching distance between them. The hypernetwork includes 11 convolutional layers followed by 7 fully- collecedndeeted layers. The hyponetwork contains 7 fully-connected layers, each of which is followed by a periodic activation as in (Sitzmann et al., 2020). A comparisons between the hyper- network architecture and the proposed context-aligned implicit function can be found in Section 5.2.
(x ) (%.))
mazes images
environment field
Figure 8: Hypernetwork architecture. Blue trapezoid is the hypernetwork that predicts the parameters of the hyponetwork (the yellow trapezoid).
C.2 TRAJECTORY SEARCHING 2D MAZES
The agent in 2D maze may be stuck at a location during the trajectory searching process. To resolve this issue, we sample from all possible locations in the next step. instead of directly taking the one with the smallest reaching distance. Specifically, for all eight possible directions, compute probability vector formed as the inverse of their reaching distance. We treat this probability vector multinomial probability distribution and sample one direction out of eight.
13

---

Published as a conference paper at ICLR 2022
C.3 TRAJECTORY SEARCHING THE BIRD'S EYE VIEW
Multiple humans navigation. Here we summarize the algorithm to navigate multiple humans simultaneously while avoiding collision with objects or other humans, as discussed in Section 4.1. As shown in Algorithm I, for each human i at each time step 4, we mark the current positions of other humans obstacles and search the optimal step based on the modified scene environment. We denote the total step number as T and total human number as H.
Algorithm 1 Multiple humans navigation.
t-1
i-1
j+1
while T do
while S H do
Initialize scene environment
while S H do
if
jtthn
Mark the position of human as an obstacle
end if end while
Predict environment field for human w.r.t the modified scene environment Search next optimal step for human
end while end while
Aligning walking sequence to searched trajectories. As discussed in Section 4.1, align random human walking sequence from the SURREAL dataset (Varol et al., 2017) to the searched trajectory from the bird's eye view. We show this process Fig. 9. At each step, we compute the angle 0 between the human torso displacement (red arrow) and the trajectory's tangent (green arrow). The human pose is then rotated by align with the trajectory's direction.
torso disp iscemeent
trajectory +
trajectory tangent
Aligned walking sequence to searched trajectory. The figure is for visualization purposes only. The human takes much smaller step in reality.
Figure 9:
C.4 TRAJECTORY SEARCHING IN 3D INDOOR ENVIRONMENT
Aligning pose sequence to searched trajectories. Given the predicted trajectory in 3D room (see Section 4.2), we align a random walking or getting-up pose sequence from the SAMP dataset (Hassan al., 2021) onto the trajectory. The alignment process of a walking sequence is the same as in the bird's eye view scenario discussed in Section C.3. While for the getting-up pose sequence, we first translate each pose by moving its torso onto the predicted trajectory. To enforce the poses to be well afforded by its surroundings, we adjust it by rotating within [-6, 6] degrees and choose the angle with the best affordance (i.e. with the most joints well supported and collision-free). Note that the fitting process is the same for trajectories predicted by our method and baseline method (Cao et al., 2020) for fair comparisons.
C.5 NETWORK TRAINING
In this work, instead of using the raw reaching distance computed by FMM supervision to learn the implicit function, we make two changes: (a) We use the reciprocal of the reaching distance from the feasible point to the goal and normalize it to [0. 1]. (b) We encourage the implicit function to predict negative one instead extremely small values for locations occupied by obstacles. this way, the network better differentiate feasible locations and obstacles.
14

---

Published as a conference paper at ICLR 2022
During the training of the VAE model discussed in Section 4.2, besides the points the training human trajectories, we also augment more points in the room. Specifically, we use a random stand- ing human pose and include all locations that afford this pose, i..., the pose can be supported by the floor and does not collide with other objects.
We learn single conditional VAE model for all training trajectories in different scenes in (Cao et al., 2020), while separate implicit function is trained for each 3D indoor scene. All models introduced in this paper are trained with the Adam solver (Kingma & Ba, 2014) using learning rate of 5 x 10-5. We implement the proposed method in the Py PyTorch framework (Paszke et al., 2019).
C.6 PATII PLANNING BY RRT AND PRM
Ashort tojectory
Do Rood map it by PIM
(b) Thee searthed by RIT
Figure 10: Visualization path planning by RRT (Bry & Roy, 2011) PRM (Kavraki al., 1996).
Ground-truth trajectories are colored in red.
For path planning, the RRT (Bry & Roy, 2011) method starts from a given point and builds a tree by adding randomly sampled collison-free points to the tree until one branch of the tree reaches close to the goal. Instead, the PRM (Kavraki al., 1996) first samples set of collision-fi -fre points in the space and builds map by establishing edges wherever possible. then searches a trajectory from given point to the goal using the Dijkstra's algorithm (Dijkstra, 1959). Both these methods less efficient than the proposed method due to the randomness in the sampling process. As shown in Fig. 10, the two methods build large road map or even when searching for short trajectory (green circles). On the contrary, the proposed method efficiently computes the next optimal step with one network forward pass and thus more efficient. quantitative comparison can be found in Table 3.
D EXPERIMENTAL RESULTS
D.1 MORE 2D MAZE NAVIGATION RESULTS
In Fig. 11, we show more results of agent navigation in 2D mazes as well as learned environment fields. As discussed in Section 5.2, the learned environment field successfully captures the reaching distance from any point to the goal (see Fig. 11(b)&(c) and effectively guides the agent from the starting position the goal in Fig. la))).
D.2 DATA-DRIVEN HUMAN MOTION MODELING
In Fig. 13, we show more results of fitting given human pose sequences searched trajectories in 3D indoor environments.
As discussed in Section 4.2, given human pose sequence, compute the step size of the human at each time step and dynamically determine the best next move w.r.t the step size. We also utilize the given human pose to avoid moving humans locations that violate physical constraints in the room, i.c., locations that cannot well support the human or lead to collision with other objects.
D.3 FAILURE CASES
Since we only focus on predicting human trajectories using the learned environment field in this work, fit given random pose sequence onto the predicted trajectory rather than model human
15

---

Published as a conference paper at ICLR 2022
Figure 11: Grid-world maze navigation results.
Figure 12: Failure cases.
pose distribution. Although assume the action of the given human pose sequence is consistent with the start and goal position (e.g. getting-up sequence to trajectory ending at a sofa's surface), however, the given pose sequence may still not well suite for the scene. For instance, The red circle shown Fig. 12 demonstrates failure case where the human collides with the floor. This collision is mainly cased by the fact that the given "getting-up" sequence does not suite the height of the sofa. We believe possible solution is to further utilize the environment field to predict human poses besides trajectories. We leave further exploration along this line future works.
D.4 TRAJECTORY 3D SCENES VISUALIZATION
In Fig. 14, we visualize the trajectories searched by our method and the baseline method Cao et al. (2020). As shown in the figure, our method produces trajectories that include less noise and align better with the ground truth trajectories better.
16

---

Published as a conference paper at ICLR 2022
Time
AAAA
Figure 13: Dynamic human motion modeling on PROX (Hassan al., 2019). We highly recommend viewing the accompanying video for dynamic visualizations.
17

---

Published as a conference paper at ICLR 2022
Figure 14: Trajectory visualization in 3D scenes. The ground truth trajectory, the trajectory searched by HMP Cao et al. (2020) and the trajectory searched by our algorithm are shown red, blue and green.
18


Reviews:
[{"review_id": "is_3wpRCYpp", "reviewer": "Reviewer_AhgQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n- I believe the main strength of the proposed solution is the ability of the learned value function to generalize to different goals & different scenes, which is ensured by conditioning the network on a target goal and learned features from the 2D layout of the specific scene in question (e.g. section 3.4 and Fig. 3, c)). It is nice to see that an MLP can indeed generalize successfully when different goals & scene layouts are fed in, as illustrated by the multiple human navigation experiment (shown in Fig. 4). Section 3 also mentions predictions are made for unseen mazes during inference, which is good.\n- The showcased value-function field evaluations in Fig. 1 and Fig. 6 also appear meaningful, capturing the correct geodesics in the scenes and showing useful predictions when the goal is switched.\n- Planning based on the learned representation seems possible, with good success rates for the considered environments, albeit with the assumed discretization of 8 (in 2D) or 27 (in 3D) directions of movement mentioned in sections 3.3 and 4.2 (see my point below).\n\n## Weaknesses\n- Looking at figure 2, c), which shows an example of obtaining a navigation plan via gradient descent through the learned representation, it seems like the trajectory is inconsistent and unstable at times. A main advantage of the proposed value function representation is that it is continuous (compared to the discrete data it is trained on), which would require meaningful interpolation in the regions between training data. I am worried that, given the example in figure 2, the gradients (and respectively predictions) of the network in these interpolation regions (i.e. between the points from FMM) might be suboptimal. Can you please comment on this? Right now I am interpreting this as signs of overfitting.\n- Along the same line of reasoning as the previous point, constructing plans from the learned network by choosing from discrete directions of movement (8 for 2D, 27 for 3D) seems inconsistent with the continuity argument. To me it seems like a step backwards towards maintaining a discrete tabular value function. Ideally, gradient descent should work.\n- The idea of learning conditional value functions for goal-reaching tasks that generalize to different goals is not exactly novel, e.g. works like [2] have already considered generic formulations in that regime that go beyond holonomic navigation in $\\mathbb{R}^2$ or $\\mathbb{R}^3$ (you can check the references section of the mentioned paper for more examples of works from the \"goal-reaching RL\" field). More generally, learning continuous value functions is standard practice in the RL and optimal control communities, and I think the link to that should be more emphasized in the paper. \n- When modeling accessible regions in 3D space with a VAE (section 4.2), what is the motivation behind using the VAE model to begin with, i.e. why not directly use the empirical distribution of human torso locations the VAE is trained on? I am asking because the trained generative part of the VAE will be a proxy for the empirical distribution it was trained to approximate. In section 4.2 there is also the note that samples from the generative model are further filtered out by checking for collisions–this makes it hard to judge if the generative model is indeed reflective of accessible regions, or if this resampling / filtering step is mostly responsible.\n- Section 4.2 mentions locations outside of the accessible region are marked with negative value, but how do you determine whether a point lies in the region or not? When using the generative part of the VAE (or the data empirical I mentioned, for that matter), this region would not be defined in a closed form. This needs further clarification.\n- NERF [3] should probably be cited in the related work section on implicit neural representations.\n\n## Other remarks\n- Looking at the supplementary material, there are some accidental flips of the human skeletons in the videos. Is this because of issues with the alignment of the movement primitives to the planned trajectories, or because the planned trajectories have occasional jitter?\n\n\n## References\n[1] James A Sethian. A fast marching level set method for monotonically advancing fronts. PNAS, 1996\n\n[2] Eysenbach, Benjamin, Ruslan Salakhutdinov, and Sergey Levine.  \"C-learning: Learning to achieve goals via recursive classification.\" ICLR 2020\n\n[3] Mildenhall, Ben, et al. \"Nerf: Representing scenes as neural radiance fields for view synthesis.\" ECCV 2020.\n\n", "summary_of_the_paper": "The paper proposes modeling reaching distance between any start position and any goal (subject to obstacle avoidance) with a neural network. This is equivalent to parameterizing a traditional path-planning (goal-reaching) continuous value function with the network, which the authors also mention in the introduction section. Different variants of the network are considered, with conditioning on goal (aiming to generalize to different goals) and conditioning on a 2D scene layout (aiming to generalize across different scenes). The network is trained in a supervised manner on data obtained from a traditional search method (fast marching method [1]) that assumes discrete states. The usefulness of the trained value network for navigation and its generalization properties are then experimentally validated in 2D and 3D environments, including an interesting navigation example with two dynamically moving agents in the same scene.", "summary_of_the_review": "Because of the empirically validated generalization properties of the presented model when the 2D scene layout & goal inputs are switched, I am leaning slightly positive, but currently the concerns listed in my review are stopping me from giving a higher score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "SsIAeTMfZE7", "reviewer": "Reviewer_fAEP", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths:**\n- Writing. The paper is well-written and is mostly easy to follow.\n- Method / motivation. Proposed method seems novel and technically sound. Generally, the framework of implicit neural representations seems like a great fit for the task at hand.\n- Experimental evaluation is conducted on several challenging tasks, a comprehensive comparison to baselines, including recent ones, is provided. Quantitatively, the method seems to be performing similarly to baselines or slightly better, while being more efficient.\n\n**Weaknesses:**\n- Reproducibility. There is not much details provided on the architectures of the networks. Figure 3 could benefit from more detailed explanations.\n- Method. Authors claim that continuity is one of the core benefits of their approach, yet in practice they seem to be resorting to a discretized solution when doing trajectory search. Is there any benefit of using continuous representation in 2D? Say, over predicting a discretized field with a convnet. \n- Experimental evaluation. It would be great to also have an idea about how the results look qualitatively for the baselines - it is hard to judge if the quantitative improvements are significant. \n- Failure cases. There are some of the failure cases presented for the human motion prediction, but not for maze navigation. In particular, I wonder if the greedy strategy ever causes issues when the agent is stuck in an infinite loop between two locations, e.g. when the reaching distance is identical in those locations, and how one can tackle that? \n\n**Misc:**\n- I wonder if there is any analogy that can be made between the gradient-based navigation algorithm (Section 3.3) and a classical sphere tracing algorithm for rendering implicit surfaces.  \n- Searching-based -> search based? \n- Supervisely -> in a supervised matter?\n- a continues field -> continuous\n", "summary_of_the_paper": "This work introduces a novel scene representation for agent navigation in 2D and 3D environments. At the core of the method is an implicit neural representation of environment - implicit environment field (IEF) - which is a neural net that maps location coordinates to its reaching distance. Several conditional variants of IEFs are proposed to allow for generalization to novel environments and goal locations. Navigation can the be achieved by gradient descent on the reaching function or a discretized greedy algorithm. \nExperimental evaluation is conducted on 2D maze navigation and 3D human motion prediction, and indicate that the proposed representation performs favorably to the baselines. ", "summary_of_the_review": "This is a well-written paper that provides an interesting solution to an important problem. I am not too familiar with the field of motion planning, thus a low confidence in the assessment. From a more general perspective, the work seems technically sound, and the experimental evaluation, including baselines seem reasonable. With all that in mind, I am quite positive about this work, and recommend accept. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Gtpnp2sblb0", "reviewer": "Reviewer_z3Y4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While the general idea, having a neural network learn to reproduce the output of an A* or Dijkstra algorithm is sensible, what is proposed in this paper is simply not comprehensible.\n\nThe issues start with it being unclear whether or not the idea is to reproduce something akin to a Dijkstra algorithm, i.e., any point to any point, or an A* algorithm, i.e., any point to a single target location. This confusion is caused by the ambiguous writing used in Figure 1 and the abstract, implying Dijkstra and the introduction, which implies A*. The related work also lacks various aspects, such as voxelgrid and tsdf representations in the scene representation section or potential field methods and rmps in the path planning aspect. The affordance prediction section also seems to be a misnomer as it focuses on trajectory prediction, which is different from affordances.\n\nThe description of the method itself is simply devoid of all required detail, such as architecture, training regime, cost function regularization, biases induced due to the discretized nature of the training data, etc. If one were to attempt to reproduce the method based on the information provided, it would be impossible to know where to start and whether one succeeded in reproducing what is described.\n\nThe section that describes the human motion prediction aspect is devoid of the required detail, just as the previous section. Furthermore, the general idea being described would likely fall into a sort of motion planning approach rather than a motion prediction method.\n\nIn many instances, the specifics of the technical detail provided appears wrong or not the commonly used form. For example, the definition of the sdf values is the opposite of what is typically used in TSDF representations. Another issue is that the paper seems to imply that RRT-like methods work in discretized representations only, which is not the case.\n\nThe experiments highlight further problems with the proposed method. While in earlier parts, the paper describes the importance of a continuous representation, all experiments are performed on discretized grid environments. Why is this? Additionally, neither the methods used in the comparison nor the metrics used are described, and the experimental setup details remain vague. Where there is information, it raises more questions. In the RRT comparison section, one of the metrics is described as \"distance between the endpoints of a searched path and the goal,\" yet the RRT method is claimed to achieve a value > 0 in these tasks. As an RRT is guaranteed to find a solution if one exists and creates a fully connected path between the start and goal location, this value can not be anything other than 0. As such, it is not clear what this metric truly measures nor that it is informative.\n\nThere is also nowhere an evaluation of the optimality or safety of the resulting paths. This is something for which theoretical information exists for RRTs, but with neural networks, especially those having the interpolate and perform out-of-domain predictions is anything but guaranteed and a big concern.", "summary_of_the_paper": "The paper proposes a method to learn an environment field that predicts the distance from any location in the map to a query location. The method builds on implicit surface representation ideas and extends it with a few other aspects for motion prediction.", "summary_of_the_review": "While the general idea is viable and something being researched, what is being proposed in this paper is simply too unclear and vague to be assessed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes environment fields, a representation that models reaching distances within a scene. Dense environment fields are learnt using a neural network, and the effectiveness of this representation is shown on 2D maze environments and 3D indoor environments. This paper received hugely contrasting reviews, with two reviewers being very supportive and one reviewer providing the lowest score of 1. In light of this, I'll start with providing my takeaways on the review and discussion with reviewer z3Y4 (rating of 1) and then proceed to the remaining discussion.

Reviewer z3Y4 has provided the score of 1 and has made strong remarks that include: "what is proposed in this paper is simply not comprehensible", "description of the method itself is simply devoid of all required detail", "The main claims of the paper are incorrect or not at all supported by theory or empirical results." and " what is being proposed in this paper is simply too unclear and vague to be assessed". **Such dismissive remarks, in my opinion, are completely unnecessary and create a toxic discussion and review environment.**

Reviewer z3Y4 has many criticisms of the submission, but the primary ones include: (a) the lack of details throughout the paper (b) the positioning of the paper in the abstract and introduction, and (c) the lack of experiments in continuous environments. Re (a): It is well understood in our research community that providing every last detail in the main submission is nearly impossible due to the restriction on the number of pages. Providing excess details in the main paper also often reduces the readability of the paper. Such details are better addressed in the appendix and crucially, the code. The authors have provided some details in the appendix and have indicated that they will release a code base.  I also agree with the authors that justifying every last detail in the network architecture such as choice of an activation function is not necessary for this submission. The same goes with describing methods in past works in detail vs referring the reader to the appropriate citation. As a result, I believe that the authors have addressed (a) well. Re (b): This has also been addressed by the authors, by pointing out relevant parts of the paper that had the necessary details. Re (c): In this regard, the paper clearly contains a well laid out experiment in 3D indoor scenes, so as far as I am concerned, this has been addressed in the main submission.

Reviewers AhgQ and fAEP have supported this submission but also laid out some concerns that include:
(1) Are the gradients suboptimal ?
(2) Positioning the paper with regards to past works
(3) Motivation behind using the VAE
(4) Qualitative analysis and failures
The authors have addressed these 4 concerns well using the rebuttal as well as via a revision of the appendix. The reviewers, post discussion have indicated their satisfaction with the revised submission.

I think this paper is interesting and proposes a novel scene representation which can be useful for others in the Embodied AI community. I am in agreement with reviewers AhgQ and fAEP, and in spite of the strong reject score by z3Y4, I recommend accepting this paper.
Decision: Accept (Poster)