Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes modeling reaching distance between any start position and any goal (subject to obstacle avoidance) with a neural network. This is equivalent to parameterizing a traditional path-planning (goal-reaching) continuous value function with the network, which the authors also mention in the introduction section. Different variants of the network are considered, with conditioning on goal (aiming to generalize to different goals) and conditioning on a 2D scene layout (aiming to generalize across different scenes). The network is trained in a supervised manner on data obtained from a traditional search method (fast marching method [1]) that assumes discrete states. The usefulness of the trained value network for navigation and its generalization properties are then experimentally validated in 2D and 3D environments, including an interesting navigation example with two dynamically moving agents in the same scene.
main_review: ## Strengths
- I believe the main strength of the proposed solution is the ability of the learned value function to generalize to different goals & different scenes, which is ensured by conditioning the network on a target goal and learned features from the 2D layout of the specific scene in question (e.g. section 3.4 and Fig. 3, c)). It is nice to see that an MLP can indeed generalize successfully when different goals & scene layouts are fed in, as illustrated by the multiple human navigation experiment (shown in Fig. 4). Section 3 also mentions predictions are made for unseen mazes during inference, which is good.
- The showcased value-function field evaluations in Fig. 1 and Fig. 6 also appear meaningful, capturing the correct geodesics in the scenes and showing useful predictions when the goal is switched.
- Planning based on the learned representation seems possible, with good success rates for the considered environments, albeit with the assumed discretization of 8 (in 2D) or 27 (in 3D) directions of movement mentioned in sections 3.3 and 4.2 (see my point below).

## Weaknesses
- Looking at figure 2, c), which shows an example of obtaining a navigation plan via gradient descent through the learned representation, it seems like the trajectory is inconsistent and unstable at times. A main advantage of the proposed value function representation is that it is continuous (compared to the discrete data it is trained on), which would require meaningful interpolation in the regions between training data. I am worried that, given the example in figure 2, the gradients (and respectively predictions) of the network in these interpolation regions (i.e. between the points from FMM) might be suboptimal. Can you please comment on this? Right now I am interpreting this as signs of overfitting.
- Along the same line of reasoning as the previous point, constructing plans from the learned network by choosing from discrete directions of movement (8 for 2D, 27 for 3D) seems inconsistent with the continuity argument. To me it seems like a step backwards towards maintaining a discrete tabular value function. Ideally, gradient descent should work.
- The idea of learning conditional value functions for goal-reaching tasks that generalize to different goals is not exactly novel, e.g. works like [2] have already considered generic formulations in that regime that go beyond holonomic navigation in $\mathbb{R}^2$ or $\mathbb{R}^3$ (you can check the references section of the mentioned paper for more examples of works from the "goal-reaching RL" field). More generally, learning continuous value functions is standard practice in the RL and optimal control communities, and I think the link to that should be more emphasized in the paper. 
- When modeling accessible regions in 3D space with a VAE (section 4.2), what is the motivation behind using the VAE model to begin with, i.e. why not directly use the empirical distribution of human torso locations the VAE is trained on? I am asking because the trained generative part of the VAE will be a proxy for the empirical distribution it was trained to approximate. In section 4.2 there is also the note that samples from the generative model are further filtered out by checking for collisions–this makes it hard to judge if the generative model is indeed reflective of accessible regions, or if this resampling / filtering step is mostly responsible.
- Section 4.2 mentions locations outside of the accessible region are marked with negative value, but how do you determine whether a point lies in the region or not? When using the generative part of the VAE (or the data empirical I mentioned, for that matter), this region would not be defined in a closed form. This needs further clarification.
- NERF [3] should probably be cited in the related work section on implicit neural representations.

## Other remarks
- Looking at the supplementary material, there are some accidental flips of the human skeletons in the videos. Is this because of issues with the alignment of the movement primitives to the planned trajectories, or because the planned trajectories have occasional jitter?


## References
[1] James A Sethian. A fast marching level set method for monotonically advancing fronts. PNAS, 1996

[2] Eysenbach, Benjamin, Ruslan Salakhutdinov, and Sergey Levine.  "C-learning: Learning to achieve goals via recursive classification." ICLR 2020

[3] Mildenhall, Ben, et al. "Nerf: Representing scenes as neural radiance fields for view synthesis." ECCV 2020.


summary_of_the_review: Because of the empirically validated generalization properties of the presented model when the 2D scene layout & goal inputs are switched, I am leaning slightly positive, but currently the concerns listed in my review are stopping me from giving a higher score.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work introduces a novel scene representation for agent navigation in 2D and 3D environments. At the core of the method is an implicit neural representation of environment - implicit environment field (IEF) - which is a neural net that maps location coordinates to its reaching distance. Several conditional variants of IEFs are proposed to allow for generalization to novel environments and goal locations. Navigation can the be achieved by gradient descent on the reaching function or a discretized greedy algorithm. 
Experimental evaluation is conducted on 2D maze navigation and 3D human motion prediction, and indicate that the proposed representation performs favorably to the baselines. 
main_review: **Strengths:**
- Writing. The paper is well-written and is mostly easy to follow.
- Method / motivation. Proposed method seems novel and technically sound. Generally, the framework of implicit neural representations seems like a great fit for the task at hand.
- Experimental evaluation is conducted on several challenging tasks, a comprehensive comparison to baselines, including recent ones, is provided. Quantitatively, the method seems to be performing similarly to baselines or slightly better, while being more efficient.

**Weaknesses:**
- Reproducibility. There is not much details provided on the architectures of the networks. Figure 3 could benefit from more detailed explanations.
- Method. Authors claim that continuity is one of the core benefits of their approach, yet in practice they seem to be resorting to a discretized solution when doing trajectory search. Is there any benefit of using continuous representation in 2D? Say, over predicting a discretized field with a convnet. 
- Experimental evaluation. It would be great to also have an idea about how the results look qualitatively for the baselines - it is hard to judge if the quantitative improvements are significant. 
- Failure cases. There are some of the failure cases presented for the human motion prediction, but not for maze navigation. In particular, I wonder if the greedy strategy ever causes issues when the agent is stuck in an infinite loop between two locations, e.g. when the reaching distance is identical in those locations, and how one can tackle that? 

**Misc:**
- I wonder if there is any analogy that can be made between the gradient-based navigation algorithm (Section 3.3) and a classical sphere tracing algorithm for rendering implicit surfaces.  
- Searching-based -> search based? 
- Supervisely -> in a supervised matter?
- a continues field -> continuous

summary_of_the_review: This is a well-written paper that provides an interesting solution to an important problem. I am not too familiar with the field of motion planning, thus a low confidence in the assessment. From a more general perspective, the work seems technically sound, and the experimental evaluation, including baselines seem reasonable. With all that in mind, I am quite positive about this work, and recommend accept.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper proposes a method to learn an environment field that predicts the distance from any location in the map to a query location. The method builds on implicit surface representation ideas and extends it with a few other aspects for motion prediction.
main_review: While the general idea, having a neural network learn to reproduce the output of an A* or Dijkstra algorithm is sensible, what is proposed in this paper is simply not comprehensible.

The issues start with it being unclear whether or not the idea is to reproduce something akin to a Dijkstra algorithm, i.e., any point to any point, or an A* algorithm, i.e., any point to a single target location. This confusion is caused by the ambiguous writing used in Figure 1 and the abstract, implying Dijkstra and the introduction, which implies A*. The related work also lacks various aspects, such as voxelgrid and tsdf representations in the scene representation section or potential field methods and rmps in the path planning aspect. The affordance prediction section also seems to be a misnomer as it focuses on trajectory prediction, which is different from affordances.

The description of the method itself is simply devoid of all required detail, such as architecture, training regime, cost function regularization, biases induced due to the discretized nature of the training data, etc. If one were to attempt to reproduce the method based on the information provided, it would be impossible to know where to start and whether one succeeded in reproducing what is described.

The section that describes the human motion prediction aspect is devoid of the required detail, just as the previous section. Furthermore, the general idea being described would likely fall into a sort of motion planning approach rather than a motion prediction method.

In many instances, the specifics of the technical detail provided appears wrong or not the commonly used form. For example, the definition of the sdf values is the opposite of what is typically used in TSDF representations. Another issue is that the paper seems to imply that RRT-like methods work in discretized representations only, which is not the case.

The experiments highlight further problems with the proposed method. While in earlier parts, the paper describes the importance of a continuous representation, all experiments are performed on discretized grid environments. Why is this? Additionally, neither the methods used in the comparison nor the metrics used are described, and the experimental setup details remain vague. Where there is information, it raises more questions. In the RRT comparison section, one of the metrics is described as "distance between the endpoints of a searched path and the goal," yet the RRT method is claimed to achieve a value > 0 in these tasks. As an RRT is guaranteed to find a solution if one exists and creates a fully connected path between the start and goal location, this value can not be anything other than 0. As such, it is not clear what this metric truly measures nor that it is informative.

There is also nowhere an evaluation of the optimality or safety of the resulting paths. This is something for which theoretical information exists for RRTs, but with neural networks, especially those having the interpolate and perform out-of-domain predictions is anything but guaranteed and a big concern.
summary_of_the_review: While the general idea is viable and something being researched, what is being proposed in this paper is simply too unclear and vague to be assessed.
review_title: null

Meta-review: This paper proposes environment fields, a representation that models reaching distances within a scene. Dense environment fields are learnt using a neural network, and the effectiveness of this representation is shown on 2D maze environments and 3D indoor environments. This paper received hugely contrasting reviews, with two reviewers being very supportive and one reviewer providing the lowest score of 1. In light of this, I'll start with providing my takeaways on the review and discussion with reviewer z3Y4 (rating of 1) and then proceed to the remaining discussion.

Reviewer z3Y4 has provided the score of 1 and has made strong remarks that include: "what is proposed in this paper is simply not comprehensible", "description of the method itself is simply devoid of all required detail", "The main claims of the paper are incorrect or not at all supported by theory or empirical results." and " what is being proposed in this paper is simply too unclear and vague to be assessed". **Such dismissive remarks, in my opinion, are completely unnecessary and create a toxic discussion and review environment.**

Reviewer z3Y4 has many criticisms of the submission, but the primary ones include: (a) the lack of details throughout the paper (b) the positioning of the paper in the abstract and introduction, and (c) the lack of experiments in continuous environments. Re (a): It is well understood in our research community that providing every last detail in the main submission is nearly impossible due to the restriction on the number of pages. Providing excess details in the main paper also often reduces the readability of the paper. Such details are better addressed in the appendix and crucially, the code. The authors have provided some details in the appendix and have indicated that they will release a code base.  I also agree with the authors that justifying every last detail in the network architecture such as choice of an activation function is not necessary for this submission. The same goes with describing methods in past works in detail vs referring the reader to the appropriate citation. As a result, I believe that the authors have addressed (a) well. Re (b): This has also been addressed by the authors, by pointing out relevant parts of the paper that had the necessary details. Re (c): In this regard, the paper clearly contains a well laid out experiment in 3D indoor scenes, so as far as I am concerned, this has been addressed in the main submission.

Reviewers AhgQ and fAEP have supported this submission but also laid out some concerns that include:
(1) Are the gradients suboptimal ?
(2) Positioning the paper with regards to past works
(3) Motivation behind using the VAE
(4) Qualitative analysis and failures
The authors have addressed these 4 concerns well using the rebuttal as well as via a revision of the appendix. The reviewers, post discussion have indicated their satisfaction with the revised submission.

I think this paper is interesting and proposes a novel scene representation which can be useful for others in the Embodied AI community. I am in agreement with reviewers AhgQ and fAEP, and in spite of the strong reject score by z3Y4, I recommend accepting this paper.
Decision: Accept (Poster)