From: ICLR.cc/2023/Conference/Paper4579/Area_Chair_BPnk
title: Meaning of Spatial Supervision
comment: Quoting the authors above (emphasis their's):

> We repeat the goal of our paper quoted by the AC with an emphasis added: "Do spatial representations emerge in non-spatial neural networks **without spatial supervision** as a natural consequence of learning to navigate?”

I find it difficult to see how one could argue that the architecture is not provided with spatial supervision. The first paragraph in Section 2 includes the following statement

> It [the agent] is equipped with an egomotion sensor providing it relative position $(\Delta x, \Delta y, \Delta z)$ and relative ‘heading’ (or yaw angle) $\Delta \theta$ between successive steps, which is integrated to keep track of the agent’s location and heading relative to start $[x_t, y_t, z_t, \theta_t]$. This is sometimes referred to as a 'GPS+Compass' sensor

The phrasing is vague, but this can be interpreted as stating that the agent is provided with knowledge of either its precise absolute position $(x_t, y_t)$[^1] and heading $\theta_t$ relative to the start at each time step (consistent with the the sensor being labeled as a 'GPS+Compass' sensor[^2]) or, at the very least, the precise change in position $(\Delta x, \Delta y)$ and heading $\Delta \theta$ between timesteps, the latter providing implicit knowledge of the agent's absolute position and heading at every timestep. It is difficult to understand how this explicit or implicit 'GPS+Compass' supervision is anything but spatial supervision, particularly since perfect (i.e., noise-free) egomotion sensors do not exist (e.g., egomotion sensors are the primary source of noise in robot mapping, localization, and SLAM).

Huynh et al., on the other hand, do not provide the network with the agent's position or heading (neither relative nor absolute). Instead, as the authors point out, Huynh et al., train the network with a loss that encourages it to remember a location (position) that it previously visited. If anything, this seems like a weaker form of spatial supervision than is provided here.

[^1] Note: I am deliberately omitting $z_t$

[^2] A GPS and compass provide absolute observations of position and heading, respectively, hence the confusion as to whether the supervision is in the form of relative or absolute position and orientation.

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Differences to Huynh et al. (Part 2)
comment: We thank the AC for their response, however there still appears to be a misunderstanding.

> The results of Huynh et al., involve the use of an auxiliary localization objective 

Not quite. Their localization objective is not an _auxiliary_ objective for their mapping results, but the _sole_ objective. The navigation policy trained in Huynh et al. is learned “on top of a fixed mapping and localization network pre-trained with random walk motion.” (Section 4.2) This may seem like a minor distinction but in this context, it is not – sole supervision means that it is not answering the question asked in our paper (quoted in AC’s response above). 

> A baseline experiment in Huynh et al. directly answers this question in the positive.

It does not. The network in this experiment was trained for localization.

We repeat the goal of our paper quoted by the AC with an emphasis added: "Do spatial representations emerge in non-spatial neural networks **without spatial supervision** as a natural consequence of learning to navigate?”

> In the case of the paper under consideration here, the goal distance-based reward also provides a form of spatial supervision, albeit one that is arguably less explicit. It doesn't seem surprising that this, coupled with the fact that the agent is "blind", encourages the model to learn spatial information.

“Surprise” is surprisingly hard to find in hindsight. 

Would it have been surprising had maps not emerged? 

We would argue that a lack of emergent maps would not be considered surprising, given that the deep learning and RL literature is replete with networks optimizing reward and metrics for the wrong reasons, enabling them to solve tasks via shortcuts. If lack of emergence wouldn’t have been surprising, then emergence is surprising. If neither is surprising, it’s likely us falling for hindsight bias. 

Finally, we also note that we are unaware of any prior work that actually formulated and tested our hypothesis. Testing and measuring previously untested hypotheses is a core importance of scientific advance and understanding.

From: ICLR.cc/2023/Conference/Paper4579/Area_Chair_BPnk
title: Re: Differences to Huynh et al.
comment: I thank the authors for following up regarding the relationship to the work of Huynh et al.

As the authors point out, Huynh et al. use an architecture that is particularly well suited to tasks such as navigation that involve spatial reasoning, whereas the paper under consideration demonstrates that an architecture without such an inductive bias is capable of learning spatial information. The results of Huynh et al., involve the use of an auxiliary localization objective, which one can reasonably expect would only encourage the emergence of spatial maps. In the case of the paper under consideration here, the goal distance-based reward also provides a form of spatial supervision, albeit one that is arguably less explicit. It doesn't seem surprising that this, coupled with the fact that the agent is "blind", encourages the model to learn spatial information.

    Our work studies the question “do spatial representations emerge in task-agnostic (non-spatial) neural networks without spatial supervision as a natural consequence of learning to navigate?” The work of Huynh et al. does not provide an answer to this questio

A baseline experiment in Huynh et al. directly answers this question in the positive.  They show that the Differentiable Neural Computer (DNC), which has a linear slot-based memory rather than a 2D spatial memory, achieves nontrivial accuracy on the following task: Given a local observation at each step in a sequence of local movements (e.g, 100 steps) in a 2D maze, answer questions about the 2D structure of the environment that has been explored (including pieces seen far in the past).  This result is in the second row of Table 1 of Huynh et al.

The fact that Huynh et al. also show the DNC (Graves et al., 2016), which does not have a 2D spatial memory architecture, is significantly outperformed by a Multigrid Memory network which does have such internal architecture, suggests there may be limits to the accuracy or learning efficiency we might expect from simpler architectures on spatial memory tasks.  But, it should not be surprising to have some degree of success with linear memory architectures.

Graves, Alex, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo et al. "Hybrid computing using a neural network with dynamic external memory." Nature 538, no. 7626 (2016): 471-476.

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Differences to Huynh et al.
comment: We thank the AC for bringing the work of Huynh et al. to our attention. This is a related work, and we will add it to the manuscript.

However, there appears to be a fundamental misunderstanding here: 
- Our work demonstrates the emergence of spatial representations in *non-spatial* or unstructured neural networks *without any inductive bias towards mapping*. 
- Hunynh et al. propose a *spatial memory explicitly trained with a localization loss*. 

The two works are operating in different experimental conditions and addressing different questions. These differences make our findings a priori unknown and a posteriori surprising. 

We describe these differences in detail below. 

Authors

-----

1) *Architectural Differences.*
Huynh et al. propose “Multigrid neural memory”, an explicitly spatial memory architecture. Huynh et al. state that mapping and localization is an “inherently spatial task” and “naturally fit to the architecturally-induced biases of [their] memory network”. Our work uses task-agnostic neural components (LSTMs and MLPs), which do not contain any explicit spatial components. 

2) *Supervision Differences.*
Huynh et al. explicitly train the mapping component of their network for a mapping+localization task. Specifically, they supervise the network with a localization loss to label all previously seen locations of a given patch of a 2D grid (environment). In contrast, the only learning signal provided in our experimental setup is the navigational reward. We do not back-propagate gradients from the probes (referred to as “readers” in Huynh et al.) into the neural memory; they are solely for diagnostic purposes. 

Cumulatively, the experimental conditions in Huynh et al. involve creating a network with “architecturally-induced biases” towards mapping and training for localization, not navigation. 

Our work studies the question “do spatial representations emerge in task-agnostic (non-spatial) neural networks without spatial supervision as a natural consequence of learning to navigate?” The work of Huynh et al. does not provide an answer to this question.

From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Accept: notable-top-5%
metareview:_summary,_strengths_and_weaknesses: The paper studies the emergence of implicit maps within standard recurrent architectures trained to navigate a "blind" agent that has access only to proprioceptive (egomotion) observations. The paper describes an extensive set of experiments that investigate the nature of the agent's implicit learned memory and the ability with which it enables the "blind" agents to navigate, and analyzes the spatial information modeled in this memory.
The paper was very well received by all four reviewers, with overall ratings that include two accepts (8) and two strong accepts (10). There is unanimous agreement among the reviewers that the paper is incredibly well written and highly thought provoking. As readily acknowledged by the authors, the contribution is not in proposing new navigation algorithms, but rather in presenting a meaningful investigation of the emergence of spatial memory in enabling navigation through a comprehensive set of results.
We agree with the reviewers regarding the interesting insight that the paper provides into learned spatial memory. However, it would be more comprehensive to discuss some related prior work; some examples below:  

T Huynh, M Maire, MR Walter. Multigrid Neural Memory. ICML 2020

Graves, Alex, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo et al. "Hybrid computing using a neural network with dynamic external memory." Nature 538, no. 7626 (2016): 471-476.

summary_of_AC-reviewer_meeting: N/A
justification_for_why_not_higher_score: N/A
justification_for_why_not_lower_score: None. This paper is worthy of at least a spotlight.
note_from_PC: if the above contains the word "oral" or "spotlight" please see: "oral" presentation means -> notable-top-5% and "spotlight" means -> notable-top-25%. As stated in our emails, we are disassociating presentation type from AC recommendations

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Response to ccRY
comment: > Tons of research has already gone into mapping for navigation with SLAM and more modern approaches. This has been worked on in robotics for decades. Claiming that future work can / should now start to use mapping as an inductive bias because of the results of this paper is a very overreaching and concerning response to me.

We see your high-level point. We agree that SLAM and structure-from-motion has a long history in robotics and vision (as we say in the paper, paragraph 2, first sentence). We certainly aren't trying to claim any role in that or its ongoing development. All we were trying to express is that perhaps our work will contribute to an increased emphasis on directions such as mapping-related inductive biases and combining implicit mapping with implicit planning. We believe our manuscript strikes a fairly balanced tone, but we do not want to over-claim so we are happy to edit any text that the reviewer may have suggestions on.

From: ICLR.cc/2023/Conference/Paper4579/Reviewer_ccRY
title: Response to authors
comment: Thanks for providing additional insight. Perhaps the work will help with cognitive science debates or help motivate work at the intersection of cognitive science and embodiment. Practically speaking, however, I still do not find the results of this work that widespread.

> Future work in AI can leverage our results in two ways. 1) By incorporating mapping-related inductive biases (architectural components, mapping supervision as an auxiliary reward, etc), or 2) using implicit maps to perform implicit planning.

Tons of research has already gone into mapping for navigation with SLAM and more modern approaches. This has been worked on in robotics for decades. Claiming that future work can / should now start to use mapping as an inductive bias because of the results of this paper is a very overreaching and concerning response to me.

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Response to ccRY
comment: We thank the reviewer for their review. We are glad they found our paper “well written and structured”, a “pleasure to read”, and that they appreciated “the time that went into the figures and outline.” That our experiments are “well thought out and convincing” and “the question posed in this work is intellectually interesting and creative.”

> However, the implications of this work don't seem very widespread or practically useful for future work. Suppose the hypothesis that model-free agents build maps without explicitly being trained to build them is true. Who cares? Will this change how you train embodied agents in the future?

We respectfully disagree. As an example of the widespread interest in the mechanisms underlying spatial navigation, John O’Keefe and the Mosers were awarded a Nobel Prize for their work on cognitive maps in animals and their discovery of cells that constitute a positioning system in the brain. Our results point towards mapping being a natural solution to navigation. This is relevant to a debate in the cognitive science, ethology, robotics, and AI communities started in (at least) 1948. Furthermore, our demonstration of the surprising efficacy of blind agents despite their limitations will serve to contextualize and provide a strong baseline for the further development of embodied agents by helping to clarify the source of agents’ success. Future work in AI can leverage our results in two ways. 1) By incorporating mapping-related inductive biases (architectural components, mapping supervision as an auxiliary reward, etc), or 2) using implicit maps to perform implicit planning.

At a higher level, our results speak to the efficacy of the research program of embodied AI. By training agents in immersive interactive environments at a large-scale, intelligence (in the form of mapping) simply emerges. This is precisely what Smith and Gasser’s embodiment hypothesis states should happen. This provides confidence in continuing to pursue this program for more challenging tasks, environments, and settings. 

Please let us know if you have any additional questions.

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Response to XPSp
comment: We thank the reviewer for their review. We are glad they found that our paper “presents many surprising findings ... that are important for the Embodied AI community”, that “understanding the internal representation of scenes from neural agents is an important direction”, and our experiments to be “detailed and thorough.”

>  I am not convinced with the point made by authors in Section 5 about the ability to properly decode the occupancy maps even in unseen locations. This might suggest the decoding model rather learns some patterns about the average structure of occupancy maps.

We agree that there is structure in the environments relevant to decoding occupancy grids -- e.g. since many environments feature hallways, the presence of one wall will increase the probability of there being a matching parallel wall. Further, the decoder must use this structure. It is tasked with predicting unseen locations and, without a prior, this is an impossible task.

However, we disagree that average structure is enough. To control for this possibility, we presented the controlled comparison -- (TrainedAgentMemory, decoder) vs. (UntrainedAgentMemory, decoder) -- with identical train/val sets and decoder architecture (the only change is the input memory). If average structure was enough and agent memory wasn’t necessary to predict unseen locations, we’d expect both to perform similarly. However, we find that (TrainedAgentMemory, decoder) achieves 32.4% IoU while (UntrainedAgentMemory, decoder) achieves 12.4% IoU (Fig. 4). This demonstrates that agent memory is necessary to predict unseen locations. In other words, the decoder is not predicting average structure, but uses the combination of structure and trained agent memory.

In summary, we show that trained agent memory is necessary for predicting unseen locations but we do not show that it is sufficient. We have updated the paper to clarify this nuance.

>  It would be interesting to quantify how biased towards some redundant structure in maps the decoding model is.

Interesting idea! We are unaware of a way to programmatically quantify the amount of redundant structure, but are happy to try something if the reviewer has suggestions. 

We have included 100 extra examples of decoder outputs in the supplementary materials in the extra-occupancy-decoder-examples folder, ground-truth is on the left and the prediction is on the right.

> It would be interesting to consider a comparison with a baseline that shares the same architecture as the blind agent, but composed of an additional explicit map with differentiable write and read operations. This could allow to compare the content of the maps with the ones decoded from the recurrent agent’s hidden state, as well as task performance.

Interesting idea! While outside the scope of this work and an entire research project in its own scope, such a model could in future be used to answer questions like “If the agent is given a map, what does it choose to record” and allow us to compare implicit vs. explicit maps. Observing what an agent with an explicit map records may also give further insight into what the implicit-map agent records by giving other things to examine. We will add this discussion to the paper. 

Please let us know if you have any additional questions.

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Response to JNho
comment: We thank the reviewer for their review. We are glad they found our paper “incredibly well written and detailed”, that “the experiments are thought provoking”, and “explores an interesting topic that should be interested to anyone ... working in the space of navigation”.


> The agent is trained in such a way that the LSTM is expecting to receive all the previous observations from the start. As such, are the experiments providing out-of-distribution signals to the agent? Can the authors comment on the possibility of poor performance being a result of this shift in the nature of the data. Similarly, if the agent were retrained with a time-limited memory horizon (e.g., 100 or 500 steps), would we expect an increase in performance...

Great question! We wondered this too and explored it in Section C.6 (on page 20), with results shown in Fig A14 (page 24). To summarize, we found that performance is better if the agent is trained to operate at that memory length, but still worse than unbounded memory length (up to a budget of 256). Due to computational constraints we weren’t able to train for lengths longer than 256 (training a memory length of 256 is 256x more computational expensive) and thus we can’t say when bounded memory would reach the performance of unbounded memory.

Overall, the results in the main paper are accurate and robust in the claim that the agent is highly dependent on memory and that its memory has long-range dependencies, however they are imprecise on the exact length of those dependencies.

We also note that while the agent was trained with an unbounded LSTM, it was trained with truncated back-prop-through-time (TBPTT). The dependencies learned certainly exceed the 256 truncation length used for training.

We have updated the paper to clarify this point.

> Relatedly, it would be helpful to have a separate curve (perhaps on the same plot) showing the success rate versus memory length in Fig. 2, as I believe this to be a key detail for understanding the performance with limited memory.

Happy to add this. We have updated Fig. 2. Success largely follows the same trend as SPL. Indicating that as the memory budget increases, the agent is able to complete longer episodes but not complete existing ones more efficiently.

> The line in Sec. 2 "but also that collisions-vs-not are linearly separable in this space": I do not believe that this claim logically follows from the previous clause. It may be true, but seems disconnected.

We believe the claim 

- collisions-vs-not is linearly separate in the space (h_t+1, c_t+1)

logically follows from the previous clause

- a linear classifier evaluated on held-out data achieves 98% class-balanced accuracy (chance: 50%) on the task of predicting from (h_t+1, c_t+1) if a_t resulted in a collusion

because the classifier is linear and we do not propagate gradients from the classifier to the agent's representations (i.e., we use frozen agent representations). To rephrase, linear separability follows from high empirical performance of a linear classifier. 

But perhaps we misunderstood a subtlety and the reviewer could clarify more?

> Writing feedback and related work.

Thank you for the feedback and pointers. We will work on incorporating it.

Please let us know if you have any additional questions. 

Finally, if you believe this paper should be highlighted at the conference, could you please consider raising your score to reflect that?

From: ICLR.cc/2023/Conference/Paper4579/Authors
title: Response to XZvt
comment: 
We thank the reviewer for their review. We are glad they found the questions we ask to be novel, our results to be meaningful, and that they “do see any weak points about the paper.”

>  Reproducibility: Although there is no source code yet, the Appendix and the paper fully explains the training/experiment procedures, so it's medium.

We will release source code upon publication.

Please let us know if you have any additional questions.

From: ICLR.cc/2023/Conference/Paper4579/Reviewer_XZvt
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: While animal navigation assumes that there should be internal spatial representations (internal mental map) built and maintained for navigation, recent machine learning community’s works show that neural network-based models for navigation do not need explicit mapping modules; indeed, many models that do not contain explicit mapping achieve good performance in the exploration/navigation problems. This paper investigates a question “is mapping an emergent phenomenon?” In this work, the authors make their AI agent “blind”, which means that only egomotion sensor is present, making the agent having minimal perceptual system. There is no visual, audio, or any other sensing available to the agent. The paper argues that, under this harsh condition, the emergence of map-like spatial representations in the agent’s unstructured memory. 

strength_and_weaknesses: - The strength of the paper comes from the novelty of the questions authors are asking. “Can blind agent navigate as well as non-blind agents?” lead to the bold, harsh conditions that agents are only given ego-motion sensors, without vision or any other sensor modalities. 
- The results are meaningful that it is possible to efficiently navigate with just ego-motion sensing. The strong performance by blind agents are explained by the role of memory, and the memory encodes the implicit maps about the environment. 
- In the section 2, the authors explain how the blind agents are becoming effective navigators (e.g. emergence of wall-following behavior and collision-detection neurons). In the section 3, the authors investigate how memory is utilized in the long-horizon settings. 
- Memory also enables shortcuts, shown by the probe agent that is given the original agent’s memory containing spatial representations of the environment. The probe agent leverages the memory to enable shortcuts. 
- I don’t see any weak points about the paper. 
clarity,_quality,_novelty_and_reproducibility: Clarity: high
Quality: high
Novelty: high
Reproducibility: Although there is no source code yet, the Appendix and the paper fully explains the training/experiment procedures, so it's medium. 
summary_of_the_review: I would give acceptance score to this paper, although i'm open to changes after the discussion with other reviewers. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 10: strong accept, should be highlighted at the conference

From: ICLR.cc/2023/Conference/Paper4579/Reviewer_JNho
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper is a study of blind point goal navigation agents---agents provided only a GPS+compass signal and expected to reach an unseen point goal---and their ability to form implicit maps and an understanding of the environment through navigation. The paper is a thoroughly interesting investigation and includes experiments to understand (1) how well blind memory-full agents can navigate despite their significant limitations, (2) how memory can be transplanted to imbue another agent with the ability to navigate more effectively (using that experience), and (3) to decode the agents' memories into occupancy grids as a means of scrutinizing their implicit maps of their surroundings.

strength_and_weaknesses: The paper is incredibly well written and detailed and the experiments are thought provoking. The paper explores an interesting topic that should be interested to anyone---including both AI/ML scientists and also perhaps cognitive scientists---working in the space of navigation. Each experiment effectively answers its respective "question" and is accompanied with convincing visuals. While the paper does not introduce new "techniques" for navigation or training navigation agents, this should not be seen as a limitation of the work and the investigation and the analyses are valuable on their own. I have only a few comments and questions.

I have some potential concerns about the results from Sec. 3 (Memory is used over long horizons). The agent is trained in such a way that the LSTM is expecting to receive all the previous observations from the start. As such, are the experiments providing out-of-distribution signals to the agent? Can the authors comment on the possibility of poor performance being a result of this shift in the nature of the data. Similarly, if the agent were retrained with a time-limited memory horizon (e.g., 100 or 500 steps), would we expect an increase in performance; new experiments are likely not necessary, though clarification on this point will be helpful.

Relatedly, it would be helpful to have a separate curve (perhaps on the same plot) showing the success rate versus memory length in Fig. 2, as I believe this to be a key detail for understanding the performance with limited memory.

Smaller questions and comments
- The line in Sec. 2 "but also that collisions-vs-not are linearly separable in this space": I do not believe that this claim logically follows from the previous clause. It may be true, but seems disconnected.
- The authors may also want to reference "Learning Deployable Navigation Policies at Kilometer Scale from a Single Traversal" by Bruce, et al 2018. The appendix of this paper tries to extract localization information from an agent's latent state, and may also be good to mention in the related work [I also believe another early paper in the VisDoom environment used a similar approach, but I am unable to find it.]
- Introduction: I understand why it is included in the introduction but that "memory" is essential for strong performance is a fairly obvious conclusion even if the results in the respective section are more detailed. Consider rephrasing this question and answer to reflect the nuance of the effect under scrutiny.
clarity,_quality,_novelty_and_reproducibility: [clarity, quality, and novelty] The paper is easy to read and provides a novel and comprehensive study of the effect(s) in question.

[reproducibility] The paper is mostly reproducible, relying on open source components for most (if not all) the navigation agent training. The authors have said they will provide the remaining analysis code in the event of publication.
summary_of_the_review: Overall, this is an excellent paper. I have only limited changes and suggestions for improvement. The investigation is detailed and thought provoking. The authors should strive to answer my question about the impact of memory on the agent's performance.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 10: strong accept, should be highlighted at the conference

From: ICLR.cc/2023/Conference/Paper4579/Reviewer_XPSp
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper studies the implicit emergence of mapping in simple recurrent blind agents. Removing the visual modality allows authors to reduce the possible strategies to reach the specified goal and to thus isolate the construction of an internal map. As mentioned in the paper, no technical novelty is proposed, but a study about an important and interesting question, i.e. whether simple recurrent agents can learn to implicitly map scenes, is studied. 

The work presents many surprising findings, such as the strong performance of blind agents, the emergence of neurons specialised in collision detection, the impressive long horizon memory developed these agents, the quality of spatial maps decoded from the agents’ hidden state, and finally the capacity to forget information that is not useful for the task at hand. Authors also show that if initialising the hidden state of an agent with the one from another agent that has finished an episode, the new episode navigates more efficiently between the starting and end points, in both directions.
strength_and_weaknesses: Strengths:
1. Understanding the internal representation of scenes from neural agents is an important direction, both to make them more interpretable and to study the strategies that can be purely learnt from data.
2. The context of the work is well defined, and the experiments are detailed and thorough.
3. This paper presents many surprising findings (stated in paper summary) that are important for the Embodied AI community. 

Weaknesses:
1. I am not convinced with the point made by authors in Section 5 about the ability to properly decode the occupancy maps even in unseen locations. This might suggest the decoding model rather learns some patterns about the average structure of occupancy maps. Results on this decoding task are still interesting, but maybe less strong. It would be interesting to quantify how biased towards some redundant structure in maps the decoding model is.
2. It would be interesting to consider a comparison with a baseline that shares the same architecture as the blind agent, but composed of an additional explicit map with differentiable write and read operations. This could allow to compare the content of the maps with the ones decoded from the recurrent agent’s hidden state, as well as task performance.
clarity,_quality,_novelty_and_reproducibility: Claims are clearly stated in the paper and well supported empirically. Experiments are detailed, allowing reproducibility. Previous work has already considered the emergence of mechanisms close to place cells in recurrent agents (Banino et al., Vector-based navigation using grid-like representations in artificial agents), but this work can still be considered as original from the direction of studying blind agents and the diverse experimental studies.
summary_of_the_review: Such empirical study is of high importance for the community. This paper provides many interesting and well supported insights. Allowing to better understand the underlying reasoning of simple recurrent agents can help develop better navigating agents in the future. I thus tend to recommend acceptance, but am still looking for answers from the authors to my remarks
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper

From: ICLR.cc/2023/Conference/Paper4579/Reviewer_ccRY
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The authors test the hypothesis of if memory-free navigation-based agents build maps of their environment. They use blind agents trained to perform PointNav, which are effectively operating in grid-world environments. They show very convincing and creative experiments controlling for the memory length of the agent, predicting occupancy maps, and probing the agent.
strength_and_weaknesses: Strengths:
- The paper is very well written and structured. It was a pleasure to read. I also appreciate the time that went into the figures and outline.
- The experiments were well thought out and convincing. Both the probing experiments and occupancy map prediction experiments were very interesting and are convincing of the phenomenon of blind agents encoding maps.
- It is surprising and impressive that blind agents are able to achieve 95% success rate on PointNav.
- The question posed in this work is intellectually interesting and creative.
- The experiments controlling for memory length are also great.

Weaknesses:
- **W1.** This paper presents a neat and interesting observation, and a set of experiments to test their hypothesis. However, the implications of this work don't seem very widespread or practically useful for future work. Suppose the hypothesis that model-free agents build maps without explicitly being trained to build them is true. Who cares? Will this change how you train embodied agents in the future?
- **W2.** It's disappointing that the experiments with vision-based agents didn't work out. The hypothesis makes sense, but effectively, the experiments in this paper are on grid-world environments. It would have been nice to show the hypothesis holds with vision-based agents or in more complex tasks like rearrangement.
clarity,_quality,_novelty_and_reproducibility: The results seem like they should be fairly easy to reproduce, generally using existing datasets and packages that have been tested many times before. I appreciate the commitment to releasing the code.

As mentioned above, the work is very easy to follow along and understand and the hypothesis that this paper attempted to solve was very creative, novel, and enjoyable. There isn't a ton of work focused on similar emergent behaviors of embodied agents.
summary_of_the_review: I found the paper very enjoyable and really interesting to read. The work isn't going to be very practically useful and won't change much from a practitioner's perspective, but tests a creative hypothesis of if maps emerge from blind navigation-based agents, which appears to be true. Overall, I found the experiments were very well thought out and convincing, and think the paper is a clear accept.
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 4: The contributions are significant, and do not exist in prior works.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper