Original: M EMO N AV : W ORKING M EMORY M ODEL FOR V ISUAL N AVIGATION
Revised: M EMO N AV : W ORKING M EMORY M ODEL FOR V ISUAL N AVIGATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present MemoNav, a novel memory model for image-goal navigation, which utilizes a working memory-inspired pipeline to improve navigation performance.
Revised: We present MemoNav, a novel memory model for image-goal navigation, which utilizes a working memory-inspired pipeline to improve navigation performance.

Original: Specifically, the node features on the topological map are stored in the short-term memory (STM), as these features are dynamically updated.
Revised: Specifically, the node features on the topological map are stored in the short-term memory (STM), as these features are dynamically updated.

Original: The MemoNav retains the informative fraction of the STM via a forgetting module to improve navigation efficiency.
Revised: The MemoNav retains the informative fraction of the STM via a forgetting module to improve navigation efficiency.

Original: To learn a global representation of 3D scenes, we introduce long-term memory (LTM) that continuously aggregates the STM.
Revised: To learn a global representation of 3D scenes, we introduce long-term memory (LTM) that continuously aggregates the STM.

Original: Afterward, a graph attention module encodes the retained STM and the LTM to generate working memory (WM).
Revised: Afterward, a graph attention module encodes the retained STM and the LTM to generate working memory (WM).

Original: After encoding, the WM contains the informative features in the retained STM and the scene-level feature in the LTM and is finally used to generate actions.
Revised: After encoding, the WM contains the informative features in the retained STM and the scene-level feature in the LTM and is finally used to generate actions.

Original: Consequently, the synergy of these three types of memory increases navigation performance by selectively retaining goal-relevant information and learning a highlevel scene feature.
Revised: Consequently, the synergy of these three types of memory increases navigation performance by selectively retaining goal-relevant information and learning a highlevel scene feature.

Original: When evaluated on multi-goal tasks, the MemoNav outperforms the SoTA methods at all difficulty levels in both Gibson and Matterport3D scenes.
Revised: When evaluated on multi-goal tasks, the MemoNav outperforms the SoTA methods at all difficulty levels in both Gibson and Matterport3D scenes.

Original: The MemoNav also achieves consistent improvements on traditional 1-goal tasks.
Revised: The MemoNav also achieves consistent improvements on traditional 1-goal tasks.

Original: Moreover, the qualitative results show that our model is less likely to be trapped in a deadlock.
Revised: Moreover, the qualitative results show that our model is less likely to be trapped in a deadlock.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: This paper studies the image-goal navigation (ImageNav) problem, which aims to steer an agent towards a destination with the goal image in unseen environments.
Revised: This paper studies the image-goal navigation (ImageNav) problem, which aims to steer an agent towards a destination with the goal image in unseen environments.

Original: ImageNav has recently received much attention due to its promising applications in robots navigating in the open world.
Revised: ImageNav has recently received much attention due to its promising applications in robots navigating in the open world.

Original: Scene memory is essential for imageNav as it provides indispensable historical information for decision-making in unseen environments (Savinov et al., 2018).
Revised: Scene memory is essential for imageNav as it provides indispensable historical information for decision-making in unseen environments (Savinov et al., 2018).

Original: During navigation, this memory typically stores both scene features and the agent’s navigation history (Kwon et al., 2021).
Revised: During navigation, this memory typically stores both scene features and the agent’s navigation history (Kwon et al., 2021).

Original: These two types of information in turn help the agent generate more reasonable navigation actions by lessening the negative impact of partial observability (Parisotto & Salakhutdinov, 2018).
Revised: These two types of information in turn help the agent generate more reasonable navigation actions by lessening the negative impact of partial observability (Parisotto & Salakhutdinov, 2018).

Original: In literature, various memory mechanisms have been introduced for ImageNav, which can be classified into three categories according to memory structure: (a) metric map-based methods (Chaplot et al., 2020a;
Revised: In literature, various memory mechanisms have been introduced for ImageNav, which can be classified into three categories according to memory structure: (a) metric map-based methods (Chaplot et al., 2020a;

Original: Chen et al., 2019) that reconstruct local top-down maps and aggregate them into a global map, (b) stacked memory-based methods (Pashevich et al., 2021;
Revised: Chen et al., 2019) that reconstruct local top-down maps and aggregate them into a global map, (b) stacked memory-based methods (Pashevich et al., 2021;

Original: Mezghani et al., 2021; Fang et al., 2019) that stack the past observations in chronological order, and (c) topological map-based methods (Kwon et al., 2021; Chaplot et al., 2020b; Beeching et al., 2020; Savinov et al., 2018) that store sparse landmark features in graph nodes.
Revised: Mezghani et al., 2021; Fang et al., 2019) that stack the past observations in chronological order, and (c) topological map-based methods (Kwon et al., 2021; Chaplot et al., 2020b; Beeching et al., 2020; Savinov et al., 2018) that store sparse landmark features in graph nodes.

Original: The topological map-based methods benefit from the memory sparsity of topological maps and have achieved impressive performance in ImageNav.
Revised: The topological map-based methods benefit from the memory sparsity of topological maps and have achieved impressive performance in ImageNav.

Original: However, existing topological map-based methods still suffer from two major limitations: (a) Unawareness of useful nodes.
Revised: However, existing topological map-based methods still suffer from two major limitations: (a) Unawareness of useful nodes.

Original: They generally use all node features for generating actions without considering the contribution of each node, thus being easily misled by redundant nodes that are uninformative of the goal.
Revised: They generally use all node features for generating actions without considering the contribution of each node, thus being easily misled by redundant nodes that are uninformative of the goal.

Original: (b) Local representation.
Revised: (b) Local representation.

Original: Each node feature only represents a small local area in a large scene, which restricts the agent’s ability to learn a higher-level semantic and geometric representation of the scene.
Revised: Each node feature only represents a small local area in a large scene, which restricts the agent’s ability to learn a higher-level semantic and geometric representation of the scene.

Original: To overcome the above two limitations, we present a novel ImageNav method named MemoNav, which is motivated by the classical concept of working memory in cognitive neuroscience (Cowan, 2008) and in loose analogy with the working memory model in human navigation (Blacker et al.,
Revised: To overcome the above two limitations, we present a novel ImageNav method named MemoNav, which is motivated by the classical concept of working memory in cognitive neuroscience (Cowan, 2008) and in loose analogy with the working memory model in human navigation (Blacker et al.,

Original: The MemoNav learns three types of scene representations: (a) Short-term memory (STM) represents the local and transient features of the nodes in a topological map.
Revised: The MemoNav learns three types of scene representations: (a) Short-term memory (STM) represents the local and transient features of the nodes in a topological map.

Original: (b) Long-term memory (LTM) is a global node that learns a scene-level representation by continuously aggregating STM.
Revised: (b) Long-term memory (LTM) is a global node that learns a scene-level representation by continuously aggregating STM.

Original: (c) Working memory (WM) learns goal-relevant features about 3D scenes and is used by a policy network to generate actions.
Revised: (c) Working memory (WM) learns goal-relevant features about 3D scenes and is used by a policy network to generate actions.

Original: The WM is formed by encoding the informative fraction of the STM and the LTM.
Revised: The WM is formed by encoding the informative fraction of the STM and the LTM.

Original: Based on the above three representations, the MemoNav navigation pipeline contains five steps: (1)
Revised: Based on the above three representations, the MemoNav navigation pipeline contains five steps: (1)

Original: STM generation.
Revised: STM generation.

Original: The map update module stores landmark features on the map as the STM.
Revised: The map update module stores landmark features on the map as the STM.

Original: Selective forgetting.
Revised: Selective forgetting.

Original: To incorporate goal-relevant STM into the WM, a forgetting module temporarily removes nodes whose attention scores assigned by a memory decoder rank below a predefined percentage.
Revised: To incorporate goal-relevant STM into the WM, a forgetting module temporarily removes nodes whose attention scores assigned by a memory decoder rank below a predefined percentage.

Original: After forgetting, the navigation pipeline will not compute the forgotten node features at subsequent time steps.
Revised: After forgetting, the navigation pipeline will not compute the forgotten node features at subsequent time steps.

Original: (3) LTM generation.
Revised: (3) LTM generation.

Original: To assist the STM, we add a global node to the map as the
Revised: To assist the STM, we add a global node to the map as the

Original: LTM.
Revised: LTM.

Original: The global node links to all map nodes and continuously aggregates the features of these nodes at each time step.
Revised: The global node links to all map nodes and continuously aggregates the features of these nodes at each time step.

Original: (4) WM generation.
Revised: (4) WM generation.

Original: A graph attention module encodes the retained STM and the
Revised: A graph attention module encodes the retained STM and the

Original: LTM to generate the WM.
Revised: LTM to generate the WM.

Original: The WM utilizes the goal-relevant information in the retained STM and the scene-level feature in the LTM, thus enabling the agent to utilize informative scene representations to improve navigation performance.
Revised: The WM utilizes the goal-relevant information in the retained STM and the scene-level feature in the LTM, thus enabling the agent to utilize informative scene representations to improve navigation performance.

Original: (5) Action generation.
Revised: (5) Action generation.

Original: Two Transformer decoders use the embeddings of the goal image and the current observation to decode the WM.
Revised: Two Transformer decoders use the embeddings of the goal image and the current observation to decode the WM.

Original: Then, the decoded features are used to generate navigation actions.
Revised: Then, the decoded features are used to generate navigation actions.

Original: Consequently, with the synergy of the three representations, the MemoNav noticeably outperforms the
Revised: Consequently, with the synergy of the three representations, the MemoNav noticeably outperforms the

Original: SoTA method (Kwon et al., 2021) in the Gibson scenes (Xia et al., 2018), increasing the navigation success rate by approximately 2.9%, 1.4%, 2.4%, and 1.7% on 1, 2, 3, and 4-goal test datasets, respectively.
Revised: SoTA method (Kwon et al., 2021) in the Gibson scenes (Xia et al., 2018), increasing the navigation success rate by approximately 2.9%, 1.4%, 2.4%, and 1.7% on 1, 2, 3, and 4-goal test datasets, respectively.

Original: The comparison in the Matterport3D scenes (Chang et al., 2017) shows that the
Revised: The comparison in the Matterport3D scenes (Chang et al., 2017) shows that the

Original: MemoNav exhibits better transferability.
Revised: MemoNav exhibits better transferability.

Original: The main contributions of this paper are as follows:
Revised: The main contributions of this paper are as follows:

Original: • We propose the MemoNav, which learns three types of scene representations (STM, LTM, and WM) to improve navigation performance in the ImageNav task.
Revised: • We propose the MemoNav, which learns three types of scene representations (STM, LTM, and WM) to improve navigation performance in the ImageNav task.

Original: • We use a forgetting module to retain the STM, thereby reducing redundancy in the map and improving navigation efficiency.
Revised: • We use a forgetting module to retain the STM, thereby reducing redundancy in the map and improving navigation efficiency.

Original: We also introduce a global node as the LTM.
Revised: We also introduce a global node as the LTM.

Original: The LTM connects to all nodes in the STM and learns a scene-level representation that provides the agent with a global view.
Revised: The LTM connects to all nodes in the STM and learns a scene-level representation that provides the agent with a global view.

Original: • We adopt a graph attention module to generate WM from the retained STM and the LTM.
Revised: • We adopt a graph attention module to generate WM from the retained STM and the LTM.

Original: This module flexibly adjusts weights used for aggregating node features, which helps the agent use adaptive WM to improve performance.
Revised: This module flexibly adjusts weights used for aggregating node features, which helps the agent use adaptive WM to improve performance.

Original: • The experimental results demonstrate that our model outperforms the SoTA baseline on both 1-goal and multi-goal tasks in two popular scene datasets.
Revised: • The experimental results demonstrate that our model outperforms the SoTA baseline on both 1-goal and multi-goal tasks in two popular scene datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: ImageNav models based on topological maps .
Revised: ImageNav methods .

Original: Cognitive research (Wang & Spelke, 2002; Foo et al., 2005; Gillner & Mallot, 1998) suggests that humans save landmark features in their memory for navigation instead of detailed scene layouts. Several methods (Savinov et al., 2018; Beeching et al., 2020; Chaplot et al., 2020b; Kwon et al., 2021; Chen et al., 2021) utilize this theory and propose topological memory for visual navigation. SPTM (Savinov et al., 2018) is an early attempt  to use topological maps inImageNav. It proposes a semi-parametric topological memory that is pre-built before navigation.
Revised: Since an early attempt (Zhu et al., 2017) to train agents in a simulator for

Original: Beeching et al. (Beeching et al., 2020) introduced a graph processing method inspired by dynamic programming-based shortest path algorithms.
Revised: ImageNav, rapid progress has been made on this task (Beeching et al., 2020; Chen et al., 2021; Wasserman et al., 2022; Al-Halah et al., 2022).

Original: This method also needs to explore the scene in advance.
Revised: Several methods have utilized topological scene representations for visual navigation, of which SPTM (Savinov et al., 2018) is an early work.

Original: NTS (Chaplot et al., 2020b) and VGM (Kwon et al., 2021) incrementally build the topological map during navigation. VGM uses no pose sensor data and is more robust to sensor noise than NTS.
Revised: NTS (Chaplot et al., 2020b) and VGM (Kwon et al., 2021) incrementally build a topological map during navigation and generalize to unseen environments without exploring the scenes in advance.

Original: The two methods utilize all features in the map, while the MemoNav flexibly utilizes the informative fraction of these features.
Revised: These methods utilize all features in the map, while the MemoNav flexibly utilizes the informative fraction of these features.

Revised: Another line of work (Yadav et al., 2022; Majumdar et al., 2022) has introduced self-supervised learning to enhance the scene representations, achieving a promising navigation success rate.

Revised: In contrast, we enhance the scene representations using a global node that aggregates the agent’s local observation features.

Original: Memory mechanisms for reinforcement learning .
Revised: Memory mechanisms for reinforcement learning .

Original: Several studies (Ritter et al., 2021; Lampinen et al., 2021; Sukhbaatar et al., 2021; Loynd et al., 2020) draw inspiration from memory mechanisms of the human brain and design reinforcement learning models for reasoning over long time horizons.
Revised: Several studies (Ritter et al., 2021; Lampinen et al., 2021; Sukhbaatar et al., 2021; Loynd et al., 2020) draw inspiration from memory mechanisms of the human brain and design reinforcement learning models for reasoning over long time horizons.

Original: Ritter et al.
Revised: Ritter et al.

Original: (Ritter et al., 2021) proposed an episodic memory storing state transitions for navigation tasks.
Revised: (Ritter et al., 2021) proposed an episodic memory storing state transitions for navigation tasks.

Original: Lampinen et al.
Revised: Lampinen et al.

Original: (Lampinen et al., 2021) presented hierarchical attention memory as a form of “mental time-travel” (Tulving, 1985), which means recovering goal-oriented information from past experiences.
Revised: (Lampinen et al., 2021) presented hierarchical attention memory as a form of “mental time-travel” (Tulving, 1985), which means recovering goal-oriented information from past experiences.

Original: Unlike this method, our model retains such information via a novel forgetting module.
Revised: Unlike this method, our model retains such information via a novel forgetting module.

Original: Expire-span (Sukhbaatar et al., 2021) predicts life spans for each memory fragment and permanently deletes expired ones.
Revised: Expire-span (Sukhbaatar et al., 2021) predicts life spans for each memory fragment and permanently deletes expired ones.

Original: Our model is different from it in that we restore forgotten memory if the agent returns to visited places.
Revised: Our model is different from it in that we restore forgotten memory if the agent returns to visited places.

Original: The most similar work is Working Memory Graph (Loynd et al., 2020) which is also inspired by working memory in cognitive neuroscience.
Revised: The most similar work is Working Memory Graph (Loynd et al., 2020) which is also inspired by working memory in cognitive neuroscience.

Original: However, its memory capacity is fixed.
Revised: However, its memory capacity is fixed.

Original: In contrast, our model retains a certain proportion of short-term memory in the working memory and adjusts the memory capacity when the navigation episode contains more goals.
Revised: In contrast, our model retains a certain proportion of short-term memory in the working memory and adjusts the memory capacity when the navigation episode contains more goals.

Original:  B ACKGROUND 3. T ASK D EFINITION
Revised:  B ACKGROUND 3. T ASK D EFINITION

Original: The objective of ImageNav (Figure 1) is to learn a policy π to reach a target, given an image I target that contains a view of the target and a series of observations { I t } captured during the navigation.
Revised: The objective of ImageNav (Figure 1) is to learn a policy π to reach a target, given an image I target that contains a view of the target and a series of observations { I t } captured during the navigation.

Original: At the beginning of navigation, the agent receives an RGB image I target of the target.
Revised: At the beginning of navigation, the agent receives an RGB image I target of the target.

Original: At each time step, the agent captures an RGB-D panoramic image I t of the current location and generates a navigational action.
Revised: At each time step, the agent captures an RGB-D panoramic image I t of the current location and generates a navigational action.

Original: Following (Kwon et al., 2021), any additional sensory data (e.g. GPS and IMU) are not available.
Revised: Following (Kwon et al., 2021), any additional sensory data (e.g. GPS and IMU) are not available.

Original: 3. B RIEF R EVIEW OF V ISUAL G RAPH M EMORY
Revised: 3. B RIEF R EVIEW OF V ISUAL G RAPH M EMORY

Original: The MemoNav is mainly built upon VGM (Kwon et al., 2021), which is briefly introduced below.
Revised: The MemoNav is mainly built upon VGM (Kwon et al., 2021), which is briefly introduced below.

Original: VGM incrementally builds a topological map G
Revised: VGM incrementally builds a topological map G

Original: =
Revised: =

Original: ( V , E ) from the agent’s past observations where V and E denote nodes and edges, respectively.
Revised: ( V , E ) from the agent’s past observations where V and E denote nodes and edges, respectively.

Original: The node features (short-term memory)
Revised: The node features (short-term memory)

Original: V ∈ R d × N t are generated from observations by a pretrained encoder F loc where d denotes the dimension of feature and N t the number of nodes at time t .
Revised: V ∈ R d × N t are generated from observations by a pretrained encoder F loc where d denotes the dimension of feature and N t the number of nodes at time t .

Original: VGM uses a graph convolutional network (GCN) to encode the topological map, obtaining the encoded memory M = GCN( V ) .
Revised: VGM uses a graph convolutional network (GCN) to encode the topological map, obtaining the encoded memory M = GCN( V ) .

Original: Before encoding, VGM obtains the target embedding e target =
Revised: Before encoding, VGM obtains the target embedding e target =

Original: F enc ( I target ) and fuses each node feature with this embedding using a linear layer.
Revised: F enc ( I target ) and fuses each node feature with this embedding using a linear layer.

Original: The encoded memory is then decoded by two Transformer (Vaswani et al., 2017) decoders, D cur and D target .
Revised: The encoded memory is then decoded by two Transformer (Vaswani et al., 2017) decoders, D cur and D target .

Original: D cur takes the current observation embedding e cur =
Revised: D cur takes the current observation embedding e cur =

Original: F enc ( I cur ) as the query and the feature vectors of the encoded memory M as the keys and values, generating a feature vector f cur .
Revised: F enc ( I cur ) as the query and the feature vectors of the encoded memory M as the keys and values, generating a feature vector f cur .

Original: Similarly, D target takes the target embedding e target as the query and generates f target .
Revised: Similarly, D target takes the target embedding e target as the query and generates f target .

Original: Lastly, an
Revised: Lastly, an

Original: LSTM-based policy network takes as input the concatenation of f cur , f target and e cur to output an action distribution.
Revised: LSTM-based policy network takes as input the concatenation of f cur , f target and e cur to output an action distribution.

Original:  M ETHOD
Revised:  M ETHOD

Original: The MemoNav comprises three key components: the forgetting module, the global node, and
Revised: The MemoNav comprises three key components: the forgetting module, the global node, and

Original: the
Revised: the

Original: GATv2 memory encoder.
Revised: GATv2 memory encoder.

Original: We show the pipeline of the MemoNav in Figure 2 and describe these components in the remainder of this section.
Revised: We show the pipeline of the MemoNav in Figure 2 and describe these components in the remainder of this section.

Original: 4. S ELECTIVE F ORGETTING M ODULE
Revised: 4. S ELECTIVE F ORGETTING M ODULE

Original: We devise a forgetting module that instructs the agent to forget uninformative experiences.
Revised: We devise a forgetting module that instructs the agent to forget uninformative experiences.

Original: Here, “forgetting” means that nodes with attention scores lower than a threshold are temporarily excluded from the navigation pipeline.
Revised: Here, “forgetting” means that nodes with attention scores lower than a threshold are temporarily excluded from the navigation pipeline.

Original: This means of forgetting via attention is evidenced by research (Anderson et al., 2018; Fukuda & Vogel, 2009) revealing that the optimal performance of working memory relies on humans’ ability to focus attention on task-relevant information.
Revised: This means of forgetting via attention is evidenced by research (Fukuda & Vogel, 2009) revealing that the optimal performance of working memory relies on humans’ ability to focus attention on task-relevant information.

Original: In Figure 3, we visualize the attention scores for the short-term memory (STM) calculated in D target of VGM.
Revised: In Figure 3, we visualize the attention scores for the short-term memory (STM) calculated in D target of VGM.

Original: The figure shows that the agent assigns high scores to nodes close to targets while paying little attention to remote ones.
Revised: The figure shows that the agent assigns high scores to nodes close to targets while paying little attention to remote ones.

Original: This phenomenon indicates that not all node features in the STM are helpful and that it is more efficient to use a small number of them for navigation.
Revised: This phenomenon indicates that not all node features in the STM are helpful and that it is more efficient to use a small number of them for navigation.

Original: The forgetting module retains a fraction of STM according to the attention scores { α i } N t i =1 in D target .
Revised: The forgetting module retains a fraction of STM according to the attention scores { α i } N t i =1 in D target .

Original: These scores denote to what extent the goal embedding e target attends to each node feature in the
Revised: These scores denote to what extent the goal embedding e target attends to each node feature in the

Original: STM.
Revised: STM.

Original: After D target finishes decoding, the agent temporarily “forgets” a fraction of nodes whose scores rank below a predefined percentage p .
Revised: After D target finishes decoding, the agent temporarily “forgets” a fraction of nodes whose scores rank below a predefined percentage p .

Original: In other words, these nodes will be disconnected from their neighbors, and not be processed by the navigation pipeline at the subsequent time steps.
Revised: In other words, these nodes will be disconnected from their neighbors, and not be processed by the navigation pipeline at the subsequent time steps.

Original: If the agent returns to a forgotten node, this node will be added to the map and processed by the pipeline again.
Revised: If the agent returns to a forgotten node, this node will be added to the map and processed by the pipeline again.

Original: In a multi-goal task, once the agent has reached a target, all forgotten nodes will be restored, as these nodes are probably useful for finding the next target.
Revised: In a multi-goal task, once the agent has reached a target, all forgotten nodes will be restored, as these nodes are probably useful for finding the next target.

Original: Theproposed forgetting module is used in a plug-and-play manner. Our default model sets p as 20%. When evaluating and deploying the agent, the forgetting module is  switched on.
Revised: The forgetting module is used in a plug-and-play manner, which means that this module is not activated during training and switched on when evaluating and deploying. p is default to 20%.

Original: With this module, the agent can selectively retain the informative STM, while avoiding misleading experiences.
Revised: With this module, the agent can selectively retain the informative STM, while avoiding misleading experiences.

Original: 4. L ONG - TERM M EMORY G ENERATION
Revised: 4. L ONG - TERM M EMORY G ENERATION

Original: In addition to STM, the information in the long-term memory (LTM) also forms part of working memory (WM) (Ericsson & Kintsch, 1995).
Revised: In addition to STM, the information in the long-term memory (LTM) also forms part of working memory (WM) (Ericsson & Kintsch, 1995).

Original: Inspired by ETC (Ainslie et al., 2020) and LongFormer (Beltagy et al., 2020), we add a zero-initialized trainable global node n global ∈ R d to the topological map as the LTM (the orange nodes in Figure 2).
Revised: Inspired by ETC (Ainslie et al., 2020) and LongFormer (Beltagy et al., 2020), we add a zero-initialized trainable global node n global ∈ R d to the topological map as the LTM (the orange nodes in Figure 2).

Original: RecBERT (Hong et al., 2021) adopts a recurrent state token similar to the proposed LTM, but their functions are different.
Revised: RecBERT (Hong et al., 2021) adopts a recurrent state token similar to the proposed LTM, but their functions are different.

Original: The token in RecBERT aggregates visual-linguistic clues while the LTM in our model summarizes the agent’s past observations by continuously fusing the STM via memory encoding (the encoder is described in the next subsection).
Revised: The token in RecBERT aggregates visual-linguistic clues while the LTM in our model summarizes the agent’s past observations by continuously fusing the STM via memory encoding (the encoder is described in the next subsection).

Original: The LTM possesses two advantages: it learns a scene-level feature and facilitates feature fusing.
Revised: The LTM possesses two advantages: it learns a scene-level feature and facilitates feature fusing.

Original: A recent study (Ramakrishnan et al., 2022) suggests that embodied agents need to learn higher-level representations of the 3D environment to overcome the partial observability caused by limited field-ofview sensors.
Revised: A recent study (Ramakrishnan et al., 2022) suggests that embodied agents need to learn higher-level representations of the 3D environment to overcome the partial observability caused by limited field-ofview sensors.

Original: From this viewpoint, the LTM stores a high-level scene representation by aggregating local node features.
Revised: From this viewpoint, the LTM stores a high-level scene representation by aggregating local node features.

Original: Moreover, another merit of the LTM is facilitating feature fusing.
Revised: Moreover, another merit of the LTM is facilitating feature fusing.

Original: The topological map is divided into several sub-graphs when forgotten nodes are removed.
Revised: The topological map is divided into several sub-graphs when forgotten nodes are removed.

Original: Consequently, direct message passing between the nodes separated in different sub-graphs no longer exists.
Revised: Consequently, direct message passing between the nodes separated in different sub-graphs no longer exists.

Original: The LTM links to every node in the map, acting as a bypath that facilitates feature fusing between these isolated sub-graphs.
Revised: The LTM links to every node in the map, acting as a bypath that facilitates feature fusing between these isolated sub-graphs.

Original: 4. W ORKING M EMORY G ENERATION
Revised: 4. W ORKING M EMORY G ENERATION

Original: The third type of scene representation WM learns goal-relevant features that are used to generate actions.
Revised: The third type of scene representation WM learns goal-relevant features that are used to generate actions.

Original: To learn adaptive WM, we utilize a graph attention module GATv2 (Brody et al., 2022) to encode the retained STM and the LTM since GATv2 is powerful when different nodes have different rankings of neighbors.
Revised: To learn adaptive WM, we utilize a graph attention module GATv2 (Brody et al., 2022) to encode the retained STM and the LTM since GATv2 is powerful when different nodes have different rankings of neighbors.

Original: GATv2 derives adaptive weights from node features for neighboring nodes, instead of from the Laplacian matrix.
Revised: GATv2 derives adaptive weights from node features for neighboring nodes, instead of from the Laplacian matrix.

Original: This design is suitable for generating WM, especially in multi-goal tasks since the STM features that contain information about the target or a path leading to the target should obtain high weights while those of irrelevant places should receive lower weights.
Revised: This design is suitable for generating WM, especially in multi-goal tasks since the STM features that contain information about the target or a path leading to the target should obtain high weights while those of irrelevant places should receive lower weights.

Original: In addition, the node features in the STM are dynamically updated during navigation, as the agent replaces these features with new ones when revisiting these nodes.
Revised: In addition, the node features in the STM are dynamically updated during navigation, as the agent replaces these features with new ones when revisiting these nodes.

Original: Therefore, it is more suitable to adaptively change the weights used to aggregate the STM features into the WM.
Revised: Therefore, it is more suitable to adaptively change the weights used to aggregate the STM features into the WM.

Original: The WM generation is formulated as: M w = { V ′ , n tglobal } =
Revised: The WM generation is formulated as: M w = { V ′ , n tglobal } =

Original: GATv2( { V , n t − 1 global } ) where M w represents the generated working memory, and V ′ the encoded STM.
Revised: GATv2( { V , n t − 1 global } ) where M w represents the generated working memory, and V ′ the encoded STM.

Original: {· , ·} denotes that the LTM (a vector) is prepended to the retained STM (a sequence of vectors).
Revised: {· , ·} denotes that the LTM (a vector) is prepended to the retained STM (a sequence of vectors).

Original: Note that the time step superscript of n global means that the LTM is recurrent through time.
Revised: Note that the time step superscript of n global means that the LTM is recurrent through time.

Original: After GATv2 encoding, the WM aggregates the goal-relevant information in the retained STM as well as the scene-level representation in the LTM.
Revised: After GATv2 encoding, the WM aggregates the goal-relevant information in the retained STM as well as the scene-level representation in the LTM.

Original: Lastly, the decoders D cur and D target take M w as keys and values, generating f cur and f target , which are further used to generate actions.
Revised: Lastly, the decoders D cur and D target take M w as keys and values, generating f cur and f target , which are further used to generate actions.

Original:  E XPERIMENTS 5. D ATASETS
Revised:  E XPERIMENTS 5. D ATASETS

Original: All experiments are conducted in the Habitat (Savva et al., 2019) simulator with the Gibson (Xia et al., 2018) or Matterpot3D (Chang et al., 2017) scene dataset.
Revised: All experiments are conducted in the Habitat (Savva et al., 2019) simulator with the Gibson (Xia et al., 2018) or Matterpot3D (Chang et al., 2017) scene dataset.

Original: We adopt the same action space as in
Revised: We adopt the same action space as in

Original: VGM (Kwon et al., 2021).
Revised: VGM (Kwon et al., 2021).

Original: 1-goal dataset .
Revised: 1-goal dataset .

Original: In the Gibson scenes, all models are trained with 72 scenes and evaluated on a public 1-goal 1 dataset (Mezghani et al., 2021) that comprises 14 unseen scenes.
Revised: In the Gibson scenes, all models are trained with 72 scenes and evaluated on a public 1-goal 1 dataset (Mezghani et al., 2021) that comprises 14 unseen scenes.

Original: Following the setting in
Revised: Following the setting in

Original: VGM (Kwon et al., 2021), 1007 out of 1400 episodes in this public dataset are used for the evaluation on 1-goal tasks, but the 1400 episodes are still used for ablation studies.
Revised: VGM (Kwon et al., 2021), 1007 out of 1400 episodes in this public dataset are used for the evaluation on 1-goal tasks, but the 1400 episodes are still used for ablation studies.

Original: Multi-goal dataset .
Revised: Multi-goal dataset .

Original: Multi-goal evaluation is more suitable for evaluating memory mechanisms used by navigation methods.
Revised: Multi-goal evaluation is more suitable for evaluating memory mechanisms used by navigation methods.

Original: By letting the agent return to visited places we can test whether memory mechanisms help the agent plan a short path.
Revised: By letting the agent return to visited places we can test whether memory mechanisms help the agent plan a short path.

Original: If not, the agent will probably waste its time re-exploring the scene or traveling randomly.
Revised: If not, the agent will probably waste its time re-exploring the scene or traveling randomly.

Original: However, recent ImageNav methods seldom conduct multi-goal evaluations.
Revised: However, recent ImageNav methods seldom conduct multi-goal evaluations.

Original: To further investigate how memory mechanisms assist the agent in navigation, we follow
Revised: To further investigate how memory mechanisms assist the agent in navigation, we follow

Original: MultiON
Revised: MultiON

Original: (Wani et al., 2020) to collect 700-episode multi-goal test datasets in the Gibson scenes (an example in Figure 1).
Revised: (Wani et al., 2020) to collect 700-episode multi-goal test datasets in the Gibson scenes (an example in Figure 1).

Original: In a multi-goal task, the agent navigates to an ordered sequence of goals.
Revised: In a multi-goal task, the agent navigates to an ordered sequence of goals.

Original: We follow five rules to set sequential goals for each sample: (1) No obstacles appear inside a circle with a radius of 0.75 meters centered at each goal.
Revised: We follow five rules to set sequential goals for each sample: (1) No obstacles appear inside a circle with a radius of 0.75 meters centered at each goal.

Original: The distance between two successive goals is no more than 10 meters.
Revised: The distance between two successive goals is no more than 10 meters.

Original: (3) All goals are placed on the same layer without altitude differences.
Revised: (3) All goals are placed on the same layer without altitude differences.

Original: All goals are reachable from each other.
Revised: All goals are reachable from each other.

Original: The final goal is placed near a certain previous one with the distance between them smaller than 1.5 meters.
Revised: The final goal is placed near a certain previous one with the distance between them smaller than 1.5 meters.

Original: Please refer to A.4.4 for more details.
Revised: Please refer to A.4.4 for more details.

Original: In the Matterpot3D scenes, we sample 1008 episodes for each difficulty level from the multi-goal test datasets used in Multi-ON
Revised: In the Matterpot3D scenes, we sample 1008 episodes for each difficulty level from the multi-goal test datasets used in Multi-ON

Original: (Wani et al., 2020).
Revised: (Wani et al., 2020).

Original: The difficulty of an episode is by the tested on other difficulty levels.
Revised: The difficulty of an episode is by the tested on other difficulty levels.

Original: Evaluation Metrics .
Revised: Evaluation Metrics .

Original: In 1-goal tasks, the success rate ( SR ) and success weighted by path length ( SPL ) (Anderson et al., 2018) are used.
Revised: In 1-goal tasks, the success rate ( SR ) and success weighted by path length ( SPL ) (Anderson et al., 2018) are used.

Original: In a multi-goal task, two metrics are borrowed from (Wani et al., 2020):
Revised: In a multi-goal task, two metrics are borrowed from (Wani et al., 2020):

Original: The progress ( PR ) is the fraction of goals successfully reached, equal to the SR for 1-goal tasks; Progress weighted by path length ( PPL ) indicates navigation efficiency and is defined as
Revised: The progress ( PR ) is the fraction of goals successfully reached, equal to the SR for 1-goal tasks; Progress weighted by path length ( PPL ) indicates navigation efficiency and is defined as

Original: PPL = 1 E
Revised: PPL = 1 E

Original: E (cid:88) i =
Revised: E (cid:88) i =

Original: Progress i l i max ( p i , l i ) , where E is the total number of test episodes, l i and p i are the shortest path distance to the final goal via midway ones, and the actual path length taken by the agent, respectively.
Revised: Progress i l i max ( p i , l i ) , where E is the total number of test episodes, l i and p i are the shortest path distance to the final goal via midway ones, and the actual path length taken by the agent, respectively.

Original: 5. C OMPARED M ETHODS AND T RAINING DETAILS
Revised: 5. C OMPARED M ETHODS AND T RAINING DETAILS

Original: We compare the MemoNav with previous methods utilizing various types of memory.
Revised: We compare the MemoNav with previous methods utilizing various types of memory.

Original: Random is an agent that samples actions from a uniform distribution and navigates with oracle stopping.
Revised: Random is an agent that samples actions from a uniform distribution and navigates with oracle stopping.

Original: CNNLSTM (Zhu et al., 2017) uses no maps but a hidden vector as implicit memory.
Revised: CNNLSTM (Zhu et al., 2017) uses no maps but a hidden vector as implicit memory.

Original: ANS (Chaplot et al., 2020a) is a metric map-based model adapted for ImageNav in the experiments of VGM (Kwon et al., 2021).
Revised: ANS (Chaplot et al., 2020a) is a metric map-based model adapted for ImageNav in the experiments of VGM (Kwon et al., 2021).

Original: NTS (Chaplot et al., 2020b) incrementally builds a topological map without pre-exploring and adopts a hierarchical navigation strategy.
Revised: NTS (Chaplot et al., 2020b) incrementally builds a topological map without pre-exploring and adopts a hierarchical navigation strategy.

Original: VGM (Kwon et al., 2021) (see Section 3.2) is the baseline the MemoNav is built on.
Revised: VGM (Kwon et al., 2021) (see Section 3.2) is the baseline the MemoNav is built on.

Original: Following VGM (Kwon et al., 2021), for all methods except ANS and NTS, the agent is first trained using imitation learning, minimizing the negative log-likelihood of the ground-truth actions.
Revised: Following VGM (Kwon et al., 2021), for all methods except ANS and NTS, the agent is first trained using imitation learning, minimizing the negative log-likelihood of the ground-truth actions.

Original: Next, the agent is finetuned with proximal policy optimization (PPO) (Schulman et al., 2017) to improve the exploratory ability.
Revised: Next, the agent is finetuned with proximal policy optimization (PPO) (Schulman et al., 2017) to improve the exploratory ability.

Original: The reward setting and auxiliary losses remain the same as in VGM.
Revised: The reward setting and auxiliary losses remain the same as in VGM.

Original: Each model is trained for five runs on a TITAN RTX GPU.
Revised: Each model is trained for five runs on a TITAN RTX GPU.

Original: The evaluation results for ANS and NTS are borrowed from the VGM paper (Kwon et al., 2021)
Revised: The evaluation results for ANS and NTS are borrowed from the VGM paper (Kwon et al., 2021)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 5. Q UANTITATIVE R ESULTS
Revised: 5. Q UANTITATIVE R ESULTS

Original: Comparison on Gibson .
Revised: Comparison on Gibson .

Original: Table 1 shows that the MemoNav outperforms all the baselines in SR on the four difficulty levels.
Revised: Table 1 shows that the MemoNav outperforms all the baselines in SR on the four difficulty levels.

Original: CNNLSTM exhibits the poorest performance as its limited memory provides insufficient information about the goal area.
Revised: CNNLSTM exhibits the poorest performance as its limited memory provides insufficient information about the goal area.

Original: The MemoNav outperforms the metric map-based method ANS which requires pre-built maps, enjoying a large improvement in SR and SPL.
Revised: The MemoNav outperforms the metric map-based method ANS which requires pre-built maps, enjoying a large improvement in SR and SPL.

Original: Compared with theSoTA VGM, our model exhibits a noticeable performance gain, increasing the SR by 0.029 , 0.014 . 0.024 , and 0.017 on the 1, 2, 3, and 4-goal tasks respectively while using 20% fewer node features.
Revised: Compared with the VGM, our model exhibits a noticeable performance gain, increasing the SR by 2.9% , 1.4% . 2.4% , and 1.7% on the 1, 2, 3, and 4-goal tasks respectively while using 20% fewer node features.

Original: This result indicates that the MemoNav benefits from the informative scene memory and the high-level scene representation contained in the WM, obtaining a higher success rate.
Revised: This result indicates that the MemoNav benefits from the informative scene memory and the high-level scene representation contained in the WM, obtaining a higher success rate.

Original: Comparison on Matterport3D .
Revised: Comparison on Matterport3D .

Original: We conduct an evaluation on the Matterport3D scenes to test the models’ ability to transfer to other scene types.
Revised: We conduct an evaluation on the Matterport3D scenes to test the models’ ability to generalize to other scene types.

Original: Table 1 shows that our method achieves consistent performance improvements on this unseen scene dataset.
Revised: Table 1 shows that our method achieves consistent performance improvements on this unseen scene dataset.

Original: Compared with VGM, the MemoNav witnesses performance gains in the success rate at the three difficulty levels, improving the SR/PR by 6.5% (from 0.244 to 0.260), 15.2% (from 0.158 to 0.182), and 10.8% (from 0.120 to 0.133) on the 1, 2, and 3-goal tasks, respectively.
Revised: Compared with VGM, the MemoNav witnesses performance gains in the success rate at the three difficulty levels, improving the SR/PR by 1.6% , 2.4% , and 1.3% on the 1, 2, and 3-goal tasks, respectively.

Original: These results show that our method is more capable of transferring to new scene styles.
Revised: These results show that our method is more capable of generalizing to new scene styles.

Original: 5. A BLATION S TUDIES AND A NALYSIS
Revised: 5. A BLATION S TUDIES AND A NALYSIS

Original: We conduct ablation studies in the Gibson scenes to analyze the impact of each proposed component.
Revised: We conduct ablation studies in the Gibson scenes to analyze the impact of each proposed component.

Original: Performance gain of each proposed component.
Revised: Performance gain of each proposed component.

Original: We ablate the three key components described in
Revised: We ablate the three key components described in

Original: Section 4 and show the results in Table 2.
Revised: Section 4 and show the results in Table 2.

Original: Comparing row 2 with row 1, we can see that the LTM brings a noticeable improvement in SR/PR over the baseline.
Revised: Comparing row 2 with row 1, we can see that the LTM brings a noticeable improvement in SR/PR over the baseline.

Original: Comparing row 3 with row 1, applying the forgetting module achieves slight improvements in the SR/PR.
Revised: Comparing row 3 with row 1, applying the forgetting module achieves slight improvements in the SR/PR.

Original: However, its cooperation with the
Revised: However, its cooperation with the

Original: LTM witnesses larger increases (row 4 vs. row 1).
Revised: LTM witnesses larger increases (row 4 vs. row 1).

Original: More importantly, compared with the baseline (row 
Revised: More importantly, compared with the baseline (row 1), the synergy of the three components (row 5) increases the SR/PR by 0.049 ( 8.8% ), 0.014 ( 3.1% ),

Original: [Table]
Revised: [Table]

Original: w/odecodingLTM
Revised: w/odecodingLTM

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: 1), the synergy of the three components (row 5) increases the SR/PR by 0.049 ( 8.8% ), 0.014 ( 3.1% ), 0.024 ( 7.2% ), 0.017 ( 6.6% ) at the 1, 2, 3 and 4-goal levels respectively.
Revised: 0.024 ( 7.2% ), 0.017 ( 6.6% ) at the 1, 2, 3 and 4-goal levels respectively.

Original: These results demonstrate that the three components help to solve long-time navigation tasks with multiple sequential goals.
Revised: These results demonstrate that the three components help to solve long-time navigation tasks with multiple sequential goals.

Original: Importance of LTM.
Revised: Importance of LTM.

Original: Table 3 presents the results of ablation experiments for the LTM (described in Section 4.2).
Revised: Table 3 presents the results of ablation experiments for the LTM (described in Section 4.2).

Original: The first ablation experiment (row 2) replaces the LTM feature with a randomly selected STM feature each time the GATv2 encoding is finished, so that the MemoNav cannot use the scene-level representation in the LTM.
Revised: The first ablation experiment (row 2) replaces the LTM feature with a randomly selected STM feature each time the GATv2 encoding is finished, so that the MemoNav cannot use the scene-level representation in the LTM.

Original: Row 2 shows that the navigation performance becomes worse at all difficulty levels compared with the full model (row 1).
Revised: Row 2 shows that the navigation performance becomes worse at all difficulty levels compared with the full model (row 1).

Original: Furthermore, when the LTM feature is not incorporated into the WM (row 3), the performance also deteriorates.
Revised: Furthermore, when the LTM feature is not incorporated into the WM (row 3), the performance also deteriorates.

Original: In summary, the LTM stores a scene-level feature essential for improving the success rate.
Revised: In summary, the LTM stores a scene-level feature essential for improving the success rate.

Original: Correlation between navigation performance and forgetting threshold .
Revised: Correlation between navigation performance and forgetting threshold .

Original: We evaluate our model with different forgetting thresholds p (see Section 4.1).
Revised: We evaluate our model with different forgetting thresholds p (see Section 4.1).

Original: The results are shown in Figure 4.
Revised: The results are shown in Figure 4.

Original: For clarity, the figure shows the performance differences between our full model (with the forgetting module, the
Revised: For clarity, the figure shows the performance differences between our full model (with the forgetting module, the

Original: LTM, and the GATv2 encoder) and the variants.
Revised: LTM, and the GATv2 encoder) and the variants.

Original: The four levels exhibit different trends.
Revised: The four levels exhibit different trends.

Original: When p increases (i.e. a larger fraction of STM is not incorporated into the WM), our model witnesses first increases and then drops in the SR/PR and SPL/PPL at the 3 and 4-goal levels while enjoying slight gains in these metrics at the 1 and 2-goal levels.
Revised: When p increases (i.e. a larger fraction of STM is not incorporated into the WM), our model witnesses first increases and then drops in the SR/PR and SPL/PPL at the 3 and 4-goal levels while enjoying slight gains in these metrics at the 1 and 2-goal levels.

Original: At the 1-goal level, our model obtains the highest SR and SPL when p = 80% , which means that a large fraction of node features in the map are useless when the navigation task is easy.
Revised: At the 1-goal level, our model obtains the highest SR and SPL when p = 80% , which means that a large fraction of node features in the map are useless when the navigation task is easy.

Original: A similar trend can also be seen at the 2-goal level.
Revised: A similar trend can also be seen at the 2-goal level.

Original: In contrast, our model exhibits an increase in the PR and PPL at the 4-goal level when p rises from 0% to 40%.
Revised: In contrast, our model exhibits an increase in the PR and PPL at the 4-goal level when p rises from 0% to 40%.

Original: However, if p rises to 80%, the two metrics see a precipitous decline, and the agent takes more than 20 steps to complete the tasks.
Revised: However, if p rises to 80%, the two metrics see a precipitous decline, and the agent takes more than 20 steps to complete the tasks.

Original: These results suggest that excluding redundant STM from the WM improves navigation performance in multi-goal tasks.
Revised: These results suggest that excluding redundant STM from the WM improves navigation performance in multi-goal tasks.

Original: However, excluding an excessive fraction forces the agent not to utilize what it has explored, thus undermining the navigation performance.
Revised: However, excluding an excessive fraction forces the agent not to utilize what it has explored, thus undermining the navigation performance.

Original: 5. V ISUALIZATION R ESULTS
Revised: 5. V ISUALIZATION R ESULTS

Original: To observe how the MemoNav improves the navigation performance, we show example episodes of
Revised: To observe how the MemoNav improves the navigation performance, we show example episodes of

Original: CNNLSTM, VGM and our model in the Gibson scenes in Fig 5 (See Appendix for more).
Revised: CNNLSTM, VGM and our model in the Gibson scenes in Fig 5 (See Appendix for more).

Original: We can see that CNNLSTM takes numerous steps to explore and tends to fail in complex scenes.
Revised: We can see that CNNLSTM takes numerous steps to explore and tends to fail in complex scenes.

Original: VGM tends to spend a large proportion of time going in circles in narrow pathways.
Revised: VGM tends to spend a large proportion of time going in circles in narrow pathways.

Original: In contrast, the trajectories of the MemoNav are shorter and smoother.
Revised: In contrast, the trajectories of the MemoNav are shorter and smoother.

Original: For instance, the 2nd and 3rd columns in Figure 5 show that the VGM agent is trapped in a bottleneck connecting two rooms, while our agent plans efficient routes.
Revised: For instance, the 2nd and 3rd columns in Figure 5 show that the VGM agent is trapped in a bottleneck connecting two rooms, while our agent plans efficient routes.

Original: This comparison shows that the MemoNav is more capable of escaping from deadlock.
Revised: This comparison shows that the MemoNav is more capable of escaping from deadlock.

Original: We next draw a histogram of the normalized path lengths of successful episodes for VGM and our model to further investigate to what degree our model escapes from deadlock.
Revised: We next draw a histogram of the normalized path lengths of successful episodes for VGM and our model to further investigate to what degree our model escapes from deadlock.

Original: The normalized path length is calculated as l norm =
Revised: The normalized path length is calculated as l norm =

Original: l i / max( p i , l i ) − 1 where p i and l i are defined in Section 5.1.
Revised: l i / max( p i , l i ) − 1 where p i and l i are defined in Section 5.1.

Original: l norm indicates how many extra steps the agent takes compared to the shortest path.
Revised: l norm indicates how many extra steps the agent takes compared to the shortest path.

Original: A larger l norm represents a less efficient navigation episode.
Revised: A larger l norm represents a less efficient navigation episode.

Original: The histogram is shown in Figure 6 in the Appendix.
Revised: The histogram is shown in Figure 6.

Original: The figure shows that the histogram of our model exhibits a less heavier-tailed distribution, especially at the 4-goal level.
Revised: The figure shows that the histogram of our model exhibits a less heavier-tailed distribution, especially at the 4-goal level.

Original: This distribution means that our model has fewer episodes with an extremely large number of steps and agrees with the qualitative comparison.
Revised: This distribution means that our model has fewer episodes with an extremely large number of steps and agrees with the qualitative comparison.

Original:  C ONCLUSION AND F UTURE W ORK
Revised:  C ONCLUSION AND F UTURE W ORK

Original: This paper proposes MemoNav, a novel memory model for ImageNav.
Revised: This paper proposes MemoNav, a novel memory model for ImageNav.

Original: This model flexibly retains the informative fraction of the short-term navigation memory via a forgetting module.
Revised: This model flexibly retains the informative fraction of the short-term navigation memory via a forgetting module.

Original: We also introduce an extra global node as long-term memory to learn a scene-level representation.
Revised: We also introduce an extra global node as long-term memory to learn a scene-level representation.

Original: The retained short-term memory and the long-term memory are encoded by a graph attention module to generate the working memory that is used for generating action.
Revised: The retained short-term memory and the long-term memory are encoded by a graph attention module to generate the working memory that is used for generating action.

Original: The experimental results show that the MemoNav outperforms the baselines in multi-goal tasks, exhibits better transferability, and is more capable of escaping from deadlock.
Revised: The experimental results show that the MemoNav outperforms the baselines in multi-goal tasks, exhibits better transferability, and is more capable of escaping from deadlock.

Original: For future work, we will try to design trainable forgetting modules to better assess which fraction of the agent’s memory is informative.
Revised: For future work, we will try to design trainable forgetting modules to better assess which fraction of the agent’s memory is informative.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We provide our source code and model configuration files in the supplementary material.
Revised: We provide our source code and model configuration files in the supplementary material.

Original: We also add the python script used for generating our multi-goal Gibson datasets to the supplementary material.
Revised: We also add the python script used for generating our multi-goal Gibson datasets to the supplementary material.

Original: Any detail about data processing can be found in this script.
Revised: Any detail about data processing can be found in this script.

Original: For example, the rules used to generate multi-goal samples (described in Section 5.1) are coded in the script.
Revised: For example, the rules used to generate multi-goal samples (described in Section 5.1) are coded in the script.

Original: L ICENSES FOR R EFERENCED D ATASETS
Revised: L ICENSES FOR R EFERENCED D ATASETS

Original: Gibson: http://svl.stanford.edu/gibson2/assets/GDS_agreement.pdf
Revised: Gibson: http://svl.stanford.edu/gibson2/assets/GDS_agreement.pdf

Original: Matterprt3D: http://kaldir.vc.in.tum.de/matterport/MP_TOS.pdf
Revised: Matterprt3D: http://kaldir.vc.in.tum.de/matterport/MP_TOS.pdf

Original: AA PPENDIX A. L IMITATIONS
Revised: A PPENDIX A. L IMITATIONS

Original: While the MemoNav witnesses a large improvement in the navigation success rate in multi-goal navigation tasks, it still encounters limitations.
Revised: While the MemoNav witnesses a large improvement in the navigation success rate in multi-goal navigation tasks, it still encounters limitations.

Original: The proposed forgetting module is a post-processing method, as it obtains the attention scores of the decoder before deciding which nodes are to be forgotten.
Revised: The proposed forgetting module is a post-processing method, as it obtains the attention scores of the decoder before deciding which nodes are to be forgotten.

Original: Future work can explore trainable forgetting modules.
Revised: Future work can explore trainable forgetting modules.

Original: The second limitation is that our forgetting module does not reduce memory footprint, since the features of the forgotten nodes still exist in the map for localization.
Revised: The second limitation is that our forgetting module does not reduce memory footprint, since the features of the forgotten nodes still exist in the map for localization.

Original: Moreover, the forgetting threshold in our experiments is fixed.
Revised: Moreover, the forgetting threshold in our experiments is fixed.

Original: Future work can merge our idea with Expire-span (Sukhbaatar et al., 2021) to learn an adaptive forgetting threshold.
Revised: Future work can merge our idea with Expire-span (Sukhbaatar et al., 2021) to learn an adaptive forgetting threshold.

Original: A. P OTENTIAL I MPACT
Revised: A. P OTENTIAL I MPACT

Original: The notable potential of negative societal impact from this work: our model is trained on 3D scans of the Gibson scenes which only contain western styles.
Revised: The notable potential of negative societal impact from this work: our model is trained on 3D scans of the Gibson scenes which only contain western styles.

Original: This inadequacy of diverse scene styles may render our model biased and incompatible with indoor environments in unseen styles.
Revised: This inadequacy of diverse scene styles may render our model biased and incompatible with indoor environments in unseen styles.

Original: As a result, our model may be only available in a small fraction of real-life scenes.
Revised: As a result, our model may be only available in a small fraction of real-life scenes.

Original: If our model is transferred to out-of-distribution scenes, the agent may take more steps and even bump on walls frequently.
Revised: If our model is transferred to out-of-distribution scenes, the agent may take more steps and even bump on walls frequently.

Original: A. R EPRESENTATIVE M ODELS OF W ORKING M EMORY IN HUMAN BRAIN
Revised: A. R EPRESENTATIVE M ODELS OF W ORKING M EMORY IN HUMAN BRAIN

Original: Cowan et al.
Revised: Cowan et al.

Original: (Cowan, 2008) proposed a typical model describing the relationships among longterm memory (LTM), short-term memory (STM), and working memory (WM) of the human brain.
Revised: (Cowan, 2008) proposed a typical model describing the relationships among longterm memory (LTM), short-term memory (STM), and working memory (WM) of the human brain.

Original: According to their definitions, LTM is a large knowledge base and a record of prior experience; STM reflects faculties of the human mind that hold a limited amount of information in a very accessible state temporarily; WM includes part of STM and other processing mechanisms that help to utilize
Revised: According to their definitions, LTM is a large knowledge base and a record of prior experience; STM reflects faculties of the human mind that hold a limited amount of information in a very accessible state temporarily; WM includes part of STM and other processing mechanisms that help to utilize

Original: STM.
Revised: STM.

Original: Cowan et al. designed a framework depicting how WM is formed from STM and LTM (shown in Figure 7).
Revised: Cowan et al. designed a framework depicting how WM is formed from STM and LTM (shown in Figure 7).

Original: This framework demonstrates that STM is derived from a temporarily activated subset of LTM.
Revised: This framework demonstrates that STM is derived from a temporarily activated subset of LTM.

Original: This activated subset may decay as a function of time unless it is refreshed.
Revised: This activated subset may decay as a function of time unless it is refreshed.

Original: The useful fraction of STM is incorporated into WM via an attention mechanism to avoid misleading distractions.
Revised: The useful fraction of STM is incorporated into WM via an attention mechanism to avoid misleading distractions.

Original: Subsequent work by Baddeley et al.
Revised: Subsequent work by Baddeley et al.

Original: (Baddeley, 2012) suggests that the central executive manipulates memory by incorporating not only part of STM but also part of LTM to assist in making a decision.
Revised: (Baddeley, 2012) suggests that the central executive manipulates memory by incorporating not only part of STM but also part of LTM to assist in making a decision.

Original: We draw inspiration from the work by Cowan et al.
Revised: We draw inspiration from the work by Cowan et al.

Original: (Cowan, 2008) and Baddeley et al.
Revised: (Cowan, 2008) and Baddeley et al.

Original: (Baddeley, 2012) and reformulate the agent’s navigation experience as the three types of memory defined above.
Revised: (Baddeley, 2012) and reformulate the agent’s navigation experience as the three types of memory defined above.

Original: The parallel between the MemoNav and the two relevant models above is shown in the following list:
Revised: The parallel between the MemoNav and the two relevant models above is shown in the following list:

Original: • The map node features are termed “STM”, since they are local and transient.
Revised: • The map node features are termed “STM”, since they are local and transient.

Original: • The topological map of the MemoNav maintains a 100-node queue to store map nodes.
Revised: • The topological map of the MemoNav maintains a 100-node queue to store map nodes.

Original: This design simulates STM that holds a limited amount of information in a very accessible state temporarily in the human brain.
Revised: This design simulates STM that holds a limited amount of information in a very accessible state temporarily in the human brain.

Original: • The MemoNav introduces a global node aggregating prior observation features stored in the topological map, thereby simulating LTM which acts as a large knowledge base.
Revised: • The MemoNav introduces a global node aggregating prior observation features stored in the topological map, thereby simulating LTM which acts as a large knowledge base.

Original: • The MemoNav utilizes a forgetting mechanism to remove a fraction of STM with attention scores lower than a threshold.
Revised: • The MemoNav utilizes a forgetting mechanism to remove a fraction of STM with attention scores lower than a threshold.

Original: This mechanism acts as a simple way of decaying STM.
Revised: This mechanism acts as a simple way of decaying STM.

Original: • The forgetting mechanism helps WM include part of STM.
Revised: • The forgetting mechanism helps WM include part of STM.

Original: • The MemoNav incorporates the retained STM and the LTM into WM, which is subsequently used to generate navigation actions.
Revised: • The MemoNav incorporates the retained STM and the LTM into WM, which is subsequently used to generate navigation actions.

Original: This design simulates the working memory model by (Baddeley, 2012).
Revised: This design simulates the working memory model by (Baddeley, 2012).

Original: A. I MPLEMENTATION D ETAILS A.4. I MPLEMENTATION OF M EMO N AV
Revised: A. I MPLEMENTATION D ETAILS A.4. I MPLEMENTATION OF M EMO N AV

Original: The structure of the memory decoding module (Figure 2(f) in the main paper) in the MemoNav remains the same as in the VGM (Kwon et al., 2021) and is shown in Figure 8.
Revised: The structure of the memory decoding module (Figure 2(f) in the main paper) in the MemoNav remains the same as in the VGM (Kwon et al., 2021) and is shown in Figure 8.

Original: We maintain the module hyper-parameters specified in the supplementary of the VGM paper.
Revised: We maintain the module hyper-parameters specified in the supplementary of the VGM paper.

Original: The forgetting module on the MemoNav requires the attention scores generated in the decoder D target .
Revised: The forgetting module on the MemoNav requires the attention scores generated in the decoder D target .

Original: Therefore, our model needs to calculate the whole navigation pipeline before deciding which fraction of the STM should be retained.
Revised: Therefore, our model needs to calculate the whole navigation pipeline before deciding which fraction of the STM should be retained.

Original: This lag means that the retained STM is incorporated into the WM at the next time step.
Revised: This lag means that the retained STM is incorporated into the WM at the next time step.

Original: The pseudo-code of the MemoNav is shown in Algorithm
Revised: The pseudo-code of the MemoNav is shown in Algorithm

Original: Algorithm 1: The implementation of the MemoNav Data: Empty topological map G = {V , E} , target image I target , current time step t , forgetting percentage p , trainable observation encoder F enc , GATv2-based encoder GATv2 , Transformer decoders Dtarget and Dcur , LSTM-based policy network LSTM
Revised: D and D

Original: Result: Navigation action a t 1 Long-term memory n global ← 0 ∈ R d ;  for graph nodes V  ; 3 while not AgentCallStop  do // Step 1:
Revised: Result: Navigation action a t 1 Long-term memory n global ← 0 ∈ R d ; 2 Attention scores for graph nodes V : α ← 0 ∈ R | V | ; 3 while not AgentCallStop () do // Step 1:

Original: Update the topological map
Revised: Update the topological map

Original: I t ← GetCurrentPanorama
Revised: I t ← GetCurrentPanorama() ;

Original: UpdateMap ; // Step 2:
Revised: G. UpdateMap( I t ) ; // Step 2:

Original: Retain the informative fraction of the STM
Revised: Retain the informative fraction of the STM

Original: Forgotten number n ← 
Revised: Forgotten number n ← Floor ( p · |V| ) ;

Original: Sorted indices i Argsort
Revised: Sorted indices i ← Argsort( α ) ;

Original: Forgotten indices i  ← 
Revised: Forgotten indices i forgotten ← i [0: n ] ;

Original: RemoveNodes ; // Step 3:
Revised: G. RemoveNodes( i forgotten ) ; // Step 3:

Original: Memory encoding and decoding
Revised: Memory encoding and decoding

Original:  ← F enc ( I target ) ; f cur ← D cur ( e cur , M w ) , f target ← D target ( e target , M w ) ; α ← D target .
Revised: V ∈ R |V|× d ← G. GetNodeFeatures () ; Working memory M w ← GATv2( { V , n global } ) ; e cur ← F enc ( I t ) , e target ← F enc ( I target ) ; f cur ← D cur ( e cur , M w ) , f target ← D target ( e target , M w ) ; α ← D target .

Original: GetAttScores() // Step 4: Action generation
Revised: GetAttScores() // Step 4: Action generation

Original: x ← LSTM(FC([ f cur , f target , e cur ]))
Revised: x ← LSTM(FC([ f cur , f target , e cur ]))

Original: ; p ( a t | x ) = σ (FC( x )) ; a t ← SampleFromDistribution( p ( at | x )) ; 18 end
Revised: ; p ( a t | x ) = σ (FC( x )) ; a t ← SampleFromDistribution( p ( a 18 end

Original: A.4. R EPRODUCTION OF CNNLSTM
Revised: A.4. R EPRODUCTION OF CNNLSTM

Original: We reproduce CNNLSTM (Zhu et al., 2017) following the description in its original paper, but we also make some modifications to keep the comparison fair.
Revised: We reproduce CNNLSTM (Zhu et al., 2017) following the description in its original paper, but we also make some modifications to keep the comparison fair.

Original: We replace the ResNet-50 in CNNLSTM with the pretrained RGB-D encoder of VGM (Kwon et al., 2021).
Revised: We replace the ResNet-50 in CNNLSTM with the pretrained RGB-D encoder of VGM (Kwon et al., 2021).

Original: We also add positional embeddings to the encoded RGB-D observations to contain temporal information.
Revised: We also add positional embeddings to the encoded RGB-D observations to contain temporal information.

Original: Moreover, we concatenate the encoded RGB-D observations with the goal image embedding and project the concatenated feature (1024D) to a 512D feature, so that CNNLSTM can utilize the information of the goal image.
Revised: Moreover, we concatenate the encoded RGB-D observations with the goal image embedding and project the concatenated feature (1024D) to a 512D feature, so that CNNLSTM can utilize the information of the goal image.

Original: The projected features of four consecutive frames are further condensed and then input to a policy network as in (Zhu et al., 2017).
Revised: The projected features of four consecutive frames are further condensed and then input to a policy network as in (Zhu et al., 2017).

Original: To use the two auxiliary tasks proposed in VGM (Kwon et al., 2021), we also introduce special linear projection layers.
Revised: To use the two auxiliary tasks proposed in VGM (Kwon et al., 2021), we also introduce the linear projection layers (Linear-ReLU-Linear) used in VGM to process the embedded goal image and embedded current observation.

Original: With these designs, the comparison between CNNLSTM and MemoNav (ours) is fair.
Revised: With these modifications, the comparison between CNNLSTM and MemoNav (ours) is fair.

Original: A.4. T RAINING D ETAILS
Revised: A.4. T RAINING D ETAILS

Original: We follow the two-step training routine and maintain the training hyper-parameters in the VGM paper (Kwon et al., 2021).
Revised: We follow the two-step training routine and maintain the training hyper-parameters in the VGM paper (Kwon et al., 2021).

Original: Firstly, CNNLSTM, VGM, and MemoNav (ours) are all trained using imitation learning for 20k steps.
Revised: Firstly, CNNLSTM, VGM, and MemoNav (ours) are all trained using imitation learning for 20k steps.

Original: Afterward, we finetune these models using PPO (Schulman et al., 2017) for 10M steps.
Revised: Afterward, we finetune these models using PPO (Schulman et al., 2017) for 10M steps.

Original: Due to the performance fluctuation intrinsic to reinforcement learning, the model at the 10M-th step is probably not the best.
Revised: Due to the performance fluctuation intrinsic to reinforcement learning, the model at the 10M-th step is probably not the best.

Original: Therefore, we evaluate all checkpoints in the step range [9M, 10.4M] and select the best one.
Revised: Therefore, we evaluate all checkpoints in the step range [9M, 10.4M] and select the best one.

Original: A.4. E XTRA INFORMATION ABOUT T HE M ULTI - GOAL D ATASETS
Revised: A.4. E XTRA INFORMATION ABOUT T HE M ULTI - GOAL D ATASETS

Original: We follow the format of the public 1-goal dataset in (Mezghani et al., 2021) and create 2-goal, 3goal, and 4-goal test datasets on the Gibson (Xia et al., 2018) scenes.
Revised: We follow the format of the public 1-goal dataset in (Mezghani et al., 2021) and create 2-goal, 3goal, and 4-goal test datasets on the Gibson (Xia et al., 2018) scenes.

Original: We generate 50 samples for each of the 14 test scenes.
Revised: We generate 50 samples for each of the 14 test scenes.

Original: In each sample, we randomly choose target positions while still following five rules: (1) no obstacles appear inside a circle with a radius of 0.75 meters centered at each target; (2) the distance between two successive targets is no more than 10 meters; (3) all targets are placed on the same layer without altitude differences.
Revised: In each sample, we randomly choose target positions while still following five rules: (1) no obstacles appear inside a circle with a radius of 0.75 meters centered at each target; (2) the distance between two successive targets is no more than 10 meters; (3) all targets are placed on the same layer without altitude differences.

Original: (4) all targets are reachable from each other.
Revised: (4) all targets are reachable from each other.

Original: The final target is placed near a certain previous one with the distance between them smaller than 1.5 meters.
Revised: The final target is placed near a certain previous one with the distance between them smaller than 1.5 meters.

Original: The distributions of the total geodesic distances for the three difficulty levels are shown in Figure 9.
Revised: The distributions of the total geodesic distances for the three difficulty levels are shown in Figure 9.

Original: A.4. C OMPUTE R EQUIREMENTS
Revised: A.4. C OMPUTE R EQUIREMENTS

Original: We utilize an RTX TITAN GPU for training and evaluating our models.
Revised: We utilize an RTX TITAN GPU for training and evaluating our models.

Original: The imitation learning phase takes 1.5 days to train while the reinforcement learning takes 5 days.
Revised: The imitation learning phase takes 1.5 days to train while the reinforcement learning takes 5 days.

Original: The computation in the GATv2-based encoder and the two Transformer decoders occupy the largest proportion of the run-time of the MemoNav.
Revised: The computation in the GATv2-based encoder and the two Transformer decoders occupy the largest proportion of the run-time of the MemoNav.

Original: The computation complexity of the encoder and the decoders are O ( |V| d 2 + |E| d ) and O ( |V| d ) , respectively.
Revised: The computation complexity of the encoder and the decoders are O ( |V| d 2 + |E| d ) and O ( |V| d ) , respectively.

Original: Using the forgetting module with a percentage threshold p , the computation complexity of the MemoNav can be flexibly decreased by reducing the number of nodes to p |V| . 
Revised: Using the forgetting module with a percentage threshold p , the computation complexity of the MemoNav can be flexibly decreased by reducing the number of nodes to p |V| . [Table] [Table]

Original: A. E XTRA A BLATION S TUDIES
Revised: A. E XTRA A BLATION S TUDIES

Original: We conduct additional ablation experiments on the forgetting module in the MemoNav to further investigate how its design affects the agent’s navigation performance.
Revised: We conduct additional ablation experiments on the forgetting module in the MemoNav to further investigate how its design affects the agent’s navigation performance.

Original: The results are shown in
Revised: The results are shown in

Original: Table 4.
Revised: Table 4.

Original: Origin of attention scores used by forgetting module .
Revised: Origin of attention scores used by forgetting module .

Original: The forgetting module in the MemoNav removes the uninformative STM according to the attention scores generated in D target , as described in Section 4.2.1.
Revised: The forgetting module in the MemoNav removes the uninformative STM according to the attention scores generated in D target , as described in Section 4.2.1.

Original: We change these scores to those generated in D cur .
Revised: We change these scores to those generated in D cur .

Original: The result (row 2) shows that using the attention scores generated in D cur leads to slightly worse performance.
Revised: The result (row 2) shows that using the attention scores generated in D cur leads to slightly worse performance.

Original: Effectiveness of forgetting module .
Revised: Effectiveness of forgetting module .

Original: This ablation investigates whether it is effective to exclude the STM with attention scores ranking below the predefined percentage p . 
Revised: This ablation investigates whether it is effective to exclude the STM with attention scores ranking below the predefined percentage p . ”

Revised: Effectiveness” here means that the forgetting module does retain useful node features.

Original: In this experiment,the forgetting module excludes a random fraction of the STM.
Revised: In this experiment,the forgetting module excludes a random fraction of the STM.

Original: We test this ablation model over five random seeds and report the average metrics.
Revised: We test this ablation model over five random seeds and report the average metrics.

Original: The result (row 3) shows that incorporating a random fraction of the STM into the WM leads to decreases in all metrics, which validates the effectiveness of our design for retaining informative STM.
Revised: The result (row 3) shows that incorporating a random fraction of the STM into the WM leads to decreases in all metrics, which validates the effectiveness of our design for retaining informative STM.

Revised: To better understand how the forgetting module works, we conduct an additional statistical experiment.

Original: A. E XTRA V ISUALIZATION R ESULTS
Revised: In this experiment, three distance metrics are calculated: (a) distance from a forgotten/retained node to the agent, (b) distance from a forgotten/retained node to the goal, and (c) distance from a forgotten/retained node to the oracle shortest path. Then the histograms of these three metrics are drawn according to the metrics records for each node at each time step so that we can see how far away the forgotten/retained nodes are from the agent, the goal, and the shortest path. We evaluate the MemoNav on the 3-goal Gibson task and draw the histograms with the average results of five different seeds, as shown in Figure 10. The figure provides two interesting findings: (1) A large proportion of the retained nodes are close to the agent, the goal, and the shortest path.

Revised: Specifically, 73 . 4 ± 0 .

Revised: 5 ± 0 .

Revised: 7% , and 90 .

Revised: 6 ± 1 .

Revised: 2% of the retained nodes are within 4 meters of the agent, the goal, and the shortest path, respectively.

Revised: In contrast, the values for the forgotten nodes are 44 .

Revised: 8 ± 1 .

Revised: 3% , 24 . 8 ± 1 .

Revised: 3% , and 67 .

Revised: 7 ± 1 .

Revised: The mean values of these three distance metrics for the retained nodes are 2 .

Revised: 99 ± 0 .

Revised: 05 m, 4 .

Revised: 72 ± 0 .

Revised: 05 m, and 1 .

Revised: 82 ± 0 .

Revised: 09 m, respectively.

Revised: In contrast, the values for the forgotten nodes are 4 .

Revised: 69 ± 0 .

Revised: 07 m, 6 .

Revised: 19 ± 0 .

Revised: 14 m, 3 .

Revised: 42 ± 0 .

Revised: 14 m.

Revised: These results empirically validate that the forgetting module retains most nodes around the agent, so that the context around the agent’s current position is still mostly utilized for decision-making.

Revised: Besides, most of the nodes near the shortest path are retained, which means that the information useful for path planning are not totally lost by the forgetting module.

Revised: Extra Validation of LTM .

Revised: In order to observe how the learned scene-level representation in the LTM benefits the MemoNav, we replace the LTM feature with the average of all STM features.

Revised: After replacing, the LTM no longer contains a scene-level representation, but it still facilitates message passing.

Revised: The results in Table 5 shows that the MemoNav without the scene-level representation in the LTM witnesses drops in all metrics.

Revised: This experiment suggests that the learned scene-level representation in the LTM contains important information that helps to improve navigation performance.

Revised: A. E XTRA V ISUALIZATION R ESULTS

Original: More examples of multi-goal episodes are displayed in Figure 10(a).
Revised: More examples of multi-goal episodes are displayed in Figure 11(a).

Original: The agent efficiently explores the scenes and finds sequential goals using the informative fraction of the node features in the map.
Revised: The agent efficiently explores the scenes and finds sequential goals using the informative fraction of the node features in the map.

Original: [Table]

Original: These examples show that the MemoNav agent focuses high attention only on a small fraction of nodes and excludes nodes that are far away from the current goal.
Revised: These examples show that the MemoNav agent focuses high attention only on a small fraction of nodes and excludes nodes that are far away from the current goal.

Original: For example, in the 3-goal example, the agent forgets the topmost node when navigating to the 1st goal since this node is the farthest from the goal; the agent forgets the nodes at the bottom left corner when navigating to the 2nd and 3rd goals since these nodes are remote and uninformative.
Revised: For example, in the 3-goal example, the agent forgets the topmost node when navigating to the 1st goal since this node is the farthest from the goal; the agent forgets the nodes at the bottom left corner when navigating to the 2nd and 3rd goals since these nodes are remote and uninformative.

Original: The comparison with the baseline for these examples is recorded in the supplementary videos.
Revised: The comparison with the baseline for these examples is recorded in the supplementary videos.

Original:  We present examples of failed episodes in Figure 10(b) and record the proportions of various failure modes at all difficulty levels.
Revised: Failure case analysis . We present examples of failed episodes in Figure 11(b) and record the proportions of various failure modes at all difficulty levels.

Original: The failure modes can mainly be categorized into four types: Stopping mistakenly , Missing the goal , Not close enough , and Over-exploring .
Revised: The failure modes can mainly be categorized into four types: Stopping mistakenly , Missing the goal , Not close enough , and Overexploring .

Original: The mode Stopping mistakenly means that the agent implements stop at the wrong place.
Revised: The mode Stopping mistakenly means that the agent implements stop at the wrong place.

Original: The mode Missing the goal means that the agent has observed the goal but passes it.
Revised: The mode Missing the goal means that the agent has observed the goal but passes it.

Original: The mode Not close enough means that the agent attempts to reach the goal it sees but implements stop outside the successful range.
Revised: The mode Not close enough means that the agent attempts to reach the goal it sees but implements stop outside the successful range.

Original: The mode Over-exploring means that the agent spends too much time exploring open areas without any goals.
Revised: The mode Over-exploring means that the agent spends too much time exploring open areas without any goals.

Original: The largest probability lies in Over-exploring cases, most of which occur when the agent explores a large proportion of the scene but still fails to get close to the target area in a limited time.
Revised: The largest probability lies in Over-exploring cases, most of which occur when the agent explores a large proportion of the scene but still fails to get close to the target area in a limited time.

Original: To investigate how the feature in the LTM changes during navigation, we calculate the L2 distance between the features at every two successive time steps and show the curves for two examples in
Revised: Visualization comparison between MemoNav with and without forgetting. We compare the trajectories of the MemoNav with and without the forgetting module to observe how this module affects the characteristics (e.g. smoothness and length) of the trajectories.

Original: Figure 11. The two curves show similar trends: the L2 difference rapidly increases and then gradually converges to 0 while several peaks appear.
Revised: Figure 12 shows that the trajectories of MemoNav using the forgetting module are smoother and shorter. Without the forgetting module, the trajectories tends to possess many tiny sharp turns and are longer.

Original: To understand why the LTM variation shows such a trend, we visualize the agent’s observations at the time steps of the peaks in Figure 11.
Revised: We hypothesize that a fraction of the STM is uninformative for navigation.

Original: Comparing these observations, we can see that the trend of the LTM variation exhibits two traits: (1) the LTM feature remains stable (the L2 difference is close to 0) if the agent is traveling in visited areas. For instance, in the 2-goal example (top row), the L2 difference steadily decreases in t = 31 ∼ 55 during which the agent turns around and travels to visited areas; (2) the LTM feature significantly changes (a peak appears) if the agent captures novel views of the scene. For instance, in the 3-goal example (bottom row), the L2 difference curve exhibits peaks at t = 68 when the agent passes a corner and at t = 88 when the agent observes a novel open area.
Revised: This fraction may acts as noise and make the policy network frequently change its output actions.

Original: These results indicate that the LTM focuses higher attention on the agent’s novel observations when aggregating the STM and stores the agent’s exploration experience.
Revised: The forgetting module adaptively excludes this disturbing fraction of STM so that the policy network is able to use helpful navigation memory for decision-making.

Original: [Figure]
Revised: [Figure]