Topological Semantic Graph Memory for Image-Goal Navigation
Authors: Nuri Kim
Obin Kwon
Hwiyeon Yoo
Yunho Choi
Jeongho Park
Songhwai Oh
Venue: CoRL 2022 Oral (2022)
Keywords: Landmark-Based Navigation
Incremental Topological Memory
Visual Navigation

Abstract: A novel framework is proposed to incrementally collect landmark-based graph memory and use the collected memory for image goal navigation. Given a target image to search, an embodied robot utilizes semantic memory to find the target in an unknown environment.
In this paper, we present a topological semantic graph memory (TSGM), which consists of (1) a graph builder that takes the observed RGB-D image to construct a topological semantic graph, (2) a cross graph mixer module that takes the collected nodes to get contextual information, and (3) a memory decoder that takes the contextual memory as an input to find an action to the target. On the task of an image goal navigation, TSGM significantly outperforms competitive baselines by +5.0-9.0% on the success rate and +7.0-23.5% on SPL, which means that the TSGM finds efficient paths. Additionally, we demonstrate our method on a mobile robot in real-world image goal scenarios.

Topological Semantic Graph Memory for Image-Goal Navigation
Nuri Kim, Obin Kwon, Hwiyeon Yoo, Yunho Choi, Jeongho Park, and Songhwai Oh Department of Electrical and Computer Engineering, ASRI, Seoul National University (firstname secondname @rllab. anu.ac.kr, songhwai@snu. ac.kr kr
Abstract: A novel framework is proposed to incrementally collect landmark- based graph memory and use the collected memory for image goal navigation. Given target image to search, to find the target in an unknown environment. In this paper, we present a topo- logical semantic graph memory (TSGM), which consists of (1) graph builder that takes the observed RGB-D image to construct a topological semantic graph, (2) a cross graph mixer module that takes the collected nodes to get contex- tual information, and (3) a memory decoder that takes the contextual memory input find action the target. On the task of image goal navi- gation, TSGM significantly outperforms competitive baselines by +5.0-9.0% on the success rate and +7.0-23.5% on SPL., which means that the TSGM finds efficient paths. Additionally, we demonstrate our method on a mobile robot in real-world image goal scenarios. Code available at https://gihub..co//llabb snu/ ThuTTTppoogccaSSemnntccCaaaaaaep
embodied robot utilizes semantic memory
Keywords: Landmark-Basee Navigation, Incremental Topological Memory
Introduction
Navigation with rich visual observations has been a critical issue in a variety of embodied agent tasks, such as exploration, image goal navigation, and object goal navigation [1-16]. A crucial in- gredient for successful visual navigation is to construct memory, which can represent the structure of the environment along with compact visual features for representing high-dimensional visual in- puts. A metric-map memory [5, 17] created with SLAM, and a graph memory [8. 9, 18-20] with nodes and edges are the two standard memory construction approaches for navigation algorithms. Even though navigation systems that use metric maps produce powerful results with exact localiza- tion and mapping, it is not practical because the navigation agent is susceptible to sensory noises. The topological map, which represents geometric properties and spatial relations of places in the form of graph, is proposed to construct map without accurate mapping. Previous visual naviga- tion methods [9, 18] with topological map exploit image features as nodes and edges connecting the nodes in proximity. Since node indicates location, the robot's position be estimated by the nodes in the topological map. Therefore, the graph can be used to navigate successfully even in a real noisy environment.
Unfortunately, using semantic information in topological graph memory presents several difficul- ties. The first challenge is to incorporate landmarks into topological graph. As demonstrated by studies [21-23] that show animals navigate utilizing contextual cues from landmarks, the problem of incorporating landmarks, such object compositions, is critical issue in visual navigation. A recent research [15] addresses the problem by making an object graph, which connects objects in field of view of directional camera to leverage object features. Although it guides agent to effective action to discover the target utilizing object relationships, it often misses 3D object con- text information since it only connects objects observed from the same viewpoint. Imagine goal is given to find a cooking pot, and you know that the cooking pot is usually kept adjacent to an oven. If the oven is nearby but not visible, the agent may miss this crucial information and pick an inefficient path. The more difficult problem is inferring contextual information from objects using geometrically arranged landmarks. There are various advantages to obtaining contextual features
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

(a)
Kitchen
Figure I: Importance of semantic contextual information. (a) When two similar items, e.g., cups, are in different locations, they can be identified individually. (b) A kitchen can be recognized when refrigerator, oven, and dining table are detected.()
from topologiaal graphs. By defining object through neighboring objects, the contextual repre- sentation helps to eliminate the ambiguity of similar but different objects. As shown in Figure 1(a), by describing cups terms of their neighboring objects, it is possible to distinguish two similar cups. For example, a cup in the kitchen can be perceived as one next to a chair and snack box, while cup in the bathroom can be shown as one that is near a toothbrush and washstand. Moreover, place can be better described through objects. A kitchen, for example, can be defined by the presence of a refrigerator, oven, and dining table (see Figure 1(b)).
To this end, this paper addresses the challenges described above using topological semantic graph memory (TSGM), which has two types of nodes: image nodes and object nodes. Image nodes rep- resent different positions that include image representations, while each object node represents a unique object with its visual representation regardless of viewpoint. If previously undiscovered image node is identified, it is connected to the previously visited image node, producing a spa- tially meaningful graph. For each image input, objects are detected using MaskRCNN [24] and then connected to image nodes where the object can be seen. A cross graph mixer, a learnable message- passing network that exchanges object and image information, is then used to make the memory contextual. Using this contextual memory, the agent determines the best strategy for finding the goal. We applied our proposed method to image goal navigation in the Gibson [25] to validate our method. As a result, the proposed method significantly outperforms competitive baselines by +5.0- 9.0% on the success rate and +7.0 +7.0-23.5% on SPL,, which means that the TSGM finds efficient paths. On a mobile jackal robot, we demonstrate our method in real-world environment. The experiment carried out using episodes where the goal location is not visible and is 5 10m away. Despite that the episodes are challenging for image goal navigation, TSGM demonstrates successful navigation.
2 Related Work
There has been lot research using memory for visual navigation. There are three types of mem- ory formation methods: implicit memory, metric memory, and topological memory.
Implicit memory. Using a recurrent neural network (RNN) as policy network is a simple method to make an implicit memory [26]. TargetDriven [26] has vanilla RI policy with CNN backbone followed by an LSTM which implicit memory. Since RNN has difficulty backpropagating for long sequence, RNN is replaced with an explicit memory structure [2, 5, 8, 9, 17, 20).
Metric memory. Active Neural SLAM [17] has hierarchical structure explore an environment: global and local policy. The global policy constructs a top-down 2D map and estimates a global goal. consists of neural network for flexible output on the input modalities. Given the global goal from the global policy module, a local policy module plans a path to the goal using the simple local navigation algorithm.Eppnnav [5] tackles the exploration problem for navigation. builds a global metric map, combining egocentric metric maps. Then, to estimate the action, it embeds images with CNN, and a recurrent policy takes the embedding of current observation and the target to output an action.
Topological memory. [2], LSTM is policy network that finds the currently located node and derives an action when a topological map is provided. The goal of NeuralPlanner [20] is primarily to

---

Graph Update
V
Vota
Exar
Ed}
Current Obs
27a
Object
Target Si
Image
Query
Cross Graph Mixer
Recurrent
mi
min
Ciey Cit
Image Memory
Action
Policy
Action
m10
m10
Cor. Co2
Object Memory
Current Obs
Target iig
Memory Attention Module
Figure 2: Overview of the proposed method. TSGM has a spatially meaningful structure that is generated and updated online. An image node Vim represented by blue circle, and object node Vos is represented by a pink triangle. The current observation Ie is shown in green, while the goal xy highlighted in red. cross graph mixer module used to update the constructed graph memory to encode the context of features. Then, the attention module chooses the updated memory by querying the current observation and target node. The action policy network determines the action a, based on the selected memory. Note that the node position is only used for visualization purposes.
explore the environment and maximize coverage. To this end, a topological map is generated from exploratory policy during the rollout. Then, pretrained neural planner calculates the path to the node most similar to the target image. Semi-parametric topological memory (SPTM) [9] forms topological memory through exploration before training agent. Then, agent navigates to the destination based on the topological memory using the Dijkstra algorithm that plans path to reach the waypoint. Scene memory transformer (SMT) [8] stacks all the visual features of the past observations as navigation memory. uses transformer network [6] to process this memory in the context of the current and target observations. other words, the authors completely replaced the RNN with an attention mechanism, which shows good performance in long-term tasks. Visual graph memory (VGM) [18] constructs topological visual memory navigate an environment while utilizing the landmark information of the scene. No RL no simulator (NRNS) [27] uses models trained using image imput without interaction with the simulator, It creates a node for an unexplored area and finds the goal by estimating the geodesic distance to the goal.
Our method builds upon prior topological memory methods handle the problem of creating and exploiting the object-based semantic graph to find the most efficient path. Once placed in a new environment, method explores while incrementally constructing topological graph, inspired by prior work [8, 18, 27J. In contrast to these previous methods, our method employs spatially meaningful landmarks without using exact positions.
3
TSGM: Topological Semantic Graph Memory
3.1 Problem Statement
We consider the problem of goal-directed exploration for efficient navigation. Given current ob- servation and a target image to search, an embodied robot aims to find the target (see Figure 2). The current observation contains RGB-D sensory input T, and detected objects {21, =K} E Z,.
3.2 Graph Memory Construction
Let us consider an agent equipped with an RGB-D camera that was dropped into a novel environ- ment that it has never been in before. We want to build and exploit graph memory that enables

---

this agent to find the goal in new environment efficiently. To this end. semantically structured graph memory, named topological semantic graph memory (TSGM), is built online while an agent traverses the new environment. For example, as shown in Figure 2, when a new node (green circle) is found, the new node is connected to the previous image node (the blue circle connected to the green circle), and the object nodes found at the new node (the green triangles). Since the graph is built with landmarks, recalling the landmarks the agent has seen before will help the agent to discover ef- ficient paths. The TSGM contains two types of nodes and edges, i... G (Vim Vol. Eim, Ee} with image nodes Vim = { J.V and its edges Eim with the affinity matrix Aim E &R-X object nodes VaL = {=x);==1 where 2; E IR1xD, and edges connecting image and object nodes &, with the affinity matrix A R RNXM NN is the number of image nodes, M is the number of object nodes, and D is the dimension of the object representation. The graph expands continuously as the agent explores the environment. The algorithm for constructing TSGM is provided in Supplementary Section 2.
Object graph construction. Objects are retrieved using MaskRCNN [24] and added to the graph memory as nodes. When the object is already in memory, the object node is updated with node with a higher object detection score. If it is not determined that the object is the same, a new object node (=x) is added the memory, connecting with the corresponding current image node (24)) using the affinity matrix (A²)- To connect objects in proximity in the graph memory, we calculate the affinity matrix for object nodes be connected to objects in the neighbors of the current image node using the image affinity and image-object affinity. Here, since we want a single object node for an object, the trained object encoder using Supervised contrastive learning [28] learns whether the objects are the same object even when the images are shot from various points of view.
Image graph construction. To construct an image graph, a pretrained similarity encoder [29] learns image similarity in an unsupervised manner and is then used to determine whether or not the node is already in memory. If observed node is new, i.e., the node feature is not similar to the existing memory, it is added to the graph memory; otherwise, it is utilized to update the corresponding memory node. While connecting image nodes, a new technique based on objects is utilized, assuming that the likelihood of being at the same place is significantly low if there are few co-visible objects between two image nodes. Therefore, the similarity between the object nodes collected at the current location and the object graph nodes is calculated. We consider two image nodes to represent different locations when there are no similar items between them. If the similarity is low, the observed image node is added to memory; otherwise, the image node is updated to the most recent image representation. When it turns out that the image node is new, it is connected to the previous image node (111), making the image affinity matrix (Aim) between 2) and z; connected.
3.3 Cross Graph Mixer
The cross graph mixer is developed from MPNNs [30] to encode scene contexts by combining in- formation from the image and object graphs The upper row of Figure depicts the image graph, while the lower row represents the object graph. The message passing phase runs for L steps and is defined of message functions M and vertex update functions U', where I E {1,..., I.). The mes- sage function M is composed of two functions, the self-update function (St) and the uross- function (C'), To begin, image and object nodes self-update to obtain contextual representations of nearby locations or objects, respectively:
m = Σ
s,(1il Aim.g), mo. = Σ kEN,(v)
(ho Aat, 9),
(1)
where No(r) and (o) denote the image/object neighbors of the vth node. The node features concatenated with the goal feature g, and it is embedded into a feature using two-layered neural
networks. Then, the self-update message function S makes mi, and mo, by aggregating connected nodes with the message passing method, which is illustrated in the left column of Figure 3. After self-update, each graph exchanges information with the others to make a complete the messages.
mity
Ci more Ad), mo! (E)
Σ
Σ wEA.(0)
Cy(moo, mey, Ac),
where C2 and Co are cross updating functions that produce mit, and mo which are cross-updated messages. The cross-update module's central concept is that images are updated through connected

---

Image graph
f
Ta
mo
Axl
rnd
Exchange nodes
Object graph
Jul
mid
f hch.
f
ma
hndw
Image node
Object node
Figure 3: Cross graph mixer. The upper row shows the image graph update while the lower row shows the object graph update.
objects, and objects are updated using connected images. To cross-update the graphs, the self- updated image/object features aggregate the connected object/image node features. Then, the update function U transfers messages from object nodes to image nodes and vice versa,
ho U. (hii,, mi),), hott1 = (ho',,
(3)
where U a two-layered neural network that maps information of an object/image to the connected objects/images. The image-object update iterates for 7, steps to create semantic contextual node features. After T. steps of the image-object update iterations, the cross-mixed memory for image graph mi and object graph mo generated for each node. The details of the module structure described in the supplementary material.
3.4 Memory Attention Module
TSGM is composed of visited image nodes and object nodes that have been observed. For this, an at- tention network is used discover the node closest to the goal among the memory. Since this module utilizes attention, interpolation between visited nodes enables goal features to be extracted from un- explored nodes. Additionally, find the relative path, the current node better be searched. The goal feature Ty is given as query q when extracting memory conditoooddd goal feature, Cig. Cog, and the current observation feature Fe given as an input when selecting the memory-come conditioned current feature, Ci., Co,, Using the decoder module of the transformer network [6], current contex- tual feature Ci,, Co, and goal context feature Ci,, Co are collected,
(Wag) [mi,, ".. min)")) (We9), Co of (Wag) [Wo))mom....... Vd
(mo mo, }T) (W..),
Ci
Vd
(4)
where W.. Wqp and Wy matrix parameters and the set (mi ..., min} is the image memory, the set [mo m0 is the object memory, and a is softmax function.
3.5 Action Policy Network
Given the contextual feature for goal and current nodes, the action policy network finds action reach the goal. For fth time step, we encode the sequential information, h; G( h h r-1 .4-1), where G is recurrent neural network and h is /th hidden state for action, and 0:-1 is the pre- vious action an agent. Then, the action a, is sampled from the Categorical distribution, i.e. at Categorical (Wh., b), where W and b are matrix parameters.
5

---

3.6 Optimization
Learning action. Proximal policy optimization (PPO) [31] is used to learn a policy for picture goal navigation. The policy is trained to maximize the expected return, which is defined as the total reward E, [R( a+)] over time trajectories T = (st, at) of the policy's 's time horizon H. When an agent takes action o at state 8t, Lypo -E, Vologro(aals)) A. where and Å₃ is an estimator of the advantage function at time step t, and pitheta is a stochastic policy. We formulated it as negative log likelihood to the oracle action, Lact = E,WDD Σl= aj (zx)] where (x,, at) for t E {1,2.... .... }.}. and aj is the oracle action at time step t. Since PPO can better estimate return after creating successful episodes, the policy is pretrained with imitation learning and then finetuned with reinforcement learning.
Auxiliary metrics. Knowing when to stop is a significant issue since an episode is considered successful if an agent calls the stop action upon seeing the goal. Fortunately, is well known that auxiliary metrics, such as a progress monitor and goal sensor, can assist an agent in determining when stop [32]. The progress monitor assesses progress based on the current observation and a target to determine when to press the stop button, which is obtained by calculating the geodesic distance between an agent and the goal. The goal sensor indicates how close the current position is to the goal. If the target is within the success criteria for image goal navigation, the goal sensor produces one. The progress monitor takes the current observation and a target as input and estimates the progress, pr. The target sensor takes the same input of the progress monitor and estimates that the current observation containing a target, Sr. Then, two estimations, Pr and 8t, are optimized with
L2 loss, Laux EEDD Σt= || šell2 + ||pt - Pell2] where st and pł are the ground truths.
Since the auxiliary loss is simply added to the action loss, the final loss function for training an agent with imitation learning Lbe Leat + ALauss where À is a balancing parameter. For reinforcement learning, the combined loss L = ppo + XLaur, is used.
Experimental Evaluation
4.1 Baselines
We compare our method with baselines with various types of memory.RGBD + RL [26] has vanilla RL policy with CNN backbone followed by an LSTM adapted from [26]. Active Neural SLAM [17] has metric memory for exploration. To adapt the algorithm to the image goal naviga- tion, we set the output of the global policy to be the relative position to the target when the target is detected using a pretrained target pose estimator. Exp4nav [5] tackles the exploration problem. To finetune the image goal navigation task model, we change the coverage reward to the image goal reward. Neural Planner [20] is a model adapted the method [20] for image goal navigation from exploration task. SPTM [9] creates a topological graph, and then the Dijkstra algorithm then creates a path to a waypoint. SMT [8] stacks all the visual features of the past observations and the pose information as a navigation memory. VGM [18] builds a topological memory to navigate envi- ronment without using landmark information. NRNS [27] trained agent without interaction with the simulator. In order to compare the method fairly, we adapted the method to use panoramic camera.
4.2 Experiment Settings
We evaluate TSGM in the Gibson environment with habitat simulator, which is photo-realistic. For image observation, we used a panoramic image, the same setting with [18], inspired by the human neuron not recognizing the heading for localizing [33]. The ground truth objects from the Gibson dataset and detected objects from the detector are used for training. For testing, a de- tector [24] pretrained with a COCO dataset used. We use a discrete action space defined as A [go forward, turn left, turn right, stop), which is common choice for navigation problems. The step size of the forward action 0.25m, and the rotation angle is set to 10 in all experiments. The reward function R is defined using the progress of an agent, using a geodesic dis- tance between an agent and the target, and +10 is given when the agent reaches the goal. Further implementation details of the proposed method are provided in the supplementary material.

---

Table 1: Comparison of TSGM with memory based baselines image goal navigation Gibson.
Method
Memory
Pose
Object
Easy
Medium
Hard
Overall
Success
SPI.
Success
SPI.
Success
SPI.
Success
SPI.
RI. [26]
implicit
72.5
53.1
45.3
Active Neural SLAM [17]
metric
X
X
74.2
20.5
68.4
22.9
29.9
11.0
57.5
18.1
Expánav
70.2
38.5
50.9
SMT [8]
grap
X
X
81.9
77.4
65.6
52.2
55.6
39.7
67.7
56.4
Neural Planner [20)
graph
X
71.7
41.3
64.7
38.5
42.0
27.0
35.6
SPTM [9]
66.5
40.6
64.2
38.5
42.1
25.4
57.6
34.8
VGM [18]
graph
86.1
79.6
81.2
68.2
60.9
45.6
76.1
64.5
TSGM (Ours)
graph
82.0
67.2
Table Comparison of TSGM with image goal navigation baselines on straight/curved episodes on
Gibson.
Easy
Medium
Hard
Overall
Path Type
Method
Success
SPL
Success
SPL
Success
SPL
Success
SPL
NRNS [27]
67.1
57.8
52.4
41.2
32.6
22.4
50.7
40.5
Straight
VGM [18]
81.0
54.4
82.0
69.9
67.3
54.4
76.7
59.6
TSGM (Ours)
94.4
92.1
92.6
84.3
70.3
62.8
85.7
79.7
NRNS [27]
31.7
13.0
29.0
13.6
19.2
10.4
26.6
12.3
Curved
VGM [18]
81.0
45.5
78.8
59.5
62.2
46.9
74.0
50.6
TSGM (Ours)
93.6
91.0
89.7
77.8
64.2
55.0
82.5
74.1
Evaluation metrics. We use Success and SPL (success rate over path length) to evaluate the naviga- tion tasks. Success is calculated by dividing the total success number by the number of test episodes. We the success criterion Im the goal, a common criterion for image goal navigation. SPL multiplies the success rate and the ratio of the shortest path length and traveled path.
Episode settings. We divide test episodes into three difficulty levels for image goal navigation: easy, medium, and hard. The difficulty is determined by the geodesic length of an episode. Following [17, 18), we set the length as follows: easy (I.5 3m), medium (3 5m), and hard 10m).
4.3 Results
TSGM outperforms baselines. Table 1 shows the performance of our TSGM and relevant memory- baselines on test splits of the Gibson dataset. Our TSGM algorithm outperforms the baseline methods in terms of Success and SPL @ Im. TSGM improves upon the implicit mem- ory model [26] across splits of Gibson by 31.8% the success rate. Compared the VGM [18], our method improved by +5% on the success rate the 4.1% on SPL. To measure performance in various situations, the test episodes are divided according to whether the path is curved or not in Table 2. Our TSGM shows surprising performance improvement in both curved and straight situ- ations. In particular, for an easy episode, the SPL increased quite a surprising amount, 45.5% to 91.0% (100%) for curved episodes and 54.4% to 92. % (69.3%) for straight episodes compared to the satt--of-the-ar result of VGM [18], which can seen that if an object existing at target position is present in the field of view, the agent can find the goal position very efficiently. In aver- aged results in Table 2, TSGM outperforms competitive baselines by 4.59.0% on the success rate and 20.2-23.555 23.5% improvement on SPL. In summary, TSGM significantly outperforms competitive baselines by 5.0-9.0% on the success rate and 7.0-23.5% improvement on SPL.
TSGM finds efficient paths. We checked that the SPL, which indicates the efficiency of the path, is improved lot. To investigate how the object graph assists the agent in choosing efficient paths, we visualized the paths Figure 4. When an object graph is not provided, an agent passes the goal without realizing it is goal, while an agent with object graph quickly understands that the goal has been achieved and terminates the search (see Figure 4(a)). Figure 4(b), the agent enters a deadlock when the object graph is not provided. In the of TSGM with object graph, on the other hand, the goal is successfully found with a more efficient path.

---

(a)
(b)
Ours
Wilhout oboject graph
Ous
Wilhout olject graphi
Figure 4: Visualization of path changes when object graph is given. Our TSGM is compared with the method that does not have the object graph. The red flag highlights the image goals. The lines with an arrow represent trajectories of the agent with the color changes with time. (a) shows that an inefficient path is created when the object graph is not given. (b) The agent is trapped in the deadlock states without the object graph.
time
Goal Image
(a)
(+)
7
Start Position
Goal Position
Image (b)
Object Node
Figure 5: Visualization of the robot path on real environment. The color gradation represents the flow of time. The blue arrow symbolizes the starting point, and the red flag indicates the destination. An image node is represented by blue circle, and object node is represented by pink triangle.
Cross update module is effective. As shown in Table 3, the ablation study on the graph update was performed, where we only used imi- tation learning results on hard episodes for the ablation experiments. The object update version indicates the update done only to the im- age node to the object node, and the visual update version shows the results when the update is the opposite. Using the semantic graph up- date shows 4.5% improvement in success rate and +5.3% on SPL. Surprisingly, the updating object node with visual node shows better results than updating graph with opposite direction, showing +8.0% on the success rate and +6.5%
Table 3: Update rules.
Update
Success
SPL
No
0.533
0.393
Visual
0.578
0.446
Object
0.613
0.458
Cross
0.627
0.471
on SPL compared to the No update version. Finally, the cross update shows the most powerful re- sults, +9.4% the success rate and +7.8% on SPL. The results indicate that cross updating improves performance by making image and object features informative.
TSGM can handle the real noisy world. Figure 5 shows paths and topological semantic graphs from navigation experiments using Jackal robot. The episodes examined in the real world are sampled from hard episodes with curves. Nonetheless, agent demonstrated successful navigation by stopping at the correct goal location. The supplementary video shows more paths, with the graph being incrementally built over time.
Conclusion and Limitation
The proposed method explores while incrementally generating semantic topological graph using landmarks such as objects. The core idea of our method is derived from animal behavior, which uses landmarks to discover an efficient path to the goal. We demonstrate that TSGM performs efficiently and ef- fectively, with significant performance improvements, particularly in SPL.. The proposed method is demonstrated using a Jackal mobile robot to show its effectiveness for practical visual navigation in the real-world.
navigational cue. Contextual information is taken from the constructed graph
While it is expected to be suitable for finding objects since object contexts are successfully trained in TSGM, a limitation of the proposed method is that experiments object-gozt -oal navigation conducted, due to time constraint. We may adapt TSGM to object-gou navigation in the future.

---

Acknowledgments
This work was supported by the Institute of Information & communications Technology Planning & Evaluation(IITP)) grant funded by the Korea governmen((MSIT) (No. 2019-0-01309 Develop- ment of AI Technology for Guidance of Mobile Robot to its Goal with Uncertain Maps in In- door/Outdoor Environments, (50%) and No. 119-0001190, [SW Star Lab] Robot Learning: Effi cient, Safe, and Socially-Aceeptaaee Machine Learning, (50%))
References
[1] Mirowski, M. Grimes, M. Malinowski, K. M. Hermann, K. Anderson, D. Teplyashin, K. Si- monyan, A. Zisserman, R. Hadsell, et al. Learning to Navigate in Cities Without Map. In Neural Information Processing Systems (NeurIPS), 2018.
[2] K. Chen, J. P. de Vicente, G. Sepulveda, F. Xia, A. Soto, M. Vázquez, and S. Savarese. A Be- havioral Approach Visual Navigation with Graph Localization Networks. Robotics: Science and Systems (RSS), 2019.
[3] D. Gordon, A. Kadian, D. Parikh, J. Hoffman, and D. Batra. SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation. In Proceedings of the IEEE International Confer- on Computer Vision (ICCV), 2019.
[4] J. Yang, Z. Ren, M. Xu, X. Chen, D. Crandall, D. Parikh, and D. Batra. Embodied Vi- sual Recognition. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.
[5] T. Chen, S. Gupta, and A. Gupta. Learning Exploration Policies for Navigation. International Conference on Learning Representations (ICLR), 2019.
[6] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L.. Jones, A. N. Gomez, t... Kaiser, and I. Polo- sukhin. Attention All You Need. In Neural Information Processing Systems (NeurIPS), 2017.
[7] A. Kumar, S. Gupta, Fouhey, S. Levine, and J. Malik. Visual Memory for Robust Path Following. In Advances in Neural Information Processing Systems (NeurlPS), 2018.
[8] K. Fang, A. Toshev, L. Fei-Fei, and Savarese. Scene Memory Transformer for Embodied Agents in Long-Horizon nition (CVPR), 2019.
Horizon Tasks. In IEEE Conference on Computer Vision and Pattern Recog-
[9] N. Savinov, A. Dosovitskiy, and V. Koltun. Semi- Ceri-Parametric Topological Memory for Naviga- tion. In International Conference on Learning Representations (ICLR), 2018
[10] S. Gupta, J. Davidson, S. Levine, R. Sukthankar, and J. Malik. Cognitive mapping and plan- ning for visual navigation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[11] E. Parisotto and R. Salakhutdinov. Neural Map: Structured Memory for Deep Reinforcement Learning. International Conference on Learning Representations (ICLR), 2018.
[12] J. F. Henriques and A. Vedaldi. MapNet: An Allocentric Spatial Memory for Mapping Envi- ronments. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[13] G. Avraham, Y. Zuo, T. Dharmasiri, and T. Drummond. EMPNet: Neural Localisation and Mapping using Embedded Memory Points. In Proceedings of the IEEE International Confer- ence on Computer Vision (ICCV), 2019.
[14] Y. Lv, N. Xie, Y. Shi, Z. Wang, and H. T. Shen. Improving Target-driven Visual Navigation with Attention 3D Spatial Relationships. arXiv preprint arXiv:2005.02155 2020.
[15] H. Du, Yu, and L.. Zheng. Learning Object Relation Graph and Tentative Policy for Visual Navigation, In European Conference Computer Vision (ECCV), 2020.

---

[16] Y. Qiu, A. Pal. and H. I. Christensen. Target driven visual navigation exploiting object rela- tionships. arxiv preprint arXiv:2003.06749, 2020.
[17] D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov. Learning to Explore using Active Neural SLAM. International Conference Learning Representations (ICLR), 2020.
[18] O. Kwon, N. Kim, Y. Choi, H. Yoo, J. Park, and S. Oh. Visual Graph Memory with Un- supervised Representation for Visual Navigation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.
[19] D. S. Chaplot, R. Salakhutdinov, A. Gupta, and S. Gupta. Neural Topological SLAM for Visual Navigation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[20] E. Beeching, J. Dibangoye, O. Simonin, and C. Wolf. Learning to Plan with Uncertain Topo- logical Maps. In European Conference on Computer Vision (ECCV), 2020.
[21] J. A. PecobaEscobar, O. Kornienko, P. Latuske, L. Kohler, and K. Allen. Visual landmarks sharpen grid cell metric and confer context specificity to neurons of the medial entorhinal cortex. Elife. 1:e16937, 2016.
[22] E. Chan, O. Baumann, M. A. Bellgrove, and J. B. Mattingley. From objects to landmarks: the function of visual location information in spatial navigation. Frontiers in psychology, 3:304, 2012.
[23] S. Collett and P. Graham. Animal navigation: path integration, visual landmarks and cogni- tive maps. Current Biology, 14(12):R475 R477, 2004.
[24] K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
[25] F. Xia, A. R. Zamir, Z. He, A. Sax, J. Malik, and S. Savarese. Gibson Env: Real-World Percep- tion for Embodied Agents. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018.
[26] Y. Zhu, R. Mottaghi, E. Kolve, J. Lim, A. Gupta, L. Fei-Fei, and A. Farhadi. Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning. In IEEE interna- tional conference on robotics and automation (ICRA), 2017.
[27] M. Hahn, D. S. Chaplot, S. Tulsiani, M. Mukadam, J. M. Rehg, and A. Gupta. No RL, No Sim- ulation: Learning to Navigate without Navigating. In Neural Information Processing Systems (NeurIPS), 2021.
[28] P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Kr- ishnan. Supervised contrastive learning. In Neural Information Processing Systems (NeurIPS), 2020.
[29] J. Li, P. Zhou, C. Xiong, and S. C. Hoi. Prototypical Contrastive Learning of Unsupervised Representations. International Conference on Learning Representations (ICLR), 2021.
[30] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural Message Passing for Quantum Chemistry. In International Conference on Machine Learning (ICML), pages
PMLR, 2017.
[31] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimovv Proximal Policy Optimization Algorithms. arXiv preprint arXiv: (okkv:/777.0060 1707006377 2017.
[32] J. Ye, D. Batra, A. Das, and E. Wijmans. Auxiliary Tasks and Exploration Enable Objectgoal Navigation. In IEEE International Conference on Computer Vision (ICCV), 2021.
[33] S. N. Weber and H. Sprekeler. Learning place cells, grid cells and invariances with excitatory and inhibitory plasticity. Elife, 7:e34560, 2018
10


Reviews:
[{"review_id": "qmqHYNfqfQ_", "reviewer": "Reviewer_T5E1", "summary": "This paper designs a novel topological semantic graph framework to represent the images and objects in the scene, and demonstrates its efficiency on the point goal navigation task in Gibson.  Further, the authors conducts physical experiments and obtain promising results. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.The novelty of the proposed method which passes information between images and objects and dynamically updates the topological semantic graph in the navigation process.\n2.On the point goal navigation task in Gibson, the proposed method outperforms competitive baselines on both SR and SPL.\n3.The physical experiments  in real world demonstrates the robustness of the proposed method.\n\nWeaknesses\n1.In the comparison experiment, the proposed method is the only one that introduces object feature, which makes it unclear that the performance gain is from the extra information or the topological semantic graph.\n2.While the proposed framework constructs a semantic representation of the indoor scene, the authors only test it on the image goal navigation task. More downstream tasks like Object Navigation and Vision Language Navigation could be included.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper designs a novel topological semantic graph framework to represent the images and objects in the scene. The proposed novel method passes information between images and objects and dynamically updates the topological semantic graph in the navigation process. On the point goal navigation task in Gibson, the proposed method outperforms competitive baselines on both SR and SPL. The significantly improvement of SPL demonstrates that the proposed method could find more efficient path. Further, the  physical experiments  in real world demonstrates the robustness of the proposed method. Some small shortcomings are: In the comparison experiment, the proposed method is the only one that introduces object feature, which makes it unclear that the performance gain is from the extra information or the topological semantic graph; While the proposed framework constructs a semantic representation of the indoor scene, the authors only test it on the image goal navigation task. More downstream tasks like Object Navigation and Vision Language Navigation could be included.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "The statement in Section 3 should be reorganized to express the training process more clearly.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}, {"review_id": "Z8qyn2pgHNK", "reviewer": "Reviewer_XjPf", "summary": "The paper proposes a Topological Semantic Graph Memory (TSGM) for image-goal navigation, which is a graph-memory based method. TSGM outperforms the competitive baselines on success rate and SPL. Physical experiments are also carried to validate the effectiveness on real-world environment.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1)\tThis paper proposes a novel topological semantic memory to improve ImageNav performance. The idea of coupling an image graph and object graph is novel and effective. This hierarchical modeling of the agent’s memory is insightful.\n2)\tMobile robot demonstration proves the effectiveness.\n\nWeaknesses:\n1)\tTSGM builds upon other topological memory methods, and it can be considered as small modification of previous algorithms.\n2)\tIt is not convincing that the so called “Cross graph mixer” can actually improve the agent’s performance. It seems that the authors simply add an extra module to VGM. Maybe the extra parameters lead to the improvements, not the “Cross graph mixer”. It would be better to analyze how the object features in the object graph help to improve navigation performance in more detail.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The proposed method is proved to be effective and reaches SOTA by adequate experiments on the specific problem of Image-Goal Navigation. However, the method is raised on the basic of prior works and is lacks of innovation. Furthermore, the paper is lack of analysis how the object features in the object graph help to improve navigation performance.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Add the analysis how the object features in the object graph help to improve navigation performance.\nAdd new innovation points.\n", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "HQqfERJBLzl", "reviewer": "Reviewer_dNHx", "summary": "This paper has proposed a topological semantic graph memory (TSGM) for the image-goal navigation tasks.\nThe proposed method explores while incrementally generating a semantic topological graph using objects in images. The experimental results showed that the proposed method could improve navigation performance using object and image graphs by cross graph mixer.\n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method can navigate without SLAM or other location estimation or metric maps.\n- TSGM has shown performance improvement in pre-experiments for an image-goal navigation task.\n\nWeaknesses:\n- The proposed method relies on pre-trained object detectors such as MaskRCNN. As such, it cannot cope with false detections and unknown objects.\n- The proposed method is that no experiments on object-goal navigation are conducted.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Image-only navigation without the use of location information is a challenging task.\nThe topological graph construction approach according to image similarity or detected object similarity seems to be effective in a simple way.\nThis method may also be useful for applications in vision and navigation tasks.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "Introduction:\n“Since a node indicates a location, the robot’s position is not required.”\nI think the author's claims are overstated. The position of the robot may affect the success of the navigation. Topological graphs have several advantages. But aren't its advantages and disadvantages complementary to metric positioning? There are visual navigation approaches that complete navigation only with topological maps. However, some approaches use robot positions.\n\nFig.1, Importance of objects:\nWhat do you mean? Is there such a thing as confusing cups in different locations? It doesn't seem difficult to determine that they are different cups because they are in different locations. Does this indicate a disadvantage in an approach that uses only topological maps? Are you saying the two cups are not identical because they exist in different locations?\nAdditionally, it is unclear at this point what exactly you mean by “update”. The object called a “cup” would not be updated. Perhaps the author is trying to say that the topological graph is updated. If that is the case, it would be good to show in Fig. 1 which elements of the topological graph are updated and how.\n\nSome words are provided only as abbreviations. What is the official name? What are these abbreviations? -> ANS, SMT, VGM, NRNS.\n\nspatially meaningful landmarks:\nWhat does it mean to use landmarks without location information? In a metric map such as SLAM, landmarks contain location information. If it is a graph structure without location information, why is the topological graph mapped on an overhead view of the environment in Fig. 2? Perhaps this representation is misleading.\n\n\nL104: topological semantic graph memory (TSGM):\nIt appears in the immediately preceding sentence. Abbreviations should be defined at the time of the first appearance.\n\nthe image affinity and image-object affinity:\nWhat are these? How are they quantified?\n\nIn object and image graph construction, the similarity of features in images is used to determine if objects or nodes are identical. Thereby, if two image nodes do not have similar items, they are determined to represent different locations. But then, whether they are different locations or not should be resolved by using the robot's location information. (As is the case with many VNL approaches.) This architecture has the limitation that it does not use robot location information. This architecture may be disadvantaged because it does not constitute a metric spatial structure. For example, when multiple rooms look almost identical, including the objects present, it seems impossible to discern that they are different rooms.\n\nL190 and 192: Typographical errors?\nThere appear to be several typographical errors in formulas and variables. The authors would do better to double-check all formulas and variables.\n\nTables 1 and 2:\nAre these results in a seen environment? Or is it an unseen environment?\nstraight/curved episodes -> What are the differences in these conditions? I could not find where these are described.\n\nWhat is the number of cross-updates for two different graphs? What is the number of training cycles for PPO, etc.? And what are their computational costs?\n"}}, {"review_id": "Fb1yQ-t5bYu", "reviewer": "Reviewer_apkv", "summary": "To tackle image goal navigation, this paper proposed a method to represent the memory of landmark images and detected objects in the environment as topological graphs, and to learn navigation action policies using a cross graph mixer and a transformer network. Experiments in a Gibson environment showed that the proposed method significantly outperforms various existing methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The paper is very readable and carefully describes the details of the proposed method. The experimental results presented in Table 1, 2, and 3 fully demonstrate the high effectiveness of the proposed method. In addition, the supplemental video shows a real-world experiment using an omnidirectional camera, which is very attractive and well demonstrates the high practicality of the method.\n\nWeaknesses: The idea of using memory and transformer networks in image-goal navigation is not new in itself but rather one of the major recent trends. For example, the following two papers (not cited in this manuscript) take such an approach:\n[] Du, Heming, Xin Yu, and Liang Zheng. \"VTNet: Visual Transformer Network for Object Goal Navigation.\" International Conference on Learning Representations. 2020.\n[] Rui Fukushima, Kei Ota, Asako Kanezaki, Yoko Sasaki, and Yusuke Yoshiyasu. “Object Memory Transformer for Object Goal Navigation.” IEEE International Conference on Robotics and Automation (ICRA), 2022.\nIn particular, the latter is similar to the proposed method in that it inputs both object and image memory to the transformer network. The main difference is that they simply store sequential frames in the time axis, whereas the proposed method in this manuscript records landmark information as nodes. The importance of this difference should be discussed in detail (showing experimental results, if possible). Also, as written in the Limitation section on page 8, the evaluation of object goal navigation experiments using such as AI2-THOR is missing. However, the way it is written, it is not clear whether the experiments have not been conducted simply due to lack of time, or whether there is a fundamental theoretical limitation that prevents the proposed method from being applied. This needs to be clarified.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "There is basically not a single issue that suppresses the significance of the paper, and I see no reason to reject it. The proposed method is interesting and based on a solid theory, and the thorough experimental results and the supplemental video provide very useful insight. For these reasons, I recommend accepting this paper.\n\n### Final Recommendation\nThe authors have provided reasonable responses in the rebuttal phase, and I was convinced by their responses. The results of the additional experiments with object-goal navigation shown in the general response are also impressive. Therefore, I keep my initial rating, Strong Accept.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "- Explanation of what is the advantage/disadvantage to use landmark-based memory against the approach to use time sequence memory.\n- Explanation of why the proposed method is not applicable to object goal navigation and the results on e.g., AI2-THOR are not provided.\n"}}]

Author rebuttal:

Decision: Accept (oral)