Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper has proposed a topological semantic graph memory (TSGM) for the image-goal navigation tasks.
The proposed method explores while incrementally generating a semantic topological graph using objects in images. The experimental results showed that the proposed method could improve navigation performance using object and image graphs by cross graph mixer.

strengths_and_weaknesses: Strengths:
- The proposed method can navigate without SLAM or other location estimation or metric maps.
- TSGM has shown performance improvement in pre-experiments for an image-goal navigation task.

Weaknesses:
- The proposed method relies on pre-trained object detectors such as MaskRCNN. As such, it cannot cope with false detections and unknown objects.
- The proposed method is that no experiments on object-goal navigation are conducted.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Image-only navigation without the use of location information is a challenging task.
The topological graph construction approach according to image similarity or detected object similarity seems to be effective in a simple way.
This method may also be useful for applications in vision and navigation tasks.
issues: Introduction:
“Since a node indicates a location, the robot’s position is not required.”
I think the author's claims are overstated. The position of the robot may affect the success of the navigation. Topological graphs have several advantages. But aren't its advantages and disadvantages complementary to metric positioning? There are visual navigation approaches that complete navigation only with topological maps. However, some approaches use robot positions.

Fig.1, Importance of objects:
What do you mean? Is there such a thing as confusing cups in different locations? It doesn't seem difficult to determine that they are different cups because they are in different locations. Does this indicate a disadvantage in an approach that uses only topological maps? Are you saying the two cups are not identical because they exist in different locations?
Additionally, it is unclear at this point what exactly you mean by “update”. The object called a “cup” would not be updated. Perhaps the author is trying to say that the topological graph is updated. If that is the case, it would be good to show in Fig. 1 which elements of the topological graph are updated and how.

Some words are provided only as abbreviations. What is the official name? What are these abbreviations? -> ANS, SMT, VGM, NRNS.

spatially meaningful landmarks:
What does it mean to use landmarks without location information? In a metric map such as SLAM, landmarks contain location information. If it is a graph structure without location information, why is the topological graph mapped on an overhead view of the environment in Fig. 2? Perhaps this representation is misleading.


L104: topological semantic graph memory (TSGM):
It appears in the immediately preceding sentence. Abbreviations should be defined at the time of the first appearance.

the image affinity and image-object affinity:
What are these? How are they quantified?

In object and image graph construction, the similarity of features in images is used to determine if objects or nodes are identical. Thereby, if two image nodes do not have similar items, they are determined to represent different locations. But then, whether they are different locations or not should be resolved by using the robot's location information. (As is the case with many VNL approaches.) This architecture has the limitation that it does not use robot location information. This architecture may be disadvantaged because it does not constitute a metric spatial structure. For example, when multiple rooms look almost identical, including the objects present, it seems impossible to discern that they are different rooms.

L190 and 192: Typographical errors?
There appear to be several typographical errors in formulas and variables. The authors would do better to double-check all formulas and variables.

Tables 1 and 2:
Are these results in a seen environment? Or is it an unseen environment?
straight/curved episodes -> What are the differences in these conditions? I could not find where these are described.

What is the number of cross-updates for two different graphs? What is the number of training cycles for PPO, etc.? And what are their computational costs?
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a Topological Semantic Graph Memory (TSGM) for image-goal navigation, which is a graph-memory based method. TSGM outperforms the competitive baselines on success rate and SPL. Physical experiments are also carried to validate the effectiveness on real-world environment.
strengths_and_weaknesses: Strengths:
1)	This paper proposes a novel topological semantic memory to improve ImageNav performance. The idea of coupling an image graph and object graph is novel and effective. This hierarchical modeling of the agent’s memory is insightful.
2)	Mobile robot demonstration proves the effectiveness.

Weaknesses:
1)	TSGM builds upon other topological memory methods, and it can be considered as small modification of previous algorithms.
2)	It is not convincing that the so called “Cross graph mixer” can actually improve the agent’s performance. It seems that the authors simply add an extra module to VGM. Maybe the extra parameters lead to the improvements, not the “Cross graph mixer”. It would be better to analyze how the object features in the object graph help to improve navigation performance in more detail.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The proposed method is proved to be effective and reaches SOTA by adequate experiments on the specific problem of Image-Goal Navigation. However, the method is raised on the basic of prior works and is lacks of innovation. Furthermore, the paper is lack of analysis how the object features in the object graph help to improve navigation performance.
issues: Add the analysis how the object features in the object graph help to improve navigation performance.
Add new innovation points.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper designs a novel topological semantic graph framework to represent the images and objects in the scene, and demonstrates its efficiency on the point goal navigation task in Gibson.  Further, the authors conducts physical experiments and obtain promising results. 
strengths_and_weaknesses: Strengths:
1.The novelty of the proposed method which passes information between images and objects and dynamically updates the topological semantic graph in the navigation process.
2.On the point goal navigation task in Gibson, the proposed method outperforms competitive baselines on both SR and SPL.
3.The physical experiments  in real world demonstrates the robustness of the proposed method.

Weaknesses
1.In the comparison experiment, the proposed method is the only one that introduces object feature, which makes it unclear that the performance gain is from the extra information or the topological semantic graph.
2.While the proposed framework constructs a semantic representation of the indoor scene, the authors only test it on the image goal navigation task. More downstream tasks like Object Navigation and Vision Language Navigation could be included.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper designs a novel topological semantic graph framework to represent the images and objects in the scene. The proposed novel method passes information between images and objects and dynamically updates the topological semantic graph in the navigation process. On the point goal navigation task in Gibson, the proposed method outperforms competitive baselines on both SR and SPL. The significantly improvement of SPL demonstrates that the proposed method could find more efficient path. Further, the  physical experiments  in real world demonstrates the robustness of the proposed method. Some small shortcomings are: In the comparison experiment, the proposed method is the only one that introduces object feature, which makes it unclear that the performance gain is from the extra information or the topological semantic graph; While the proposed framework constructs a semantic representation of the indoor scene, the authors only test it on the image goal navigation task. More downstream tasks like Object Navigation and Vision Language Navigation could be included.
issues: The statement in Section 3 should be reorganized to express the training process more clearly.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: To tackle image goal navigation, this paper proposed a method to represent the memory of landmark images and detected objects in the environment as topological graphs, and to learn navigation action policies using a cross graph mixer and a transformer network. Experiments in a Gibson environment showed that the proposed method significantly outperforms various existing methods.
strengths_and_weaknesses: Strengths: The paper is very readable and carefully describes the details of the proposed method. The experimental results presented in Table 1, 2, and 3 fully demonstrate the high effectiveness of the proposed method. In addition, the supplemental video shows a real-world experiment using an omnidirectional camera, which is very attractive and well demonstrates the high practicality of the method.

Weaknesses: The idea of using memory and transformer networks in image-goal navigation is not new in itself but rather one of the major recent trends. For example, the following two papers (not cited in this manuscript) take such an approach:
[] Du, Heming, Xin Yu, and Liang Zheng. "VTNet: Visual Transformer Network for Object Goal Navigation." International Conference on Learning Representations. 2020.
[] Rui Fukushima, Kei Ota, Asako Kanezaki, Yoko Sasaki, and Yusuke Yoshiyasu. “Object Memory Transformer for Object Goal Navigation.” IEEE International Conference on Robotics and Automation (ICRA), 2022.
In particular, the latter is similar to the proposed method in that it inputs both object and image memory to the transformer network. The main difference is that they simply store sequential frames in the time axis, whereas the proposed method in this manuscript records landmark information as nodes. The importance of this difference should be discussed in detail (showing experimental results, if possible). Also, as written in the Limitation section on page 8, the evaluation of object goal navigation experiments using such as AI2-THOR is missing. However, the way it is written, it is not clear whether the experiments have not been conducted simply due to lack of time, or whether there is a fundamental theoretical limitation that prevents the proposed method from being applied. This needs to be clarified.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: There is basically not a single issue that suppresses the significance of the paper, and I see no reason to reject it. The proposed method is interesting and based on a solid theory, and the thorough experimental results and the supplemental video provide very useful insight. For these reasons, I recommend accepting this paper.

### Final Recommendation
The authors have provided reasonable responses in the rebuttal phase, and I was convinced by their responses. The results of the additional experiments with object-goal navigation shown in the general response are also impressive. Therefore, I keep my initial rating, Strong Accept.
issues: - Explanation of what is the advantage/disadvantage to use landmark-based memory against the approach to use time sequence memory.
- Explanation of why the proposed method is not applicable to object goal navigation and the results on e.g., AI2-THOR are not provided.
review_title: null

Meta-review: metareview: Summary 
This research proposed a technique to represent landmark images and identified objects as topological graphs and to develop navigation action policies using a cross graph mixer and a transformer network. The suggested method generates an image-based semantic topological graph progressively.  The proposed strategy outperforms existing methods in a Gibson context. 

Strengths 

1. The proposed technique can traverse without SLAM or other location estimate or metric maps. 
 2. This study proposes a topological semantic memory for ImageNav. Combining an image graph and object graph and the  hierarchical memory model is commendable. 
3. They provide an effective mobile robot demonstration. 
4. The proposed technique reaches SOTA in Image-Goal Navigation trials. 
5. The suggested method transmits information between images and objects and dynamically updates the topological semantic graph during navigation. 
6.The suggested technique outperforms competitive baselines in SR and SPL for Gibson point goal navigation.
 7. Real-world experiments show the method's robustness. 
8.  The paper is well-written and discusses the proposed method. Tables 1, 2, and 3 show the method's effectiveness. The companion movie depicts a real-world experiment utilizing an omnidirectional camera, which is highly appealing and demonstrates the method's feasibility.

Weaknesses
• The proposed approach uses MaskRCNN however false detections and unknown objects can't be handled via this approach. 
• No object-goal navigation experiments are proposed. 
• It is not clear how many graphs cross-update and how many PPO training cycles and how expensive they are. 
What's image affinity and image-object affinity? What's their size? 
• TSGM modifies earlier topological memory approaches, however the "Cross graph mixer" doesn't seem to boost agent performance. More insight is needed on how object graph features aid navigation. 
• Only the proposed method introduces object features in the comparison experiment, thus it's unclear if the performance boost is from the extra information or the topological semantic graph. 
. Although the suggested system creates a semantic indoor scene representation, the authors only test it for picture goal navigation. Object Navigation and Vision Language Navigation could be added. 
• Image-goal navigation utilizing memory and transformer networks is a recent technique the reviewers have identified two uncited papers that utilize this approach:
. The evaluation of object goal navigation experiments using AI2-THOR is missing from page 8. The way it's stated, it's unclear whether the experiments haven't been done due to lack of time or a theoretical limitation that prevents the recommended method from being used. 

Update: Thank you for the clear explanation to some concerns. The issues raised have been addressed

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Oral)