Paper: IG-Net: Image-Goal Network for Offline Visual Navigation on A Large-Scale Game Map (ICLR-2024)

Reviewer comment (Novelty):
Section 2 does not provide a clear discussion of why the proposed model (IG-Net) is different from previously published models. It simply mentions that 'IG-Net is meticulously designed to navigate significantly larger maps,' but that is not sufficient.

Author response:
A1: IG-Net stands out from previous models because it builds an implicit map comprehension through training the model with the several proposed auxiliary tasks: relative and absolute position prediction, distance prediction and navigation path prediction. Specifically, we show the implicit map understanding is especially helpful in large game environments like ShooterGame as compared to VGM, ViNT, and other baselines.

Reviewer comment (Novelty):
This is not the first model to propose to solve the navigation problem with offline data. First and foremost, the manuscript overlooks conducting a review of the literature related to imitation learning (behavior cloning) and even recent offline RL, where comparisons with many previous models that have addressed the problem of navigation in virtual environments under the same constraint—using only offline data—could be made.

Author response:
A2: We thank your comment for mentioning those related works which we don’t fully cover, and we will add these works in the discussions of our related work part. In the meantime, our method mainly focuses on designing auxiliary tasks for implicit map understanding that benefits the performance for behavior cloning methods, which is different from the main contributions of offline RL algorithms. Also, our method does not leverage reward information, which is in a different setting compared with offline RL methods.

Reviewer comment (Reproducibility):
Sections 4.2 - 4.4 need improvement in terms of clarity. Details are missing on how the training of the entire model is structured, and especially, details of the proposed architecture.

Author response:
A3: We add the network architecture in Section 4.3 in our revision. Briefly, IG-Net uses MAE-pretrained ViT to separately encode the observation image and goal image, and feed the embedding into four additional transformer layers, and concat the final joint embedding to feed into four different encoders for corresponding tasks.

Reviewer comment (Experiments):
It is an offline model that just needs a small dataset of 200 trajectories. That is not too much. I wonder how this impacts the performance of the model.

Author response:
A4: We agree that it is important to test the scaling-up ability of our dataset and algorithm, which will be an important part of study in our future work. We also believe that 200 trajectories are sufficient to validate the effectiveness of our proposed approach as our model can successfully reduce all losses on both training and validation sets.

Reviewer comment (Experiments):
More baselines are needed. The selection of VGM is adequate but other approaches have to be considered. Offline-RL models should be considered as well (see [MR4]).

Author response:
A5: We have added 4 baselines to the experiments including ViNT, GNM, NoMaD, and NoMaD-EMA, shown in Table 2. These baselines all use RGB observations and can be trained without RL. Results show that IG-Net outperforms all the baselines in ShooterGame. We believe the added baselines suffer significant performance loss due to lack of a z-axis, lack of long-horizon training, and lack of a low-level navigator.

Reviewer comment (Experiments):
How does the proposed approach perform in standard embodied AI navigation benchmarks? Habitat datasets, for example. There is offline data that can be used (see PIRLNav paper dataset of 77k navigation trajectories) or the authors can maybe generate new data (it seems that with just 200 trajectories the IG-Net model has enough information to be trained).

Author response:
A6: We have added Gibson experiments where IG-Net is evaluated on different seen and unseen floorplans. Since Gibson environments are much smaller (Table 1), IG-Net doesn’t have an advantage compared to previous approaches. This experiment validates IG-Net has generalization ability to novel environments and the implicit training is especially important in larger game experiments.

Reviewer comment (Experiments):
The authors perform the testing on the seen environment during training. I believe that demonstrating the performance in unseen environments will consolidate the work.

Author response:
A3: Please refer to our new experiments on Gibson, where the agent is evaluated on 14 unseen environments for short horizon navigation.  Results show that IG-Net can navigate well in unseen environments and outperforms no auxiliary task baselines, showing the generalizability of IG-Net.

Reviewer comment (Reproducibility):
The authors do not provide the details for the architecture of the IG-Net. I am a little bit confused about the input/output of each network component.

Author response:
A2: We add the network architecture in Section 4.3 in our revision. Briefly, IG-Net uses MAE-pretrained ViT to separately encode the observation image and goal image, and feed the embedding into four additional transformer layers, and concat the final joint embedding to feed into four different encoders for corresponding tasks.

Reviewer comment (Experiments):
The ablation study in Table 4 shows an experiment "IG-Net (no position)." I am curious that the experiment demonstrates the performance of IG-Net without both absolute/relative position prediction or either of them. A similar question about the ablation experiment "IG-Net (no path and dist)", why did the authors remove two auxiliary tasks at the same time in the ablation study?

Author response:
A4: “No position” ablation study simultaneously removes both relative position prediction and absolute position prediction. We remove two auxiliary tasks at the same time to verify the joint effect of position loss as a whole. We have realized that removing each single auxiliary task for ablation study is better and will perform the experiment in our future work.

Reviewer comment (Experiments):
Although the coverage of the ShooterGame environment is much larger than other environments, it also seems to be more empty than other environments. Could the authors discuss the difficulty differences in navigation among these environments except the coverage area? Meanwhile, I am curious about the number of different states in the ShooterGame environment.

Author response:
A1: Other works use metrics including specific surface area and navigation complexity to show the complexity of the environment (see Table 1 in the paper of Gibson dataset https://arxiv.org/pdf/1808.10654.pdf). We believe that these two metrics overlook the scale of the environment which greatly affects the navigation difficulty in the environment. We think that the minimum angle of turns alongside the navigation episode is a great metric. We will add more metrics for comparing navigation complexity in our future work.

Reviewer comment (Reproducibility):
In the third part of the architecture design, the authors mentioned "Utilizing position, path, distance, and action decoders." I am curious whether the historical information, apart from visual embedding, is included in the action prediction. If the authors do not use historical action/state representations, how does the agent maintain the search efficiency instead of being stuck in a small area?

Author response:
A5: We really appreciate the comment. Currently, IG-Net only uses one current image and one goal image as inputs, which we also observe failure searching behavior around the corner of the map and hope to try adding historical states in the inputs. In the future work, we will carefully design experiments to verify the design for historical states.

Reviewer comment (Experiments):
Suppose we put the trained agent in an unseen environment, how will the agent perform? In the unseen environment, will the agent still be able to perform these auxiliary tasks? How does the agent perform on these auxiliary tasks, e.g., absolute/relative position prediction?

Author response:
A3: Please refer to our new experiments on Gibson, where the agent is evaluated on 14 unseen environments for short horizon navigation.  Results show that IG-Net can navigate well in unseen environments and outperforms no auxiliary task baselines, showing the generalizability of IG-Net.

Reviewer comment (Experiments):
The experiments section doesn’t show comparison with state-of-the-art methods on ImageNav. There has been some recent works in ImageNav which use end-to-end RL training using pretrained representations (visual and pose) [1, 2, 3, 4] which achieve significant performance on the ImageNav benchmark. The current state-of-the-art method [1] acheives ~92% SR using simple concatenation of goal and observation image being fed to a single ResNet model to generate a observation embedding. OVRL [3] and OVRL-v2 [2] have been using pretrained visual representations using data from omnidata which leads to improved perfomance. It is essential that the authors compare IG-Net to the state-of-the-art baselines. I’d suggest adding comparison to FGPrompt-EF baseline from [1] and OVRL and OVRL-v2 pretrained baselines from [2, 3]. [4] seems to be a concurrent work with no code released so no need to add comparison with it.

Author response:
A1: We have included comparison between ViNT, GNM, and NoMaD baselines, shown in Table 2. We thank the reviewers for suggesting SOTA methods such as FGPrompt and OVRL. However, to keep the comparison fair and due to the restrictiveness of our environment in simulation speed and perception domains, we have to focus on non-RL algorithms that use RGB observations only.

Reviewer comment (Presentation):
Based on description in section 4.3 it is unclear what the policy architecture of IG-Net looks like. Can authors please share a policy architecture figure?

Author response:
A3: We add the network architecture in Section 4.3 in our revision. Briefly, IG-Net uses MAE-pretrained ViT to separately encode the observation image and goal image, and feed the embedding into four additional transformer layers, and concat the final joint embedding to feed into four different encoders for corresponding tasks.

Reviewer comment (Experiments):
Ablations in section 5.5 need to ablate each component of auxiliary task separately. In the current results it looks like row 2 only uses relative and absolute pose prediction and row 3 doesn’t use navigation path prediction and distance aux tasks. In each ablation authors should sequentially remove every single loss from list of auxiliary losses to clearly demonstrate effectiveness of each components.

Author response:
A4: We really appreciate the comment, and will perform the experiment for ablating each auxiliary task in our future revision of this work.

Reviewer comment (Experiments):
The authors test their method on a single environment which is a fairly restrictive experimental setup. We don’t know if the model is capable of handling / memorising multiple environments. I would have liked to see experiments in which a single model can be learned for multiple environments.

Author response:
We perform additional experiments of IG-Net on new maps in Gibson experiments, where the agent is trained on 396 environments and evaluated on 14 novel ones, shown in Table 1. Results show that IG-Net can navigate well in the easy setting and outperforms no auxiliary task baselines, showing the generalizability of IG-Net.

Reviewer comment (Experiments):
The experiments also assume perfect actuation — left/right and forward will always lead to the same amount of actuation. I wonder how does the agent behave when the actuation during test-time is noisy / different from those used during training. Such a situation is fairly common in the real world and it would be nice to show experiments for the more general setting.

Author response:
Actuation noise is a very important problem for real world applications, and it can be simulated in game engines by adding action noise. Exploring how IG-Net can handle actuation noise is a good research direction and we will explore it in future work. Observation noise requires specific design in the camera, and we will test the performance in the future work.

Reviewer comment (Evaluation):
More importantly, the paper doesn’t have any analysis and discussion about the train-time vs test-time distribution. How different are the trajectories used at train-time compared to test-time trajectories. This analysis is important to understand the model’s generalisation capability and to actually test if it has build an efficient implicit representation of the map or not.

Author response:
We verify the difference of test-time state-goal pair and training state-goal pair by defining a metric of minimum state-goal distance. For each evaluation state-goal pair $(s,g)$, the minimum state-goal distance is defined to be $\min_{i} \|s_i - s\|_2 + \|g_i - g\|_2$, where $(s_1, g_1), …, (s_N, g_N)$ are $N=200$ state-goal pairs for navigation training data. Minimum state-goal distance describes the lowest difference in the training dataset of an evaluation state-goal pair. The mean value of minimum state-goal distance on the whole evaluation data is $2130$ on $D<4000$ difficulty, $2260$ on the $4000<D<8000$ difficulty, and $2375$ on the $D>8000$ difficulty, which is much larger than the distance threshold for success judgment ($800$). This demonstrates that the evaluation state-goal pairs have a small overlap with the training dataset.

Reviewer comment (Experiments):
I would have liked to see some more experiments to probe this implicit understanding of the map. For instance, if the agent has only been trained on a complex trajectory, can it learn to output the shortest path between two pair of observations?

Author response:
This is a great direction for testing the internal scene understanding in the IG-Net. We will perform detailed analysis on designing probing tasks for pretrained IG-Net to verify this in the future work.

Reviewer comment (Theory):
According to my understanding, none of the losses in Section 4.2 actually involves predicting actions. How does the method learn to predict actions?

Author response:
Actions are categorical {Forward, Left, Right} and are learned through Cross-Entropy loss between expert training data. We have added the description for action prediction in Section 4.2 in our revisions.

Reviewer comment (Evaluation):
Can the authors comment on how different test time trajectories are to the ones seen during training? For example, for each location of the agent for a trajectory at test time, what is the nearest location observed during training?

Author response:
We verify the difference of test-time state-goal pair and training state-goal pair by defining a metric of minimum state-goal distance. For each evaluation state-goal pair $(s,g)$, the minimum state-goal distance is defined to be $\min_{i} \|s_i - s\|_2 + \|g_i - g\|_2$, where $(s_1, g_1), …, (s_N, g_N)$ are $N=200$ state-goal pairs for navigation training data. Minimum state-goal distance describes the lowest difference in the training dataset of an evaluation state-goal pair. The mean value of minimum state-goal distance on the whole evaluation data is $2130$ on $D<4000$ difficulty, $2260$ on the $4000<D<8000$ difficulty, and $2375$ on the $D>8000$ difficulty, which is much larger than the distance threshold for success judgment ($800$). This demonstrates that the evaluation state-goal pairs have a small overlap with the training dataset.