Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces AVLEN, Audio-Visual-Language Embodied Navigation. In this setup, an embodied agent aims to find the location of an audio event by navigating in a simulated 3D visual world by interacting with the environment and an oracle. The oracle provides free-form natural language utterances to guide the agent. To address this task, a hierarchical RL system is proposed. Query policy \pi_q learns whether to use audio-goal policy \pi_g or query the oracle and use language-feedback policy \pi_l. Experiments show that the proposed method surpasses audio-only methods and successfully learns when to query the oracle.  
strengths_and_weaknesses: * S1 paper contributes to the literature by showing that audio-only methods can be improved with interactive agents when they are provided with natural language instructions

* S2 paper shows that rather than using the model's uncertainty as the query policy, we can train the system to learn when to query.

* W1 While reporting results, the paper misses some essential information or experiments which could be misleading or hard to evaluate fairly the merit of the work. 


questions: 
* L87 we --> our

* L193 please eliminate epsilon notation for policies. 

* L243 First, please clarify the intuition behind this formulation for the reward function. Second please add equation number. Also, please clarify how you choose hyperparameters r_neg and r_f

* L251 Please explain the entropy loss. It is not clear to me.

* Table1: Please add AVLEN only \pi_l where budget is infinity. also add a column with checkmarks for language input available or not. AFAIK none of the models you compare have access to the oracle.

* L278 in front --> in front

* L281-282 How do you ensure that the sound comes from a unique object? There are several objects of the same type in the environment. 

* Figure 3a: please report  \pi_g, \pi_l only models here as well. 

* Figure 3b: please report K=0,1,2 as well

* L328 it's --> its

limitations: Please discuss limitations in terms of language i.e. you only worked with English annotations also visual diversity i.e. MatterPort images come from North American houses.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In many scenarios, to effectively navigate in the real-world, agents need to combine a variety of sensory information including audio, visual, language, etc. VL agents navigate based on only vision and language input. More recently, SoundSpaces introduced the task of navigating conditioned on an audio source but without language guidance. This paper presents AVLEN, an embodied agent that leverages audio-visual cues in addition to natural language instructions from an oracle for navigation. The agent is trained using hierarchical RL and consists of one high-level policy for deciding when to query the oracle and two low-level navigation policies conditioned on either audio-visual cues or language instructions. The paper demonstrates that the proposed multimodal agent performs better across several evaluation metrics compared to SOTA models on several challenging sound settings in the AudioGoal task. Several ablations are conducted to analysis the usefulness of querying for language guidance from an oracle, using RL for learning when to query, and robustness of the navigation agent to distracting sounds. 
strengths_and_weaknesses: Strengths: 
- First to combine audio-visual navigation with natural language instruction guidance from oracle. Introduces a navigational agent that can leverage full range of modalities, and not just a subset.
- Paper is well motivated both in terms of the task setting and technical approach. 
- The authors report SOTA metrics on the benchmark task across all metrics and compare their method against several strong baselines. 
- Good set of ablations, demonstrating that the performance gain is bottlenecked by the speaker model and also how the agent learns to distribute the queries throughout the trajectory unlike the MU baseline

Weaknesses:
- There have already been many prior works focused on solving the individual subproblems of AV navigation and separately agents that can interface with an oracle to ask for help. 
- Could use an explanation for why n=4 is used for the number of segments used for producing the oracle instruction. Was it chosen arbitrarily? 
- Could also compare against a non-hierarchical baseline that uses a single network fusing together the different modalities and producing action to show that the hierarchical abstraction is indeed helpful
- The ablation with distractor sounds makes sense, but I'd imagine that the language guidance should help to disambiguate the distracting sounds a bit more. It might be challenging with only audio-visual, but language instructions should provide a stronger discerning signal. 

Minor:
- L330 - "the agents interaction ability is would come useful". wording
- Typo in equation after L234
- For Figure 3c is this a distribution of queries over multiple rollouts? 
questions: - The paper claims that AVLEN solves VL navigation as a subtask of the SAVi task. It'd make a stronger argument if the paper also included experiments to support this claim by investigating the performance of the AVLEN agent in traditional VL benchmarks like RxR, CVDN, etc.
- Is there an explanation as to why the random model's performance decreased with a larger budget for number of queries? Also the performance gain with more queries seems a bit negligible. It seems that if an agent had access to more oracle guidance, the performance should increase more linearly. 
- Is there a breakdown of SR for different goal categories? Wondering if the SR is pretty similar across all categories of sounds or if there are certain categories that are better than others. 
limitations: The authors point out several limitations of their work. Specifically they mention the assumptions that the agent is able to query anywhere in the environment and a pretrained speaker model is used for generating instructions. In terms of societal impact, the authors acknowledge that language-guided / instruction following agents that interface with humans are increasingly relevant and will soon permeate our everyday lives. They state that such agents facilitate automation of challenging human tasks, but neglect to address the negative aspects of such a trend, e.g. putting people out of work. The authors briefly mention that perhaps the simulated environment the agents are trained in, which are hand selected by humans, may be biasing the agent's behavior. It may also be interesting to (which may be quite far-fetched for the current state of the field) consider the negative consequences of introducing sound as an additional modality and equipping robots with audio-capturing sensors which may be used for nefarious purposes.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work proposes a new audio-visual-language embodied navigation task, where the agent not only hears the audio from the sounding object but also is able to query the environment of the goal location. To deal with the multimodal navigation task, this paper propses a multimodal hierarchical policy that consists a high-level pocliy that decides which low-level policy and two low-level policies that either navigates with audio-visual cues or navigates with queried instructions. In the experiment, authors compare with the SOTA model in semantic auido-visual navigation and show that the proposed model outperforms the SOTA model.
strengths_and_weaknesses: The proposed task is interesting and adds one more modality to the audio-visual navigation task, although the ability to query is not very realistic in the real world.

The multimodal hierarchical model is also pretty novel in that the hierarchy is for in modality selection rather than the common goal prediction/actuation command hierarchy.

Using RxR dataset to train a speaker model as the oracle is inspiring and this show a promising way to bridge the gap between the audio-visual navigation task and the vision-language navigation task.

The paper is clearly written and easy to understand. The experiments and analysis are also pretty sufficient.

I expected the success rate and SPL improvement be higher than just a couple percents. I wonder if authors have answer for that.
questions: The trained oracle speaker seems to be useful for reproduce the results. Will you release the model and code?

In the supp video, it looks like a couple successful cases were because of the query of stop. Is it possible most the performance improvement comes from the stop query?

Do authors have an intution of why the performance improvement is not very big? It seems like the language instructions provide pretty strong guidance for navigation.

limitations: Both the limitations and societal impact were discussed in the conclusion.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The manuscript modifies the semantic audio-visual navigation task to incorporate oracle feedback, intermittently, throughout episodes. The approach leverages an options framework, in order to encourage the policy to learn when to query the oracle.
strengths_and_weaknesses: Strengths

The manuscript provides an options framework and extends the SAVi baseline to incorporate natural language. 

The manuscript is reasonably well-written.

Weaknesses

Section 2: Consider other interaction paradigms — see some of the new embodied AI surveys, e.g., https://www.jair.org/index.php/jair/article/download/13646/26807/

Section 3: More analysis necessary: I would have liked to see a discussion that justifies that the reward signal is sufficiently expressive for appropriately biasing the high-level policy.

Section 3: Whereas predicating the reward design on distance-to-goal may have been essential to get started on applying RL frameworks to these embodied tasks, it is now important to move beyond this. Methods should pursue self-supervisory alternatives as reward signals, which would narrow the gap to practical real-world deployment.

Section 4: The manuscript is missing sufficient ablations, to confirm that the performance improvements are indeed resulting from the querying framework, as opposed to the more-expressive CLIP context, transformer representations, and VLN pre-training tasks; these are not present in the previous baselines. Despite the addition of those components, with unsupported contribution, the improvements are admittedly not so dramatic, as the manuscript self-admits.
questions: Why is assuming access to an oracle considered a reasonable problem setting?

In order to more flexibly accommodate the penalties, why not formulate the problem as a constrained POMDP, with, among other signals, the queries as costs? Indeed, it would provide more control over the query budget. Can you discuss the tradeoff(s) in doing so?
limitations: See above — i.e., reward design, problem formulation, (lack of) ablations
review_title: null

Meta-review: metareview: The authors present a model for soundscapes that is able to request guidance from an oracle. This requires that the agent know when to query the user (oracle) for help. This sits somewhere between the work on oracle guidance and language instruction following.  Ablations and comparisons are provided but do not fully explore the natural set of questions about why/where the model performs best.  Note that this oracle deviates from that in the most relevant prior work by Nguyen and these instructions differ from those generated by a model in papers by Thomason.

Overall, the work makes a nice contribution to bridging existing literatures within EAI.  
confidence: Certain
Decision: Accept