Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduced a method to learn a visual-audio conditioned control policy. The core idea is to learn a joint visual-audio representation and derive a self-supervised reward for training task policies. To acquire a high-quality visual-audio representation, they improved the existing VAR algorithm by adopting a batch-based supervised constrastive loss to improve the data efficiency. The policy reward is then defined as the difference between the embedding of the current image and the embedding of a sound signal provided at the beginning to indicate the task goal. The method was applied to two control tasks and one navigation-related task.
strength_and_weaknesses: Strengths:
- Improvement over the existing VAR representation, which can be potentially helpful beyond the presented setting.
- Self supervised RL training method appears solid and effective.
- Enable robots to follow audio command is a useful and essential skill.

Weaknesses:
- Fine-tuning doesn’t seem very efficient in general: 2400 labels and 1M self-supervised RL training for fine-tuning seems a lot. Does the -user need to provide 2400 labels (image/audio pairs)? Providing the label and collecting 1M samples in the real-world seems infeasible.
- Not all images have intentions in real environments, which would lead to sparse reward problems and make the method potentially difficult to handle long-range tasks.
- No examples of handling dynamics gap were presented even though it is claimed to handle that. For robotic control tasks they are in general simple: e.g. the manipulator only needs to reach certain location without interactions like grasping. It's not clear if the method can effectively adapt to like manipulation of unseen objects.

clarity,_quality,_novelty_and_reproducibility: good
summary_of_the_review: I think the paper is in general interesting and can be a good work for the venue, though I do have a couple concerns as listed in the weaknesses above. 

In addition I also have some clarification questions for the work: 
1) For E2E was it trained with 1M RL steps as well? Or it was terminated after doing a pass through the 2400 new labels?
2) For the intrinsic reward, was eq 4 or eq 5 used in the final system? 
3) For the training process, how were the sound labels as well as intentions generated?
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents VAR++, that is representation that generates intrinsic rewards by associating images and sound commands. This model is used to perform command-following robot tasks. The method builds on top of VAR (Visual-audio representation), is evaluated in 3 different tasks in simulation, and is compared against other 2 baselines in addition to the basic version.
strength_and_weaknesses: The paper is well structured and written in a clear way. The problem of interest is introduced and motivated in a comprehensive way, mentioning relevant literature in the field. The "list of contributions" reads more as the list of steps describing the approach and its advantages, rather than highlighting the novelties of this particular piece of work. The methodology is presented in a clear way and the diagrams are clarifying and well presented as well. The experimental setup is also described fairly well, however there is very little visualisation (only Fig. 4) of the simulated environment and setup. There is also no visualisation of the data and of the results obtained. The tables do report quantitative numbers, however they seem sometimes incomplete: for example why is ASR+NLU+RL not included in Table 2 and 4? Also, what is the impact of the transcription step (which is not part of the ASR+NLU+RL per se) on the performance obtained from applying ASR+NLU+RL? What are the results if you used (correct) text directly?
The limitations of the proposed work are not discussed. Can you describe in which cases the proposed method would fail or perform just as good as other methods? Is there an evaluation on time efficiency? Is the proposed method also more efficient with respect to other alternatives?

clarity,_quality,_novelty_and_reproducibility: The paper is generally written clearly, including the formulation of the problem and of the proposed solution. The novelty is, in my understanding, although interesting, marginal. The method introduced mainly adds on top of an existing method. The ideas proposed to improve the existing method are interesting, although I cannot evaluate their novelty with respect to other existing works.
The results are not reproducible, as there are very little details of the implemented models and of the simulated environments used.
summary_of_the_review: The proposed method is interesting and the paper well written. The novelty and contribution of the paper are incremental. The experimental evaluation does not include enough details to be reproducible, and the quantitative evaluation is missing some results (eg relative to one of the baselines considered) and some discussion about the limitations of the proposed method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an architecture for audio visual representations that can be used to embed multi-model representations useful for downstream robot learning tasks.
strength_and_weaknesses: Strenghs:
- very relevant problem space: this is an important area of research,
- the model is well-motivated, and described in sufficient detail to be generally useful,
- the quantitative results are compelling.

Weaknesses:
- The whole paper can be summarized as: 'we took VAR, replaced the triplet loss with a contrastive loss, and things worked better'. While a solid result to be publishing, the novelty is limited: the general idea that contrastive losses generally improve over triplets is well documented in the literature.
- The model is well engineered and reasonable for the setting that's explored, but the paper doesn't provide any unique, novel insight beyond 'here is an architecture that works well for this problem'. This means there is little one will learn from reading this paper beyond this exact setting.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and detailed. The appendix could be enhanced by providing all hyperparameter settings or a link to the open-source codebase.
summary_of_the_review: Nice contribution to a very specific problem, which is clear and correct, but likely too incremental to meet the bar for a venue like ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper considers the problem of robot instruction-following with visual inputs when commands are provided as audio inputs as opposed to text (i.e., typed or provided by a speech recognizer), which is typically assumed to be available by existing methods. The paper proposes the use of a contrastive loss to learn a joint audio-visual representation that is then used to define a form of intrinsic reward for policy learning. The method is evaluated in comparison to different baselines on two simulated control and one navigation task.

Instruction-following has received significant renewed interest within the robotics and the broader machine learning communities of-late. As such, the topic of the paper is of interest to many in the ICLR community. The ability to handle audio inputs as opposed to requiring that text be provided to the agent is refreshing. It opens the door to practical challenges and opportunities (e.g., handling audio observations that do not correspond to instructions) that are ignored/abstracted away by text-based methods. The novelty of this work primarily lies in replacing the loss used to train an existing audio-visual architecture with a contrastive loss. As the reviewers point out, the significance of this contribution is limited. The use of a contrastive loss to improve the quality of the joint audio-visual representation is certainly sensible, but also predictable. The paper provides a nice validation of the benefits of contrastive learning for an important problem, but without further novelty in this domain or clear implications for other domains, the contributions are limited.
justification_for_why_not_higher_score: The problem is interesting and relevant to the community, but the novelty is limited as acknowledged by the reviewers. All three reviewers read the author response and while one increased their score (to 6), they found that the authors' response improved the paper, but did not address their primary concerns.
Decision: Reject