Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposed a system which learns depth in an unsupervised manner and semantic segmentation in a self-supervised manner from object segmentations from Mask R-CNN and task demonstrates and language instructions. Using the resulting predicted depth and semantic segmentations, the approach builds an uncertainty-aware semantic 3D map which can continually update information from new views. This map can be used for downstream task solving or question answering.
strengths_and_weaknesses: Strengths:
 - Interesting approach to fusing semantic and spatial uncertainty into one unified 3D scene representation for planning.
 - Impressive quantitative results on Alfred benchmark.

Weaknesses:
- Key assumption for self-supervised semantic segmentation matching may not apply on other simulators or real robots. This assumption is also not addressed in the limitations section.
- Some training details unclear from writing.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Overall an interesting system which combines depth and semantic segmentation self-supervised learning with 3D map building for task solving. While their quantitative results demonstrate impressive performance on the ALFRED benchmark, I'm doubtful whether their key assumption allows the system to work on real robots. 
issues:  - A key limitation of this work are their core assumptions, which is that "1) If a subgoal completes, ... until the robot drops it" (L137-143). This is an artifact of the AI2THOR environment, and may not generally be applicable in other simulators and in the real world. 
- From section 3.2, it is not clear how the self-supervised semantic model is trained. For instance, after reading the paper, I still have questions about why hungarian matching over a randomly initiallized MLP might make sense during the beginning of training, and how is the MLP's supervision signal derived in a self-supervised manner. Please provide more details.
- When the robot is deployed in home level tasks, achieving high enough resolution using voxel representations may cause prohibitive memory usage. While this 3D representation makes it straight forward to aggregate semantic and spatial uncertainties on a grid, it is also not scalable. 
- Does it make more sense to allow ECL-Oracle access to ground truth segmentation and depth rather than to ground truth segmentation and depth supervision? This would help gauge ECL's performance if its self-supervised depth and semantic segmentation was perfect.
- Mistake in L231 about objects that appear infrequently in the environment.
- I'm curious to see exposure frequency v.s. concept learning accuracy of object classes to back up statements in L229 - 232.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents embodied concept learner that can ground the objects included in the instructions and constructs a semantic map. Grounding is achieved by a minimum cost assignment problem between visual features and word embeddings. The semantic map is constructed by assigning the object probabilities to 3D space. Based on the semantic map and parsed instructions, the agent can predict actions.
strengths_and_weaknesses: # Strengths
- ECL achieved high performance in unseen test tasks of ALFRED. This result indicates that the ECL has high generalizability. 

# Weaknesses and suggestions
- First of all, I have a question about the purpose of this study written in lines 47-56. This study aims to learn knowledge from expert demonstrations without supervision. However, this study uses the labels in ALFRD, and this is considered different from learning by a baby. 
- For a similar reason, if demonstrations mean the labels in ALFRED in this study, I think these are not demonstrations but dis ambiguated labels. 
- Sec 3.2: How were the MLP that projects visual representation into semantic space of word learned? If this projection is already learned, this leads that the agent already has knowledge about the objects and their names. 
-  Figure 3: I do not understand the difference between learned encoding and word embedding. 
- Lines 229-231: typos?

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think this paper should be revised to be accepted. 
issues: Described in the strengths and weaknesses section.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents Embodied Concept Learner (ECL), a self-supervised framework for grounding concepts and following instructions in 3D environments. The framework is set in the ALFRED instruction following environment. Instead of relying on ground-truth segmentations, ECL makes connections between instructions, observations, and actions to learn object-centric concepts like “knife” and “soap bottle”. Similarly, for estimating depth, the method uses an unsupervised-learning approach. Together, the learnt concepts and depth predictions are fused with a Bayes filter to build a spatial-semantic map of the scene. This map is used with an executor module to solve instruction following tasks. The experiments include evaluations on the ALFRED benchmark with metrics like success and path-weighed success on seen and unseen scenes. While the proposed approach does not achieve state-of-the-art performance, it achieves compelling results without using any ground-truth segmentations or depth. 
strengths_and_weaknesses: **Strengths**

+ Overall, the idea of using self-supervised learning to learn object-centric concepts to aid embodied agents is an interesting and novel approach. Most existing methods for the ALFRED challenge make heavy use of ground-truth segmentation masks and perfect depthmaps from the simulator. Such supervision is often hard to obtain in real-world robotics where it’s difficult to pre-define a set of object categories, or obtain noiseless depth estimates from onboard sensors. As such, ECL takes a good step towards realizing approaches that have been prevalent in the Embodied AI community for instruction following tasks. 

+ The experiments include a good set of comparisons to prior work on the ALFRED benchmark. Additionally, the results also include ablations investigating various aspects of ECL like Bayesian Filtering, word embeddings, and an analysis on common failure cases. If the authors open-source their implementation, these results should be easily reproducible.

+ The results in Section 4.2 and 4.3 are interesting. Without any explicit object-level supervision, the learnt concepts can be re-used to query questions like “how many sofas are in the room?”. Future works could follow up these investigations by studying the generalizability of bottom-up perceptual representations that are learnt not through explicit supervision, but from environmental interactions.

**Weaknesses**

- One major concern is that ECL’s approach to self-supervised concept learning might be a bit specific to the ALFRED environment. As described in Section 3.2, the training data is composed of frames from before and after interacting with objects. These interactions are observed through discrete changes in image observations, e.g. picking up an object immediately zaps the object to the agent’s visual frame in the center. Such obvious visual cues might not be apparent in real-world settings, especially if the agent is interacting with objects through a manipulator. 

- Section 3.2 is missing details on how exactly the concept learner is trained. The word embeddings and visual representations are fine-tuned with MLP layers, but they are matched with the Hungarian algorithm. How is gradient descent used with the Hungarian algorithm? And what is the training loss? Without these details, it’s hard to understand how exactly ECL works in practice. Similarly, is the Program Executor (in Section 3.4) a hand-coded policy specifically designed for ALFRED tasks? If there is no learning or planning involved here, again, such hand-coded policies might be specific to ALFRED-only tasks. 

- One minor concern regarding the prose is the use of frequent analogies comparing ECL’s approach to “human-like” and “baby-like” learning. While it’s great that ECL is inspired by theories in cognitive science, and a lot of its core ideas are rooted in cognitive development, some references like “Humans … group[ing] pixels regions without knowing their semantics” and making an “agent behave like a baby” with a Bayes filter for mapping – is unsubstantiated. These analogies could be rephrased or included with proper citations to cognitive science literature to back-up the claims. 

robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: ECL presents a novel approach for learning object-centric concepts without explicit segmentation masks or depth supervision. ECL’s contributions might be limited to ALFRED tasks, but nonetheless they might be broadly useful to the embodied AI and robotics community. 

**Post Rebuttal**  
These responses address most of my concerns. I am still a bit skeptical about some of the cogsci claims made in the paper like "humans ... [group]ing pixels" and equating Bayes-filter mapping to how humans map and navigate. I checked out the references the authors mentioned, and I couldn't find any evidence of "pixel" representations or Bayesian-filtering. But overall, I think the contributions here are still of interest to the embodied-AI/robotics community, and so I will keep my score of Weak Accept.
issues: See above on the concern regarding the approach being specific to ALFRED and missing details.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper addresses the problem of embodied concept learning, whose aim is to learn concepts through interaction in an interactive 3D environment. Specifically, this work focuses on learning to ground visual concepts from human demonstrations and language instructions and enabling embodied instruction following. To this end, the paper proposes a framework that parses an instruction into a sequence of subtasks, grounds visual concepts based on language descriptions, leverages learned concepts to estimate the depth and construct semantic maps, and fulfills each subtask with a crafted rule-based policy. The experiments on the ALFRED benchmark show that the proposed framework outperforms baselines when no depth and semantic label is provided in terms of success rate and goal-condition success. I believe this work explores a promising research direction (i.e. embodied concept learning) and presents an effective framework to address the problem. 
strengths_and_weaknesses: ## Paper strengths and contributions
**Motivation and intuition**
- The motivation for studying embodied concept learning is convincing.
- Studying embodied concept learning from and for embodied instruction following task seems like a decent choice.

**Technical contribution**
- Utilizing an objects proposal network to obtain detected objects makes sense and seems to be effective
- Using the Hungarian maximum matching algorithm seems effective for matching detected objects and extracted word embeddings.
- The map constructor that builds up semantic maps based on estimated depths and grounded visual concepts seems to incorporate the information well.

**Clarity**
The overall writing is very clear. The authors utilize figures and examples well to illustrate the ideas. 

**Experimental results**
- The presentation of the experimental results is very clear.
- The experimental results on ALFRED show that the proposed framework (1) outperforms baselines when no depth or semantic label is available, (2) generalizes better between test-seen and test-unseen, and (3) achieves competitive performance when depth supervision is used.
- The qualitative results in both the main paper and the appendix are well-analyzed and interesting.

**Ablation study**
Ablation studies justify the effect of the language representation in concept learning and the mapping strategy in map construction. 

## Paper weaknesses and questions

**Oversell**
I feel the authors oversell the paper a little bit. After reading the abstract and the introduction, 
I thought the proposed framework learns from scratch from only interactions and human demonstrations.
Yet, word embeddings from a pre-trained BERT model are utilized which can provide relationships among objects and an object proposal network is leveraged which has learned low-level visual features and how to group pixels.
In my opinion, the authors can tone down the key motivation a little bit so that the readers do not get disappointed.

**Instruction-program pairs**
This work fine-tunes a pre-trained BERT model to convert an instruction to a program (i.e. a sequence of subtasks) using paired data from the subtask sequence annotations in ALFRED. I wonder if the instruction parser can learn from the task completion signal when there is no instruction-program pair available, or if paired data is just required.

**Instruction parser's accuracy**
I wonder how accurate the instruction parser is. It would be great if this information can be provided.

**Training object proposal network**
How was the object proposal network trained? Was it trained on data from ALFRED? I wonder how accurate it has to be to allow the whole framework to work. Also, is it fine-tuned during the learning?

**Learning continuous concepts**
This work presents an effective framework to ground visual concepts by matching them to language descriptions in a "discrete" manner. I wonder how we can learn continuous concepts or even dynamic concepts.

**Deterministic (rule-based) policy**
This work incorporates a simple deterministic policy. I wonder how this can scale up to environments that require longer-term exploration.

**Related work**
I feel the authors can include the following papers to make the related work more comprehensive:
- Synthesizing Environment-Aware Activities via Activity Sketches: it generates program sketches from instructions. 
- Learning to Synthesize Programs as Interpretable and Generalizable Policies: it discusses interpretability and generalizability. 

====== After rebuttal ======

I thank the authors for the response and the revision, which addresses of my concerns. I have carefully read other reviews and decided to keep my original rating.


robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I believe this work explores a promising research direction (i.e. embodied concept learning) and presents an effective framework to address the problem. While I have a few minor concerns stated above, I enjoy reading this paper and would love to see this paper included at the conference.
issues: Described in the Strengths And Weaknesses section.
review_title: null

Meta-review: metareview: The novel interesting approach for self-supervised learning of object-centric concepts in challenging 3D environments is recognized and appreciated. Also the presented results in ALFRED are promising, especially the generalization to unseen tasks. Moreover the paper is presented in an understandable, well-structured way, supported by nice and helpful figures.

Nevertheless, there are also some concerns raised. Mainly on clarifications on the training scheme, the underlying assumptions and how tight the approach is tailored to ALRED only. Please see the following proposed action items based on those (more details in the corresponding reviews).

Proposal of action items:
- Clarifying the training procedures and details, the usage and accuracy of the pre-trained models
- Discussing the assumptions and transfer to other simulators/real world, or adding an additional experiment 
- Clarifying the usage of ‘labels’ in ALFRED
- L231 the same objects are referenced as hard to learn, but were easy to learn before?
- Adding cognitive science literature to support the human-like statements
- Rethinking the claims and wording with respect to the method, assumptions, and usage of pre-trained models

-----

I’d like to thank all the reviewers and the authors for constructive discussions and provided updates. There is a consensus that the presented work is tackling a challenging and important problem in the field of embodied AI, and an effective and promising framework is presented. The overall recommendations are to accept the paper.


confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)