Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The paper addresses the problem of using multiple modalities for learning from demonstration. Approaches that take in task or joint space data to learn a policy for replicating that task are numerous. Doing the same with multiple modalities involved, in particular vision, language and motion, has only been recently considered, so this is a timely paper. 

The core contribution is pretty well summarised by the architecture in figure 1, which involves a combination of encodings of the words and sentences, images and parameters of a DMP in order to generate movement commands from a high level instruction. 

Unless I have missed something in the experimental setup, all of the considered task variations are movement commands of the form <Move> to <Object>. The network setup allows for synonyms of two kinds, so <Move> can be replaced by numerous verbal synonyms such as advance and go, and the object can be specified in terms of shapes, colors and so on, but otherwise this is the only specification of the task. This has been addressed in the recent literature using neural network architectures similar to the one being proposed here, e.g., see the following papers. These papers already solve the proposed problem and provide similar explanations. It would be helpful to see comparative discussion with respect to those methods and a clear statement of novelty with respect to such prior work:
[R1] M. Burke, S. Penkov, S. Ramamoorthy, From explanation to synthesis: Compositional program induction for learning from demonstration, Robotics: Science and Systems (R:SS), 2019.
[R2] Y. Hristov, D. Angelov, A.Lascarides, M. Burke, S. Ramamoorthy, Disentangled Relational Representations for Explaining and Learning from Demonstration, Conference on Robot Learning (CoRL), 2019. 

An interesting feature in R2 that the authors do not explicitly address here is the issue of relational specifications in the language, e.g., in addition to saying "move to the red bowl", we may also wish to say "place on top of red block". In the way that MPN is currently set up to map from the language input directly to hyperparameters of the DMP, and considering the embedding structure, it is not clear if MPN is capable of handling such specifications. If so, the claim of generalisation on the language input should be stated more clearly.

The ablation study is setup somewhat differently than what I would have expected. The authors consider the effect of changing the training set size and if the language input includes synonyms or not. Those two aspects seem to produce the expected results. It would also be interesting to see an ablation study in the sense of replacing or removing aspects of the architecture to see its relative effect on the overall model performance. So, for instance, if one did not have a DMP with the hyperparameters being estimated by a network and instead had a more straightforward encoding of where to move to - does it make a difference and how much? Likewise, how much performance benefit, if any, is being derived from an uninterpreted image I being combined as described in the embedding as opposed to an alternative that detects an objects and combines that position differently. The paper would have been stronger if such architectural choices were better justified and also demonstrated in the experiments.



review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: *Summary 

The paper describes a new end-to-end imitation learning method combining language, vision, and motion.
A neural network architecture called Multimodal Policy Network is proposed. That can extract internal representations from language and vision to condition the generated motions. 
It enables an end-user to influence a robot's policy through verbal communication.
The experiments demonstrate the generalization performance of the method. That can generate behaviors towards different goals depending on different sentences. 

*Decision and supporting arguments

I think the paper is just below the borderline. The reason is as follows.

The concern is about evaluation. They demonstrated the method could work, and the robot can move to appropriate goals. However, there is no comparative methods in the experiment.
Related to this point, the problem was not identified in the Introduction.
The authors might assume that introducing language into behavioral cloning itself is qualitatively new work. However, such a study has a long history. 
For example, please refer to Tani's pioneering works.
Sugita, Yuuya, and Jun Tani. "Learning semantic combinatoriality from the interaction between linguistic and behavioral processes." Adaptive behavior 13.1 (2005): 33-52.

The author should specify a current challenge or problem in pre-existing studies about imitation learning with language input, clarify their claim, and give empirical support for the claim.   

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This work uses imitations learning (from synthetic data) to train a deep model which takes a natural language instruction, and a visual representation of a robot's environment, and outputs a trajectory for the robot to follow which executes this instruction.  The work focuses on a robotic pick-and-place task, where the instruction indicates which of the available bins an item should be placed in.  In addition to the trajectory model, a second model is trained which allows the agent to predict whether a given command is actually feasible (i.e. whether the target bin exists).  Empirical results show a reasonably high success rate in placing objects in the bin specified by the instruction, though there is still room for improvement in cases where the shape o a combination of features is important to the selection of the correct bin. 

Rather than mapping directly from instructions and observations to control signals, the model trained in this work translates from an instruction, and an image of the agent's environment, to the parameters of a DMP controller.  The network therefore outputs the entire motion for the task in a single inference pass.  This approach would have advantages and disadvantages.  The DMP formulation ensures that the resulting trajectory is relatively smooth.  It also means that the network outputs a distinct goal configuration, which the DMP should reach (assuming the goal is feasible) regardless of the other motion parameters.  The use of a DMP output space, however, limits the model to generating relatively simple, goal-directed motions, and does not allow the agent to adapt to changes in the layout of the environment (which would only be observed in the static visual input).

As other work has considered visual instruction following (e.g. Misra et. al. "Mapping Instructions and Visual Observations to Actions with Reinforcement Learning") it would strengthen this work considerably to see a direct comparison between this method and existing approaches.  It is likely that the approach presented in this work is better suited to the specific problem of robot control, but it would be helpful to see if learning a low-level control policy directly can be successful in this context.

The work needs to expand on the discussion in the second paragraph of section 4, where human annotators were used to generate natural language instructions for different tasks.  The paper suggests that this data was not used directly to train the model, but was instead used to build a template for generating natural language instructions.  What this template looks like, and how it was constructed based on the human-generated data, remains unclear, and needs to be described in much more detail.
review_title: Official Blind Review #1

Meta-review: The present paper addresses the problem of imitation learning in multi-modal settings, combining vision, language and motion. The proposed approach learns an abstract task representation, and the goal is to use this as a basis for generalization. This paper was subject to considerable discussion, and the authors clarified several issues that reviewers raised during the rebuttal phase. Overall, the empirical study presented in the paper remains limited, for example in terms of ablations (which components of the proposed model have what effect on performance) and placement in the context of prior work. As a result, the depth of insights is not yet sufficient for publication.
Decision: Reject