Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper describes a new method for collecting demonstrations for long-horizon tasks in mobile manipulation. The approach manages the gap between demonstrator observations and the systems observations. The authors use this to collect a large dataset of long-horizon demonstrations and achieve state-of-the-art results by imitating them. The authors also propose an error detection method to deal with the potentially large shift in states observed between the expert demonstrations and the learned policy. 
strengths_and_weaknesses: Strengths
--------------

The paper is well-written and gives a good characterization of related work. 

The demonstration collection method looks novel, non-trivial, and effective. The dataset collected looks like it could be quite useful to the community. 

The policies the authors train go beyond the state-of-the-art in policy learning for long-horizon tasks. This is an important area for the community and the results are impressive. 

Weaknesses
-------------------

While the error detection method looks to be useful and clearly helps with imitating the data, the error detection method itself is not really evaluated. In particular, the authors could evaluate their design choices for the error detection. This is important if the authors wish to claim the error detection method as a contribution.

I found the discussion of the few-shot generalization results to be too short. I recommend the authors expand this section of the text (l.307-322). In particular, including some details about the drop in performance to reduce the burden on the reader.

Minor Comments
------------------------
[l.173] "110 successful demonstration per from" -- missing word
[l.188] in a POMDP, $\pi$ is a function of the full history, not just the last observation
[l.240] "Algorithm 1 formalizes this method" --- the paper does not appear to have an algorithm 1


summary_of_recommendation: As my review indicates, I'm a big fan of this paper. It makes an important step forward and contributes resources that will help the community progress. I will advocate for the paper's acceptance, if only selfishly so that I can build on the dataset and tools they have provided. 


Update after rebuttal
----------------------------
I thank the authors for addressing my comments and I am happy they have included more discussion of the error detection method they propose. 
issues: In my read through the paper, I did not see discussion of publishing the dataset of demonstrations. Can the authors please clarify how they will share this resource (or point me to the place in the paper where I have missed it)?

Please address the minor comments.

Please provide more analysis of the few-shot results. 

Consider adding evaluations of the design choices in the error detection method.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper proposed imitation learning (behavioral cloning (BC) more specifically) with an error detector by variational autoencoder (VAE).
In order to automate mobile manipulation, the authors first developed a teleoperation framework, MOMART, for collecting huge dataset, which is utilized for imitation learning.
Since mobile manipulation is a long-term operation, the effect of covariate shift, which is a shortcoming of BC, becomes more pronounced.
To prevent malfunctions caused by this, the authors additionally implemented the error detection mechanism based on the recovery accuracy of VAE.
Although it is unclear how high the success rate should be, the authors showed a relatively high percentage of error detection.
strengths_and_weaknesses: Strengths:
- The authors collected a huge dataset of mobile manipulation that had not yet been constructed, although it was simulated.
- Focusing on that the observation of the dataset is from POMDP, the authors combined BC with TieredRNN to use the observation history appropriately, which makes the task successful with some probability.
- The VAE-based error detection mechanism achieved the high error detection rate even in the generalized setting.

Weaknesses:
- The newly introduced error detection mechanism has not improved the success rate of the task itself.
- Its implementation is also naive, and there is room for improvement.
- The authors have only verified the developed framework, and have not sufficiently examined and compared other possible implementations.
summary_of_recommendation: There is not much novelty in the technology, and it is not a good proposal because various patterns of error detection by VAE have been proposed so far.
On the other hand, the construction of a large dataset of long-horizon mobile manipulation tasks, which is closer to real-world problems, is very useful, and I commend the authors for this.
If qualitative or experimental comparisons on error detection are added, the paper seems to be worthy enough to be accepted.
issues: - I think the error detection based on the reconstruction accuracy has a disadvantage that the threshold setting can be complicated depending on the observation information.
There are several existing methods based on latent variables, and I would like to see a comparison with such methods.
- The benefits of error detection to the task accomplishment were not sufficiently demonstrated. 
I would like to see a comparison for indicators other than success rate (such as the load caused by the misoperation).
- The latent prior distribution and the output distribution of BC are assumed to be mixture distributions, but how do the authors give the number of mixtures? Isn't it task-dependent?
- What kind of observations are included in the constructed dataset?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents an innovative imitation learning method with error detection in the context of mobile manipulation. It proposes a novel architecture, TieredRNN, which is a multi-layered variant of BC-RNN. It is able to perform significantly better than the current SOTA methods for a series of real-world mobile manipulation tasks. On top of that, it presents a way to detect out-of-distribution (OOD) inputs using VAE reconstruction error and perform intervening error corrections for the agents.
strengths_and_weaknesses: Strengths:

- It is quite impressive that BC based methods perform much better than RL based baselines in a challenging domain like mobile manipulation. It shows the effectiveness of BC + RNN in long sequential tasks and gives room for future exploration into this line of work.

- Successfully evaluated the effect of suboptimal demonstration data

- Presented a novel error detection methods based on VAE construction error

Weaknesses:

- On choice of model architectures: from the experimental results, it is not entirely clear to me how TieredRNN perform better than BC-RNN, and it seems that BC-RNN (which I assume to be a simpler structure) should already give quite impressive results. Is there any reason that the authors choose the more complex model structure of TieredRNN over simplicity? It would be better if there could be more explanations in the paper.

- On experimental results: According to Table 2, the error detection results are quite impressive, but they do not help success rate that much. In fact in many sections, the success rate decreases because of the error detection. How might the authors explain the lack of improvement over task success rates?

summary_of_recommendation: The paper is a novel imitation learning method that is applied to mobile manipulation which comes with effective error detection mechanisms for out of distribution inputs. Future improvements could be made from clearer explanations of the significance of experimental results, choice of model architectures, and more insights into error correction strategies.  
issues: Related to the Weaknesses section above:

- More detailed explanation and justification of presenting the TieredRNN methods

- More justified explanation on how error correction relates to success rate

On error correction strategies: 

- The current interventions when an error is detected are hardcoded programming if I interpret it correctly - only "recover" and "terminate". I'm curious of whether the authors have considered more versatile intervention methods, e.g. human monitoring and tele-operated corrections?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper addresses learning mobile manipulation tasks by 1) building a smartphone teleoperation interface that enables human operators to demonstrate mobile manipulation tasks in simulation, 2) training a visuomotor policy on such collected demonstrations, and 3) training an error-detection classifier to detect when the learned policy had incurred an unseen state and either attempts to recover or terminates task execution. Experiments were done in a simulated kitchen domain, where a fetch robot was tasked to transport objects by interacting with a variety of articulated objects. A total of 1200 demonstrations were collected, and the learned policies achieved task success rate of 45%.
strengths_and_weaknesses: Strengths:
-	The paper’s motivation is well founded – learning long-horizon mobile manipulation tasks could benefit from more robust human priors.
-	The fact that the demonstrator is not using privileged information is a plus – both the demonstrator and the robot is given the same single-camera view.
-	An easy to use interface for providing mobile manipulation demonstration data is clearly beneficial for the field.
-	The paper is clearly written.

Weaknesses:
-	My main criticism is the paper’s seeming lack of focus. While it builds a system that can collect mobile manipulation demonstrations and train imitation learning policies, it does not study these components in depth. 
-	On the teleoperation interface, a more thorough user study would have made the paper more convincing. There are many questions such a study can help answer. What is special about giving demonstrations for mobile manipulation as compared to stationary demonstration? Were task strategies pre-determined or did each demonstrator come up with their own? How varied were the demonstrations, and how noisy were the collected trajectories? Were any data cleaning and filtering done to preprocess demonstration data? What’s the success rate of human demonstrators? Could the camera be moved independently of the robot? How many trials did it take for a new human operator to achieve task success? 
-	On imitation learning, while the use of RNN seems appropriate given the partial observability aspect of single-view camera inputs, it would still be helpful to run a baseline comparison against a non-RNN version with a window of observations as input. The claim that the Tiered-RNN architecture performs better than the generic RNN architecture is also not obvious, at least not from Table 2.
-	On covariate shift - the use of error detection to address covariate shift is not entirely satisfying. There are many well established and simple-to-implement methods for reducing covariate shift, from the DAgger algorithm to noise injection during demonstrations, and it is unclear why the authors did not experiment with them. One potential justification may be that the paper is not completely focused on robust imitation learning, but that further highlights the lack of focus in the paper

summary_of_recommendation: I recommend weak reject. In my opinion, the only novel part about the paper is its mobile manipulation teleoperation interface (and perhaps the 1200 collected demonstrations), but the paper does not perform thorough experiments and analyses of this interface. The results from the trained manipulation policies, while interesting, do not bring a new understanding of the problem.

**Update** After the author's response with additional clarifications and the new user study, which helped to address my original concerns and better contextualize the paper, I am changing my recommendation to a weak accept.
issues: Please see Strength and Weaknesses for revision suggestions. The website link was not operational at the time of review.
review_title: null

Meta-review: **After rebuttal**
I thank the authors for their thorough responses, my concerns have been addressed, and the reviewers concerns seems to have been addressed to the reviewers satisfaction.
Thus I recommend accept.

**Initial Meta-review**
**Summary**

This work presents a teleoperation system for long horizon tasks performed by mobile manipulation robots. This teleoperation framework is utilized to collect a continuous control dataset in a simulated kitchen domain consisting of over 1200 successful demonstrations across five long-horizon tasks.  The author then present initial imitation learning results on this dataset. The imitation learning process is augmented with a learned error detector which can detect when the agent is in a failure state.

**Strengths**:
This work contributes a system for mobile manipulation data collection, which is an important and active research are in the robotics community

The manuscript is well written and the work is well motivated 

The work introduces a variant of behavior cloning that leverages hierarchical abstractions, and show that imitation learning with this variant outperforms some relevant baselines 

**Weaknesses**:

This work also introduces an error detection method, however it is barely evaluated. 
All reviewers are missing some more detailed analysis and discussion on various aspects of the evaluation
Decision: Accept (Poster)