Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a method for learning skills from multiple diverse demonstrations using an approach that infers multi-modal distributions in a geometric feature space.  It is motivated by skills that can inherently be performed/taught in different ways and a robot may be (and perhaps should be) learning from a dataset that leverages multiple (human) demonstrators.  The method is evaluated against several imitation learning baselines on a reaching task and a pushing task.
strengths_and_weaknesses: This paper addresses an important, open problem in the Learning from Demonstration (LfD) literature.  It is very relevant in the age of big data, as we can much more feasibly scale up the number of humans to collect data from.  The method proposed is principled and builds from existing literature.  My primary questions are around the evaluation of the method.  

Overall strengths and suggestions for improvement are summarised below.

More detailed strengths of paper:
+  The method proposed is mathematically principled and allows for adaptability to multiple learning contexts.  It models a set of demonstrations as multi-modal distributions (Gaussian Mixture Models) in order to capture diversity/versatility in the way humans provide successful demonstrations of the task. Furthermore, it learns multi-modal distributions conditioned upon learning context and trains a discriminator to infer the correct context. This allows the method to account for more diverse contexts. 
+ The paper evaluates its method on real robot platforms.
+  Figure 4 (right) is a nice result.  It empirically confirms that the proposed method can better leverage (and hence take advantage of) variation in training contexts, as compared to baseline imitation learning methods tested.

I would suggest the following to improve the paper submission:
-  RE the evaluation: How and why were these particular tasks (reaching and pushing) selected?  What interesting properties does each exhibit with respect to what it highlights about the difficulty of the problem or insights about this method as compared to other imitation learning methods, on inherently multimodal learning tasks?  What hypotheses were being tested with each of the experiments? 
-  RE the experimental results shown in Figures 3 & 4: Any insights as to why there was significantly more variance in how the methods performed in the reaching tasks as compared to the pushing task?  For the reaching tasks, I also was a bit unclear. Is the entire demonstration trajectory used for learning or just the final goal position? If the latter, I’m not entirely clear on where the multi-modality in the data is expected to derive from and would request more clarity on this.  In particular, if only the goal position is used as learning signal for the task, what geometric features are expected to vary sufficiently as to induce a multi-modal distribution?  
-  Regarding the step of the method that performs distribution matching of behavioural descriptors (subsection 4.2), this seems related to the idea of optimising weights for feature expectation matching in Inverse Reinforcement Learning. I would be interested to understand how these are similar and different?  Or perhaps, does the proposed method build upon feature expectation matching imitation learning approaches?  And if so, in what ways? Why is generalising to geometric representations a difficult challenge?
-  The first sentence under Section 4 (page 4) is causing confusion for me.  Does saying that the approach estimates weights for all training and test configurations imply that the method has access to test configurations at training time?  That would seem like “cheating”, from the perspective of generalising the learned model.  I would recommend clarifying what is intended here.

Additional minor comments:
-  On page 1 (line 30), what does context-insensitive mean in this context?  Is this synonymous with saying context-independent?
-  In the Experiments section (page 5, line 175), why n=5 components? How was this selected?
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I would recommend accepting the paper, based upon its contribution of a method for addressing inherent multi-modality in skill learning from demonstration.  This is an important and (to my knowledge) currently open problem in the Imitation Learning literature.
issues: See limitations above.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes an imitation learning approach to capture the multi-modality (versatility) of expert demonstrations. This paper focuses on the trajectory generation aspect of imitation learning while conditioning on relative geometric features such as relative distance to the goal positions.
strengths_and_weaknesses: Strengths:
- I can’t tell the strengths of the approach given that it’s missing  a very important baseline, as mentioned in the “weakness” and “issues”

Weakness:
- The work is about learning behaviors / skills / motion primitives, but not a plan technically. The title itself is very misleading.
- It’s missing a very important baseline, behavior cloning + gaussian mixture model (BC-GMM) [1]. This previous approach can outputs GMM actions, and the model is trained with a very simple supervision loss. Solely comparing against vanilla BC is insufficient.
- It’s very vague about the term “context”. What is exactly the context here? Different goals? Then why not just say “goal-conditioned”? There is no clear semantic of “context” in any of the tasks shown. Please be more specific about the definition of “context” (possibly with rigorous definition in the text) or use other terms without ambiguity.
- The choice of comparison baselines is also ambiguous. What is the purpose of comparing against IRL? The authors are not learning rewards for their approach, then why compare against IRL? 
- all the tasks involvely only free motion generation (reaching, and pushing but not really involve contact-rich behaviors), and not in the closed-loop manner. How does the approach generalize to pick-place behaviors, and more complicated pushing behaviors?

[1] Mandlekar et al. What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Given that one of the basic baselines is missing to showcase the proposed formulation is indeed necessary to learn versatile behaviors, I vote for weak reject but will change the review if the authors address all my comments and issues properly.
issues: - revise the title to be precise about the choice of words.
- missing related works, especially related to the reference that has BC-GMM. And lack comparison to this baseline
- Revise the experiment session to explain the reason of the specific choices of baselines. This also helps to illustrate what the contribution is exactly.  
- The last paragraph of the related work is not well-written. The paragraph name needs a more precise summary other than “further relevant approaches”. 
- Some grammar issues: “Few modes” -> “a few modes”. 
- Explicitly state whether this approach can be applied/extended to pick/place behaviors. If it can, what are the major changes to the approach the authors need? Otherwise, this approach would only be incremental to solve pushing / reaching tasks that are not really complicated not all.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper is concerned with the problem of using imitation learning (IL) to find behavior policies that allow machines to accomplish tasks when the set of demonstrations that should be imitated are "versatile," meaning that it contains multiple ways to accomplish a particular task. The paper proposes a new IL method called VIGOR which is based on probabilistic motion primitives and distribution matching over "geometric features," and presents an evaluation of this new method for both simulated and real robot arm tasks.
strengths_and_weaknesses: (+) The paper does an excellent job of illustrating the problem of "versatile" demonstrations and performs experiments using real human demonstrations, which is compelling. 

(+) Experiments are performed on both simulated and real robots. 

 

(-) It's not clear to me how strong the contribution of the paper is here. The introduction of the paper makes it seems as though the authors are claiming that the use of geometric features constitutes the novelty of the proposed method, but merely representing the state features of a problem in terms of, eg, distance to goal, seems commonplace in the community. If the authors disagree, could they provide citations to important recent papers that do _not_ do this? Additionally, from the ablation studies, it also seems important for VIGOR that the demonstrations be "filtered" by first fitting a ProMP; are the authors claiming that doing this is also a novel part of their approach, or has this been done in other work? An explicit list of claimed contributions—along with justifications for why these are indeed contributions—would go a long way to improving the paper. 

(-) The technical discussion needs to be improved for clarity. For example, Section 3 starts off by defining a trajectory $\tau = {\tau_1, …}$, but never actually defines for the reader what type of mathematical object $\tau_t$ is (eg, is it a scalar? a vector? what specifically does it encode? a state?). The same goes for the precise meaning of $\phi$ and $O$. This missing information makes it hard for me to evaluate the proposed algorithm. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The problem of performing imitation learning in the presence of "versatile demonstrations is interesting and the paper's illustration of this phenomenon, along with interesting experiments is good. However, the specific contributions to the literature are unclear to me and the technical discussion also needs to be improved for clarity.

POST-DISCUSSION UPDATE: During the discussion phase, the authors addressed several of my questions, and made corresponding revisions to the paper. I now think I have a better understanding of the novelty of the paper here, and am raising my score to "Weak Accept." However, I still think that the authors can do a better job in the presentation here by more explicitly focusing on the truly novel portion of their algorithm as opposed to the features that it relies on having access to. I understand this is an important point, but the authors have provided very little here to the community regarding how to obtain these features.
issues: (see weaknesses identified above)
review_title: null

Meta-review: metareview: A method is proposed for learning skills from diverse demonstrations, with a focus on inferring
multi-modal distributions and grounded in geometric features.  This allows the method to learn
multiple ways of performing a task, as learned from diverse human demonstrations.

The paper has three reviews, with recommendations of:  weak accept, weak accept, weak reject. Two of the reviewers actively engaged in the discussion.  The reviewers ultimately point to a lack of clarity in positioning the work as being a primary weakness of the paper.
The paper is borderline based on the current reviews and reviewer discussion.  

My recommendation is to Accept (Poster), based on importance of the problem (encoding geometric features in a meaningful way into robotic tasks is important, as is multimodality), that a key benchmark had in fact already been included, and that the presentation has now been significantly improved.

Strengths:
- principled approach to an important open problem in learning-from-demonstration
- evaluated on real robot platforms, and with real human demonstrations
- demonstration of advantage of the training variation, as compared to baselines

Weaknesses:
- missing behavior cloning + gaussian mixture model (BC-GMM) baseline comparison, e.g., see
  "What Matters in Learning from Offline Human Demonstrations for Robot Manipulation";
  Comparing against a vanilla BC method is insufficient.  Relatedly, the choice of baselines used
  should be justified  (rebuttal:  this was included, will rename to make it more evident)
- not clear if it generalizes beyond free-motion generation (rebuttal:  demonstrated push tasks are already quite different)
- contribution is somewhat unclear:  needs explicit list of contributions;  
  is the use of geometric features part of the claimed novelty?  (rebuttal: provided;  first to use concise geometric features for Adversarial Imitation Learning)
- title may be misleading, as the work is about motion primitives and not really about planning  (new title: "Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors")
- paper would benefit from further discussion & insights on various points  (many details added)

confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)