Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work provides a temporal logic imitation method for robot operations, which can deal with both motion-level perturbations and task-level perturbations. In detail, a dynamic system is implemented along with collision boundaries for avoiding motion failure, and linear temporal logic is used for task-level failure detection and recovery. The experiments on real robots show the effectiveness of the proposed method, and the theoretical analysis of the dynamic system provides the guarantee of the G.A.S. of the DS and the satisfaction of the constraints (cuts).
strengths_and_weaknesses: This work provides a robot skill imitation method with the ability to recover from failures and perturbations. The proposed method successfully combines DS and LTL and provides theoretical analysis on the guarantee of system reliability. The detained diagram that illustrates the principle of the proposed method is clear and intuitive, which helps the reader quickly understand the main idea of this work, and the video presentation is also very impressive.

However, as maintained by the author, this method itself relies on well-designed sensor states for successfully classifying the mode, which would cause some problems when applying to more complex robot tasks. Also, the experiments are mainly on the soup-scooping task, further evaluation on more complex tasks is welcomed.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This work is well organized and easy to read, the provided video presentation and diagram can illustrate the proposed method clearly and intuitively. The experiments on the soup-scooping task are a little bit simple but impressive, where the analysis of the results is sufficient and can well support the main claim of this work.
issues: 1. How to design and choose the binary sensor state in practice?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents an approach to train sets of controllers to imitate demonstrations. The demonstrations can be segmented into sequences of steps separated by sensory events, leading to a representation of the tasks in the form of hybrid automata with sequences of sensory "modes". The method combines synergistically prior work on dynamical systems to obtain convergent controllers for each of the segments/steps, and prior work on temporal-logic synthesis to map the demonstrations into hybrid automata, to obtain convergent policies both in the motion level and in the task level (sequences of modes). The method is tested on synthetic data and with a proof of concept on a real robot.

strengths_and_weaknesses: Strengths:
- The paper is well written, although very tightly packed
- The community is in need of methods that move away from “just” action imitation into some forms of imitation with better abstractions and objectives
- The combination of control with convergence warranties with a hybrid automata is novel and interesting for the community.
- It includes a demonstration of the approach on a real-robot

Weaknesses:
- The experimental evaluation is rather limited to understand the strengths and limitations. Many of the concepts are “evaluated” in simple synthetic 2D point-agent domains that, while illustrative, are more a proof of concept. There is only one task with full-robotic complexity. This makes it hard to understand the weaknesses of the proposed approach. For example, how well would it scale to more complex automata? What is the complexity of finding and perceiving the sensor events to switch between modes (partially mentioned in the limitations)? How stable is the boundary in the switch between states of the automata (usually, the problematic parts)?
- There are a lot of concepts that are inherited from the theory of dynamic systems and temporal logic and that were presented before. While I appreciate the effort to make the paper self-contained with these explanations, the paper does not draw a clear line on what is novel and what is prior work applied here. A suggestion is to create a section “Preliminaries” to explain the used prior concepts that would make very clear what is new.
- As far as I understand, the videos/experiments with the robot show the same person that created the solution (and knows the limitations) perturbing the agent. Some of the perturbations seem to be studied and I don't know if they are similar to the ones applied to the (failing) baselines. While this is good as a proof of concept, it would be great to perform some experiments with unbiased participants. 


robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: It is a good contribution, well written and well motivated. The paper is technically competent, although it is hard to understand what is novel contribution and what is prior work in dynamical systems and temporal-logic. The experiments are illustrative but additional robot experiments (more tasks, unbiased perturbations) would help understand the strengths and limits of the presented approach.
issues: Mentioned in the weaknesses:
- Make clear what is prior work and what is novel contribution, possibly with a "Preliminaries" section
- Additional tasks and evaluation conditions on the real robot
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: Current learning from demonstration (LfD) techniques, do not guarantee success in the presence of significant perturbations, and rarely have the ability to recover from task failures.  In this paper, the authors frame this deficiency as a failure to satisfy the high-level task specification which is implicitly present in the given demonstrations.

By leveraging task "modes" (the sets of sensor states which correspond to relevant components of the task) in combination with a LTL-automaton and dynamical system, their method is able to satisfy the task in the presence of arbitrary perturbation, and even recover from perturbations which cause previous steps of a task-plan to be undone. 

In contrast with work in temporal logic planning, these "modes" are not given, but learned from the data with a quadratically constrained quadratic program to learn the boundary cutting planes of each mode.

Their experiments demonstrate their method with a soup-scooping task on a Franka robot arm.
strengths_and_weaknesses: # Strengths

The presentation is clear, and the formalizations are clean & understandable. The figures do a good job providing intuitive explanations of algorithm operation, and how it contrasts with traditional imitation learning methods.

The work is well situated --- alternative solutions to their problem are addressed and discussed (e.g., framing the problem as covariate shift and simply asking for more human demonstrations). The related work covers the relevant literature on temporal logic motion planning, imitation learning, and subgoal/long-horizon manipulation.

The contribution itself is strong, representing progress in augmenting LfD frameworks to incorporate task specification knowledge in a way that both robust to failures and aids recovery. In addition to empirical evidence for the value of their technique, the theoretic proofs clearly outline the assumptions under which their method can manage arbitrary perturbations.

The experiments were implemented on a real panda robot, and fair comparisons are made to NN-Behaviour Cloning policies to demonstrate the advantages of their method.

# Weaknesses

The weaknesses I can identify mostly come from a desire to see this work extended.

For one, while the soup scooping example is cleanly demonstrates their idea, I would have liked to see it applied to a range of varied tasks with expressive modes. 

Additionally, while the modes of the task are learned, the high-level sensor abstractions themselves used to learn these modes are given. It would be interesting to see if this approach could be applied to high-dimensional data-driven perception (e.g., streaming video pushed through a perception network). I note that this is mentioned by the authors in their own limitation section. 

Third, their use of GR(1) LTL as a constraint language inherently limits the types of cyber-physical problem that can be addressed with this approach. For plans involving continuous signals, timing constraints, or looser probabilistic guarantees, a more expressive language would likely be needed. I am unsure if transferring these techniques to something like P-STL would be a non-trivial endeavor.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I feel confident in recommending this paper for publication --- the presentation is clear, the techniques novel, it has a strong formal grounding, and is well placed within the existing LfD literature. 
issues: Paper is essentially good in its current form. I was left wondering about the limitations and expressive potential of the author's chosen constraint language (the GR(1) fragment of LTL). How broadly does this apply to robotic missions in general as compared to say Signal Temporal Logic, or Probabilistic Signal Temporal Logic? Perhaps this could get a brief mention in the limitations section.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents a temporal logic imitation approach which generates policies that satisfy a given linear temporal logic (LTL) specification.
Classical learning from demonstration approaches (LfD) can often not ensure task satisfaction in the presence of perturbations. 
To address this issue, the presented approach proposes a mode-based approach to satisfy the task while optimizing controllers that ensure stability. 
The task is decomposed into a set of modes, e.g., moving a spoon to a soup, scooping the soup, and moving to the goal pose.
Local controllers within each mode ensure stability in the presence of perturbations.  
The paper shows that the presented approach satisfies invariance and reachability for the modes. 
The approach has been evaluated in simulation and on a real robot in a soup-scooping task.
Results indicate that the approach significantly improves performance under various perturbations.
strengths_and_weaknesses: Robustly performing tasks under various perturbations is challenging. 
This paper presents an interesting and promising control-theoretic approach to address this challenge.
The approach builds upon sophisticated control theory.
The proposed task abstraction into modes is compelling and helps to decompose the task.
Constructing stable controllers for each mode to create plans is a key feature of this work.
The authors also provide various proofs and details on how to create the controllers and the convex operating regions in each mode.
The approach can work with various user-defined LTL formulas that specify how modes can be entered and how the system should transition from one mode to another. 
The authors provide detailed explanations on the mode and how to construct the controllers.
The approach has been evaluated in a small simulation with various controlled perturbations and on a real robot which is great to see. 
The submitted videos help to understand the approach better and showcase various aspects of the approach.

Although the paper is interesting, its scope is not really fitting the learning theme of the conference. 
The paper claims to address learning from demonstration. 
Yet, the approach focuses on control-theoretic aspects (both main contributions are Theorem 1 and 2) and learning is left out.
The only reference to learning is on learning the linearized dynamics through Bayesian non-parametric GMMs.
The demonstrations as well as the LTL specifications are assumed to be provided to the approach.
It would have been great to see more integration into learning from demonstration frameworks. 
The control theoretic details are great, but the paper does not really fit the scope of CoRL.
Moreover, the paper claims to focus on LTL specifications and it introduces the necessary foundations.
Except for one experiment, the paper does not talk about LTL anymore. 
Instead, specifications are assumed to be provided by experts. 
How can one create specifications that can be used in the approach?
Which conditions need to be fulfilled to specify transitions between modes?
Which limitations does the approach have?
In its current form, the paper does not provide enough details to justify the general claims of temporal logic imitation (the paper focuses on modes and constructing controllers).
Considering the applicability, the paper has focuses on a rather simple task for which the mode decomposition is straightforward.
How does the approach translate to other tasks?
What if the task contains time-variant elements, such as dynamic objects?
In the control sections, the paper needs to provide more details.
For instance, how are attractors chosen?
What kind of assumptions are made on the system/sensors?
What happens if the system's dynamics are highly non-linear and the approximation in the paper leads to large linearization errors?
The paper also misses to explain the motivation behind many modeling steps, e.g., why are the controllers chosen as presented?
Lastly, the experiment section is very short, and it is not possible for a reader to follow the details and contributions of the experiments.  
It would be nice if the paper can properly introduce the considered environment, system, and task.
The limitations sections can also be improved by not just stating straightforward future work, but discussing the limitations of the approach and its applicability to a wide range of tasks.

Considering the paper's style, it is mostly well written. 
Yet, the control-theoretic sections are very dense due to the page limit. 
The authors could remove the unused LTL definitions to gain space and add more explanations.
Moreover, the figures are very small, and it would be nice if the authors could add labels to them for easier reference. 
The numbering of equations 3 and 4 can also be improved since it is not directly clear where Eq. 3 is. 
robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: In summary, the paper presents an interesting control-theoretic approach that shows promising results.
However, the paper has no real focus on learning and seems out of scope for CoRL. 
The integration with learning from demonstrations is not really described in the paper.
Moreover, the approach does not really demonstrate temporal logic imitation.
The specifications are assumed to be provided and the paper does not explain how to create specifications for the use in the approach.
THe paper also does not discuss the temporal logic aspects in detail. 
The experimental section also needs to be revised to properly demonstrate the approach and claims in various environments and tasks.
issues: * Temporal logic imitation: How can one create specifications that can be used in the approach?
Which conditions need to be fulfilled to specify transitions between modes?
* Assumptions: which assumptions do the authors make in their approach? Which tasks can this approach be applied to?
* Experiments: More details on the task, the specification, and the evaluation criteria. 
* Approach: More details on choosing the attractors. 
* Limitations: What are limitations of the approach? When does the approach fail? Can it be applied to all tasks?
review_title: null

Meta-review: metareview: 
I invite the authors to respond to all issues and questions by the reviewers and make modifications accordingly. 

Excerpt of the reviews: 

Strengths:
* (OTy6) The community is in need of methods that move away from “just” action imitation into some forms of imitation with better abstractions and objectives
* (QTy6) The combination of control with convergence warranties with hybrid automata is novel and interesting for the community.
* (all) real robot experiments

Weaknesses / Issues:
* (QTy6) Make clear what is prior work and what is a novel contribution, possibly with a "Preliminaries" section
* (QTy6) Additional tasks and evaluation conditions on the real robot
* (VK5r, L7Wt) mainly one task: soup-scooping -> more tasks would be better
* (p7Tc) little to no relation to learning
* (p7Tc) not enough details (supply more information in the appendix)
----- post rebuttal
The authors have addressed weaknesses by improving the relation to prior work, adding more (unbiased) evaluation, and adding 2 different tasks. 
The paper should definitely be accepted and can be considered for an oral presentation (weak endorsement). 

I ask the authors to incorporate the changes into the main text as much as possible and make references to the new material in the supplementary at the appropriate places. 

confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Oral)