Temporal Logic Imitation: Learning Plan-Satisﬁcing Motion Policies from Demonstrations
Abstract: Learning from demonstration (LfD) has successfully solved tasks featuring a long time horizon.
However, when the problem complexity also includes human-in-the-loop perturbations, state-of-the-art approaches do not guarantee the successful reproduction of a task.
In this work, we identify the roots of this challenge as the failure of a learned continuous policy to satisfy the discrete plan implicit in the demonstration.
By utilizing modes (rather than subgoals) as the discrete abstraction and motion policies with both mode invariance and goal reachability properties, we prove our learned continuous policy can simulate any discrete plan speciﬁed by a linear temporal logic (LTL) formula.
Consequently, an imitator is robust to both task- and motion-level perturbations and guaranteed to achieve task success.
Project page: https://sites.google.com/view/ltl-ds
Introduction
In prior work, learning from demonstration (LfD) [1, 2] has successfully enabled robots to accomplish multi-step tasks by segmenting demonstrations (primarily of robot end-effector or tool trajectories) into sub-tasks/goals [3, 4, 5, 6, 7, 8], phases [9, 10], keyframes [11, 12] or skills/primitives/options [13, 14, 15, 16].
Most of these abstractions assume reaching subgoals sequentially will deliver the desired outcomes; however, successful imitation of many manipulation tasks with spatial/temporal constraints cannot be reduced to imitation at the motion level unless the learned motion policy also satisﬁes these constraints.
This becomes highly relevant if we want robots to not only imitate, but also generalize, adapt and be robust to perturbations imposed by humans who are in the loop of task learning and execution.
LfD techniques that learn stable motion policies with convergence guarantees (e.g., Dynamic Movement Primitives (DMP)
[17], Dynamical
System (DS) [18]) are capable of providing such desired properties but only at the motion level.
As shown in Fig.
1 (a-b) the robot can successfully replay a soup-scooping task while being robust to physical perturbations with a learned DS.
Nevertheless, if the spoon orientation is perturbed to a state where the material is dropped, Fig.
1 (c), the motion policy will still lead the robot to the target, unaware of the task-level failure or how to recover from it.
To alleviate this, in this work, we introduce an imitation learning approach that is capable of i) reacting to such task-level failures with
Linear Temporal Logic (LTL) speciﬁcations, and ii) modulating the learned DS motion policies to avoid repeating those failures as shown in Fig.
1 (d).
Example We demonstrate that successfully reaching a goal via pure motion-level imitation does not imply successful task execution.
The illustrations in Fig.
2 represent a 2D simpliﬁcation of the soup-scooping task, where task success requires a continuous trajectory to simulate a discrete plan of consecutive transitions through the colored regions.
Human demonstrations, shown in Fig.
2 (a) , are employed to learn a DS policy [19], depicted by the streamlines in Fig.
2 (b) .
The policy is stress-tested by applying external perturbations, displacing the starting states of the policy rollouts.
As shown, only blue trajectories succeed in the task, while the red ones fail due to discrete transitions that are not physically realizable (e.g., white ) pink).
As shown in Fig.
2 (c-f) , even if the demonstrations are further segmented by subgoals (and corresponding DS policies are learned), this issue is not mitigated.
While one could treat this problem as covariate shift and solve it by asking the humans for more demonstrations [20], in this work, we frame it as the mismatch between a learned continuous policy and a discrete task plan speciﬁed by the human in terms of a logical formula.
Speciﬁcally, the core challenges illustrated by this example are two-fold: 1) subgoals only impose point constraints that are insufﬁcient to represent the boundary of a discrete abstraction; and 2) the continuous policy can deviate from a demonstrated discrete plan when perturbed to unseen parts of the state space, and is incapable of replanning to ensure all discrete transitions are valid.
To address these challenges, our proposed approach employs “modes” as the discrete abstractions.
We deﬁne a mode as a set of robot and environment conﬁgurations that share the same sensor reading [21, 22]; e.g., in Fig.
2 each colored region is a unique mode, and every mode has a boundary that imposes path constraints on motion policies.
Additionally, we use a task automaton as a receding-horizon controller that replans when a perturbation causes the system to travel outside a mode boundary and triggers an unexpected sensor change; e.g., detecting a transition from yellow ) white instead of the desired yellow ) pink will result in a new plan: white ) yellow ) pink ) green.
In this work, we synthesize a task automaton from a linear temporal logic formula (LTL) that speciﬁes all valid mode transitions.
We denote the problem of learning a policy that respects these mode transitions from demonstrations as temporal logic imitation (TLI).
In contrast to temporal logic planning (TLP) [23], where the workspace is typically partitioned into connected convex cells with known boundaries, we do not know the precise boundaries of modes; consequently, the learned policy might prematurely exit the same mode repeatedly, causing the task automaton to loop without termination.
To ensure any discrete plan generated by the automaton is feasible for the continuous policy, the bisimulation criteria [24, 25] must hold for the policy associated for each mode.
Speciﬁcally, every state starting in the same mode should stay in the mode (invariance) until eventually reaching the next mode (reachability) .
The violations of these conditions are referred to as invariance failures and reachability failures respectively.
Contributions First, we investigate TLP in the setting of LfD, and introduce TLI as a novel formulation to address covariate shift by proposing imitation with respect to a mode sequence instead of a motion sequence.
Second, leveraging modes as the discrete abstraction, we prove that a statebased continuous behavior cloning (BC) policy with a global stability guarantee can be modulated to simulate any LTL-satisﬁcing discrete plan.
Third, we demonstrate that our approach LTL-DS, adapts to task-level perturbations via a LTL-satisﬁcing automaton’s replanning and recovers from motion-level perturbations via DS’ stability during a multi-step, non-prehensile manipulation task.
Related Works
Temporal Logic Motion Planning LTL is a task speciﬁcation language widely used in robot motion planning [26, 27, 28, 23].
Its ease of use and efﬁcient conversion [29] to an automaton have spurred substantial research into TLP [25, 30, 31], which studies how to plan a continuous trajectory that satisﬁes a LTL formula.
However, TLP typically assumes known workspace partitioning and boundaries a priori , both of which are unknown in the rarely explored TLI setting.
While a robot can still plan in uncertain environments [32, 33], LfD bypasses the expensive search in high-dimensional space.
Recent works [34, 35] have considered temporal logic formulas as sideinformation to demonstrations, but these formulas are treated as additional loss terms or rewards, and are not guaranteed to be satisﬁed.
The key motivation for using LTL is to generate a reactive discrete plan, which can also be achieved by a ﬁnite state machine [14] or behavior tree [36].
Behavior Cloning We consider a subclass of LfD methods called state-based behavior cloning (BC) that learns the state-action distribution observed during demonstrations [37].
DAGGER [20], a BCvariant ﬁxing covariate shift, could reduce the invariance failures depicted in Fig. 2, but requires online data collection, which our framework avoids with a LTL speciﬁcation.
To satisfy goal reachability, we employ a DS-based LfD technique [38].
Alternatives to this choice include certiﬁed
NN-based methods [39, 40], DMPs [41], partially contracting DS [42] and Euclideanizing-ﬂows [43].
To satisfy mode invariance, we modulate the learned DS to avoid invariance failure as statespace boundaries [44], similar to how barrier functions are learned to bound a controller [45, 46, 47].
Multi-Step Manipulation Prior LfD works [13, 14, 10, 48] tackle multi-step manipulation by segmenting demonstrations via a hidden Markov model.
Using segmented motion trajectories, [13] learned a skill tree, [14] learned DMPs, [10] learned phase transitions, and [49] learned a task model.
Most of these works assume a linear sequence of prehensile subtasks (pick-and-place) without considering how to replan when unexpected mode transitions happen.
[48, 49] considered a non-prehensile scooping task similar to ours, but their reactivity only concerned collision avoidance in a single mode.
[50, 6] improved BC policies with RL, but offered no guarantee of task success.
Temporal Logic Imitation: Problem Formulation
Let x 2 R n represent the n -dimensional continuous state of a robotic system; e.g., the robot’s endeffector state in this work.
Let ↵ =
[ ↵ 1 , ..., ↵ m ] T 2 { 0 , 1 } m be an m -dimensional discrete sensor state that uniquely identiﬁes a mode  =
L ( ↵ ) .
We deﬁne a system state as a tuple, s = ( x, ↵ )
R n ⇥ { 0 , 1 } m .
Overloading the notation, we use  2 ⌃ , where
⌃ =
{  i } M i =1 , to represent the set of all system states within the same mode—i.e.,  i = { s = ( x, ↵ )
| L ( ↵ ) =
i } .
In contrast, we use  i =
{ x | s = ( x, ↵ ) 2  i } to represent the corresponding set of robot states.
Note x cannot be one-to-one mapped to s , e.g., a level spoon can be either empty or holding soup.
Each mode is associated by a goal-oriented policy, with goal x
⇤ i 2 R n .
A successful policy that accomplishes a multi-step task ⌧ with a corresponding LTL speciﬁcation  can be written in the form:
[Equation]
with  ⌦  ( ↵ )  i being the Kronecker delta that activates a mode policy
f i ( x ; ✓ i , x ⇤ i ) : R n + m !
R n encoded by a set of learnable parameters ✓ i and goal x
⇤ i .
Mode activation is guided by an LTLequivalent automaton ⌦  ( ↵ ) !
i choosing mode  i based on current sensor reading ↵ .
Demonstrations Let demonstrations for a task ⌧ be ⌅ = {{ x t,d , ˙ x t,d , ↵ t,d } T d t =1 } Dd =1 where x t,d , ˙ x t,d , ↵ t,d are robot state, velocity, and sensor state at time t in demonstration d , respectively, and T d is the length of each d -th trajectory.
A demonstration is successful if the continuous motion traces through a sequence of discrete modes that satisﬁes the corresponding LTL task speciﬁcation.
Perturbations External perturbations, which many works in Sec.
2 avoid, constitute an integral part of our task complexity.
Speciﬁcally, we consider: (1) motion-level perturbations that displace the continuous motion within the same mode, and (2) task-level perturbations that drive the robot outside of the current mode.
Critically, motion-level perturbations do not cause a plan change instantaneously, but they can lead to future unwanted mode transitions due to covariate shift.
Environmental stochasticity is ignored, as its cumulative effects can also be simulated by external perturbations.
Problem Statement Given (1) an LTL formula  specifying valid mode transitions for a task ⌧ , and (2) successful demonstrations ⌅ , we seek to learn a policy deﬁned in Eq. 1 that generates continuous trajectories guaranteed to satisfy the LTL speciﬁcation despite arbitrary external perturbations.
Preliminaries 4.1 LTL Task Speciﬁcation
LTL formulas consist of atomic propositions (AP), logical operators, and temporal operators [51, 23].
Let ⇧ be a set of Boolean variables; an inﬁnite sequence of truth assignments to all APs in ⇧ is called the trace [ ⇧ ] .
The notation [ ⇧ ] , t | =
means the truth assignment at time t satisﬁes the LTL formula  .
Given ⇧ , the minimal syntax of LTL can be described as:
[Equation]
where p is any AP in ⇧ , and  1 and  2 are valid LTL formulas constructed from p using Eq.
The operator X is read as ‘next,’ and X  1 intuitively means the truth assignment to APs at the next time step sets  1 as true.
U is read as ‘until’ and, intuitively,  1 U  2 means the truth assignment to APs sets  1 as true until  2 becomes true.
Additionally, ﬁrst-order logic operators ¬ (not), ^ (and), _ (or), and ! (implies), as well as higher-order temporal operators F (eventually), and G (globally), are incorporated.
Intuitively, F  1 means the truth assignment to APs eventually renders  1 true and
G  1 means truth assignment to APs renders  1 always true from this time step onward.
4.2 Task-Level Reactivity in LTL
To capture the reactive nature of a system given sensor measurements, the generalized reactivity (1) (GR(1)) fragment of LTL [29, 30] can be used.
Let the set of all APs be ⇧ = X
[ Y , where sensor states form environment APs X
= { ↵ 1 , ..., ↵ m } and mode symbols form system APs Y =
{  1 , ...,  l } .
A GR(1) formula is of the form  =
(  e !
s ) [29], where  e models the assumed environment behavior and  s models the desired system behavior.
Speciﬁcally,
[Equation]
ei and  si are non-temporal Boolean formulas that constrain the initial truth assignments of X and
Y (e.g., the starting mode).
st and  et are LTL formulas categorized as safety speciﬁcations that describe how the system and environment should always behave (e.g., valid mode transitions).
sg and  eg are LTL formulas categorized as liveness speciﬁcations that describe what goal the system and environment should eventually achieve (e.g., task completion) [23].
The formula  guarantees the desired system behavior speciﬁed by  s if the environment is admissible —i.e.,  e is true—and can be converted to an automaton ⌦  that plans a mode sequence satisfying  by construction [30].
4.3 Motion-Level Reactivity in DS
LPV-DS [19] can be learned in minutes from as few as a single demonstration and has form: ˙
[Equation]
where A k 2 R n ⇥ n , b k 2 R n are the k-th linear system parameters, and  k ( x ) : R n !
R + is the mixing function.
To certify global asymptotic stability (G.A.S.) of Eq. 4, a Lyapunov function
V ( x ) =
( x  x ⇤ ) T P ( x  x ⇤ ) with P = P T  0 , is used to derive the stability constraints in Eq.
Minimizing the ﬁtting error of Eq.
4 with respect to demonstrations
⌅ subject to constraints in Eq.
5 yields a non-linear DS with a stability guarantee [19].
To learn the optimal number K and mixing function  k ( x ) we use the Bayesian non-parametric GMM ﬁtting approach presented in [19].
4.4 Bisimulation between Discrete Plan and Continuous Policy
To certify a continuous policy will satisfy a LTL formula  , one can show the policy can simulate any LTL-satisﬁcing discrete plan of mode sequence generated by ⌦  .
To that end, every mode’s associated policy must satisfy the following bisimulation conditions [25, 23]:
Condition 1 ( Invariance ) .
All states starting in a mode must remain within the same mode when following that mode’s policy; i.e., 8 i 8 t ( s 0 2  i !
s t 2  i )
Condition 2 ( Reachability ) .
All states starting in a mode must reach their next modes while following the current mode’s policy; i.e., 8 i 9 T ( s 0 2  i !
s T 2  j )
LTL-DS: Methodology
To solve the TLI problem in Sec. 3, we introduce a mode-based imitation policy—LTL-DS:
[Equation]
=
⌃ M
[Equation]
⌦  ( ↵ )  i | {z } ofﬂine learning
M i  x
[Equation]
i (
[Equation]
x
[Equation]
i  | {
[Equation]
} online learning f i (
[Equation]
) | {
[Equation]
} ofﬂine learning , (6)
During ofﬂine learning, we synthesize the automaton ⌦  from  as outlined in Sec.
4.2 and learn
DS policies f i from ⌅ according to Sec.
While the choice of DS satisﬁes the reachability condition as explained later, DS rollouts are not necessarily bounded within any region.
Neither do we know mode boundaries in TLI.
Therefore, an online learning phase is necessary where for each mode policy f i we learn an implicit function,  i ( x ) : R n !
R + , that inner-approximates the mode boundary in the state-space of the robot x 2 R N .
With a learned  i ( x ) for each mode, we can construct a modulation matrix M i that ensures each M i f i to be mode invariant as discussed below.
5.1 Ofﬂine Learning Phase
In the ofﬂine learning phase, the user provides successful demonstrations ⌅ and a LTL formula  as described in Sec.
4.2, which necessarily includes the deﬁnition of sensors and AP regions.
Synthesis of LTL-Satisﬁcing Automaton Given  , we use [52] to convert it into an automaton, which plans a mode sequence that satisﬁes  by construction.
Assuming all possible initial conditions for the system are speciﬁed in the LTL, the automaton is always deployed from a legal state.
Sensor-based Motion Segmentation and Attractor Identiﬁcation Given the demonstration set ⌅ and AP regions related to the set of M modes we can automatically segment the trajectories into M clusters and corresponding attractor set X ⇤ .
For more details, refer to Appendix C.
Ensuring Goal Reachability with Learned DS Mode Policies While any BC variant with a stability guarantee can satisfy reachability (see Sec. 2), we focus on the G.A.S. DS formulation and learning approach deﬁned in Section 4.3 that ensures every x 2 R n is guaranteed to reach x
⇤ i .
By placing x
⇤ i within the boundary set of  j for a mode  j , we ensure mode  j is reachable from every s in mode  i .
Note f
( x ) cannot model sensor dynamics in ↵ .
Yet, we employ mode abstraction to reduce the imitation of a system state trajectory in s —which includes the evolution of both the robot and sensor state—to just a robot state trajectory in x .
5.2 Online Learning Phase
Iterative Mode Boundary Estimation via Invariance Failures As shown in Fig.
2, DS can suffer from invariance failures in regions without data coverage.
Instead of querying humans for more data in those regions [20], we leverage sparse events of mode exits detected by sensors to estimate the unknown mode boundary.
Speciﬁcally, for each invariance failure, we construct a cut that separates the failure state, x T f , from the mode-entry state, x 0 , the last in-mode state, x T f  1 , and the mode attractor, x ⇤ .
We ensure this separation constraint with a quadratically constrained quadratic program (QCQP) that searches for the normal direction (pointing away from the mode) of a hyperplane that passes through x T f  1 such that the plane’s distance to x ⇤ is minimized.
The intersection of half-spaces cut by the hyper-planes inner-approximates a convex mode boundary, as seen in Fig.
Adding cuts yields better boundary estimation, but is not necessary unless the original vector ﬁeld ﬂows out of the mode around those cuts.
For more details, refer to Appendix E.3.
Ensuring Mode Invariance by Modulating DS We treat each cut as a collision boundary that deﬂects the DS ﬂows following the approach in [44, 53].
In our problem setting the mode boundary is analogous to a workspace enclosure rather than a random task-space object.
Let existing cuts form an implicit function,  ( x ) : R n !
R + , denote the estimated interior with  ( x ) < 1 ,  ( x )
= 1 the boundary and  ( x ) > 1 the exterior of a mode.
0 <  ( x ) < 1 monotonically increases as x moves away from a reference point x r inside the mode.
For x outside the cuts, or inside but moving away from the cuts, we leave f ( x ) unchanged; otherwise, we modulate f ( x ) to not collide with any cuts as ˙ x = M ( x ) f
( x ) by constructing a modulation matrix M ( x ) through eigenvalue decomposition:
[Equation]
The full-rank basis E ( x ) consists of a reference direction r ( x ) stemming from x r toward x , and d  1 directions spanning the hyperplane orthogonal to r  ( x ) , which in this case is the closest cut to x .
In other words, all directions e 1 ( x ) ...
e d  1 ( x ) are tangent to the closest cut, except r ( x ) .
By modulating only the diagonal component,  r ( x ) , with  ( x ) , we have  r ( x ) !
as x approaches the closest cut, effectively zeroing out the velocity penetrating the cut while preserving velocity tangent to the cut.
Consequently, as long as there are cuts bounding the mode, the modulated DS will not experience invariance failures where the original DS would.
Notice this modulation strategy is not limited to DS, and can apply to any state-based BC methods to achieve mode invariance.
Proof
Next, we prove LTL-DS produces a continuous trajectory that satisﬁes a LTL speciﬁcation.
We start with assumptions and end with theorems.
Detailed proofs are provided in Appendix A.
Assumption 1.
All modes are convex.
This assumption leads to the existence of at least one cut—i.e., the supporting plane [54], which can separate a failure state on the boundary from any states within the mode.
A corollary is that a boundary shared by two modes, which we call a guard surface, G ij =
i \  j , is also convex.
Since all transitions out of a mode observed during demonstrations reside on the mode boundary, their average location—which we use as the attractor for the mode—will also be on the boundary.
Assumption 2.
There are a ﬁnite number of externally exerted motion- and task-level perturbations of arbitrary magnitude.
Given zero perturbation, all BC methods should succeed in any task replay, as the policy rollout will always be in distribution.
If there are inﬁnitely many arbitrary perturbations, no BC methods will be able to reach a goal.
Here, we study the setting between these extremes—where there are a ﬁnite number of external perturbations causing unexpected mode exits.
Assumption 3.
Every unexpected mode transition only results in sensor states that have been seen in the demonstrations.
While demonstrations of all valid mode transitions are not required, they must minimally cover all possible modes.
If the system encounters a completely new sensor state during online interaction, it is reasonable to assume that no BC methods could recover from the mode unless more information about the environment were provided.
Theorem 1.
(Key Contribution 1)
A nonlinear DS deﬁned by Eq. 4, learned from demonstrations, and modulated by cutting planes as described in Section 5.2 with the reference point x r set at the attractor x
⇤ , will never penetrate the cuts and is G.A.S. at x ⇤ .
Proof: See Appendix A.
Theorem 2.
(Key Contribution 2)
The continuous trace of system states generated by LTL-DS deﬁned in Eq.
6 satisﬁes any LTL speciﬁcation  under Asm. 1, 2, and 3.
Proof: See Appendix A.
Experiments 7.1 Single-Mode Invariance and Reachability
We show quantitatively both reachability and invariance are necessary for task success.
We compare
DS and a NN-based BC policy (denoted as BC) to represent policies with and without a stability guarantee respectively.
Fig.
5 shows that policy rollouts start to fail (turn red) as increasingly larger perturbations are applied to the starting states; however, DS only suffers from invariance failures, while BC suffers from both invariance and reachability failures (due to diverging ﬂows and spurious attractors).
Fig.
5 (right) shows that all ﬂows are bounded within the mode for both DS and BC after two cuts.
In the case of DS, ﬂows originally leaving the mode are now redirected to the attractor by the cuts; in the case of BC, while no ﬂows leave the mode after modulation, spurious attractors are created, leading to reachability failures.
This is a counterfactual illustration of Thm. 1, that policies without a stability guarantee are not G.A.S. after modulation.
Fig.
6 veriﬁes this claim quantitatively
and we empirically demonstrate that a stable policy requires only four modulation cuts to achieve a perfect success rate—which an unstable policy cannot be modulated to achieve.
[Figure]
7.2 Multi-Modal Reactivity and Generalization to New Tasks
We now empirically demonstrate that a reactive discrete plan alone is insufﬁcient to guarantee task success without mode invariance for tasks with multiple modes.
Consider the multi-modal soupscooping task introduced in Fig.
Formally, we deﬁne three environment APs, r, s, t , sensing the spoon is in contact with the soup, has soup on it, and has arrived at a target location respectively.
Given successful demonstrations, sensors will record discrete transitions ( ¬ r ^ ¬ s ^ ¬ t ) )
( r ^ ¬ s ^ ¬ t ) )
( ¬ r ^ s ^ ¬ t ) )
( ¬ r ^ ¬ s ^ t ) , from which four unique sensor states are identiﬁed.
We label each sensor state as a mode with robot AP a (reaching) )
b (scooping) )
c (transporting) )
d (done) .
Invariance of mode b enforces contact with soup during scooping, and invariance of mode c constrains the spoon’s orientation in order to avoid spilling.
We follow the TLP convention to assume LTL formulas are provided by domain experts (although they can also be learned from demonstrations [51, 55].)
The speciﬁc LTL for the soup-scooping task is detailed in Appendix F, and can be converted into a task automaton as shown in Fig.
One might assume the automaton is sufﬁcient to guarantee task success without modulation, as it only needs to replan a ﬁnite number of times assuming a ﬁnite number of perturbations; however, not enforcing mode invariance can lead to looping at the discrete level, and ultimately renders the goal unreachable, as depicted in the top row of Fig.
In contrast, looping is prevented when modulation is enabled, as the system experiences each invariance failure only once.
We found 50% of the learned policies without modulation could get stuck in looping after a ﬁnite number of perturbations, while all experiments with modulation succeeded in the task replay.
Robot Experiments First, we implemented the soup-scooping task on a Franka Emika robot arm as shown in Fig.
We show in videos on our website that (1) DS allowed our system to compliantly react to motion-level perturbations while ensuring system stability; (2) LTL allowed our system to replan in order to recover from task-level perturbations; and (3) our modulation ensured the robot learned from previous invariance failures in order to avoid repeating them.
To test robustness against unbiased perturbations, we collected 25 trials from 5 humans as seen in in Appendix H. All trials succeed eventually in videos.
We did not cherry-pick these results, and the empirical 100% success rate further corroborates our theoretic success guarantee.
Second, we implemented an inspection task as a permanent interactive exhibition at MIT museum, with details documented in Appendix I.
Lastly, we show a third color tracing task testing different automaton structures on our website.
Generalization Once a DS is learned we can generalize to a new task sharing the same set of modes observed in demonstrations given a new LTL formula.
Consider another multi-step task of adding chicken and broccoli to a pot.
Different humans might give demonstrations with different modal structures (e.g., adding chicken vs adding broccoli ﬁrst).
LTL-DS could learn individual DS which can be ﬂexibly combined to solve new tasks with new task automatons.
To get these different task automatons, a human just needs to edit the  st portion of the LTL formulas differently.
We provide further details of this analysis in Appendix G.
Limitations
Our approach of integrating logic formulas into LfD requires deﬁning an appropriate abstraction for a task.
For example, allocating a sensor to detect contact events requires domain knowledge.
Our work is based on the assumption that for well-deﬁned tasks (e.g., assembly tasks in factory settings), domain expertise in the form of a logic formula is a cheaper knowledge source than collecting hundreds of motion trajectories to avoid covariate shift (we use up to 5 demonstrations in all experiments).
However, even when abstractions for a task are given by an oracle, a LfD method without either the invariance or the reachability property will not have a formal guarantee of successful task replay, which is this work’s focus.
In future work, we will learn such abstractions directly from sensor streams such as videos so that our approach gains more autonomy without losing reactivity.
Conclusion
In this paper, we formally introduce the problem of temporal logic imitation as imitating continuous motions that satisfy a LTL speciﬁcation.
We identify the fact that learned policies do not necessarily satisfy the bisimulation criteria as the main challenge of applying LfD methods to multi-step tasks.
To address this issue, we propose a DS-based approach that can iteratively estimate mode boundaries to ensure invariance and reachability.
Combining the task-level reactivity of LTL and the motionlevel reactivity of DS, we arrive at an imitation learning system able to robustly perform various multi-step tasks under arbitrary perturbations given only a small number of demonstrations, and demonstrate our system’s practicality on a real Franka robot.
Acknowledgments
We would like to thank Jon DeCastro, Chuchu Fan, Terry Suh, Rachel Holladay, Rohan Chitnis,
Tom Silver, Yilun Zhou, Naomi Schurr, and Yuanzhen Pan for their invaluable advice and generous help.