Patent ID: 12208522

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

The following detailed description refers to the figures that show, by way of illustration, specific details and aspects of this disclosure in which the present invention may be practiced. Other aspects may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the present invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure can be combined with one or more other aspects of this disclosure to form new aspects of the present invention.

In the following, various examples will be described in more detail.

FIG.1shows a robot100.

The robot100includes a robot arm101, for example an industrial robot arm for handling or assembling a work piece (or one or more other objects). The robot arm101includes manipulators102,103,104and a base (or support)105by which the manipulators102,103,104are supported. The term “manipulator” refers to the movable members of the robot arm101, the actuation of which enables physical interaction with the environment, e.g. to carry out a task. For control, the robot100includes a (robot) controller106configured to implement the interaction with the environment according to a control program. The last member104(furthest from the support105) of the manipulators102,103,104is also referred to as the end-effector104and may include one or more tools such as a welding torch, gripping instrument, painting equipment, or the like.

The other manipulators102,103(closer to the support105) may form a positioning device such that, together with the end-effector104, the robot arm101with the end-effector104at its end is provided. The robot arm101is a mechanical arm that can provide similar functions as a human arm (possibly with a tool at its end).

The robot arm101may include joint elements107,108,109interconnecting the manipulators102,103,104with each other and with the support105. A joint element107,108,109may have one or more joints, each of which may provide rotatable motion (i.e. rotational motion) and/or translatory motion (i.e. displacement) to associated manipulators relative to each other. The movement of the manipulators102,103,104may be initiated by means of actuators controlled by the controller106.

The term “actuator” may be understood as a component adapted to affect a mechanism or process in response to be driven. The actuator can implement instructions issued by the controller106(the so-called activation) into mechanical movements. The actuator, e.g. an electromechanical converter, may be configured to convert electrical energy into mechanical energy in response to driving.

The term “controller” may be understood as any type of logic implementing entity, which may include, for example, a circuit and/or a processor capable of executing software stored in a storage medium, firmware, or a combination thereof, and which can issue instructions, e.g. to an actuator in the present example. The controller may be configured, for example, by program code (e.g., software) to control the operation of a system, a robot in the present example.

In the present example, the controller106includes one or more processors110and a memory111storing code and data based on which the processor110controls the robot arm101. According to various embodiments, the controller106controls the robot arm101on the basis of a statistical model112stored in the memory111.

A robot100can take advantage of learning-from-demonstration (LfD) approaches to learn to execute a task or collaborate with a human partner. Human demonstrations can be encoded by a probabilistic model (also referred to as statistical model) that represents the nominal plan of the task for the robot. The controller106can subsequently use the statistical model, which is also referred to as robot trajectory model, to generate the desired robot movements, possibly as a function of the state of both the human partner and the environment.

The basic idea of LfD is to fit a prescribed skill model such as GMMs to a handful of demonstrations. Let there be M demonstrations, each of which contains Tmdata points for a dataset of N=ΣmTmtotal observations ξ={ξ}t=1N, where ξt∈d. Also, it is assumed that the same demonstrations are recorded from the perspective of P different coordinate systems (given by the task parameters such as local coordinate systems or frames of objects of interest). One common way to obtain such data is to transform the demonstrations from a static global frame to frame p by ξt(p)=A(p)−1(ξt−b(p)). Here, {(b(p),A(P))}p=1Pis the translation and rotation of (local) frame p w.r.t. the world (i.e. global) frame. Then, a TP-GMM is described by the model parameters {πk,{μk(p),Σk(p)}p=1P}k=1Kwhere K represents the number of Gaussian components in the mixture model, πkis the prior probability of each component, and {μk(p),Σk(p)}p=1Pare the parameters of the k-th Gaussian component within frame p.

Differently from standard GMM, the mixture model above cannot be learned independently for each frame. Indeed, the mixing coefficients πkare shared by all frames and the k-th component in frame p must map to the corresponding k-th component in the global frame. Expectation-Maximization (EM) is a well-established method to learn such models.

Once learned, the TP-GMM can be used during execution to reproduce a trajectory for the learned skill. Namely, given the observed frames {b(p),A(p)}p=1P, the learned TP-GMM is converted into one single GMM with parameters {πk, ({circumflex over (μ)}k,{circumflex over (Σ)}k)}k=1K, by multiplying the affine-transformed Gaussian components across different frames, as follows

Σ^k=[∑p=1P(Σ^k(p))-1]-1,µ^k=Σ^k[∑p=1P(Σ^k(p))-1⁢µ^k(p)],(1)
where the parameters of the updated Gaussian at each frame p are computed as
{circumflex over (μ)}k(p)=A(p)μk(p)+b(p)and {circumflex over (ξ)}k(p)=A(p)Σk(p)A(p)T. While the task parameters may vary over time, the time index is dropped for the sake of notation.

Hidden semi-Markov Models (HSMMs) extend standard hidden Markov Models (HMMs) by embedding temporal information of the underlying stochastic process. That is, while in HMM the underlying hidden process is assumed to be Markov, i.e., the probability of transitioning to the next state depends only on the current state, in HSMM the state process is assumed semi-Markov. This means that a transition to the next state depends on the current state as well as on the elapsed time since the state was entered. They can be applied, in combination with TP-GMMs, for robot skill encoding to learn spatio-temporal features of the demonstrations. More specifically, a task-parameterized HSMM (TP-HSMM) model is defined as:
Θ={{ahk}h=1K,(μkD,σkD),πk,{(μk(p),Σk(p))}p=1P}k=1K,
where ahkis the transition probability from state h to k; (μkD,σkD) describe the Gaussian distributions for the duration of state k, i.e., the probability of staying in state k for a certain number of consecutive steps; {πk,{μk(p),Σk(p)}p=1P}k=1Kequal the TP-GMM introduced earlier, representing the observation probability corresponding to state k. Note that herein the number of states corresponds to the number of Gaussian components in the “attached” TP-GMM.

Given a certain (partial) sequence of observed data points, assume that the associated sequence of states in Θ is given by st=s1s2. . . st. The probability of data point ξtbelonging to state k (i.e., st=k) is given by the forward variable αt(k)=p(st=k,):
αt(k)=Στ=1t−1Σh=1Kαt−τ(h)ahk(τ|μkD,σkD)oτt,  (2)

where oτt=(|{circumflex over (μ)}k,{circumflex over (Σ)}k) is the emission probability and ({circumflex over (μ)}k,{circumflex over (Σ)}k) are derived from (1) given the task parameters. Furthermore, the same forward variable can also be used during reproduction to predict future steps until Tm.

In this case however, since future observations are not available, only transition and duration information are used, i.e., by setting(|{circumflex over (μ)}k,{circumflex over (Σ)}k)=1 for all k and>t in (2). At last, the sequence of the most-likely states sTm*=s1*s2* . . . sTm* is determined by choosing st*=argmaxkαt(k), ∀1≤t≤Tm.

Let now a desired final observation of the robot state be given as ξT, where T is the skill time horizon (e.g. the average length over the demonstrations). Moreover, the initial robot state is observed as ξ1. For execution of the skill (i.e. skill reproduction) given the learned model Θa, the most-likely state sequence sT* given only ξ1and ξTis constructed.

Reproduction using the forward variable cannot be directly in that case since the forward variable in equation (2) computes the sequence of marginally most probable states, while what is desired is the jointly most probable sequence of states given ξ1and ξT. As a result, when using (2) there is no guaranteeing that the returned sequence sT* matches both the spatio-temporal patterns of the demonstrations and the final observation. In terms of an example of picking up an object, it may return a most likely sequence corresponding to “pick from the side”, even if the desired final configuration is that the end-effector is on the top of object.

To overcome this issue, according to one embodiment, a modification of the Viterbi algorithm is used. The classical Viterbi algorithm can be used to find the most likely sequence of states (also called the Viterbi path) in HMMs that result in a given stream of observed events. According to one embodiment, an approach is used which differs from that in two main aspects: (a) it works on HSMM instead of HMM; and more importantly (b) most observations except the first and the last ones are missing. Specifically, in the absence of observations the Viterbi algorithm becomes

δt(j)=maxd∈𝒟maxi≠jδt-d(i)⁢aij⁢pj(d)⁢∏t⁢′=t-d+1tb~j(ξt⁢′),(3)δ1(j)⁢bj(ξ1)⁢πj⁢pj(1),
where pj(d)=(d|μjD,σjD) is the duration probability of state j, δt(j) is the likelihood of the system being in state j at time t and not in state j at t+1; and

b~j(ξt⁢′)={𝒩⁡(ξt⁢′|µ^j,Σ^j),t=1⋁t=T;1,1<t<T.
where ({circumflex over (μ)}j,{circumflex over (Σ)}j) is the global Gaussian component j in Θafrom (1) given ξt. Namely, at each time t and for each state j, the two arguments that maximize equation δt(j) are recorded, and a simple backtracking procedure is used to find the most likely state sequence sT*. In other words, the above algorithm derives the most-likely sequence sT* for skill a that yields the final observation ξT, starting from ξ1.

As the robot task space is represented by time-varying poses (with position and orientation) of the end-effector, classical Euclidean-based methods are inadequate for processing such data. Therefore, according to various embodiments, the robot task space is endowed with a Riemannian manifold. Briefly, for each point x in the manifold, there exists a tangent space. This allows us to carry out Euclidean operations locally, while being geometrically consistent with manifold constraints.

Exponential and logarithmic maps may be used to map points betweenand. The exponential map Expx:→maps a point in the tangent space of point x to a point on the manifold, while maintaining the geodesic distance. The inverse operation is called the logarithmic map Logx:→. Another useful operation is the parallel transport→, which moves elements between tangent spaces without introducing distortion. The exact form of the aforementioned operations depends on the Riemannian metric associated to the manifold. According to various embodiments, Riemannian manifolds are used to properly compute statistics overusing Riemannian normal distributions that encode the observed motion patterns and retrieve the control actions corresponding to the task plan (i.e., sequenced skills) using a Riemannian optimal controller.

For the following explanations, a multi-DoF robotic arm101is considered as example, whose end-effector104has state xe∈3×3×1(describing the Cartesian position, orientation quaternion and gripper state), that operates within a static and known workspace. Also, within the reach of the arm101, there are objects of interest denoted by O={o1, o2, . . . , oj}, each of which has state xoj∈3×3. For simplicity, the overall system state is denoted by x={xe,{xoj,∀oj∈O}}.

Within this setup, a human user performs several kinaesthetic demonstrations on the arm to manipulate one or several objects for certain manipulation skills. Let the set of demonstrated skills be denoted by A={a1, a2, . . . , aH}. Moreover, for skill a∈A, the set of objects involved is given by Oaand the set of available demonstrations is denoted by Da. It should be noted that all demonstrations follow the object-centric structure introduced above, i.e., they are recorded from multiple frames, normally associated to the objects in Oa, which often represent the object pose in the workspace. For example, the skill “insert the peg in the cylinder” involves the objects “peg” and “cylinder”, and the associated demonstrations are recorded from both the robot, the “peg” and the “cylinder” frames.

The (manipulation) task that is considered in the following consists of a sequence of skills a* chosen from the demonstrated skills A. For example, an insertion task involves “pick the cap, re-orient the cap, pick the cap again and the insert the cap”. In the end of the task, a goal configuration G is reached as the desired final state of the system, including the robot and the objects.

The common way of organizing manipulation tasks in factory is via a diagram or flowchart. They are commonly defined via drag-and-drop in a GUI (graphical user interface). Such an approach is sufficient if two conditions hold: (1) the task is simple and specified as a linear sequence of skills; (2) each skill is simple without branches. In this way, each skill can be triggered and executed in sequence as specified.

However, in many cases, either one of the above conditions does not hold. For instance, the desired task has multiple choices of execution in various workspace situations or some skills inside have multiple choices of execution in various situations.

FIG.2shows a flow diagram (or task diagram)200illustrating a manipulation task including skills in sequence and in branches.

For example, the manipulation task includes, as first job201, to pick up an object. This may mean that the robot has to execute the skill “pick up object from top” in202, “pick up object from left” in203or “pick up object from the right” in204depending on the initial configuration (i.e. state) of the object. So, the task includes the execution of these skills in branches, i.e. they are to be executed alternatively, i.e. only one of them is to be executed. The first operation201, i.e. the execution of one of the skills in202, in203or in204is followed by one or more skills in sequence. For example, if the skill “pick up object from top” was executed in202, this is followed (in sequence) by the skill “attach object” in205.

If the respective skill in203or in204was executed, this has to be followed by a reorientation job206, i.e., for each case by execution of a re-orientation skill in207or in208. The re-orientation skills may differ in the re-orientation direction. The re-orientation operation206is then followed by execution of the “pick up object from top” in skill209and finally by the execution of the skill “attach object” in210.

Branching may be addressed by manually specifying the branching conditions211,212,213, commonly as “if” conditions, e.g. “if object is standing”211, “if object is lying toward left”212and “if object is lying toward right”213. To design such conditions, a region of system states may be manually measured as the region of validity for this condition to hold.

This means that the robot may have a set of manipulation skills pre-installed (pre-programmed from the factory or taught via demonstration) and for a particular assembly task, the operator constructs a diagram manually that specifies this task (e.g. as inFIG.2), where the building blocks are the set of learned skills. Because of the branches (possibly both at the task-level and at the skill-level), the operator is required to manually defined the branching conditions for each branch.

According to various embodiments, approaches are provided which in particular allow avoiding the necessity for manually defining branching conditions.

FIG.3shows a flow diagram illustrating a method for controlling a robot according to an embodiment.

In301, demonstrations of skills are performed.

The skills include at least those skills which are needed for the execution of a task given by a task diagram303.

For one demonstrated skill a∈A, as described above, the set of available demonstrations is given by Da={ξt}t=1N, recorded in P frames. It should be noted that such frames are directly attached to the objects in Oa.

In302, a robot trajectory model (also denoted as “robot behaviour model”), is learned for each skill.

For example, as described above, given a properly chosen number of components K, the TP-HSMM model Θaabstracting the spatio-temporal features of trajectories related to skill a, can be learned using an EM (Expectation Maximization)-like algorithm.

In304, a composed robot trajectory model is generated from the robot trajectory models learned in302.

For this, the learning of the skill modes further includes the learning of a pre-condition model, a final condition and an effect model for each skill. In304, using these models, a composition model of the specified task is constructed and the choices on the task-level and the skill-level can be then made automatically depending on the workspace situation. Simply speaking, the pre-condition model encapsulates how the system should be before executing the skill, while the effect model and the final condition model encapsulate how the system should be changed after executing the skill. These models are an important part for computing the composition model as they measure the compatibility between skills and keep track of the evolution of system state. It should be noted that the term “skill model” may be understood to include all of the robot trajectory model, the precondition model, the final condition model and the effect model for the skill.

As described with reference toFIG.2, a task may include execution of skills in branches (i.e. as alternatives) and in sequence.

Accordingly, the generation of the composed model includes recursively applying combination operations for combining skills in sequence and an operation for combining skills in parallel.

FIG.4illustrates a cascading operation for cascading robot trajectory models of skills401,402,403which are to be executed in sequence to a composed robot trajectory model404.

FIG.5illustrates a combination operation for combining robot trajectory models of skills501,502,503,504which are to be executed in branches (i.e. alternatively) to a composed robot trajectory model505.

The combination operation of cascading a sequence of skills as illustrated inFIG.4includes one or more applications of an operation of cascading two skills which are to be executed in sequence. Similarly, the combination operation of combining a sequence of skills as illustrated inFIG.5includes one or more applications of an operation of combining two skills which are to be executed in branches.

For the combination of two skills which are to be executed in sequence, the trajectory models of the two skills are cascaded into one composed trajectory model as follows.

Considering two TP-HSMMs Θa1and Θa2of two skills in sequence, the operation for cascading them into {circumflex over (Θ)} is summarized in Algorithm 1.

Algorithm 1: Cascading a pair of TP-HSMMsInput: (Θa1, Γa1) and (Θa2, Γa2).Output: ({circumflex over (Θ)}, {circumflex over (Γ)})1forall final component kf∈ Θa1do2|Create copy of Θa2as Θa2kf.3|Compute {akf,ki} for all initial ki∈ Θa2kf4|Update Θa2kfand Γ1T,a2kf5|—Cascade Θa1and Θa2kf. Add to {circumflex over (Θ)}.6Set additional parameters of {circumflex over (Θ)}.7{circumflex over (Γ)} = {{circumflex over (Γ)}1, {circumflex over (Γ)}T, {circumflex over (Γ)}1T} = {Γ1,a1, ΓT,a2, {Γ1T,a2kf, ∀kf}}.

It should be noted that the computation and the update of lines 3 and 4 of algorithm 1 may be performed according to equations (4) and (5) given below, respectively.

A key insight can be seen in that the same model Θa2is updated differently depending on the final component (i.e. HSMM state) of Θa1to which Θa2is cascaded to. This is because each final component encodes different transformations of the task parameters of Θa1after executing a1, which in turn results in different ways to update the components in Θa2. Consequently, the composed model {circumflex over (Θ)} has size K1+K1,f·K2, where K1and K2are the number of components of Θa1and Θa2, respectively, while K1,fis the number of final components in Θa1. More specifically, algorithm 2 consists of two main operations: (a) compute the transition probability from each final component in Θa1to each initial component in Θa2; (b) modify all components of Θa2for each final component in Θa1that Θa2is cascaded to.

According to one embodiment, a precondition model and an effect model as described in reference [1] are used. In particular, the learned precondition model, denoted by Γ1,a, contains TP-GMMs for the initial robot state (i.e. the initial configuration (e.g. position and/or pose) of the robot), i.e., Γ1,a={({circumflex over (μ)}1(p),{circumflex over (Σ)}1(p)), ∀p∈P1,a}, where P1,ais the chosen set of task parameters, derived from the initial system state (i.e. the initial configuration (e.g. position and/or pose) of the robot and/or objects). In addition, a final condition model is introduced here, denoted by ΓT,a, which is learned in a similar way as Γ1,a, but for the final robot state, i.e., ΓT,a={({circumflex over (μ)}T(p),{circumflex over (Σ)}T(p)), ∀p ∈PT,a}, where PT,ais the chosen set of frames, derived from the final system state. Simply speaking, Γ1,amodels the initial configuration before executing skill a, while ΓT,amodels the final configuration afterwards. Furthermore, the learned effect model, denoted by Γ1T,a, contains TP-GMMs for the predicted final system state, i.e., Γ1T,a={{({circumflex over (μ)}1,o(p),{circumflex over (Σ)}1,o(p), ∀p∈P1,a}, ∀o∈Oa∪e}, where P1,ais defined in Γ1,a. It is worth noting the differences among these three models: the task parameters for ΓT,aare computed from the final system state (after performing a), while those for Γ1,aand Γ1T,aare extracted from the initial system state (before performing a). For the sake of notation Γa{Γ1,a,ΓT,a,Γ1T,a}.

Then, the transition probability from one final component kfof Θa1to one initial component kiof Θa2is:
αkfki∝exp(−Σp∈PcKL(ΓT,a1(p)(kf)∥Γ1,a2(p)(ki))),  (4)

where KL(·∥·) is the KL(Kullback-Leibler)-divergence, ΓT,a1(p)(kf) is the GMM associated with component kffor frame p, Γ1,a2(p)(ki) is the GMM associated with component kifor frame p; Pc=PT,a1∩P1,a2is the set of common frames shared by these two models, which can be forced to be nonempty by always adding the global frame. This process is repeated for all pairs of final components in Θa1and initial components in Θa2. It should be noted that the out-going probability of any final component in Θa1should be normalized.

Secondly, given one final component kfof Θa1, each component k of Θa2should be affine-transformed as follows:
({circumflex over (μ)}k({circumflex over (p)}),{circumflex over (Σ)}k({circumflex over (p)})(μk(p),Σk(p))⊗(bkf({circumflex over (p)}),Akf({circumflex over (p)})),  (5)

where the operation ⊗ is defined as the same operation of (1); (bkf({circumflex over (p)}), Akf({circumflex over (p)})) is the task parameter computed from the mean of Γ1T,a1({circumflex over (p)}),o(kf), where o is the object associated with the old frame p in Θa1and {circumflex over (p)} is the new frame in Γ1T,a1o(kf). It should be noted that the change of frames is important to compute directly all components of Θa2given an initial system state of Θa1. The same process is also applied to each component of r by changing its frames based on Γ1T,a1o(kf).

Lastly, as stated in Algorithm 1, other model parameters of {circumflex over (Θ)} such as duration probabilities, initial and final distributions are set trivially with minor changes from Θa1and Θa2. For instance, the duration probability of Θa2is duplicated to kfmultiple copies; the initial distributions Θa2are set to zero as the initial states of {circumflex over (Θ)} correspond to those of the first model Θa1; the final components of Θa1are removed since the final states of {circumflex over (Θ)} are now the final components of Θa2updated to its multiple instances.

For the combination of two skills which are to be executed in parallel, the trajectory models of the two skills are combined into one composed trajectory model as follows.

Consider two TP-HSMMs Θa1and Θa2of two skills in parallel, the operation for combining them into {circumflex over (Θ)} is summarized in Algorithm 2.

Algorithm 2: Composing a pair of TP-HSMMs in parallelInput: (Θa1, Γa1) and (Θa2, Γa2).Output: ({circumflex over (Θ)}, {circumflex over (Γ)})1Re-index the components of Θ2by incrementing K1./* Compute TPHSMM {circumflex over (Θ)}*/2Copy the all components of Θ2and their duration distributions into {circumflex over (Θ)}.3Append {akh}1of Θ1and {akh}2of Θ2diagonally into {akh} of {circumflex over (Θ)}.4Fill other entries in {akh} of {circumflex over (Θ)} by 0./* Compute precondition and effects TPGMM {circumflex over (Γ)}*/5Copy Γ1into {circumflex over (Γ)}.6Copy Γ2(with the updated components name) into {circumflex over (Γ)}.

Algorithm 2 consists of two parts: one part to compute the composed TP-HSMM model {circumflex over (Θ)} and another part to compute the composed TPGMM model {circumflex over (Γ)}. The first and most important step is to update component indices of Θ2by the total number of components in Θ1. This is to avoid multiple components form different skills with the same index. After this, all associated TPGMM model, duration model, precondition and effect model have be updated accordingly. Last but not least, when computing the composed transition matrix {akh}, we need to append {akh}1of Θ1and {akh}2of Θ2diagonally into {akh} while filling the remaining entries by zero. This means that no additional transitions are added from Θ1to Θ2, as they are composed in parallel (i.e., not in sequence).

In summary, generating the composed model in304includes repeated application of the following operations of two skills:1) Operation 1 (cascading skills in sequence): if “skill#1” and “skill#2” are connected in sequence (as illustrated inFIG.4), calculate composed skill model according to algorithm 12) Operation 2 (combining skills in branches): if “skiing” and “skill#2” are connected in parallel (as illustrated inFIG.5), calculate composed skill model according to algorithm 2.

Specifically, these two operations are repeatedly performed as follows:A) For each branch or sub-branch within the task diagram303, apply operation 1 recursively to derive the composed skill for each branch.B) For all parallel branches, apply operation 2 recursively to) derive the composed skill for all branches. Note that after A, each branch should only have one composed skill.C) Recursively, apply A to all sequences of skills, and B to all parallels of skills.

For example, for the task illustrated inFIG.4, the process for generation of the composed model includes an application of A, then B, then again A and then again B as illustrated inFIGS.6to9.

FIG.6shows the result after a first application of A to the task as given by the task diagram ofFIG.2.

The skill models201and205have been combined to601, the skill models203,207have been combined to602and the skill models204,208have been combined to skill model603.

FIG.7shows the result after a first application of B. The skill models602and603have been combined to skill model701.

FIG.8shows the result after a second application of A. The skill models701,209and210have been combined to skill model801.

FIG.9shows the result after a second application of B. The skill models601and801have been combined to skill model901.

As can be seen, inFIG.9, the whole task is represented by a single composed skill model901. No “if” conditions are needed.

When the composed robot trajectory model has been generated in304, a task may be executed in a given situation.

For this, the initial system state (configuration) in the given situation is observed in305and, e.g. by applying equation (3) to the composed robot trajectory model, the most-likely sequence of components within the composed robot trajectory model is determined, which drives the system (including the robot and objects) to the goal state with the highest probability.

The determination of the sequence of components also outputs the actual sequence of skills that need to be executed under the given situation. This is of importance because the sequence of skills is different when different branches are chosen by the algorithm.

In306, during execution, the optimal sequence of skills contained in this output is executed by following the optimal sequence of components. Given the state sequence linear quadratic tracking (LQT) may for example be used to retrieve the optimal trajectory.

In summary, according to various embodiments, a method is provided as illustrated inFIG.10.

FIG.10shows a flow diagram1000illustrating a method for controlling a robot according to an embodiment.

In1001, demonstrations are provided for performing each of a plurality of skills.

In1002, a robot trajectory model is trained for each skill from the demonstrations, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states.

In1003, a precondition model including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and a final condition model including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill, are trained from the demonstrations for each skill.

In1004, a description of a task is received, wherein the task includes performing the skills of the plurality of skills in sequence and/or branches.

In1005, a composed robot trajectory model is generated byWhen two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills byIncluding the states of the trajectory models of the two skills in the composed robot trajectory model andCalculating a transition probability between each final state of the trajectory model of the first skill of the two skills and each initial state of the trajectory model of the second skill of the two skills as a function of the similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skillWhen two skills are performed in branches in the task, combining the robot trajectory models of the skills byIncluding the states of the trajectory models of the two skills in the composed robot trajectory model andSetting the transition probability between states of the first skill and states of the second skill to zero.

In1006, the robot is controlled in accordance with the composed robot trajectory model to execute the task.

According to various embodiments, in other words, models for a robot are trained for a plurality of skills and when a task is to be carried out which involves multiple of executions of those skills in branches or in sequence, the models are cascaded and/or combined to a composed model. The composed model may then be used for controlling the robot as if it was a model for a single skill, i.e. for example by determining an optimal state sequence for the task (and the initial configuration of robot and objects where the task is to be executed) and controlling the robot accordingly.

The method ofFIG.10may be performed by one or more computers including one or more data processing units. The term “data processing unit” can be understood as any type of entity that allows the processing of data or signals. For example, the data or signals may be treated according to at least one (i.e., one or more than one) specific function performed by the data processing unit. A data processing unit may include an analogue circuit, a digital circuit, a composite signal circuit, a logic circuit, a microprocessor, a micro controller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable gate array (FPGA) integrated circuit or any combination thereof or be formed from it. Any other way of implementing the respective functions, which will be described in more detail below, may also be understood as data processing unit or logic circuitry. It will be understood that one or more of the method steps described in detail herein may be executed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

The term “robot” can be understood to refer to any physical system (with a mechanical part whose movement is controlled), such as a computer-controlled machine, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant or an access control system.

Various embodiments may receive and use sensor signals from various sensors such as video, radar, LiDAR, ultrasonic, motion, thermal imaging etc., for example to obtain sensor data regarding demonstrations or system (robot and object) states and configurations and scenarios. The sensor data may be processed. This may include classification of the sensor data or performing a semantic segmentation on the sensor data, for example to detect the presence of objects (in the environment in which the sensor data was obtained). Embodiments may be used for training a machine learning system and controlling a robot, e.g. a robotic manipulators autonomously to achieve various manipulation tasks under different scenarios. In particular, embodiments are applicable to the control and monitoring of execution of manipulation tasks, e.g., in assembly lines. They can for example be seamlessly integrated with a traditional GUI for a controlling process.

Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein.