2011 IEEE/RSJ International Conference onIntelligent Robots and Systems
September 25-30, 2011. San Francisco, CA, USA

Motion Generation by Reference-Point-Dependent Trajectory HMMs
Komei Sugiura, Naoto Iwahashi and Hideki Kashioka

et al. used multiple Gaussian distributions to model the timeseries of spatiotemporal values [6]. Manipulation trajectories
by a couple of robot arms were generated based on Gaussian
Mixture Regression (GMR).
These methods use relative trajectories between two objects under an assumption that the two objects are specified
by the user. Therefore, there is a limitation that the user
has to specify which motion needs information on the two
objects and which does not. For example, learning the“place an object on another” motion requires the position
sequences of two objects, while learning the “raise an object”
motion does not. Thus, overcoming this limitation enables
the learning of various types of object manipulation fromnatural interactions between the user and robot.
On the other hand, we have investigated imitation learning methods for reference-point-dependent motions [7]–[9].
The number of objects necessary to describe the motion is
automatically determined, and the relevant object(s) in eachdemonstration is estimated as well. In this paper, we present
a motion generation method for reference-point-dependent
motions. Our key contributions are as follows:• Motion
generation method by reference-pointdependent trajectory HMMs. The HMMs are trainedin the intrinsic coordinate system (see Section III)and transformed to the world coordinate system. The
transformation and motion generation methods are
explained in Section IV and Appendix.
• Smooth trajectories are generated by using trajectory
HMMs (see Fig. 4). The generated trajectories are evaluated in Section VI.
In the paper, the method is evaluated with a fixed manipulator
robot; however, the method is also applicable to wheeledhumanoid robots. Indeed, the proposed method was successfully applied to learning to throw an object into another inthe RoboCup@Home [10] environment.

Abstract— This paper presents an imitation learning method
for object manipulation such as rotating an object or placing
one object on another. In the proposed method, motions arelearned using reference-point-dependent probabilistic models.
Trajectory hidden Markov models (HMMs) are used as theprobabilistic models so that smooth trajectories can be generated from the HMMs. The method was evaluated in physicalexperiments in terms of motion generation. In the experiments,a robot learned motions from observation, and it generated
motions under different object placement. Experimental results
showed that appropriate motions were generated even when theobject placement was changed.

I. I NTRODUCTION
Object-manipulating motions such as “put the dishes in
the cupboard” are fundamental for robots aimed at homeenvironments, but difficult to program beforehand. This isbecause the desired motion depends on elements specificto each home: the size and shape of the dishes and thecupboard, and whether the cupboard has a door. In contrast,
if a robot can learn motions from multiple observations ofuser motions, obtaining a controller for motions adapted toeach home is possible.
One of the main difficulties in learning object manipulation is that clustering trajectories in a fixed coordinate
system is not effective, since the trajectories are then notreusable in other conditions. An example is learning to “place
an object on another.” Initial object placements are usually
not fixed within the user’s demonstrations, and in addition
the robot has to generate appropriate motions under various
object placements. Therefore, it is necessary to generalize
relative motions between objects in the learning phase, andsynthesize an appropriate motion trajectory for the given
object placement in the generation phase.
Some recent studies have attempted to solve the difficulties
in learning object manipulation [1]. Regier investigated amodel describing the spatial relationship between two objects[2]. He proposed to model motions as the time evolution of
the spatial relationship between a trajector and a landmark.
Ogawara et al. attempted to model the relative trajectories between two objects by using hidden Markov models(HMMs) [3], [4]. Billard et al. developed a method which
extracted important features from task and joint spaces byintroducing a cost function, which was the weighted sumof task- and joint-space-based reproduction errors. In [5],relative trajectories were modeled by HMMs and optimal
state numbers were determined by model selection. Calinon

II. T HE TASK
A. Learning Reference-Point-Dependent Motions
In cognitive linguistics, a trajector is defined as a participant (object) being focused on. A landmark has a secondary
focus and a trajector is characterized with respect to thelandmark. Words representing spatial relationships such as“away” and “left of” are described in terms of a relationshipbetween a trajector and a landmark [11]. An example of thisis shown in the left-hand figure of Fig. 1. The figure depicts acamera image in which the green puppet (Kermit) is moved
along the dotted line. In the example shown in Fig. 1, thetrajector is Kermit. On the other hand, there are multiple

Komei Sugiura, Naoto Iwahashi and Hideki Kashioka arewith the National Institute of Information and Communications
Technology, 3-5 Hikaridai, Seika, Soraku, Kyoto 619-0289, Japan.

komei.sugiura@nict.go.jp
978-1-61284-455-8/11/$26.00 ©2011 IEEE350

Fig. 1. Left: Example shot of an image stream. Right: Preprocessed visualfeatures obtained from the image stream.

Fig. 2. Relationship among a trajector/landmark, a reference point, andan intrinsic coordinate system. The spheres, ellipsoids, and box represent
objects, and the arrows represent the axes of the intrinsic coordinate systems.
Left: “raise.” Right: “move-closer.”interpretations of the landmark. If the blue box is consideredas the landmark, the label “place-on” can be provided to the
trajectory. However, the label can become “Let Kermit jumpover the green box (jump-over)” if the green box is thoughtof as the landmark.
Motions such as “place-on” and “jump-over” are dependent on landmarks. Other motions, such as “raise an
object,” do not depend on a landmark; however, this motionmeans that the object is moved higher than its original
position. Therefore, “place-on” and “raise” are both modeled
as motions dependent on particular references. In this paper,
those motions are called reference-point-dependent motions.
Here the problem of learning reference-point-dependent
motions in a learning-from-demonstration framework is considered. The reference point of each demonstration is not
specified by the user in the learning phase. This setup isbased on the fact that it is not always possible to specify thereference point for the given object placement only throughlinguistic expressions. Moreover, an easy-to-use interactionof teaching motions is possible by using this setup. Anotherunspecified element is the intrinsic coordinate system type.
Object manipulation involves a spatial relationship between
objects, and it is important to select an appropriate intrinsic
coordinate system, which is the frame of reference [12]intrinsic to a motion, to describe the evolution of the spatial
relationship.
Two examples of this can be shown by “raise” and “movecloser” (Fig. 2). The reference point of “raise” can reasonably
be assumed to be the trajector’s center of gravity. The
intrinsic coordinate system can be a Cartesian coordinate
system, as shown in the left-hand figure. However, in thecase of “move-closer,” another type of intrinsic coordinate
system is necessary. In this case, the x axis of the coordinate
system passes through the centers of gravity of the trajector
and the landmark. The point is that the reference point andintrinsic coordinate system are unobservable in the problem
trying to be solved, and so they must be estimated.

Fig. 3.

Hardware platform used in the experiments.

camera was fixed in the environment. The recorded data wereused for the learning and recognition of motions.
Motion generation experiments were conducted with a
Mitsubishi Heavy Industries PA-10 manipulator with 7 degrees of freedom (DOFs). The manipulator was equippedwith a BarrettHand, a four-DOF multifingered grasper. Motion generation results were examined in an environment
using the manipulator and physical objects such as puppets
and toys. The hardware platform was used for motion recognition/generation experiments as well.
III. M OTION L EARNING BY
R EFERENCE -P OINT-D EPENDENT T RAJECTORY HMM S
A. Modeling Trajectory by Delta Parameters
Let V denote a single training sample for a type of motion
such as “raise” or “place-on.” V consists of the trajectory of
trajector, Ξ, and a set of positions of the static objects, O S ,as follows:V = (Ξ, O S ),
]>
[
>
>
,
Ξ = ξ>
1 , ξ 2 , ..., ξ T ,
[
]>
(1) >
ξ t = x>
xt , ∆(2) x>
,
t ,∆
t

(1)
(2)
(3)

where xt denotes the position vector at time t. The dimensionof xt is denoted by m. We assume that m = 2; however, themethod can be applied to three-dimensional space. ∆(k) xt(k = 0, 1, 2) is called a delta parameter in the speechprocessing community. Delta parameters, which are used forgenerating smooth trajectories, are defined as:

B. Hardware Platform
The experiments were conducted with the platform shownin Fig. 3. The user’s movements were recorded by a Bumblebee 2 stereo vision camera at a rate of 30 [frame/s]. The sizeof each camera image was 320 × 240 pixels. The left-handfigure of Fig. 1 shows an example shot of an image stream,
and the right-hand figure shows its internal representation.
The transformation matrix between the camera coordinate
system and the world coordinate system was fixed since the

∆

(k)

xt =

(k)
L
∑

τ =−L(k)351w(k) (τ )xt+τ ,(k = 0, 1, 2),(4)independent from the positions of objects. Therefore,{}
O R = {O S , x(0), xcenter } , xr r = 1, 2, ..., |O R | ,(7)
where |O R | denotes the number of elements contained inOR .
C. Training Reference-Point-Dependent Trajectory HMMs

Fig. 4. Examples of generated trajectory for motion “place-on” using(a) xt , (b) (xt , ∆(1) xt ), and (c) (xt , ∆(1) xt , ∆(2) xt ), Left: t-y plot.
Right:x-y plot.

r

Let Ck (x ) Ξ denote the trajectory in intrinsic coordinate
system Ck (xr ), where r is the index of the reference point
in Ck (xr ). Henceforth parameters in a specific coordinate
system are represented by using such left superscript. Now,
the optimal k, the optimal set of reference points r, and the
optimal parameters of a probabilistic model λ are searched
for using the following maximum likelihood criterion:

where L(0) = 0 and w(0) (0) = 1. We set L(k) as L(1) =L(2) = 1. w(k) (τ ) is set as w(1) (τ ) = τ2 and w(2) (τ ) =3 2
1
4 τ − 2 , and thus we obtain
11∆(1) xt = − xt−1 + xt+1 ,
2
2
1
1
1
(2)
∆ xt = xt−1 − xt + xt+1 .
4
2
4

(5)

(λ̂, k̂, r̂) = argmax

(6)

λ,k,r

The effect of the delta parameters is shown in Fig. 4. Smooth
trajectories are generated with them.
A left-to-right HMM is used for modeling trajectory Ξ. Let
λ denote the parameters of the N -state HMM. λ consists ofinitial state probability π = {πi }, state transition probability
A, and output probability density functions (OPDFs) b ={bi (·)}. A single-mixture Gaussian distribution is used for
the OPDF.

L
∑

r

r

log P (Ck (x ) Ξl ; Ck (x ) λ),

(8)

l=1

where Ξl represents the trajectory in the lth training sample.
The solution to Equation (8) is explained in [9] in detail.
To estimate the optimal number of states of HMMs, we
use leave-one-out cross-validation [13]. In the first stage ofthe cross-validation process, training samples were divided
into real training set (L − 1 samples) and validation set (onesample).
We obtain state sequence q̄ by averaging the state durations:

B. Intrinsic Coordinate System Typesq̄ = (qinit , q1 , q2 , ..., qT̄ , qfinal )= (Sinit , 1, ..., 1, 2, ..., 2, ..., N, ..., N , Sfinal ),| {z }| {z } | {z }We assume that there are several types of intrinsic coordinate systems, and these are given by the designer. The typeof intrinsic coordinate system is denoted by k. The following
types of intrinsic coordinate systems were defined:

d¯1

d¯2

(9)

d¯N

where d¯i denotes the average duration of the ith state, andT̄ represents the sum of d¯i . q̄ is later used for motion
generation.

C1 A translated camera coordinate system with its origin
at the landmark position. The direction of the y axis
is always vertical. The x axis is inverted when the xcoordinate of x1 is negative after translation.
C2 An orthogonal coordinate system with its origin at thelandmark position. The direction of the x axis is fromthe landmark position toward x1 .
C3 A translated camera coordinate system with its origin
at x1 .
C4 A translated camera coordinate system with its origin
at xcenter .

IV. M OTION G ENERATION BY
R EFERENCE -P OINT-D EPENDENT T RAJECTORY HMM S
In this section, we consider the problem of generating trajectories from reference-point-dependent trajectory HMMs.
We obtain the following trajectory that maximizes the probability of W Ξ̂ conditioned by state sequence q̄ and HMM
parameters W λ as follows:W

In C3 - or C4 -type intrinsic coordinate systems, the reference
point does not have to be estimated since it is fixed to x1 or
xcenter .
From the estimation of k and the reference point xr , theintrinsic coordinate system is obtained in the training sample.
The candidate set of reference points is denoted by O R . The
set of the positions of static objects, O S , has to be includedin O R , since these objects are the landmark candidates. The
first position of the trajector, x(0), is included in O R so thata motion which is dependent only on the object’s trajectory
can be described. In addition, the center of the camera
image, xcenter , is put in O R to describe motion conceptsΞ̂ = argmax P (W Ξ| q̄, W λ)

(10)

WΞ

= argmax P (W Ξ| q̄, C λ, xrtraj , xr ),

(11)

WΞ

where C and W respectively represent the intrinsic and world
coordinate systems, xrtraj denotes the trajector position, andxr denotes the reference point.
Since HMM parameters λ are trained in intrinsic coordinate system C, we first conduct the transformation from C to
W . Let Cµ(k) (s) and C Σ(k) (s) denote the mean vector andthe covariance matrix of the k(= 0, 1, 2)th delta parameters
at state s, respectively.
352

Mean position vector Cµ(0) (s) is transformed by thefollowing homogeneous transformation matrix:[W (0) ] [] [C (0)]R xrtrajµ (s)µ (s) − Cµ(0) (1)=, (12)01
1
1(s = 1, 2, ..., N )Fig. 5.
Output probability density functions for HMM corresponding
to motion “place-on.” Left-hand, middle, and right-hand figures show the
OPDFs of the k(= 0, 1, 2)-th delta parameters, respectively. Vertical andhorizontal axes show the C1 -coordinate system [pixel].

where R denotes the rotation matrix. R is also used forrotating the other mean vectors and the covariance matrices:
W (k)

µ

W (k)

Σ

(s) = R Cµ(k) (s)C (k)(s) = R Σ(s) R

(k = 1, 2)>(k = 0, 1, 2)(13)
B. Training Reference-Point-Dependent Trajectory HMMs

(14)

Fig. 5 shows the output probability density functions(OPDFs) for an HMM trained with trajectories corresponding
to motion label “place-on.” The HMM for which the optimal
state number was estimated as 18 was obtained using TA9. The left-hand, middle, and right-hand figures of Fig. 5
illustrate the distributions corresponding to the k(= 0, 1, 2)th delta parameters, respectively. The center and semimajor(or semiminor) axis of each ellipse stand for the mean andstandard deviation of each distribution. The direction of thestate transition is indicated by the darkness of the color. From
Fig. 5, we can see that the first and last state for position havelarge and small variances, respectively. This indicates that the
trajectory of “place-on” started from an arbitrary point but
converged to a certain point in the end.
Fig. 6 compares the log likelihood of intrinsic coordinate
systems under the condition that TA-9 was used for training.
The figure shows that appropriate coordinate systems wereselected for the motions. Table II summarizes the relationshipof the motions and the corresponding types of their intrinsic
coordinate systems. Table II reveals that C4 was not selected
for any motion, indicating that the learning trajectories ofthese motions in the camera coordinate system were inappropriate. Except for “put-down,” the optimal state number
was 18, and it was 10 for “put-down.”

To generate a trajectory which starts from xrtraj , a positive
small number ε  1 is set as the position variance at thefirst state:WΣ(0) (1) = εI.

(15)

Thus, we obtained W λ from C λ. A solution to Equation (10)is explained in Appendix.
V. E XPERIMENTAL S ETUP
A. Training and Test Sets
In this subsection, we explain the setup of the experiments conducted to obtain trained reference-point-dependent
HMMs for our subsequent motion generation experiments.
A subject was asked to demonstrate motions that manipulated objects placed on a desk. Each motion was given to thesubject in the form of a motion label (verb) such as “raise”
or “rotate.” The environment is illustrated in the left-handfigure of Fig. 1. A training sample was obtained from the
manipulation by preprocessing the recorded image stream ofthe manipulation (see the right-hand figure of Fig. 1).
All trajectories were obtained from the demonstration by
the same subject, Subject A. The following motion labels
were used:jump-over, move-away, move-closer, place-on, putdown, raise, rotate

TABLE IT RAINING AND TEST SETS .

The training sets obtained in the experiments are summarized in Table I. TA-2 to TA-9 were obtained by increasing
the number of training samples, L. The average number ofobjects contained in a sample in TA-9 was 3.1, which meansa typical scene contained one trajector and two landmark
candidates on average.
To avoid generating motions in any of the trained object
placement, another set was obtained from Subject A. Test set
RA-5 consisted of samples of seven kinds of motions, andeach motion has five trajectories.
While the number of states is fixed in most imitation
learning studies, we select the optimal number of statesby cross-validation. We used four kinds of initial HMMs,
each of which had n ∈ {6, 10, 14, 18} states. The averagecomputation time for learning a motion in TA-9 was 9.8
seconds with a quad-core 2.66GHz computer.

Data set

Subject# of motion labels

TA-2
TA-3
..
.
TA-9
RA-5

A
A
..
.
A
A

7
7
..
.
7
7

# of trajectories
per motion label
2
3
..
.
9
5

TABLE IIS ELECTED TYPE OF INTRINSIC COORDINATE SYSTEM .

Type
C1
C2
C3
C4
353

Motion labels
jump-over, place-on
move-away, move-closer
put-down, raise, rotate
N.A.

Fig. 6. Average log likelihood of intrinsic coordinate systems. (a)“placeon,” (b)“jump-over,” (c)“move-closer,” (d)“move-away,” (e)“raise,” (f)“putdown,” (g)“rotate.”

Fig. 7. Generated trajectories from models trained by 2, 4, 6, and 8 samples.

VI. E XPERIMENTAL R ESULTS
The objective of the motion generation experiments was
to show how the generated trajectories improved when thesize of the training set increased.
A. Qualitative Results
The qualitative results regarding the relationship between
the generated trajectories and the training set size are shownin Fig. 7. The objects were placed as in the figure, and the
trajectories were generated for placing Object 3 on Object 2.
“Subject A” represents the trajectory performed by Subject
A under the same condition, and “TA-n” represents the
trajectories generated from models trained with n samples.
The figure shows that the last position of TA-2’s trajectory
was not close to the position where it was supposed to be;
however, the last positions of TA-8’s and Subject A’s were
almost the same. This is explained by the variances of thelast states of the trained models. Indeed, the variances of thevertical positions in the models trained by TA-2, TA-4, TA-6,
and TA-8 were 26.6, 23.5, 5.41, and 7.44, respectively. This
clearly supports the result shown in Fig. 7.
From Fig. 7, we can see differences in the height dimension between the test sample (Subject A’s trajectory) and thegenerated trajectories. This can be explained by the fact thatthe subject’s trajectory was affected by Object 1, however
the generated trajectories were not. In other words, thesubject performed object manipulation with avoiding object
collision, while the proposed method simply generated trajectories from probabilistic models representing appearancelevel object manipulation. Fig. 8 also qualitatively illustrates
that appropriate trajectories were generated for various combinations of relevant objects. For example, trajectories (a)and (b) were originally generated from the same HMM, butthey ended in opposite directions.

Fig. 8. Generated trajectories for various combinations of relevant objects.
The following sets of [motion label, trajector ID, landmark ID] were used:(a) [jump-over, 3, 2], (b) [jump-over, 3, 1], (c) [move-closer, 4, 1], (d)[move-closer, 1, 3], and (e) [rotate, 1].

performed/generated given (rtraj , vi , r), where rtraj denotes
the trajector ID, vi denotes the motion label, and r denotes
the reference point ID.
To evaluate generation error D(Ξ, Ξ̂), Euclidean distancewas used since it is a widely used criterion for time series
data [14]. Let xt and x̂t denote the position at frame t intrajectories Ξ and Ξ̂, respectively. Generation error D(Ξ, Ξ̂)was defined as follows:
1 ∑√|xt − x̂t |2 ,T t=1
TD(Ξ, Ξ̂) =(16)

where D(Ξ, Ξ̂) is normalized by frame length T , and x̂t
is resampled so that x̂t contained the same number ofpoints with xt . Note that D(Ξ, Ξ̂) evaluates the similarityof the two position sequences without explicitly evaluating
the similarity in terms of the velocity and acceleration1 .
Fig. 9 shows a quantitative evaluation of the proposed
method. In the figure, D(Ξ, Ξ̂), which was averaged by
the number of trajectories, is plotted against the number oftraining samples. Fig. 9 reveals that D(Ξ, Ξ̂) for motions“place-on,” “jump-over,” “move-closer” and “move-away”
decreased with the increase in the number of training samples. The generation error converged after seven samples
were used for training. This indicates that seven samples

B. Quantitative Results
To investigate the generation error decrease with theincrease in training set size, trajectories were generated
from the models obtained from training sets TA-2 to TA9. A trajectory performed by Subject A, Ξ, and generated
trajectory Ξ̂ can be compared since both Ξ and Ξ̂ were

1 The same is equally true of most standard techniques for measuring
similarity between two time series such as simple Dynamic Time Warping.

354

were sufficient for the proposed method to learn motions
performed by Subject A in the experiments.
Fig. 9 illustrates that the “move-away” trajectory has larger
error than other trajectories for the following reason. In the
training and test sets, each trajectory shown by the subject
started from the trajector’s initial position and ended atvarious points, increasing the distance from the landmark.
Therefore, the difference between the generated and Subject
A’s trajectories increased particularly at the final position.
The large variances in the trained OPDFs of the “moveaway” HMM also support this explanation. On the other
hand, the error of “move-closer” was smallest among othermotions when the sample number was nine. This indicatesthat the OPDFs of the “move-closer” HMM had smaller
variances.

method did not deal with the correspondence problem, since
the main focus is the estimation of relevant objects.
B. Related Work
Synthesizing human-like animation and programming humanoid robots to perform new tasks are attractive applications of methods based on the learning-from-observation
framework [16]. Inamura et al. used HMMs to model motion
patterns such as “swing” and “walking” [18]. The trained
HMMs were mapped in a space by applying multidimensional scaling (MDS) to Kullback-Leibler divergences between the parameters of the HMMs. Novel motions weregenerated by the interpolation and the extrapolation of the
HMMs in the mapped space [19]. A hidden semi-Markov
model (HSMM), which is an HMM with state duration
probability distributions, was used for learning walking patterns in [20]. In [21], a Gaussian Process Dynamical Model(GPDM) was proposed and applied to human motion capture
data. A GPDM was comprised of a low-dimensional latent
space with associated dynamics and a map from the latent
space to an observation space.
These studies focused on the learning motions in a fixed
coordinate system. In contrast, the proposed method is able toestimate the optimal intrinsic coordinate system to generalize
motion trajectories.
VIII. C ONCLUSION

Fig. 9. Evolution of generation error D(Ξ, Ξ̂) plotted against number oftraining samples.

This paper presented an imitation learning method for
object-manipulating motions. Conventional methods forlearning object manipulation from demonstrations had trouble dealing with motions that did and did not depend on alandmark with the same learning framework. In contrast, oneof the contributions of this paper is providing an algorithm
for estimating reference points to overcome such a limitation.
The proposed method was applied to object-manipulationand evaluated its generation errors. We also obtained
an accuracy of 90% for motion recognition, which was
not presented due to space limitations. The motion
recognition results are explained in detail in [9]. Even
though the results in this paper are promising, the learned
motions were limited to an appearance level. In the future
the application of machine learning to the problem ofunderstanding the intent of an action will be explored,since that is necessary to develop a robot which imitates
the goal of an action. Demo video clips are availableat
http://mastarpj.nict.go.jp/˜ksugiura/
video_gallery/video_gallery_en.html.

Fig. 10 shows sequential photographs of the manipulator
executing trajectories generated by the proposed method. The
grasping motions were not obtained through learning but
were implemented by the designer.
VII. D ISCUSSION
A. Future Work
Motion generation from trajectory HMMs is not limited
to left-to-right HMMs. The method explained in Section III
and Appendix can be applied to periodic motions such as
walking and waving hands. While the state sequences wereobtained from training samples in this study, estimating state
sequences by an EM algorithm is possible [15]. However, itneeds more computation cost than the method adopted in ourstudy.
Arguments have swirled around a number of key issues inimitation learning [1], [16]. Among them, what to imitate andhow to imitate are fundamental questions. The first issue is todetermine what perceptual aspects are relevant to a task whenattempting to imitate a human [16]. The proposed method
focused on the estimation of relevant objects in objectmanipulation tasks based on the idea that the referencesof motions performed by humans were implicit. On theother hand, the correspondence problem [17], or the “how toimitate” question, poses a problem when a user is trying totransfer skills to robots with different bodies. The proposed

A PPENDIX
This appendix explains the trajectory-generation method
proposed in [15]. For simplicity, the notation of the world
coordinate system shown by the left superscript is omitted
in the below, so for example W λ is written as λ.
355

Fig. 10.

Sequential photographs of manipulator executing “place-on” manipulation trajectories.

The OPDFs of the HMM λ are Gaussian distributions,
therefore Equation (10) can be rewritten as[3] K. Ogawara, J. Takamatsu, H. Kimura, and K. Ikeuchi, “Generation ofa task model by integrating multiple observations of human demonstrations,” in Proceedings of the 2002 IEEE International Conferenceon Robotics and Automation, 2002, pp. 1545–1550.
Ξ̂ = argmax log P (Ξ|q, λ)(17)[4] ——, “Modeling manipulation interactions by hidden Markov modΞ{}els,” in Proceedings of the 2002 IEEE/RSJ International Conference1
on Intelligent Robots and Systems, 2002, pp. 1096–1101.
= argmax − Ξ> Σ−1 Ξ + Ξ> Σ−1 µ + K ,(18)[5] A. Billard, S. Calinon, and F. Guenter, “Discriminative and adap2
Ξtive imitation in uni-manual and bi-manual tasks,” Robotics andAutonomous Systems, vol. 54, no. 5, pp. 370–384, 2006.
where
[
]
[6] S. Calinon and A. Billard, “A probabilistic programming by demon−1−1stration framework handling constraints in joint space and task space,”
Σ−1 = diag Σ−1(19)
qL0 +1 , ΣqL0 +2 , ..., ΣqT −L0
in Proceedings of 2008 IEEE/RSJ International Conference on Intel[]>
ligent Robots and Systems, 2008, pp. 367–372.
>>>µ = µqL0 +1 , µqL0 +2 , ..., µqT −L0(20)[7] K. Sugiura and N. Iwahashi, “Learning object-manipulation verbs forhuman-robot communication,” in Proceedings of the 2007 workshop
L0 = max L(k)(21)
on Multimodal interfaces in semantic interaction, 2007, pp. 32–38.
k[8] ——, “Motion recognition and generation by combining referencepoint-dependent probabilistic models,” in Proceedings of 2008
µqi and Σqi denote the 3m × 1 mean vector and the 3m ×IEEE/RSJ International Conference on Intelligent Robots and Systems,
3m covariance matrix, respectively, and K is a constant. q
2008, pp. 852–857.
denotes an arbitrary state sequence, which is set as q = q̄ in[9] K. Sugiura, N. Iwahashi, H. Kashioka, and S. Nakamura, “Learning,
generation, and recognition of motions by reference-point-dependent
this study.
probabilistic models,” Advanced Robotics, vol. 25, no. 6-7, pp. 825–
Equation (4) can be rewritten in matrix form:
848, 2011.
[10] “Robocup@home rules & regulations.” [Online]. Available: http:Ξ = W x,(22)//www.ai.rug.nl/robocupathome/[11] R. W. Langacker, Foundations of Cognitive Grammar: Theoretical
where
Prerequisites. Stanford Univ Pr, 1987.
[12] L. A. Carlson-Radvansky and D. E. Irwin, “Frames of reference in
[ > >
]
> >
x = x1 , x2 , ..., xT ,vision and language: where is above?” Cognition, vol. 46, no. 3, pp.
223–244, 1993.
>
W = [wL0 +1 , wL0 +2 , ..., wT −L0 ][13] C. M. Bishop, Pattern Recognition and Machine Learning. Springer,
[
]
2006.
(0)
(1)
(2)
wt = wt , wt , wt
[14] E. Keogh and S. Kasetty, “On the need for time series data mining[
benchmarks: A survey and empirical demonstration,” Data Mining and(k)Knowledge Discovery, vol. 7, no. 4, pp. 349–371, 2003.
wt = 0m×(t−L(k) −1)m , w(k) (−L(k) )Im×m , ..., w(k) (0)Im×m ,[15] K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura,]>“Speech parameter generation algorithms for HMM-based speech..., w(k) (L(k) )Im×m , 0m×(T −t−L(k) )msynthesis,” in Proceedings of ICASSP, 2000, pp. 1315–1318.
[16] C. Breazeal and B. Scassellati, “Robots that imitate humans,” Trends
in Cognitive Science, vol. 6, pp. 481–487, 2002.
To solve Equation (17) under condition (22), we solve the[17] C. Nehaniv and K. Dautenhahn, “The correspondence problem,” infollowing equation:
Imitation in Animals and Artifacts. MIT Press, 2002, pp. 41–61.
[18] T. Inamura, I. Toshima, H. Tanie, and Y. Nakamura, “Embodied∂ log P (W x|q, λ)=0(23)
symbol emergence based on mimesis theory,” International Journal∂xof Robotics Research, vol. 23, no. 4, pp. 363–377, 2004.
[19] T. Inamura and T. Shibata, “Geometric proto-symbol manipulation
From Equations (18), (22) and (23) we obtaintowards language-based motion pattern synthesis and recognition,” inProceedings of 2008 IEEE/RSJ International Conference on Intelligent
W > Σ−1 W x = W > Σ−1 µ(24)Robots and Systems, 2008, pp. 334–339.
[20] N. Niwase, J. Yamagishi, and T. Kobayashi, “Human walking motion
3
Equation (24) is solved with O(T m ) operations by usingsynthesis with desired pace and stride length based on HSMM,” IEICE
Cholesky decomposition.
transactions on information and systems, vol. 88, no. 11, pp. 2492–
2499, 2005.
[21] J. M. Wang, D. J. Fleet, and A. Hertzmann, “Gaussian process
R EFERENCES
dynamical models,” in Advances in Neural Information Processing[1] V. Krüger, D. Kragic, A. Ude, and C. Geib, “The meaning of action:
Systems, vol. 18. MIT Press, 2006, pp. 1441–1448.
a review on action recognition and mapping,” Advanced Robotics,
vol. 21, no. 13, pp. 1473–1501, 2007.
[2] T. Regier, The Human Semantic Potential: Spatial Language andConstrained Connectionism. Bradford Books, 1996.

356