Generating Natural Motion in an Android
by Mapping Human Motion
Daisuke Matsui1 , Takashi Minato1 , Karl F. MacDorman2 , and Hiroshi Ishiguro1
1

Department of Adaptive Machine Systems, Graduate School of Engineering, Osaka University
2

School of Informatics, Indiana University
daisuke.matsui@ams.eng.osaka-u.ac.jp

Abstract
One of the main aims of humanoid robotics is to develop robots that are capable of interacting
naturally with people. However, to understand the essence of human interaction, it is crucial toinvestigate the contribution of behavior and appearance. Our group’s research explores these relationships by developing androids that closely resemble human beings in both aspects. If humanlike
appearance causes us to evaluate an android’s behavior from a human standard, we are more likely
to be sensitive to deviations from human norms. Therefore, the android’s motions must closely
match human performance to avoid looking strange. This paper proposes a method to implement
motions that look human by mapping their three-dimensional appearance from a human subject tothe android and then evaluating the verisimilitude of the visible motions using a motion capturesystem. This approach has several advantages over current research, which has focused on copyinga person’s moving joint angles to a robot: (1) in an android robot with many degrees of freedom
and kinematics that differs from that of a human being, it is difficult to calculate which joint angles
would make the robot’s posture appear similar to the human subject; and (2) the motion that weperceive is at the robot’s surface, not necessarily at its joints, which are often hidden from view.

keywords: Learning control systems, motion analysis, humanlike motion, human-robot imitation,
android science, appearance and behavior problem.

1

Introduction

Much effort in recent years has focused on the development of such mechanical-looking humanoid robots
as Honda’s Asimo and Sony’s Qrio with the goal of partnering them with people in daily situations.
Just as an industrial robot’s purpose determines its appearance, a partner robot’s purpose will also
determine its appearance. Partner robots generally adopt a roughly humanoid appearance to facilitatecommunication with people, because natural interaction is the only task that requires a humanlike
appearance. In other words, humanoid robots mainly have significance insofar as they can interact
1naturally with people. Therefore, it is necessary to discover the principles underlying natural interaction
to establish a methodology for designing interactive humanoid robots.
Kanda et al. (Kanda et al., 2002) have tackled this problem by evaluating how the behavior of thehumanoid robot “Robovie” affects human-robot interaction. But Robovie’s machine-like appearance
distorts our interpretation of its behavior because of the way the complex relationship between appearance and behavior influences the interaction. Most research on interactive robots has not evaluated theeffect of appearance (for exceptions, see (Goetz et al., 2003; DiSalvo et al., 2002)) — and especially not
in a robot that closely resembles a person . Thus, it is not yet clear whether the most comfortable andeffective human-robot communication would come from a robot that looks mechanical or human. However, we may infer a humanlike appearance is important from the fact that human beings have developed
neural centers specialized for the detection and interpretation of hands and faces (Grill-Spector et al.,
2004; Farah et al., 2000; Carmel and Bentin, 2002). A robot that closely resembles humans in both looks
and behavior may prove to be the ultimate communication device insofar as it can interact with humans
the most naturally.1 We refer to such a device as an android to distinguish it from mechanical-looking
humanoid robots. When we investigate the essence of how we recognize human beings as human, it
will become clearer how to produce natural interaction. Our study tackles the appearance and behavior
problem with the objective of realizing an android and having it be accepted as a human being (Minato
et al., 2006).
Ideally, to generate humanlike movement, an android’s kinematics should be functionally equivalent
to the human musculoskeletal system. Some researchers have developed a joint system that simulates
shoulder movement (Okada et al., 2002) and a muscle-tendon system to generate humanlike movement(Yoshikai et al., 2003). However, these systems are too bulky to be embedded in an android withoutcompromising its humanlike appearance. Given current technology, we embed as many actuators as
possible to provide many degrees of freedom insofar as this does not interfere with making the android
look as human as possible (Minato et al., 2006). Under these constraints, the main issue concerns how
to move the android in a natural way so that its movement may be perceived as human.
A straightforward way to make a robot’s movement more humanlike is to imitate human motion.
Kashima and Isurugi (Kashima and Isurugi, 1998) extracted essential properties of human arm trajectories and designed an evaluation function to generate robot arm trajectories accordingly. Anothermethod is to copy human motion as measured by a motion capture system to a humanoid robot. Riley
et al. (Riley et al., 2000) and Nakaoka et al. (Nakaoka et al., 2003) calculated a subject’s joint trajectories from the measured positions of markers attached to the body and fed them to the joints ofa humanoid robot. In these studies the authors assumed the kinematics of the robot to be similar tothat of a human body. However, since the actual kinematics and joint structures are different between
human and robot bodies, calculating the joint angles from only the human motion data could in somecases result in visibly different motion. This is especially a risk for androids because their humanlike
form makes us more sensitive to deviations from human ways of moving. Thus, slight differences could
1 We use the term natural to denote communication that flows without seeming stilted, forced, bizarre, or inhuman.

2

familiarity

100%

uncanny valley

healthy
personhumanoid robot

toysimilarity

100%
moving corpse

Figure 1: Uncanny valley (Mori, 1970; Fong et al., 2003).

strongly influence whether the android’s movement is perceived as natural or human. Furthermore,these studies did not evaluate the naturalness of robot motions.
Hale et al.( Hale et al., 2003) proposed several evaluation functions to generate a joint trajectory (e.g.,
minimization of jerk) and evaluated the naturalness of generated humanoid robot movements according
to how human subjects rated their naturalness. In the computer animation domain, researchers havetackled a motion synthesis with motion capture data (e.g., (Gleicher, 1998)). However, we cannot apply
their results directly; we must instead repeat their experiment with an android because the resultsfrom an android testbed could be quite different from those of a humanoid testbed. For example, Mori
described a phenomenon he termed the “uncanny valley” (Mori, 1970; Fong et al., 2003), which relates
to the relationship between how humanlike a robot appears and a subject’s perception of familiarity.
According to Mori, a robot’s familiarity increases with its similarity until a certain point is reached atwhich slight “nonhuman” imperfections cause the robot to appear repulsive (Fig. 1). This would be
an issue if the similarity of androids fell into the chasm. (Mori believes mechanical-looking humanoid
robots lie on the left of the first peak.) This nonmonotonic relationship can distort the evaluation
proposed in existing studies. Therefore, it is necessary to develop a motion generation method in which
the generated “android motion” is perceived as human.
This paper proposes a method to transfer human motion measured by a motion capture system tothe android by copying changes in the positions of body surfaces. This method is called for because theandroid’s appearance demands movements that look human, but its kinematics is sufficiently different
that copying joint-angle information would not yield good results. Comparing the similarity of theandroid’s visible movement to that of a human being enables us to develop more natural movements forthe android.
In the following sections, we describe the developed android and mention the problem of motion
transfer and our basic idea about the way to solve it. Then we describe the proposed method in detailand show experimental results from applying it to the android.

3

Figure 2: The developed android “Repliee Q2”.

Figure 3: Examples of motion and facial expressions.

2

The Developed Android

Fig. 2 shows the developed android called Repliee Q2. The android is modeled after a Japanese woman.
The standing height is about 160 cm. The skin is composed of a kind of silicone that feels like humanskin. The silicone skin covers the neck, head, and forearms, with clothing covering other body parts.
Unlike Repliee R1 (Minato et al., 2004), the silicone skin does not cover the entire body so as to facilitate
flexibility and a maximal range of motion. Forty-two highly sensitive tactile sensors composed of PVDF
film are mounted under the android’s skin and clothes over the entire body, except for the shins, calves,and feet. Since the output value of each sensor corresponds to its deforming rate, the sensors candistinguish different kinds of touch ranging from stroking to hitting. The soft skin and tactile sensors
give the android a human appearance and enable natural tactile interaction.
The android is driven by air actuators (air cylinders and air motors) that give it 42 degrees of
freedom (DoFs) from the waist up. The legs and feet are not powered; it can neither stand up nor movefrom a chair. A high power-to-weight ratio is necessary for the air actuator in order to mount multiple

4

1

eyes

2
parallel link
neck

16

3

15
14

5
6

17
right arm

7

13

4
18
22

8

19
torso 23

20
parallel link

21

left arm

25
parallel link

9
24
10

right hand

12

11
parallel link

left hand

Figure 4: Kinematic structure of the android.

actuators in the human-sized body.
The configuration of the DoFs is shown in Table 1. Fig. 4 shows the kinematic structure of the body,excluding the face and fingers. Some joints are driven by the air motors and others adopt a slider-crankmechanism. The DoFs of the shoulders enable them to move up and down and backwards and forwards;
this shoulder structure is more complicated than that of most existing humanoid robots. Moreover,parallel link mechanisms adopted in some parts complicate the kinematics of the android, for example
in the waist. The android can generate a wide range of motions and gestures as well as various kindsof micro-motions such as the shoulder movements typically caused by human breathing. Furthermore,
the android can make some facial expressions and mouth shapes, as shown in Fig. 3. Because theandroid has servo controllers, it can be controlled by sending data on the desired joint angles (cylinder
positions and rotor angles) from a host computer. The compliance of the air actuator makes for safer
interaction, with movements that are generally smoother than other systems typically used. Because of
the complicated dynamics of the air actuator, executing the trajectory tracking control is difficult.

5

Table 1: The DoF configuration of Repliee Q2.
Degree of freedom
Eyes

pan×2 + tilt×1

Face

eyebrows×1 + eyelids×1 + cheeks×1

Mouth

7 (including the upper and lower lips)

Neck

3

Shoulder

5×2

Elbow

2×2

Wrist

2×2

Fingers

2×2

Torso

4

Feedforward
control inputfFeedforward
controller

Humanmarker pos.
h

x

Error
signalEstimated
Error ofhuman
joint angle joint angle
Transform

T: xh q^ h

q^ h + ∆q^ d

q

a∆qb

Feedback controller^qa
Estimated
androids
joint angle

Transform+

Androids
joint angleaq∆qb + qd

Feedback
control input

Android

ControlinputxaAndroidsmarker pos.

T: xa q^ aFigure 5: The android control system.

3

Transferring Human Motion

3.1

The basic idea

One method to realize humanlike motion in a humanoid robot is through imitation. Thus, we consider
how to map human motion to the android. Most previous research assumes the kinematics of thehuman body is similar to that of the robot except for the scale. Thus, they aim to reproduce humanmotion by reproducing kinematic relations across time and, in particular, joint angles between links.
For example, the three-dimensional locations of markers attached to the skin are measured by a motion
capture system, the angles of the body’s joints are calculated from these positions, and these angles
are transferred to the joints of the humanoid robot. It is assumed that by using a joint angle space(which does not represent link lengths), morphological differences between the human subject and the6Error ofmarker pos.

Humanmarker pos.
h

x

+

∆ xd

-

Androidsmarker pos.

Feedback controller+∆qb + qd

Androidxa(a) Feedback of maker position error

Humanmarker pos.
hx

Estimated
Error ofhuman
joint angle joint angle
Transform

T: xh q^ h

q^ h + ∆q^ d-Androids
joint angle

Feedback controller+∆qb + qd

Androidqa(b) Error estimation with the androids joint angle measured by the potentiometerHumanmarker pos.
hx

Estimated
Error ofhuman
joint angle joint angle
Transform^q^ h + ∆q^ d

T: xh qh-

Androidsmarker pos.

Feedback controllerq^ aEstimated
androids
joint angle+∆qb + qd

Androidxa

Transform

T: xa q^ a(c) Error estimation with the androids joint angle estimated from the androids marker position
Figure 6: The feedback controller with and without the estimation of the android’s joint angle.

humanoid robot can be ignored.
However, there is potential for error in calculating a joint angle from motion capture data. The joint
positions are assumed to be the same between a humanoid robot and the human subject who serves asa model; however, the kinematics in fact differs. For example, the kinematics of Repliee Q2’s shoulder
differs significantly from those of human beings. Moreover, as human joints rotate, each joint’s center ofrotation changes, but joint-based approaches generally assume this is not so. These errors are perhaps
more pronounced in Repliee Q2, because the android has many degrees of freedom and the shoulder
has a more complex kinematics than existing humanoid robots. These errors are more problematic foran android than a mechanical-looking humanoid robot because we expect natural human motion fromsomething that looks human and are disturbed when the motion instead looks inhuman.
To create movement that appears human, we focus on reproducing positional changes at the body’s
surface rather than changes in the joint angles. We then measure the postures of a person and theandroid using a motion capture system and find the control input to the android so that the postures7

of person and android become similar to each other.

3.2

The method to transfer human motion

We use a motion capture system to measure the postures of a human subject and the android. Thissystem can measure the three-dimensional positions of markers attached to the surface of bodies in a
global coordinate space. First, some markers are attached to the android so that all joint motions can be
estimated. The reason for this will become clear later. Then the same number of markers are attached
to corresponding positions on the subject’s body. We must assume the android’s surface morphology is
not too different from the subject’s.
We use a three-layer neural network to construct a mapping from the subject’s posture to theandroid’s control input, which is the desired joint angle. The reason for the network is that it is difficult
to obtain the mapping analytically. To train a neural network to map from xh to q a would require
thousands of pairs of xh , q a as training data, and the subject would need to assume the posture of theandroid for each pair. We avoid this prohibitively lengthy task in data collection by adopting feedback
error learning (FEL) to train the neural network. Kawato et al. (Kawato et al., 1987) proposed feedback
error learning as a principle for learning motor control in the brain. This employs an approximate wayof mapping sensory errors to motor errors that subsequently can be used to train a neural network (orother method) by supervised learning. Feedback-error learning neither prescribes the type of neuralnetwork employed in the control system nor the exact layout of the control circuitry. We use it toestimate the error between the postures of the subject and the android and feed the error back to the
network.
Fig. 5 shows the block diagram of the control system, where the network mapping is shown as
the feedforward controller. The weights of the feedforward neural network are learned by means of a
feedback controller. The method has a two-degrees-of-freedom control architecture. The network tunes
the feedforward controller to be the inverse model of the plant. Thus, the feedback error signal isemployed as a teaching signal for learning the inverse model. If the inverse model is learned exactly, theoutput of the plant tracks the reference signal by feedforward control. The subject and android’s marker
positions are represented in their local coordinates xh , xa ∈ R3m ; the android’s joint angles q a ∈ Rn
can be observed by a motion capture system and a potentiometer, where m is the number of markers
and n is the number of DoFs of the android.
The feedback controller is required to output the feedback control input ∆q b so that the error
in the marker’s position ∆xd = xa − xh converges to zero (Fig. 6(a)). However, it is difficult toobtain ∆q b from ∆xd . To overcome this, we assume the subject has roughly the same kinematics as
the android and obtain the estimated joint angle q̂ h simply by calculating the Euler angles (hereafter
the transformation from marker positions to joint angles is described as T ).2 Converging q̂ a to q h
2 There are alternatives to using the Euler angles such as angle decomposition (Grood and Suntay, 1983), which has theadvantage of providing a sequence independent representation, or least squares, to calculate the helical axis and rotational
angle (Challis, 1995; Veldpaus et al., 1988). This last method provides higher accuracy when many markers are used but

8Marker

Subject

Android

Figure 7: The marker positions corresponding to each other.

does not always produce identical postures because q̂ h is an approximate joint angle that may include
transformation error (Fig. 6(b)). Then we obtain the estimated joint angle of the android q̂ a using thesame transformation T and the feedback control input to converge q̂ a to q̂ h (Fig. 6(c)). This techniqueenables xa to approach xh . The feedback control input approaches zero as learning progresses, while theneural network constructs the mapping from xh to the control input q d . We can evaluate the apparent
posture by measuring the android posture.
In this system we could have made another neural network for the mapping from xa to q a using
only the android. As long as the android’s body surfaces are reasonably close to the subject’s, we canuse the mapping to make the control input from xh . Ideally, the mapping must learn every possible
posture, but this is quite difficult. Therefore, it is still necessary for the system to evaluate the error in
the apparent posture.

4

Experiment to Transfer Human Motion

4.1

Experimental setting

To verify the proposed method, we conducted an experiment to transfer human motion to the android
Repliee Q2. We used 21 of the android’s 42 DoFs (n = 21) by excluding the 13 DoFs of the face, the4 of the wrists (the cylinders 11, 12, 20, and 21 in Fig. 4), and the 4 of the fingers. We used a Hawk

has an increased risk of marker crossover.

9

Position vectors on theshoulder and chest (5)MarkerPosition
vectorp1,...,p3

MarkerPosition
vectorp4,...,p8

Coordinates fixed to the chest
Position vectors
on the head (3)Coordinates
fixed to the chest

Coordinates
fixed to the waist

Position
vectorp9,...,p20
a vector connecting
two neighboring markers

MarkerPosition vectors on
the arms (12)

[]

p1
.
x= .
.
p20

Figure 8: The representation of the marker positions. A marker’s diameter is about 18 mm.

Digital System,3 which can track more than 50 markers in real-time. The system is highly accuratewith a measurement error of less than 1 mm. Twenty markers were attached to the subject and another
20 to the android as shown in Fig. 7 (m = 20). Because the android’s waist is fixed, the markers on
the waist set the frame of reference for an android-centered coordinate space. To facilitate learning, we
introduce a representation of the marker position xh , xa as shown in Fig. 8. The effect of waist motions
are removed with respect to the markers on the head. To avoid accumulating the position errors at theend of the arms, vectors connecting neighboring pairs of markers represent the positions of the markers
on the arms. We used arc tangents for the transformation T , in which the joint angle is an angle between
two neighboring links where a link consists of a straight line between two markers.
The feedback controller outputs ∆q b = K∆q̂ d , where the gain K consists of a diagonal matrix.
There are 60 nodes in the input layer (20 markers × x, y, z), 300 in the hidden layer, and 21 in the
output layer (for the 21 DoFs). Using 300 units in the hidden layer provided a good balance between
computational efficiency and accuracy. Using significantly fewer units resulted in too much error, while
using significantly more units provided only marginally higher accuracy but at the cost of slower convergence. The error signal to the network is t = α∆q b , where the gain α is a small number. The samplingtime for capturing the marker positions and controlling the android is 60 ms. Another neural network
which has the same structure previously learned the mapping from xa to q a to set the initial values of
the weights. We obtained 50,000 samples of training data (xa and q a ) by moving the android randomly.
The learned network is used to set the initial weights of the feedforward network.
3 Motion Analysis Corporation, Santa Rosa, California.

http://www.motionanalysis.com/

10

70
65

Average of feedback error

60
55
50
45
40
35
30
25
20

0

100

200

300

400

500

600

700

Step

Figure 9: The change of the feedback control input with learning of the network.

Average error of marker position [mm]

140

120

100

80

60

40

20

0

20

40

60

80

100

120

140

160

Step

Figure 10: The change of the position error with learning of the network.

4.2

Experimental results and analysis

4.2.1

Surface similarity between the android and subject

The proposed method assumes a surface similarity between the android and the subject. However, themale subject whom the android imitates in the experiments was 15 cm taller than the women after
whom the android was modeled. To check the similarity, we measured the average distance between
corresponding pairs of markers when the android and subject make each of the given postures; the value
was 31 mm (see the Fig. 7). The gap is small compared to the size of their bodies, but it is not smallenough.

11

Height of the fingers [mm]

500

400
Subject
Android
Error

300

200

100

0

0

5

10

15

20

25

30

35

40

Time [x60 msec]

Figure 11: The step response of the android.

4.2.2

The learning of the feedforward network

To show the effect of the feedforward controller, we plot the feedback control input averaged among thejoints while learning from the initial weights in Fig. 9. The abscissa denotes the time step (the samplingtime is 60 ms.) Although the value of the ordinate does not have a direct physical interpretation,
it corresponds to a particular joint angle. The subject exhibited various fixed postures. When thesubject started to make the posture at step 0, error increased rapidly because network learning had
not yet converged. The control input decreases as learning progresses. This shows that the feedforward
controller learned so that the feedback control input converges to zero.
Fig. 10 shows the average position error of a pair of corresponding markers. The subject also gave
an arbitrary fixed posture. The position errors and the feedback control input both decreased as thelearning of the feedforward network converged. The result shows the feedforward network learned themapping from the subject’s posture to the android control input, which allows the android to adopt thesame posture. The android’s posture could not match the subject’s posture when the weights of thefeedforward network were left at their initial values. This is because the initial network was not given
every possible posture in the pre-learning phase. The result shows the effectiveness of the method toevaluate the apparent posture.
4.2.3

Performance of the system at following fast movements

To investigate the performance of the system, we obtained a step response using the feedforward network
after it had learned enough. The subject put his right hand on his knee and quickly raised the hand rightabove his head. Fig. 11 shows the height of the fingers of the subject and android. The subject started
to move at step 5 and reached the final position at step 9, approximately 0.24 seconds later. In thiscase the delay is 26 steps or 1.56 seconds. The arm moved at roughly the maximum speed permitted by
the hardware. The android arm cannot quite reach the subject’s position because the subject’s position12

was outside of the android’s range of motion. Clearly, the speed of the subject’s movement exceeds theandroid’s capabilities. This experiment is an extreme case. For less extreme gestures, the delay will
be much less. For example, for the sequence in Fig. 12, the delay was on average seven steps or 0.42
seconds.
4.2.4

The generated android motion

Fig. 12 shows the subject’s postures during a movement and the corresponding postures of the android.
The value denotes the time step. The android followed the subject’s movement with some delay (themaximum is 15 steps, that is, 0.9 seconds). The trajectories of the positions of the android’s markers
are considered to be similar to those of the subject, but errors still remain, and they cannot be ignored.
While we can recognize that the android is making the same gesture as the subject, the quality of the
movement is not the same. There are a couple of major causes of this:• The kinematics of the android is too complicated to represent with an ordinary neural network.
To avoid this limitation, it is possible to introduce the constraint of the body’s branching in
the network connections. Another idea is to introduce a hierarchical representation of the mapping. A human motion can be decomposed into a dominant motion that is at least partly driven
consciously and secondary motions that are mainly nonconscious (e.g., contingent movements tomaintain balance, such autonomic responses as breathing). We are trying to construct a hierarchical representation of motion not only to reduce the computational complexity of learning but
to make the movement appear more natural.
• The method deals with a motion as a sequence of postures; it does not precisely reproduce higher
order properties of motion such as velocity and acceleration because varying delays can occur
between the subject’s movement and the android’s imitation of it. If the subject moves very
quickly, the apparent motion of the android differs. Moreover, a lack of higher order properties
prevents the system from adequately compensating for the dynamic characteristics of the android
and the delay of the feedforward network.
• The proposed method is limited by the speed of motion. It is necessary to consider the properties
to overcome the restriction, although the android has absolute physical limitations such as a fixedcompliance and a maximum speed that is less than that of a typical human being.
Although physical limitations cannot be overcome by any control method, there are ways of finessing
them to ensure movements still look natural. For example, although the android lacks the opponent
musculature of human beings, which affords a variable compliance of the joints, the wobbly appearanceof such movements as rapid waving, which are high in both speed and frequency, can be overcome byslowing the movement and removing repeated closed curves in the joint angle space to eliminate lag
caused by the slowed movement. If the goal is humanlike movement, one approach may be to query a
database of movements that are known to be humanlike to find the one most similar to the movement13

made by the subject, although this begs the question of where those movements came from in the firstplace. Another method is to establish criteria for evaluating the naturalness of a movement (Kashima
and Isurugi, 1998). This is an area for future study.

4.3

Required improvement and future work

In this paper we focus on reproducing positional changes at the body’s surface rather than changes in
the joint angles to generate the android’s movement. Fig. 6(a) is a straightforward method to implement
the idea. This paper has adopted the transformation T from marker positions to estimated joint angles
because it is difficult to derive a feedback controller which produces the control input ∆q b only fromthe error in the marker’s positional error ∆xd analytically. We actually do not know which joints
should be moved to remove a positional error at the body’s surface. This relation must be learned,
however, the transformation T could disturb the learing. Hence, it is not generally guaranteed that the
feedback controller which converges the estimated joint angle q̂ a to q̂ h enables the marker’s position xa
to approach xh . The assumption that the android’s body surfaces are reasonably close to the subject’s
could avoid this problem, but the feedback controller shown in Fig. 6(a) is essentially necessary formapping the apparent motion. It is possible to find out how the joint changes relate to the movementsof body surfaces by analyzing the weights of the neural network of the feedforward controller. A feedback
controller could be designed to output the control input based on the error in the marker’s position withthe analyzed relation. Concerning the design of the feedback controller, Oyama et al. (Oyama et al.,2001b; Oyama et al., 2001c; Oyama et al., 2001a) proposed several methods for learning both of feedback
and feedforward controllers using neural networks. This is one potential method to obtain the feedback
controller shown in Fig. 6(a). Assessment of and compensation for deformation and displacement of thehuman skin, which cause marker movement with respect to the underlying bone (Leardini et al., 2005),
are also useful in designing the feedback controller.
We have not dealt with the android’s gaze and facial expressions in the experiment; however, if gazeand facial expressions are unrelated to hand gestures and body movements, the appearance is often
unnatural, as we have found in our experiments. Therefore, to make the android’s movement appear
more natural, we have to consider a method to implement the android’s eye movements and facial
expressions.

5

Conclusion

This paper has proposed a method of implementing humanlike motions by mapping their three-dimensional
appearance to the android using a motion capture system. By measuring the android’s posture and comparing it to the posture of a human subject, we propose a new method to evaluate motion sequences
along bodily surfaces. Unlike other approaches that focus on reducing joint angle errors, we consider
how to evaluate differences in the android’s apparent motion, that is, motion at its visible surfaces. The

14

experimental results show the effectiveness of the evaluation: the method can transfer human motion.
However, the method is restricted by the speed of the motion. We have to introduce a method to deal
with the dynamic characteristics (Ben=Amor et al., 2007) and physical limitations of the android. We
also have to evaluate the method with different subjects. We would expect to generate the most naturaland accurate movements using a female subject who is about the same height as the original woman onwhich the android is based. Moreover, we have to evaluate the human likeness of the visible motions by
the subjective impressions the android gives experimental subjects and the responses it elicits, such aseye contact (Minato et al., 2006; Shimada et al., 2006), autonomic responses, and so on. Research inthese areas is in progress.

Acknowledgment
We developed the android in collaboration with Kokoro Company, Ltd.

References
Ben=Amor, H., Ikemoto, S., Minato, T., Jung, B., and Ishiguro, H. (2007). A neural framework for
robot motor learning based on memory consolidation. In Proceedings of International Conference
on Adaptive and Natural Computing Algorithms.
Carmel, D. and Bentin, S. (2002). Domain specificity versus expertise: Factors influencing distinct
processing of faces. Cognition, 83:1–29.
Challis, J. H. (1995). A procedure for determining rigid body transformation parameters. Journal of
Biomechanics, 28:733–737.
DiSalvo, C. F., Gemperle, F., Forlizzi, J., and Kiesler, S. (2002). All robots are not created equal: The
design and perception of humanoid robot heads. In Proceedings of the Symposium on Designing
Interactive Systems, pages 321–326.
Farah, M. J., Rabinowitz, C., Quinn, G. E., and Liu, G. T. (2000). Early commitment of neuralsubstrates for face recognition. Cognitive Neuropsychology, 17:117–123.
Fong, T., Nourbakhsh, I., and Dautenhahn, K. (2003). A survey of socially interactive robots. Robotics
and Autonomous Systems, 42:143–166.
Gleicher, M. (1998). Retargetting motion to new characters. In Proceedings of the International Conference on Computer Graphics andInteractive Techniques, pages 33–42.
Goetz, J., Kiesler, S., and Powers, A. (2003). Matching robot appearance and behavior to tasks to
improve human-robot cooperation. In Proceedings of the Workshop on Robot and Human Interactive
Communication, pages 55–60.
15

Grill-Spector, K., Knouf, N., and Kanwisher, N. (2004). The fusiform face area subserves face perception,
not generic within-category identification. Nature Neuroscience, 7(5):555–562.
Grood, E. S. and Suntay, W. J. (1983). A joint coordinate system for the clinical description of threedimensional motions: Application to the knee. Journal of Biomechanical Engineering, 105:136–144.
Hale, J. G., Pollick, F. E., and Tzoneva, M. (2003). The visual categorization of humanoid movement
as natural. In Proceedings of the Third IEEE International Conference on Humanoid Robotics.
Kanda, T., Ishiguro, H., Ono, T., Imai, M., and Mase, K. (2002). Development and evaluation of aninteractive robot “Robovie”. In Proceedings of the IEEE International Conference on Robotics andAutomation, pages 1848–1855.
Kashima, T. and Isurugi, Y. (1998). Trajectory formation based on physiological characteristics ofskeletal muscles. Biological Cybernetics, 78(6):413–422.
Kawato, M., Furukawa, K., and Suzuki, R. (1987). A hierarchical neural network model for control andlearning of voluntary movement. Biological Cybernetics, 57:169–185.
Leardini, A., Chiari, L., Croce, U. D., and Cappozzo, A. (2005). Human movement analysis using
stereophotogrammetry Part 3. Soft tissue artifact assessment and compensation. Gait and Posture,
21:212–225.
Minato, T., Shimada, M., Ishiguro, H., and Itakura, S. (2004). Development of an android robot forstudying human-robot interaction. In Proceedings of the 17th International Conference on Industrial& Engineering Applications of Artificial Intelligence & Expert Systems, pages 424–434.
Minato, T., Shimada, M., Itakura, S., Lee, K., and Ishiguro, H. (2006). Evaluating the human likenessof an android by comparing gaze behaviors elicited by the android and a person. Advanced Robotics,
20(10):1147–1163.
Mori, M. (1970). Bukimi no tani [the uncanny valley] (in Japanese). Energy, 7(4):33–35.
Nakaoka, S., Nakazawa, A., Yokoi, K., Hirukawa, H., and Ikeuchi, K. (2003). Generating whole body
motions for a biped humanoid robot from captured human dances. In Proceedings of the 2003 IEEE
International Conference on Robotics and Automation.
Okada, M., Ban, S., and Nakamura, Y. (2002). Skill of compliance with controlled charging/discharging
of kinetic energy. In Proceeding of the IEEE International Conference on Robotics and Automation,
pages 2455–2460.
Oyama, E., Agah, A., MacDorman, K. F., Maeda, T., and Tachi, S. (2001a). A modular neural networkarchitecture for inverse kinematics model learning. Neurocomputing, 38–40(1–4):797–805.

16

Oyama, E., Chong, N. Y., Agah, A., Maeda, T., Tachi, S., and MacDorman, K. F. (2001b). Learning acoordinate transformation for a human visual feedback controller based on disturbance noise andthe feedback error signal. In Proceedings of the IEEE International Conference on Robotics andAutomation.
Oyama, E., MacDorman, K. F., Agah, A., Maeda, T., and Tachi, S. (2001c). Coordinate transformation
learning of a hand position feedback controller with time delay. Neurocomputing, 38–40(1–4).
Riley, M., Ude, A., and Atkeson, C. G. (2000). Methods for motion generation and interaction with ahumanoid robot: Case studies of dancing and catching. In Proceedings of AAAI and CMU Workshop
on Interactive Robotics and Entertainment.
Shimada, M., Minato, T., Itakura, S., and Ishiguro, H. (2006). Evaluation of android using unconscious
recognition. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots, pages
157–162.
Veldpaus, F. E., Woltring, H. J., and Dortmans, L. J. M. G. (1988). A least squares algorithm for theequiform transformation from spatial marker co-ordinates. Journal of Biomechanics, 21:45–54.
Yoshikai, T., Mizuuchi, I., Sato, D., Yoshida, S., Inaba, M., and Inoue, H. (2003). Behavior systemdesign and implementation in spined musle-tendon humanoid ”Kenta”. Journal of Robotics andMechatronics, 15(2):143–152.

17

0

35

80

2

35

90

140

215

260

154

215

270

305

380

410

320

387

415

Figure 12: The generated android’s motion compared to the subject’s motion. The number represents
the step.

18