Proceedings of the 2006 IEEE/RSJ
International Conference on Intelligent Robots and Systems
October 9 - 15, 2006, Beijing, China

Adaptive Human-Robot Interaction Systemusing Interactive EC
Yuki Suga,
Chihiro Endo, Daizo Kobayashi,
Takeshi Matsumoto, Shigeki Sugano

Tetsuya Ogata

School of Science and Engineering,
Waseda Univ.,Tokyo, Japan.
{ysuga, cendo, kobayashi, mattake, sugano}@sugano.mech.waseda.ac.jp

Abstract— We created a human-robot communication systemthat can adapt to user preferences that can easily change
through communication. Even if any learning algorithms areused, evaluating the human-robot interaction is indispensableand difficult. To solve this problem, we installed a machinelearning algorithm called Interactive Evolutionary Computation(IEC) into a communication robot named WAMOEBA-3. IEC isa kind of evolutionary computation like a genetic algorithm. With
IEC, the fitness function is performed by each user. We carriedout experiments on the communication learning system using an
advanced IEC system named HMHE. Before the experiments,
we did not tell the subjects anything about the robot, so theinteraction differed among the experimental subjects. We could
observe mutual adaptation, because some subjects noticed therobot’s functions and changed their interaction. From the results,
we confirmed that, in spite of the changes of the preferences, thesystem can adapt to the interaction of multiple users.

I. INTRODUCTION
In coming years, robots are expected to contribute muchto our lives through nursing, house-keeping, and other areas.
These robots should be able to communicate with humans
effectively. Human-robot communication is not only for giving
commands to the robot, but also for entertaining or relaxing
the user, so communication should have variety and flexibility.
However, most of robots can only communicate using the
scenarios designed by their developers. Though scenariobased communication is practical for the context-sensitive
communication like verbal communication, it lacks variety andflexibility.
On the contrary, we think that describing entire scenarios
is not necessary for human-robot communication. We areinterested in a behavior-based technique. Various, complex
behaviors can be generated through interaction between arobot and its surroundings, including humans. Though abehavior-based technique is not yet suited to context-sensitive
communication, such a technique would make human-robot
interaction very flexible. However, even though entire scenarios are not described, a robot’s behavior which corresponds
to the specific sensory input is defined by the designer apriori. Ideally, to achieve user-friendly robot communication,
the robot’s behavior should be configured not by the designer
but by the user. However, most of users do not have any

1-4244-0259-X/06/$20.00 ©2006 IEEE

Graduate School of Infomatics
Kyoto Univ., Kyoto, Japan.
ogata@i.kyoto-u.ac.jp

programming skills, so it is desirable that robots should havefunctions that enable them to learn.
Therefore, our goal is to create a robot with user-adaptive
communication. Using machine-learning algorithms, such asa neural network, reinforcement learning, and genetic algorithms, the robot can changes its behavior through interaction,
which prevent users from boredom. However, those learningalgorithm when using in the human-machine interaction havepotential problem, that is, quantitative evaluation problem.
Though it is indispensable for the learning algorithm to evaluate how successful the behavior was, the subjective evaluationof the interaction is quite difficult to model. For example,
Ishiguro et al. conducted some experiments of behavior adaptation using Policy Gradient Reinforcement Learning [15]. In
this experiment, they hypothised that a human’s gaze, motion
speed, and distance between a human and a robot indicate
how well the interaction was. However, the human’s evaluationand interaction are quite complicated and varied, so that thehumans sometimes behave beyond the developer’s expectation.
To address this problem, we introduced Interactive Evolutionary Computatiaon (IEC), a kind of evolutionary computation, such as a genetic algorithm. In conventional EC,
designers have to define a fitness function that evaluates how
successful genes are. On the other hand, the fitness function
in IEC is performed by the user, so we do not have to model
the user’s preferences, and we can apply a machine-learning
to the problems of human subjective preferences.
Moreover, IEC is able to keep the user’s interest, because
it can generate various behaviors using a diverse of thegenetic pool. To improve this advantage, we developed anadvanced evaluation technique called Human-Machine Hybrid
Evaluation (HMHE). In HMHE, some representative genes aremanually evaluated by the user, while the others are evaluatedautomatically using the results of the manual evaluation.
This technique can increase population size of genetic pool
without increasing human fatigue. We installed HMHE in acommunication robot named WAMOEBA-3, and confirmedthat the method will not bore the user [13]. However, in theformer experiments, we told the experimental subjects thefunction of the robot, so the interactions were restricted.
In this paper, we show the further experiments in the3663

real world. To realize various interactions, we installed extra
sensors and actuators in the robot and did not tell experimental
subjects anything about the robot before their interaction.
Eventually, the interactions differed remarkably among the
subjets, but the robot could adapt to their preferences. In thenext section, HMHE is described in detail. Then, in section
III, the experimental settings are described. In section IV, weshow the experimetal results, and we discuss them in section
V. Finally, we summarize this study and indicate areas for
future work.

pool (see Figure 2(a)). Next, seven genes are selected; eachselected gene has a best matching dataset among each of theseven neurons placed at the positions shown in Figure 2(b).
Therefore, we can select individuals that have datasets of genesthat are distant from each other, which is efficient for theestimation function.
Self-Organizing Map

II. HUMAN - MACHINE HYBRID EVALUATION
Though IEC can be used to address preferences, it causes
human fatigue. Since the user must evaluate all of the genes ingenetic pool, the population size and generations are strictly
limited. This limitation causes an early convergence problem,
which bores the user. This is because the robot’s behavior
is simplified when the genetic pool converges. To solve thisproblem, the diversity of the genetic pool must be kept at ahigh level during interaction without increasing human fatigue.
We developed a Human-Machine Hybrid Evalution(HMHE) technique, where the genes are evaluated both manually and automatically. Figure 1 shows the flowchart of the
HMHE.
Newest Genetic Pool

Genetic PoolFig. 2.

nαik = 1 −Estimation
Functionrimanual − rk n∑l=1 rlmanual − rk 

(2)Here, Eiauto is the fitness value of gene i, and Ekmanual is thefitness value of gene k evaluated manually. n is the numberof the genes evaluated by the user. αik is the weight, whichis calculated according to the distance between genes i and kusing equation (2). r is a dataset of a gene.
This technique can increase the number of genes without
increasing human fatigue. Therefore, the system can maintaingene diversity and the users’ interest.
We carried out some simulations to compare the HMHE tothe conventional IEC [12]. In the simulation, the HMHE had
a higher fitness value than the plain IEC.

Human-Robot
Interaction

Human Subjective
Evaluation

Fig. 1.

(1)

k=1

MotionGenerator

Communication Robot

Genetic Operators(Selection)(Crossover)(Mutation)

Positions of selected genes on SOM

Eiauto = ∑ αik Ekmaunal

Translator

The other genes(b) Selecting Period

Then, each representative gene is translated into pheno-type(the robot’s parameter), and the robot interacts with a user.
Next, the user evaluates the robot’s behavior. This sequence
continues until all of the representative genes are evaluated.
After that, the fitness values of the other genes are estimatedautomatically. The estimation function is as follows:

Selected Genes

Self Organizing Map

Genetic Pool(a) Training PeriodNo
All genes
evaluated?

Yes
Human-Machine Hybrid Evaluation

III. EXPERIMENTAL SETTINGSFirst, all of the generated genes are analyzed with a selforganizing map (SOM). An SOM is useful for analyzing
a multi-dimensional dataset. In the SOM, the genes with asimilar dataset are placed near each other, while dissimilar
genes are placed away from each other. We used the SOM
algorithm because the algorithm was very simple and if we
define the distance between two dataset, it can eventually
analyze a bunch of the sample dataset. Using the SOM, thevarious representative genes are selected as follows. First, theSOM is trained by the dataset of all genes in the genetic

A. WAMOEBA-3
Our IEC experiment took a considerable amount of time,
because each subject evaluates a number of genes. Therefore,
the robot must have a variety of behaviors to keep the subject’s
interest, harmless to people, and be easy to maintain andcustomize. The robot must also be able to move without
cables for power supply or control, because cables prevent
easy interaction with humans.
We used a communication robot called the Waseda Artificial Mind On Emotion Base (WAMOEBA-3, Figure 3).

3664

TABLE I
S PECIFICATIONS OF WAMOEBA-3

WAMOEBA-3 is an independent, wheeled robot, with built-in
batteries and a PC. This robot was developed as a platform
for communication experiments. Its upper body is similar to ahuman one and its size is about the average size of Japanesechildren: 825 mm wide, 1316 mm tall. WAMOEBA-3 weighsapproximately 105 kg. It is equipped with two arms (7 degrees
of freedom) and a head (8 degrees of freedom). Each joint
has a torque sensor to measure the stress on the arm andhead. WAMOEBA-3 is also equipped with an omni-directional
vehicle for locomotion, which can move in any direction
without actually turning at any stage. This is advantageous
for both the variety of its behavior and for safety.

Dimensions
mm
Total Weightkg
Max speed
km/h
Payloadkgf/hand
Drive Time
hours
Drive
Camera DOF
MemberEar DOF
Neck DOF
Vehicle DOF
Arm DOF
Hand DOF
OutsideVisionSensors1316 (H) x 825 (L) x 656 (W)
105
3.5
5.0
1.5
1+1 x 2=32
3
3
6 x 2=12
1 x 2=2
CCD Color camera x 2(x10 Optical zoom,
x4 Degital zoom)
Microphone x 2(Directional hearing,
Voice recognition)
Speaker(Voice synthesis)
Ultrasonic sensor x 8
Pyroelectric sensor x 3
Bumper switch x 8
Torque sensor x 14
6-Axis force sensor x 2
Extra super duralumin
Titanium alloy (Ti-6Al-4V)
aluminium (52S)
Pentium4 (3.2GHz)
VR5550-ATOM×5
Linux
Lead-Acid Battery for EV

Sound input
Sound output
Distance
Human Body
Collision
Joint stressShoulder
Structural material

CPU
MicrocomputersOS
Battery

Equation (3) shows the output decision of the actuators.

θ̇iFig. 3.

= λi σi (x) − δi θi
σi (xi ) = exp −γi (xi − c)2− exp −γi (xi + c)2

WAMOEBA-3

The WAMOEBA-3 has also a lot of sensors. It has shoulder
covers installed with 6-axis force sensors to detect touches
on the shoulders. The head has two CCD cameras and two
microphones. Each camera can independently move verticallyand horizontally, and each ear can rotate horizontally, whichenable the robot to indicate the direction of its attention. The
vehicle has eight bumper sensors, three infrared sensors, andeight ultrasonic sensors. Table I shows the specifications of the
WAMOBA-3 in more detail. We did not use all of the internalsensors, like the battery voltage or thermal sensors.

B. Robot Motion Generator
We used the motor-agent model (MA model) proposed informer studies as a motion generator of the WAMOEBA3 [14]. The MA model is a distributed control algorithm
in which all the actuators and sensors are linked to eachother with connections. Each actuator autonomously collectssensor inputs through the network and determines its motion.

(3)

(4)

j

xi

=

∑ ω ji s j

(5)

m=0

Here θ̇i is the rotation velocity of the motor agent i. The
first term of quation (3) is the calculation result from sensor
inputs, and the second term is for the stabilization. σi is asigmoid function defined by equation (4). Here, the γ affects
the linearity of the function. If the γ is larger, the linearity
is smaller. x is the summation of the inputs from the sensor
network, defined by equation (5). ω is the connection weight
between the agents, and s is the sensor input like the encoders
or switches.
Using the MA model, the network has to be defined. We
used the very simple network shown in Figure 4.
The characteristic behaviors achieved by this network are
as follows.
• If someone waves his hand in front of the camera, therobot detects a moving region of the image, and the eyes
move vertically or horizontally.
• If someone touches a bumper sensor placed on thevehicle, the vehicle either turns towards or away from
him.

3665

Ear

7

Ear
Eye

7

Eye
Neck
ShoulderShoulderInfraredInfrared

Sonar
Bumper

Sonar
Vehicle(a)(b)

BumperFig. 5.

Fig. 4. Motor-agent network: filled boxes and open boxes represent motoragents and sensor-agents, respectively. A motor agent has both an actuatorand a sensor (encoder), and sensor-agent has a sensor only.

•

If someone claps his hands or shouts, the robot’s ears turnhorizontally and the eyes move.
• If someone stands near the robot, the sonars measure
the distance or the infrared sensors detect him, and thevehicle moves.
• If someone touches the shoulder covers, the vehicle
moves.
C. Genetic Settings
The MA model enables WAMOEBA-3 to move only reactively. However, the direction and amount of movements
are based on human interpretation. Therefore, we configured
the connection weights between the agents (ω in Equation(5)) with IEC. The weights were encoded into genes using
numerical encoding, which is easy to analyze. The dimensionof the dataset of each gene was 40. The probability of mutation
was 0.5%. If a mutation occurred, the value of the gene wasadded by a random value. We also used multipoint crossoverand elitism (the best 40% of the genes were preserved to thenext generation). The population size was 30. The experimentcontinued until the 7th generation. Without HMHE, eachsubject had to evaluate all the genes (240 genes). Since there
must be mostly the same genes in a genetic pool, evaluating
all of them is not efficient. Using HMHE, each subject had toevaluate only 56 in total.
D. Interaction and Other settings
The experiment was carried out in a conference roomin our university (Figure 5). The robot (Figure 5 (a)-A)and the subject (Figure 5 (a)-B) interacted with each otheruntil he/she wanted to stop. After that, the subject evaluated
how interesting the interaction was and gave a score to theexperimenter (Figure 5 (a)-C).
The number of subjects was 10. 4 of them were the membersof our laboratory, so they had more knowledge of robots
than the others did. The others were also the students at ouruniversity, one female student was included in this group.
In former studies, the interactive experiments were conducted after announcing the robot’s function, so the interaction
was restricted [13]. In this study, we wanted to observe themutual adaptation of both the robot and humans. Therefore,

Environment

we did not tell the subjects anything about the robot to thesubjects, so they could interact freely with the robot andevaluate it without bias.
After the 4th and 7th generations, we carried out a survey
about the subject’s impressions of the robot. The questionnaire
items were as follows:
1) Is the robot friendly?
2) Is the robot funny?
3) Do you want to interact with the robot?
4) Do you feel a sense of unity with the robot?
5) Do you and the robot have similar behavior?
6) Is the robot interesting?
7) Can you communicate with the robot?
8) Does the robot do various behaviors?
9) Is the robot’s reaction dynamic?
10) Can the robot communicate with you?
11) Is the robot sensitive?
12) Does the robot seem to be a living creature?
13) Is the robot boring?
14) Is the robot easy to interact with?
15) Is the robot warm-hearted?
16) Is the robot’s bahviour normal?
IV. RESULTS
The experiment was continued until the 7th generation(including the initial evaluation). Each experiment took 1.5
hours on average. The longest time was 2.5 hours and theshortest was 46 minutes. Each pheno-types interacted with asubject for 1.5 minutes on average.
Figure 6 shows the average fitness values of all subjects’ experiments. The fitness value tended to increase throughout theexperiments. The diversity of the fitness value was kept during
the experiment because HMHE could maintain a diversity of
behavior.
Figure 7 shows the questionnaire results. We could see
significant differences for “Friendliness,” “Sense of Unity,”and “Dynamic Reaction.”
During the experiment, we shot videos and analyzed theinteraction. We counted interaction such as touching theshoulders, waving hands, clapping hands, and so on. Figure
8 shows the percentage of interactions by body part. The
horizontal axis and the vertical axis, respectively, indicate theexperimental subjects and the percentage. Cool colors indicate
proximal interaction like touching, and warm colors show

3666

Fitness Value

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

1

Fig. 6.

2

3
4
5
Generation

axes indicate the generation and the percentage, respectively,of J’s interaction. The subject J was not a member of ourlaboratory, so he had less knowledge of the robot we used.
This is an example of the proximal interaction; J continued tointeract with the robot arms and shoulders. We did not use any
controls on the arms. Though the arms moved like a doll’s, J
did not become bored of manipulating them. He also continued
to push or pull on the robot’s shoulders, and eventually the
robot started to follow the pushing on the shoulders.

6

Fitness Value (Average)

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

4th
7th

*

Friendly?
Funny?
Smooth behavior? 6
5
Do you want to interact?
Warm-hearted?
4
3
Do you feel sense of unity?
Easy to contact?
2
1
You and Robot have0
Boring?
similar behavior?

**

Is Robot natural
creature?

0Interesting?

Can robot communicate?
Various behavior?
Dynamic reaction?

*( :P < 0.05
Fig. 7.

: P < 0.01)

80%
60%
40%
20%

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

0%
D

E

F

G

H

I

0

J

Experimental Subject

Fig. 8.

Subject J (Interaction)

On the other hand, the subject D changed her interaction
through the generations. Figure 10 shows the percentages ofher interaction. The subject D was not a member of ourlaboratory, either. In the first generation, touching the robot’s
arms accounted for almost 50% of her interaction. However,in later generations, touching decreased and distant interaction
increased.

ETC.
BumperArm
ShoulderHead (Touch)Head (Observe)
Sonar
CameraSound

C

6

Questionnaire Results

100%

B

5

Head (Observe)
Sonar
CameraSound

Fig. 9.

distant interaction like waving hands or clapping hands. Theinteraction greatly differs among the subjects.

A

2
3
4
Generation

ETC.
BumperArm
ShoulderHead (Touch)Can you communicate?

Sensitive?

1

1

2
3
4
Generation

ETC.
BumperArm
ShoulderHead (Touch)

Subjects’ Interactions

We used IEC to create an adaptive interaction system. Since
we did not tell the subjects anything about the robot, we could
observe various kinds of interction in these experiments. However, the fitness values tended to increase in the experiments
for almost all of the subjects. This may be because the robot
had enough actuators and sensors to realize various kinds ofinteraction so IEC could adapt to the subject’s preferences.
On the other hand, it is important to discuss the subjects’
adaptation. They might have learnt through the interactions,and changed their behaviors. Therefore, in this section, we focus on the temporal changes of the interaction. Figure 9 shows
the interaction of the subject J. The horizontal and vertical

6

Head (Observe)
Sonar
CameraSound

Fig. 10.

V. DISCUSSION

5

Subject D (Interaction)

This is clearly because we did not prepare any control on
the robot’s arms. When D subject noticed that, she changed
her interest to other parts of the robot. In the later part of theexperiment, she tended to watch the robot’s behavior, standing
at a distance of about 1 meter from it. The best individual
moved so that it stood directly in front of her.
Figure 11 shows the interaction of subject B. First he waved
his hands in front of the cameras many times, but gradually
he changed his interaction in the same manner as D.
In this experiment, we implemented moving area detectionin the camera agents. Therefore, the robot was capable of de-

3667

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0

1

2
3
4
Generation

ETC.
BumperArm
ShoulderHead (Touch)5

To achieve longer-term interaction, we think that, to improve human-robot communication, we must change a robot’sinteraction more drastically. Using IEC, we can achieve this byincreasing the mutation rate. A sudden change in the robot’s
behavior would make the interaction more interesting. We are
hopeful that IEC will be achieve this eventually.
In our future work, we will improve the internal control
algorithm to achieve more dynamic changes in the interaction.
We have already tested some algorithms, but the number ofparameters to be configured has increased. Therefore we areusing incremental evolution techniques to configure a robot’s
behavior.

6

Head (Observe)
Sonar
CameraSound

Fig. 11.

Subject B (Interaction)

ACKNOWLEDGMENTS

tecting his waving hands. Nevertheless, the subject B changed
his interaction from using cameras to observing.
Figure 12 shows absolute values of the connection weights
in the MA model. The circle plots represent the connection
weights between the front sonar and the vehicle’s sliding
motion. The square plots represent the connection weights
between the moving area detection and the eye motion.
Throughout the experiment, the connection weights between
the sonar and the vehicle were larger than those between thecamera and eye motion. Therefore, the eye motion was smaller
than the vehicle’s. On the other hand, the vehicle motion waslarge and visible to the subject. As a result, the subject started
to observe the robot’s motion from a distance.
Absolute value of Connection
Parameter (Average)

3
2.5
2
1.5
Sonar to Slide
Camera to Eye

1
0.5
0
0

Fig. 12.

1

2

3
4
Generation

5

6

Absolute values of connections

These subjects demonstrated that interaction changes during
the experiments. Despite of the temporal changes of interaction, the fitness values of them increased.
VI. C ONCLUSION AND F UTURE WORKIn this paper, we proposed an adaptive human-robot interaction system using Interactive Evolutionary Computation.
Using IEC, a robot can configure its behavior according to thesubjective evaluation of its user. We also proposed HumanMachine Hybrid Evaluation as an advanced technique of IEC.
In the experiments, we did not tell the experimental subjects
anything about the robot, so they interacted with the robot
and evaluated it freely. In our results, we confirmed theadaptability of IEC. We also discussed temporal changes in
the interaction. Despite the changes, IEC could adapt to theirchanging preferences.

This research was supported in part by a Grant-in-Aid for
the WABOT-HOUSE Project by Gifu Prefecture. This work
was also supported in part by the gThe innovative researchon symbiosis technologies for human and robots in the elderly
dominated societyh, 21st Century Center of Excellence (COE)
ProgramCJapan Society for the Promotion of Science.
R EFERENCES[1] Pfeifer, R., Scheier, C.: Understanding Intelligence; MIT Press (1999).
[2] Dautenhahn, K., Werry, I.: Towards interactive robots in autism therapy;
Pragmatics and Cognition 12(1), pp. 1-35.
[3] Kanda, T., Ishiguro, H., Imai, M., Ono, T.: Body Movement Analysis
of Human-Robot Interaction; Proc. of Int’l Joint Conf. on Artificial
Intelligence (IJCAI2003), pp. 177-182, 2003.
[4] Nolfi, S., Floreano, D.: Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines (Intelligent
Robotics and Autonomous Agents); Bradford Books, 2000.
[5] Lund, H., Miglino, O., Pagliarini, L., Billard, A., Ijspeert, A.F Evolutionary Robotics - A Children’s Game; IEEE Int’l Conf. on Evolutionary
Computation (ICEC ’98), pp.154-158 ,1998.
[6] Takagi, H.: Interactive Evolutionary Computation : Fusion of the Capabilities of EC Optimization and Human Evaluation; Proc. of the IEEE,
Supecial Issue on Industrial Innovations Using Soft Computing, Vol. 89,
No. 9, September, 2001.
[7] Kohonen, T.: Self-Organizing Maps; Springer-Verlag, Berlin Heidelberg,
1995.
[8] Holland, J.: Adaptation in Natural and Artificial System; MIT Press,
1992.
[9] Dawkins, R.: The Blind Watchmaker; Essex:Longman, 1986.
[10] Ogata, T., Sugano, S.: Emotional Communication Between Humans andthe Autonomous Robot which has the Emotion Model; Proc. of IEEE
Int’l Conf. on Robotics and Automation (ICRA’99), pp.3177-3182, 1999.
[11] Ogata, T., Matsunaga, M., Sugano, S., Tani, J.: Human Robot Collaboration Using Behavioral Primitives; Proc. of IEEE/RSJ Int’l Conf. onIntelligent Robots and Systems (IROS 2004), pp.1592-1597, Sept. 2004.
[12] Suga, Y., Ogata, T., Sugano, S.: Aquisition of Reactive Motion forCommunication Robots Using Interactive EC; Proc. of IEEE/RSJ Int’l
Conf. on Intelligent Robots and Systems (IROS 2004), pp.1198-1203,
Sept. 2004.
[13] Suga, Y., Ikuma, Y., Nagao, D., Ogata, T., Sugano, S.; Interactive
Evolution of Human-Robot Communication in Real World; Proc. ofIEEE/RSJ Int’l Conf. on Intelligent Robots and Systems (IROS2005),
August, 2005[14] Ogata, T., Komiya, T., Sugano, S.: Motion Generation of the Autonomous Robot based on Body Structure, in Proc. of IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS2001),
pp. 2338-2343, Oct. 2001.
[15] Mitsunaga, M., Smith, C., Kanda, T., Ishiguro, H., Hagita, N.: Robot
Behavior Adaptation for Human-Robot Interaction based on Policy
Gradient Reinforcement Learning, in Proc. of the 2005 IEEE/RSJ Int’l
Conf. on Intelligent Robots and Systems, pp.1594-1601, 2005.

3668