Interactive Evolution of Human-Robot Communication
in Real World
Yuki SUGA, Yoshinori IKUMA,
Daisuke NAGAO and Shigeki SUGANO

Tetsuya OGATA

Department of Mechanical Engineering
Waseda University#59-325 3-4-1 Okubo Shinjuku, Tokyo, Japan{ysuga, ikuma, nagao, sugano}@sugano.mech.waseda.ac.jp

Abstract— This paper describes how to implement interactive
evolutionary computation (IEC) into a human-robot communication system. IEC is an evolutionary computation (EC) in
which the fitness function is performed by human assessors.
We used IEC to configure the human-robot communication
system. We have already simulated IEC’s application. In thispaper, we implemented IEC into a real robot. Since thisexperiment leads considerable burdens on both the robot andexperimental subjects, we propose the human-machine hybrid
evaluation (HMHE) to increase the diversity within the genetic
pool without increasing the number of interactions. We used acommunication robot, WAMOEBA-3 (Waseda Artificial Mind
On Emotion BAse), which is appropriate for this experiment. In
the experiment, human assessors interacted with WAMOEBA3 in various ways. The fitness values increased gradually,
and assessors felt the robot learnt the motions they desired.
Therefore, it was confirmed that the IEC is most suitable as thecommunication learning system.
Index Terms— Human-Robot Interaction, Communication
Robot, Interactive Evolutionary Computation

I. I NTRODUCTION
In recent years, many companies and researchers have beendeveloping many different kinds of robots, which are expected
to contribute to our lives. These robots should be able
to communicate with people, however the communication
capabilities of current robots are poor when compared tohumans.
Most communication robots are developed using ‘modelbased’ techniques. They have their own models (scenarios) ofcommunication, and can only communicate with people basedon the models. Although these techniques are currently suitable to achieve the context sensitive communication, such asverbal communication, the variety of behaviors is physicallylimited.
Therefore, it is strongly required that the robots are able
to adapt to the situations changing continuously. We thinkthat evolutionary computation (EC) is suitable for learning
communicative motions. In EC, various possible solutions are
generated and tested. Since the diversity of behaviors can be
kept high, it can prevent people from boredom throughout thelearning experiment.
However, there is a potential problem, even if we use any
kind of learning algorithms. It is a problem of quantitative

Graduate School of Infomatics
Kyoto University
Yoshida-honmachi, Sakyo-ku, Kyoto, Japanogata@i.kyoto-u.ac.jp

evaluation. Though it is indispensable to evaluate the robot’s
behaviors, it is quite difficult to evaluate the robot’s communicative behaviors quantitatively rather than qualitatively.
To solve this problem, we used interactive evolutionary
computation (IEC), which is constructed of techniques ofinteractive learning and EC.
In this paper, we discuss how to implement IEC into areal robot system. In the next section, we describe IEC in
detail. We also show a problem of IEC in communication
learning. In section III, we propose the method of humanmachine hybrid evaluation (HMHE), which reduces humanfatigue without decreasing population size. In section IV, we
describe the robot we tested. We used a communication robot,
WAMOEBA-3, which was designed as a platform robot forcommunication experiments. In section V, we describe theimplementation of the IEC with HMHE is described in detail.
In section VI, we describe the experimental setting. In section
VII, we show results of this experiment and in section VIII,
we discuss the results. Finally, we present our conclusion anddescribe our future work.
II. I NTERACTIVE E VOLUTIONARY C OMPUTATIONInteractive evolutionary computation (IEC) is an evolutionary computation whose fitness function is provided by humanassessors. IEC, therefore, makes it possible to apply EC into
human subjective optimization [5]. IEC has been applied toaesthetic areas, such as art, music, and so on. It has also been
applied to robots [4] [13].
On the other hand, we aimed to use IEC for human-robot
communication. Since we believe that communication can belearnt only via mutual interaction between a human being anda robot, our system consists of both of them. Therefore, in our
experiment, assessors interact with a robot and simultaneously
evaluate its response.
However, in our case, there are two problems. One ishuman fatigue, and the other is hardware error. IEC process
takes very long time to learn, since the assessor must cooperate with a tireless robot to evaluate individuals. This is aburden for both human beings and a robot.

III. H UMAN -M ACHINE H YBRID E VALUATION

Newest Genetic Pool

To minimize human fatigue, we need to reduce the numberof evaluation, that is, we should decrease the population sizeand generations. However, that usually results in the genetic
pool’s convergence which makes the assessors bored. One
way to avoid the assessor’s boredom is ‘mutation’ because
it diverses the genetic pool. Since the mutation is a random
process, however, mutation makes it difficult to refrec humanassessor’s preference.
To solve this problem, we proposed the human-machine
hybrid evaluation (HMHE). In IEC with HMHE, an assessor
tests only some selected individuals. As for the others, thesystem automatically estimates the fitness values of thembased on the results the assessor tested before. With thismethod, we can increase the population size of the genetic
pool without increasing the assessor’s fatigue.
Moreover, to improve the HMHE, we deviced the selection
method. If the selected genes have similar data to each other,
the estimation of the method becomes poorer. In order toanalyze the genes, we used a self-organization map (SOM)[6]. The SOM is an algorithm that is suitable for analyzinglarge multi-dimensional dataset. In the SOM, similar vectors
are placed close to each other and dissimilar vectors areplaced further away from each other. The individuals are
selected using it as follows: first, the SOM is trained by thedataset of genes in the genetic pool (see Fig.1(a)). Next, sevengenes are selected; each selected gene has the best matchingdataset with each of the seven neurons placed at the positions
shown in Fig.1(b). In this way, we can select individuals
that have distant dataset of genes from each other, which isefficient for the estimation function.

Selected Genes

Self Organizing Map

Parameters
MotionGenerator

The other genes

Communication Robot
Estimation
Function
Human Machine
Communication

Human Subjective
Evaluation
Genetic Operators(Selection)(Crossover)(Mutation)

Fig. 2.

If all genes areevaluated

No

Yes

IEC with HMHEof the Euclidian distance from the dataset of the genes given
the assessor’s subjective fitness values previously.
After all the genes are evaluated both automaticallyand manually, the EC applies genetic operators (selection,
crossover, and mutation) to the genetic pool and generates
more appropriate genes.
We have already confirmed efficiency of this algorithm in asimulation. The simulation confirmed that this algorithm cankeep diversity of the genetic pool at a high level, and stop thefitness value’s drops which is caused by the human boredom[11].
IV. WAMOEBA-3

Self-Organizing Map

Genetic Pool

Genetic Pool

Training Period
Fig. 1.

(a)(b) Selecting Period

Positions of selected genes on SOM

Our proposed algorithm is shown in Figure 2.
First, initial genes are generated at random. Next, some
genes are selected using the SOM. Then, the selected genes
are translated into pheno-type (parameters of a motion generator) and installed into a robot. An assessor interacts withthe robot, and evaluates the gene individually. This sequenceis repeated until all the selected genes are evaluated. Whereas
unselected genes are automatically evaluated by the estimation function. Their fitness values are estimated on the basis

As described above, our IEC experiment takes a considerable amount of time, because an assessor individually
evaluates many individuals. Therefore, the robot must havea variety of behaviors to keep the assessor’s interest. It
also must be harmless for people and easy to maintain andcustomize. Another important feature that the robot requires
is independency, that is, it can move without the cables for energy supply or control, because those cables are inconvenientwhen the robot moves dynamically.
In this paper, we used a communication robot named“WAMOEBA-3”(Waseda Artificial Mind On Emotion Base)(Fig. 3). WAMOEBA-3 is an independent, wheeled robot,
with inbuilt batteries and a PC. Its upper body is similar tohuman ones and its size is similar to the average size of aJapanese child: 656 [mm] long, 825 [mm] wide, 1316 [mm]tall. It weighs approximately 105 [kg]. It is equipped with
two arms (7 degrees of freedom) and a head (8 degrees of
freedom). Each joint has a torque sensor to measure the stresson the arm and head. The head is equipped with 2 CCD
cameras and 2 ears. Each camera can independently movehorizontally, and each ear can rotate horizontally, too.
WAMOEBA-3 is also equipped with an omni-directional
vehicle for locomotion. The vehicle moves in any direction

V. I MPLEMENTATION
Using HMHE and WAMOEBA-3, we were able to carryout an IEC experiment in a real world. In this section, weshow how we implemented IEC into the robot.
A. Motion Generator of the Robot

Fig. 3.

WAMOEBA-3

without actually turning at any stage. It has 8 bumper sensors
and 8 ultrasonic sensors on the vehicle.
TABLE IS PECIFICATIONS OF WAMOEBA-3
Dimensions
mm
Total Weightkg
Max speed
km/h
Payloadkgf/hand
Drive Time
hours
Drive
Camera DOF
MemberEar DOF
Neck DOF
Vehicle DOF
Arm DOF
Hand DOF
OutsideVisionSensors
Sound input
Sound output
Distance
Collission
Joint stress
Structural material

CPU
Micro ComputersOS
Battery1316(H)×825(L)×656(W)
105
3.5
5.0
1.5
1+1×2=32
3
3
6×2=12
1×2=2
CCD Color camera × 2(10×Optical zoom,
4×Degital zoom)
microphone×2(Directional hearing,
Voice recognition)
Speeker(Voice synthesis)
Ultra sonic sensor×8
Bumper switch×8
Torque sensor×14
Extra super duralumin
Titanium alloy(Ti-6Al-4V)
aluminium(52S)
Pentium4(3.2GHz)
VR5550-ATOM×5
Linux
Lead-Acid Battery for EV

WAMOEBA-3 was developed as a platform for communication experiments. Therefore, we designed some devices
which make it easy to maintain and customize. For example,
sliding mechanisms are installed on the battery case andpower supply unit to allow easy access.It has a distributed
control system constructed of 5 microcomputers and a Dos/V
PC. The control system contains enough space capacity tocustomize the robot in the future.

The motion generation algorithm for WAMOEBA-3 was
the motor-agent (MA) model. The MA model is a distributed
control algorithm. Each actuator (or sensor) is regarded as anautonomous agent, and each agent collects data from anotheragents, and determines its own actions autonomously andindependently. Therefore, the connections between the agents
create a network in WAMOEBA’s body, and the network
topology stands on the robot’s bodily structure [12].
Each actuator moves according to the output of the following equations.
The output of the motors is described in terms of angularvelocity, and movement of the whole body is generated byintegrating motions caused by input from the sensors, suchas the rotary encoders, bumper switches, and so on.
θ˙i

=

n

j=0

γji α(θj ) +m

γki sk + δθi

(1)

k=0

Here, θ is the joint angle of actuator, and s is the input fromsensor. α is the function that determines the influence fromthe other actuators.The 3rd term is for the stability of theoutput of the function. α is given by the following function
which changes based on the angle of another joint.
2α(θj ) = exp −ωj (θj − cconst )


2
(2)
− exp −ωj (θj + cconst )Here, cconst is the threshold, and ω is the degree of sensitivityof θ. If the value of ω is large, the change of the θ strongly
influences the output of the function.
In this experiment, we used a very simple MA network, asshown in Figure 4. To simplify the experiment, the images
captured using the cameras were not in use, and the joints on
the arms and ears weren’t used, either. Agents with actuation
capabilities included the eye agents, the neck agents, and theomni-directional vehicle agent, and available sensors were
the bumper switches, the ears (microphones), and the sonars(ultra sonic sensors). In this case, the eye agent autonomously
determined its rotation direction and speed based on the angleof the neck joint and data from the ear sensors (microphones)
and the bumper sensors. The neck agent determines its action
based on the angles of both eye joints and action of the omnidirectional vehicle. The omni-directional vehicle determined
its translational motion on the basis of input from the bumper
switches and ultrasonic range sensors, and its rotationalmotion on the basis of input from the angle of the neck joint
as well as the input from the bumper and sonar sensors.
The characteristic behaviors achieved by this network wereas follows.

Ear

Eye

Eye

between the simulation and the real world, we made thebehavior acquisition experiment using IEC with HMHE. In
this section, our experimental settings are described in detail.

Neck

Sensors

Actuators

Sonar

Sonar

VehicleBumperBumperFig. 4. Motor-agent network: blue circle and white circle represent motoragents and sensor-agents respectively.

If someone made a noise (shout, clapping), the microphones detected the sound’s direction. Then the eyes
and neck moved horizontally, and the vehicle moved tofollow (or escape from) him/her.
• If someone touched the bumper sensors, the sensors
detected the collision. Then the eyes moved up or down,
and the vehicle moved.
• If someone came up to the robot, the ultrasonic sensors
detected the person approaching. Then the robot moved.
•

B. Genetic Settings
The MA enables WAMOEBA-3 to move just reactively.
However, the direction and amount of movements are basedon human interpretation. Therefore, the connection weights
between the agents (γ in Equation 1) were encoded into genes.
The genes were encoded using numerical data, which are easy
to analyze. The dataset dimension of a gene was 32. The
probability of mutation was 0.3%. If a mutation occurred,
the value of the gene was added by a random value.
We also used multi point crossover and elitisum (the best
40% individuals were preserved and made their offsprings).
C. IEC with HMHE
Since the period of the experiment is limited by humanfatigue, we set the parameters according to the result of thesimulation so that the experiment ends in at the most 3 hours.
The population size was 30 and the experiment continued
until the 7th generations. There were 37 neurons in the SOM
network, and 1000 training cycles. The system also displayed
the best individual from the previous generation, which madesure the learning progressed and eased an assessor’s mind.
Without HMHE, each assessor must evaluate all the genes.
Therefore, the assessor would have had to interact with 240
individuals. Since there must be almost the same genes in thegenetic pool, this is an inefficient method. Using the HMHE,
each assessor needed to evaluate only 56 individuals in total.
VI. E XPERIMENTAs described above, we’ve already made a simulation toconfirm the effectivity of the HMHE. However, we thoughtthat human-robot interaction in real world should be different from simulated one. In order to confirm the difference

A. Experimental Subjects
Experimental subjects (assessors) were 6 students in ourlaboratory. They were young students (21-23 years old), andhad some knowledge of computers and robots.
Before the experiments, we informed them the following.
• The robot’s name is “WAMOEBA”.
• The robot has bump sensors, ultra-sonic sensors, andears. The robot’s cameras and arms are not in use.
• Please tell us the total point of the robot’s behaviors
whenever you want.
They were simply informed of the robot’s abilities, and wereallowed to evaluate the robot’s behavior freely.
B. Experimental Design
Usually, communication has its own specified task or goal(eg. escape from labyrinth). In our experiment, the task was“play”, because communication robots like WAMOEBA-3 are
expected to entertain or may be used in psycho-therapy in
the near future [1]. Therefore, in this study, WAMOEBA-3
interacted with a human assessor to entertain him/her.
C. Environment
The experimental environment is shown in Figure 5. We
used a conference room where there were no objects or
obstacles. The assessors interacted freely with the robot, andevaluated its behaviors. Each individual started to move fromthe center of the room.
7

7

Ear

Fig. 5. Experimental environment: (A) WAMOEBA-3. (B) human assessor.
(C) experimenter.

D. Evaluation
In this experiment, the robots’ behaviors were evaluatedusing a scoring method. The assessors were able to evaluate
the robot at any time. When they wanted to input a score,
they said it to an experimenter and the experimenter inputted
the score into a laptop.
We did not supply any evaluation criteria to the assessors.
They gave their own subjective evaluation to the robot’s
behaviors.

E. Questionnaire Surveys

3

After the experiment, the assessors completed questionnaires to survey their impression of the experiment. Someof the questionnaire items used are shown as follows.

1• Did you feel the WAMOEBA tried to communicate with you?
• How did you feel the WAMOEBA in the early generations?
• How did you feel the WAMOEBA in the last generation?
• How did you evaluate the robot’s behaviors?
• Did your evaluation criteria change?
• Did you think the robot learnt?
• Were you satisfied with the result?

-1

3

3

1
-1

(b)

Value

-1
-3
0 1 2 3 4 5 6 7Generation(b) Eye to Neck

A. Interaction

(a)

1

Value

-3
01234567
Generation(a) Ear to Eye

VII. R ESULTS
Each individual interacted with a human assessor for an
average of 3 minutes. The experiment took about 3 hours intotal.
Figure 6 shows the typical interactions between a humanassessor and WAMOEBA-3. The assessor clapped their hands
or called the robot’s name (Fig.6-a), kicked the bumper
switches on WAMOEBA-3’s vehicle (Fig.6-b), touched or
pushed WAMOEBA-3’s body or shoulders (Fig.6-c), andposed in front of the robot (Fig.6-d).

3

1

Value

-1

25-30
20-25
15-20
10-15
5-10
0-5

Value

-3
01234567
Generation-301234567
Generation(c) Sonar to Vehicle(d) Bumper to Vehicle

Fig. 7. Population distribution of each connection weight and generation:
Colored region represents population of genes. These are examples from one
assessor.

its eye in the direction of the sound. In this case, throughout
the experiment, the population of the genes which had a valueof 1 was the largest.
Figure 7 b) shows the connection weight between the eyeand neck agents. If the value is negative, WAMOEBA-3
turned its head in the direction of the gaze. Until the 3rd
generation, the population of 1 was the largest. However, fromthe 4th generation, the population of -3 increased drastically.
In Fig. 7 c) (the connection weight between the sonar andvehicle agents), the data of 0 was the most popular.
The connection weight between the bumper and vehicle,
as shown in Fig. 7-d), shifted from 0 to -2 in 4th generation.
This suggests that the vehicle started to move to avoid the
obstacles.
C. Fitness Values

(c)

(d)

Fig. 6. Interaction: An assessor clapped his hands (a), kicked the bumpswitches (b), touched the robot’s shoulders (c), and posed in front of therobot (d).

Figure 8 shows the average and maximum fitness values.
The horizontal and vertical axis respectively indicate generations and fitness values.
0.8
0.7
0.6

B. Acquired Motions
Figure 7 shows the population distribution of each genetic
parameter and for each generation. In the graphs, horizontal
and vertical axis respectively indicate generation and the dataof each gene. Colored regions represent the population. If
the population size of the gene has a specified data that islarge, the assessor highly evaluated the genes in the previous
generation. Therefore, we can see the acquired motions andthe assessor’s preference from the graphs.
Figure 7 a) shows the connection weight between the ear
and eye agents. If the value is positive, WAMOEBA-3 turnedeu
la 0.5V
ss 0.4
net 0.3
iF
0.2

avgFitness
maxFitness

0.1
0
0

1

2

Fig. 8.

3

4
Generation

5

6

7

8

Fitness Value

Both the average and maximum fitness values gradually
increased. Although the variance is not indicated in Fig. 8, it
remained constant throughout the experiment.

D. Questionnaire surveys
As for the results of the questionnaire, we obtained many
comments from the subjects.
In earlier generations, they felt that the robot did not move
dynamically, or moved wildly. On the contrary, WAMOEBA3 moved actively and achieved motions the assessors desiredin the last generation. They said that they scored high fitness
values when the robot moved actively and safely. Most of
the individuals which they scored high fitness values escapedfrom the assessors, or turned their head to the direction of
the sound.

In this experiment, future works of this project werealso discussed. Since the human assessors interacted with
WAMOEBA-3 in a variety of ways, the WAMOEBA-3 required more sensors. The visual impression of the robot isvery important, so we should prepare all sensors in the next
experiment.
We also believe that the results will not improve as long as
the robot behaves reactively. We will develop a system thatchanges WAMOEBA’s motions dynamically. In the former
researches, we proposed an endocrine model for the robot
system [9]. We think that this endocrine model could be
configured using IEC.

VIII. D ISCUSSION
At first, we discuss the interaction between humans androbot. Although the assessors were informed of the robot’s
capabilities, they interacted with the robot in a variety ofways. The assessors sometimes touched the robot’s shoulders,though there were no sensors on the shoulders. They evenposed in front of the robot although they knew that thecameras are not in use in this experiment. Considering theseresults, we determined that human-robot interaction has various and infinite ways, and it is quite difficult for developers
to predict because the interaction is greatly affected by the
behaviors of the robot as well as its form.
We should have prepared more sensors and actuations, sothat we could provide more space for interesting communication (redundancy).
Then, we considered the acquired behaviors and assessor’s evaluation. In earlier generations, the behaviors of the
WAMOEBA-3 were small and slow because the absolutevalues of the initial genes were very small. In contrast,
WAMOEBA-3 moved more dynamically in the last half ofthe experiment. In the final generation, most of the acquired
motions seemed to be compliant and governable. For example,
when the assessor stood in front of WAMOEBA-3, it moved
backward.
The assessors may have placed importance on safety ofboth the robot and environment. The robots that rushed at
obstacles were evaluated as low. However, these rushingrobots were sometimes highly evaluated because the motion
surprised the assessors and they scored accordingly. Therefore, the robot should be equipped with sensors that candistinguish human beings from other obstacles.
According to the questionnaire surveys, the assessors felt
WAMOEBA-3 had learnt motions they desired. This canbe also observed in the gradually increasing fitness value.
Therefore, we determined that IEC with HMHE is effective
for the real environment.
IX. C ONCLUSION AND F UTURE W ORKS
In this paper, we demonstrated that interactive evolutionary
computation (IEC) is suitable for communication learning
because fitness functions are performed by a human assessor.
Using IEC, we configured the behaviors of a communication
robot, WAMOEBA-3, and determined that IEC could beapplied to the real robot.

ACKNOWLEDGEMENT

This research was supported in part by a Grant-in-Aid for
the WABOT-HOUSE Project by Gifu Prefecture.
R EFERENCES[1] K. Dautenhahn and Iain Werry: Towards interactive robots in autism
therapy, in Pragmatics and Cognition 12(1), pp. 1-35.
[2] T. Kanda, H. Ishiguro, M. Imai and T. Ono: Body Movement Analysis
of Human-Robot Interaction, Proceedings of International Joint Conference on Artificial Intelligence (IJCAI2003), pp. 177-182, 2003.
[3] S. Nolfi and D. Floreano : Evolutionary Robotics: The Biology,
Intelligence, and Technology of Self-Organizing Machines (Intelligent
Robotics and Autonomous Agents), Bradford Books, 2000.
[4] H. H. Lund, O. Miglino, L. Pagliarini, A. Billard, and A. IjspeertF
Evolutionary Robotics - A Children’s Game, in IEEE Int’l Conf. on
Evolutionary Computation (ICEC ’98), pp.154-158 ,1998.
[5] H. Takagi: Interactive Evolutionary Computation : Fusion of the Capabilities of EC Optimization and Human Evaluation, in Proc. of theIEEE, Supecial Issue on Industrial Innovations Using Soft Computing,Vol. 89, No. 9, September, 2001.
[6] T.Kohonen: Self-Organizing Maps, Springer-Verlag, Berlin Heidelberg,
1995.
[7] J. H. Holland: Adaptation in Natural and Artificial System, The MIT
Press, 1992.
[8] R. Dawkins: The Blind Watchmaker, Essex:Longman, 1986.
[9] T. Ogata, and S. Sugano: Emotional Communication Between Humans
and the Autonomous Robot which has the Emotion Model, in Proc. ofIEEE Int. Conf. on Robotics and Automation (ICRA’99), pp.3177-3182,
1999.
[10] T. OGATA, M. MATSUNAGA, S. SUGANO, and J. TANI: Human
Robot Collaboration Using Behavioral Primitives, in Proc. of IEEE/RSJ
Int’l Conf. on Intelligent Robots and Systems (IROS 2004), pp.15921597, Sept. 2004.
[11] Y. Suga, T. Ogata, and S. Sugano: Aquisition of Reactive Motion forCommunication Robots Using Interactive EC, in Proc. of IEEE/RSJ
Int’l Conf. on Intelligent Robots and Systems (IROS 2004), pp.11981203, Sept. 2004.
[12] T. Ogata, T. Komiya and S. Sugano: Motion Generation of the Autonomous Robot based on Body Structure, in Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2001),
pp. 2338-2343, Oct. 2001.
[13] Y. Katurada and Y. Maeda: Support system for automatically emotional
generation using interactive EC, in 16th Fuzzy System Symposium(Akita, Japan), pp. 297-300, September, 2000. (in Japanese).
[14] T. Shimokawa and T. Sawaragi: Acquiring Communicative Motor Actsof Social Robot Using Interactive Evolutionary Computation, in Proc.
of 2001 IEEE SMC Conference, Tucson, pp.1396-1401, 2001.