Training AIBO like a Dog– Preliminary results –YAMADA Seiji
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda
Tokyo, 101-8430, Japanseiji@nii.ac.jp

Abstract
This paper describes a method to facilitate humanadaptation to a pet robot. A pet robot learns which behavior it should execute when stimulus are given and ahuman user learns how to give commands to the robot
through its various sensors. A pet robot utilizes a computational classical conditioning model for learning tointerpret human’s commands. We propose and implement such a mutual adaptation framework, and develop like-dog heuristics to facilitate the human’s adaptation to a pet robot AIBO. Finally we evaluate theheuristics through preliminary experiments.

1

Introduction

Home robots like AIBO[2], Roomba are gradually
spreading for entertainment and housework as progression of robotics technology. When home robots
come in house, people inevitably have interaction withthem and design of such interaction is becoming important research issues in robotics. Thus human-robot
interaction, especially the interaction between homerobots and naive people to it, is a very signiﬁcant research target.
A human adapts to objects like machines, tools innature. Thus when a human works and plays with arobot at home, he/she tries to adapt it by modelingit, predicting its behavior and teaching it something.
In contrast, a robot should be designed to adapt toa human by learning the model and the action selection through interaction with him/her. Because arobot needs to be able to interpret user’s commands
and execute suitable actions depending on user’s mental state. Hence mutual adaptation between a humanand a pet robot, in which a human and a learning
robot adapt each other, naturally occurs in humanrobot interaction. In such situation, eﬃciency of themutual adaptation between a human and a learning
robot signiﬁcantly depends on design of interaction between them. In this literature, the interaction design

YAMAGUCHI Tomohiro
Nara National College of Technology
22 Yata-cho, Yamato-Koriyama
Nara, 639-1080, Japan
yamaguch@info.nara-k.ac.jp

includes not only design of modality, representationof information which a human and a robot communicate, but also design of robot’s behaviors, learningalgorithm on the human-robot interaction.
The objective of our study is to develop the
methodology of designing human-robot interaction forfacilitating mutual adaptation between a human anda robot. In this paper, ﬁrst we design and implement a framework in which a human trains a pet robot
AIBO to achieve a simple task. Classical conditioning
is utilized to behavior learning of an AIBO. A human teacher trains an AIBO by simultaneously giving unconditioned stimulus and conditioned stimulus
to it, and the AIBO gradually becomes able to execute suitable actions with only conditioned stimulus.
For facilitating human’s adaptation in this framework,
we propose like-dog heuristics to design behavior of apet robot, and actually describe behavior of an AIBO
with the heuristics. Finally we conduct preliminary
experiments to investigate the eﬀectiveness of the likedog heuristics and show the results supporting our approach.
Human-robot interaction has thus far been studied between a human and mobile robots, humanoid
robots. In particular, Ono and Imai proposed a cognitive model to describe how a human reads a robot
mind and investigated its validity experimentally[9].
Though their work is interesting, it has no mutual
learning of mind reading like our work.
We have researched mutual adaptation between ahuman and a life-like software agent[10]. We try to extend our mutual adaptation framework in which a human and a life-like agent adapt each other to humanrobot interaction,
Mutual adaptation between two humans is also interesting research issue. Komatsu et al. pointed out
mutual adaptation between a human teacher and an
human operator who cooperate each other to scorein TV game[8]. Their study is closely related to ourTable 1 Speciﬁcation of AIBO
Dimension
WeightCPU
Sensors

Actuators

Wireless LAN

180 (W) × 278 (H) × 319 (D) mm
Approx. 1.65kg
MIPS R7000 (576MHz)
Temperature Sensor
Distance Sensor (head)
Edge detection sensor (chest)
Acceleration Sensor
Tactile Sensor (head, back)Pressure Sensor (chin, paws (4))
CMOS camera 350,000 pixels
Mouth: 1 dof
Head: 3 dof
Leg: 3 dof × 4 legs
Ear: 1 dof × 2
Tail: 2 dof
IEEE 802.11b

research, however they do not deal with mutual adaptation between a human and an agent like a pet robot.
Kaplan developed clicker training of AIBO[6] withreinforcement learning. By using a clicker in the similar way to a real dog training, a human can gradually
train an AIBO. Dorigo also utilized shaping of operant conditioning[4]. However, a main issue of these researchs are not concerned with adaptation of a humantrainer to a robot during training, and they assumed
a human trainer to be well trained. Our research objective is to facilitate mutual adaptation, not to implement an interactive learning pet robot which canlearn to achieve complicated tasks.

2

AIBO: pet robot

AIBO[2] is a popular pet robot having appearance
like a dog. It has tactile sensors, a camera, a wireless
LAN unit and so on. Fig.1 and Fig.2 show front viewand rear view of AIBO ERS-7 respectively, and thepositions of sensors, indicators attached on AIBO. The
detail speciﬁcation is shown in Table 1.
We employ OPEN-R SDK[1] programming environment for implementing sensing and executing actions
on AIBO, and Ruby language to control an AIBO on
a PC by remote brain approach.

3

Task: easy operation of AIBO

In this study, a task of a human and a pet robot isthat a human operates an AIBO by using only threebuttons(tactile back sensors in Fig.2) on the back toachieve a simple task that going forward a little way,sitting down there and going backward around a start
position. Because no action is assigned to the threebuttons at the beginning, a human needs to train an
AIBO to execute actions by the three buttons, andthis is human’s adaptation to a robot. In contrast,

Figure 1

Sensors and indicators in front of an AIBO

Figure 2

Sensors and indicators in back of an AIBOan AIBO should learn to interpret commands from ahuman using the three buttons and execute adequate
actions, and this is robot’s adaptation to a human.
Thus there occurs mutual adaptation between a human and a robot, and our research objective is to facilitate the mutual adaptation. We try to accelerate
human’s adaptation to a robot by designing suitablerobot’s behavior for a dog-type pet robot.

4

Robot’s adaptation to a human: behavior learning

In order to achieve the task mentioned before, we
utilize classical conditioning as learning algorithm of
AIBO. Though there are a lot of machine leaning algorithm like reinforcement learning, GA, GP, we consider classical conditioning is suitable for our study
because it is a simple learning algorithm based on animal behavior to achieve our task eﬃciently. Reinforcement learning is also based on operant conditioning ofanimals, however the learning needs a large numberof trial and error to converges to the optimal policy
and human subjects can not bear such heavy teaching. Note that our research objective is to develop amethod to facilitate mutual adaptation, not to build

CS (Conditioned Stimulus)[ Bell ]CR (Conditioned Response)

UR (Unconditioned Response)[ Salivation ]US (Unconditioned Stimulus)［Food ]s1 (t)

w1 j (t)s2 (t)

w2 j (t)m j (t)Unconditioned Relationship
Conditioned Relationship

Figure 3Classical conditioningsn-1 (t)w(n-1) j (t)sn (t)wnj (t)a robot to learn many tasks.

4.1

Classical conditioning[5] is well known learning ofan animal as experiments of Pavlov’s dog. Fig.3 showsa framework of classical conditioning. At the beginning, a dog does not respond to ringing bell(CS:
Conditioned Stimulus), and salivates(UR: Unconditioned Response) responding to food(US: Unconditioned Stimulus) in front of it. After ringing bell andfood were given to the dog several times, CS and US
are associated and the dog eventually salivates by only
ringing bell, not food. This is a process of classicalconditioning.
Thought tasks that classical conditioning achieves
is more restricted than reinforcement learning, it has
a simple procedure, quick convergence of learning andsuﬃcient learning ability to achieve the task in ourframework. Thus we utilize classical conditioning as
behavior learning of a pet robot.

4.2

Figure 4Classical conditioning

sensitively to input changes and more eﬃciently.
We assign a neuron to an action, and sensor data
are inputed to each neuron and the output of a neuron is activity. Fig.5 shows the whole neural network
used for action selection of an AIBO indicating sensed
data and actions to be mentioned later. The weights
of inputs for US are ﬁxed, and those for CS are updated by formula (2)-(6) every action execution. The
AS unit is a function which outputs the action withthe maximum activity only when the activity is morethan activity threshold θ. A cycle of sensing, action
selection, action execution and learning is repeated.
mj (t)=nwij (t)si (t)

(1)

i=1

Computational model

Various computational models have been proposed
and their ability are experimentally investigated[3].
We utilize a Klopf model[7] as classical conditioningof a pet robot because it has simple structure and oneof the best models to describe classical conditioning
phenomena of an animal[3].
Fig.4 shows a neuron model of Klopf. A neuron
has multiple inputs from other neurons and a single
output to other neurons. Each input has weight to be
modiﬁed by learning and the output value(activity)
is determined by a function of the inputs. The following formula show the output function (1) and thelearning procedure (2)-(6) and of a neuron j, where
si is a sensed data, mj (t) is activity, cj is a learning
rate to adjust the degree of updating weights and τ
controls how many past data the update is computedwith. The parameters τ and ck are determined experimentally. We slightly modiﬁed the original Klopf
learning equations so that learning can be done more

Klopf neuron model∆wij (t)= wij (t − 1) + ∆wij (t)(2)τ
= ∆mj (t) ck |wij (t−k)|∆si (t−k)(3)∆wij (t)= wij (t + 1) − wij (t)wij (t)k=1∆mj (t) = mj (t) − mj (t − 1)∆si (t−k) = si (t−k) − si (t−k − 1)

4.3

(4)
(5)
(6)

Sensors and motor commands

Since the task §3 needs three actions; go forward, go
backward and sit down, we use three motor commands
corresponding to the three actions for AIBO and noother redundant one is not employed. Sensor data weused are in the following.
• Pressing the head sensed by a tactile sensor on
the head.
• Pressing the back sensed by tactilesors(buttons on the back) on the back.

sen-

5.2

Pressing the head
Pressing the back
Go foreward
Showing a pink ballMotorcommandGo backward

AS

Sit down

Figure 5

Action selection of our AIBO

• Pressing the chest sensed by an edge dectection
sensor on the chest.
• Pressing the chin sensed by a touch sensor on thechin.

Facilitating adaptation with like-dog
heuristics

We propose like-dog heuristics to facilitate mutual adaptation between a human and a dog-type petrobot. This heuristics means that when a humanhas interaction with a dog-type pet robot, he/she inevitably models it as a dog and he/she behaves to it in
the similar way to interact with a dog. This like-dog
heuristics provides us the policy of designing interaction between a human and a dog-type pet robot. In
this study, we can apply like-dog heuristics to AIBO’s
behavior for US/UR.
As mentioned earlier, a human needs to search US
by trial and error because he/she does not know which
stimulus is US at the beginning. In order to facilitate this search, we can apply like-dog heuristics bydesigning AIBO’s behavior for US/UR similarly to adog. By like-dog heuristics, we can expect that a human is likely to ﬁnd the similar US to real dog’s ones.
Thus, by designing AIBO’s behavior for US like a real
dog’s ones, we can facilitate human’s adaptation to an
AIBO.
Hence we design natural behavior of an AIBO for
US/UR in the following, and they are used in experiments.
• If an AIBO ﬁnds a pink ball, it’ll go forward.

• Showing a pink ball sensed by a CMOS camera
image sensor on the top of a nose.

• If a human presses the back of an AIBO, it’ll sit
down.

Interaction design for facilitating
adaptation• If a human presses the chest of an AIBO, it’ll gobackward.

We identify the human’s adaptation in mutual
adaptation of our framework and develop a method
to facilitate the human’s adaptation by using like-dog
heuristics.

These behavior are considered to be natural as adog’s one. By designing such dog-like behavior for US,
we expect to facilitate the human’s search for US andclassical conditioning. The eﬀort of this interaction
design will be evaluated experimentally in the next
section.

5

5.1

Human’s adaptation to a robot

In our framework, the adaptation of a pet robot isjust behavior learning by classical conditioning, andthe human’s adaptation is mainly consisting of thefollowing searches because a human does not know
US and CS at the beginning.
• Searching US: A human need to search US by trial
and error.
• Searching CS: A human need to search CS by trial
and error in the similar to unconditioned.

6

6.1
However, searching CS is not necessary because wetell a human to use three buttons on the AIBO’s backas CS. Thus we attempt to facilitate human’s searching US.

Experiments

We conduct preliminary experiments for evaluating the like-dog heuristics and verifying feasibility of
training an AIBO by classical conditioning.
To evaluate like-dog heuristics, we compare eﬃciency of ﬁnding US with or without like-dog heuristics. Meanwhile, we investigate a human can actually
train an AIBO by classical conditioning for its feasibility.

Experimental setting

We used four subjects(male graduate students in
a computer science department) and an AIBO ERS7 on which classical conditioning was implemented in
the way of §4 for all experiments.

Pink ballFigure 7

Classical conditioning for a pink ball

• An AIBO has three actions, go forward, go backward and sit down which can be executed by certain stimulus.
Press back• Please ﬁnd the three stimulus to execute actions
by touching it, showing it something and so on.
• An AIBO likes a pink ball.

Press chest

Figure 6 Dog-like US

To compare with natural behavior for US/UR withlike-dog heuristics in §5.2, we use unnatural behavior
for US/UR in the following. The ﬁrst and third behavior have opposite actions to natural ones. Notethat the condition of the second behavior, press thehead, is unnatural though it is easily satisﬁed only byhead touch.
• If an AIBO ﬁnds a pink ball, it’ll go backward.
• If a human presses the head of an AIBO, it’ll sit
down.
• If a human presses the chest of an AIBO, it’ll go
forward.
The experiments consist of two phases like the following.
6.1.1

Finding US

Before starting experiments, we give subjects the following instructions.

After the instructions, six behavior consisting of
three natural and three unnatural behavior are in implemented on an AIBO one by one in random ordering
for each subject, and they try to ﬁnd the stimulus of
behavior one by one. We shoot a video and investigate
the time for a subject to ﬁnd each stimulus.
6.1.2

Classical conditioning

After the ﬁrst phase, we ask the same subject to train
an AIBO by classical conditioning, and the following
are instructions.
• We explain principle and procedure of classicalconditioning as described in §4.1.
• We explain a task as described in §3.
For each subject, we implement a set of three natural behavior and a set of three unnatural behavior on
an AIBO, and an subject trains an AIBO twice with
diﬀerent AIBO’s behavior sets. Because the three buttons on the back are used to CS, we modiﬁed the condition of natural behavior for sitting down into touching the chin. We shoot a video and investigate thetime for a subject to ﬁnish learning to achieve the task.
Fig.7 shows a snapshot in which an subject trains an
AIBO with a pink ball. From this ﬁgure, you can see
he simultaneously gives CS(pushing a button in theback) and US(showing a pink ball) to an AIBO.

6.2

Experimental results and discussion

Fig.8 shows experimental results of the ﬁrst phaseto ﬁnd US. The average time to ﬁnd US over subjects

Table 2 Experimental results of classical conditioning: Average time(sec.)

30

Time do find US (sec.)25

Natural behavior
198.5 (40.88)

20

15

10

5

0

Natural Unnatural

Natural Unnatural

Natural Unnatural

Go forward

Go backward

Sit down

Figure 8

Experimental results of ﬁnding US

and the standard deviation are indicated. Seeing fromthis graph, the time to ﬁnd stimulus by the natural
behavior is shorter than that by the unnatural behavior for all three actions. Hence, though the sample
number is small and statistic test is not available, wealmost conclude the natural behavior makes a human
to ﬁnd US more eﬃciently than the unnatural behavior and like-dog heuristics is promising to facilitate
human’s adaptation to an AIBO in our framework.
Currently we are trying to conduct more systematic
experiments with more subjects.
Table 2 shows average and the standard deviationin parentheses of the time to ﬁnish learning. The
training of all subjects ﬁnished about four minutes
and these facts support the feasibility of classical conditioning to train a pet robot for a simple task. The
subjects understood AIBO’s US through the previous
experiments, and this might training more eﬃcient.
Training with unnatural behavior slightly outperforms
one with natural behavior, however a subject with unnatural behavior misled an AIBO by confusion caused
from the unnatural behavior.
Through the experiments, we found out the familiarity of a subject to a real dog, a pet robot, might
have signiﬁcant inﬂuence on whether he/she feels agiven behavior natural or not. Thus, in order to obtain precise results, subjects should be controlled andgrouped or the results are normalized depending onsuch familiarity.

7

Conclusion

We proposed and implemented a mutual adaptation
framework of a human and a pet robot, and developed
like-dog heuristics to facilitate the human’s adaptation

Unnatural behavior
177.75 (28.34)

to a pet robot. In our framework, a pet robot does behavior learning and a human learns how to train it. A
pet robot utilizes classical conditioning, and we proposed like-dog heuristics to facilitate human’s adaptation to a pet robot. Finally we conducted preliminary
experiments using an AIBO to evaluate the heuristics,
and veriﬁed eﬀectiveness of like-dog heuristics.
The experimental results were obtained with asmall number of subjects. Thus we will conduct more
systematic experiments with more subjects, and verify the eﬀectiveness of like-dog heuristics. Furthermore, we have a plan to develop various interaction
designs using like-dog heuristics and evaluate them experimentally.

References[1] Aibo sde web site. http://openr.aibo.com/.
[2] Aibo web site. http://www.sony.net/Products/aibo/.
[3] C. Balkenius and J. Morn. Computational models
of classical conditioning: a comparative study. In
Proceedings of the Fifth International Conference on
Simulation of Adaptive Behavior, 1998.
[4] M. Dorigo and M. Colombetti. Robot Shaping:
An Experiment in Behavior Engineering.
MIT
Press/Bradford Books, 1998.
[5] W. Hill. Learning: A survey of psychological interpretations. Harper and Row, 1985.
[6] F. Kaplan, P.-Y. Oudeyer, E. Kubinyi, and A. Miklosi. Robotic clicker training. Robotics and Autonomous Systems, 38(3-4):197–206, 2002.
[7] A. H. Klopf. A neuronal model of classical conditioning. Psychobiology, 16(2):85–123, 1988.
[8] T. Komatsu, K. Suzuki, K. Ueda, K. Hiraki, andN. Oka. Mutual adaptive meaning acquisition by
paralanguage information: Experimental analysis ofcommunication establishing process. In The Proceedings of the 24th Annual Meeting of the Cognitive Science Society, pages 548–553, 2002.
[9] T. Ono and M. Imai. Reading a robot’s mind: Amodel of utterance understanding based on the theory of mind mechanism. In Proceedings of the Seventeenth National Conference on Artificial Intelligence,
pages 142–148, 2000.
[10] S. Yamada and T. Yamaguchi. Mutual adaptation
to mind mapping in human-agent interaction. In
IEEE International Workshop on Robot-Human Interaction, pages 105–110, 2002.