Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, Technische
Universität München, Munich, Germany, August 1-3, 2008

“Good Robot”, “Bad Robot” –Analyzing Users’ Feedback in a
Human-Robot Teaching Task
Anja Austermann, Seiji Yamada

The work presented in this paper tries to answer threequestions which are a basis for our research:- Is there a benefit in having a robot learn multimodal
feedback from its user? This is only true, if different
people give reward in different ways which are hard tohandle by using hard-coded feedback patterns.
- Is it possible to learn multimodal user feedback in a
training phase in a reasonable amount of time? Only if
reward behavior used by a single person does not vary
excessively and is similar between different tasks, it
can be learned effectively in a training phase- Which features of the training tasks are important forlearning natural user feedback?
In our study four different tasks are used to provoke
positive and negative feedback from the user. The tasks aremodeled to resemble simple games, suitable for young
children, such as “Pairs” and “Connect Four”.
As the tasks are used for automatically recording andlearning users’ feedback, they must be designed in a way thatthey allow the robot to provoke natural feedback from the user
while not boring or stressing him or her. Moreover, the robot
needs to be able to explore its user’s reward behaviorautonomously without remote control. We decided to use anapproach for provoking user’s feedback that is inspired by the
Wizard-of-Oz principle, which is widely used in research on

Abstract—This paper describes an experimental study inwhich we analyze how users give multimodal positive andnegative feedback by speech, gesture and touch when teachingeasy game-tasks to a pet robot. The tasks are designed to allow
the robot to freely explore and provoke human reward behavior.
By choosing game-based tasks, we ensure that the training canbe carried out without stressing or boring the user. This way, we
can observe natural, situated reward behavior.

I. INTRODUCTIONW

HEN teaching a robot by natural interaction one
important task for the human teacher is to evaluate therobot’s performance and give positive and negative reward.
This is especially true in case of reinforcement learning, where
positive and negative reward is the only means for the user toguide the behavior of the robot, but feedback is also usedfrequently in general teaching tasks.
In this paper we describe a recently conducted experimental
study which aimed at finding out, how users employ different
modalities to give positive and negative feedback to a robot.
We analyzed how speech, touch and gestures were useed forgiving reward. We also assessed the variability of positive andnegative feedback between users and between different tasks
that the robot performs. By analyzing and comparing users’
feedback in different training tasks, we investigated whichfeatures of a task encourage or hinder the user to give
feedback in a natural way.
The long-term-goal of our research, which motivated thisstudy, is to develop a method for learning commands as wellas positive and negative rewards that can be used forcontrolling the service and entertainment functions ofservice-robots and pet-robots through the interaction with auser. We chose the learning of positive and negative rewards
as the starting point of our work, because they are the smallest
useful set of commands, that can be used to teach a robot, forexample by reinforcement learning. We are using a
combination of Hidden Markov Models [11] and classicalconditioning [3] for enabling the robot to learn its user’s
preferred ways of giving reward and instruction. Details of thelearning algorithm are given in [1] and [2].
Manuscript received May, 20. 2008. A. Austermann is with the Graduate
University for Advanced Studies (SOKENDAI), Tokyo, 101-8430 Japan(e-mail:anja@nii.ac.jp)
S. Yamada is with the National Institute of Informatics and the Graduate
University for Advanced Studies (SOKENDAI) Tokyo, 101-8430 Japan(e-mail: seiji@nii.ac.jp)

978-1-4244-2213-5/08/$25.00 ©2008 IEEE

Figure 1: AIBO playing. The goal of the task, is to find the picturethat corresponds to the sample above

41

Human-Robot-Interaction [4]. As in a Wizard-of-Oz scenario,
we aim at giving the user the impression that the robot
adequately reacts to his or her commands at a stage, where it
actually does not understand them. However, different from a
regular Wizard-of-Oz scenario, the training can be performed
without remote controlling the robot.
Instead of that, the tasks and the behavior of the robot aredesigned with the objective to ensure that the user and therobot share the same understanding of the goal of the task aswell as the way to reach it. This is done in two ways: The rulesof the game task as well as an evaluation function to calculate
the quality of all possible moves are hard-coded in the robot,
so that it knows whether a move is good or bad in a certain
situation. On the other hand, we chose tasks, which are easy
enough to allow the user an instant and correct evaluation of
the moves of the robot and give feedback accordingly. So therobot does not need to actually understand the user’scommands to know how to correctly perform the task and candeliberately provoke feedback from the user by good or bad
performance.
In our experiment, three out of four training tasks are“virtual” tasks. The robot plays on a computer-generated
game board, which is projected from the back to a white
screen, as can be seen in figure 1. All relevant information on
the state of the training task can be accessed directly from thegame server. This way, we can ensure that the task state isalways interpreted correctly because the robot does notdepend on potentially erroneous processing of sensor data.
Further explanations on the training tasks are given in section
III of this paper.

well as pressing the robots head or back sensor.
Thomaz et al. described an experimental setting forassessing human reward behavior and its contingency [8]. The
participants of the study could give positive as well as
negative reward to teach the virtual character Sophie to bake acake in the “Sophie’s World” scenario. Reward could be given
by an interactive reward interface that allowed the user toassign any reward on a scale from -1 to +1 either to a certain
object or to the world state. In their experiments they found astrong bias towards positive reward and discovered aphenomenon that they described as anticipatory rewards,
positive rewards that were assigned to an object that thecharacter has to use in a later step. This kind of reward can beinterpreted as guidance for the character.
III. TRAINING TASKSIn order to allow the robot to explore the way, its user gives
reward and in order to record positive and negative feedback
for learning and further analysis, we use specially designed
training tasks. The main difficulty that has to be addressed
during the training is that the robot needs to performautonomously while not having any prior knowledge about its
user and his or her way of interacting. This leads to a numberof requirements that need to be met by the training task.
A. Requirements for suitable training tasks
Deliberately provoking positive and negative rewards froma user is only possible for the robot within a task where thehuman and the robot have the same understanding of whichmoves are desirable or undesirable. This allows the robot toanticipate the user’s commands and explore his or her reward
behavior by performing compliant or non-compliant actions.
In our experiments, two of the tasks allowed the user to give
reward as well as instructions, while in the remaining twotasks, the users were asked to teach the robot by only giving
positive and negative reward. Tasks that only allow positive
and negative feedback from the user for training the robot canbe designed relatively freely as long as their goal is known toboth, the user and the robot. However, tasks that allow the user
to give commands to the robot have to be designed carefully toensure, that the user and the robot maintain the sameunderstanding of whether an action of the robot is positive or
negative. When users feel, their current level of instruction is
not interpreted correctly by the robot they tend to give morefine-grained instructions. E.g. when the robot did notunderstand “Put the red stone to the yellow field”, theparticipants of our preliminary experiments typically changed
their instructions to “go forward… stop… pick up the redstone…”. In tasks where the order of their subtasks can bechanged, fine grained instruction can lead to a situation where
the robot’s actions do not comply with the user’s instructions,although they lead to the correct goal state. Therefore, whenallowing instructions from the user, tasks have to be designed
in a way that knowing the goal state imposes an unambiguousorder on their subtasks.

II. RELATED WORK
Various studies have been conducted to investigate howusers apply different modalities when interacting with acomputer system. In [6] Oviatt gives an overview of somemajor results of recent research in multimodality and points
out ten of the most common misconceptions about how peopleinteract with multimodal voice- and gesture-based systems.
However, all quantitative analyses on multimodal user
behavior, that we are aware of, have been conducted using
GUI-based systems. Although several implementations ofmultimodal systems for Human-Robot-Interaction can befound in literature [5][7] little or no information exists on user
preferences for natural, unrestricted multimodal interaction
with robots. As embodiment needs to be considered as animportant factor in Human-Robot-Interaction [10], it isunclear in how far results from the interaction with GUI-based
systems can be transferred to Human-Robot-Interaction.
Several studies on how people teach robots or otherartificial creatures, such as virtual characters through positive
and negative reward have been conducted in recent years.
Ullerstam and Mizukawa [9] developed a method to teachcomplex behavior patterns to an AIBO pet robot based on
reinforcement learning. In their system, reward is given
through two predefined positive and negative utterances, as

42

However, the user’s judgment of the situation remains a
potential source of errors, so the task must be designed in a
way that the situation is easy to evaluate by the user. This istrue for three out of our four tasks that we used in our
experiments. In the “Pairs” game and the “Same Image” gamethe user only had to evaluate whether the two cards, chosen by
the robot, show the same picture or not. In the “Dog Training”
game, the users had to evaluate, whether the motion of therobot corresponded to the command given. In case of the“connect four” game, evaluation is more difficult for the user
and there are possible moves that are not clearly good or bad.
This allowed us to assess changes in the participants’ behavior
depending on whether they are confident in the feedback, they
provide, or not.

focus on the latter kind of interaction patterns.
We further assume that each user has a limited inventory ofinteraction patterns to express a certain command or reward.
The interaction patterns, that are typically used, can change
slowly over time. Moreover, we assume that interaction
patterns used by one user for the same instruction do not vary
excessively between different tasks. Otherwise, it would not
be possible to learn these patterns within a training task in areasonable amount of time.
V. EXPERIMENTS

B. Using “virtual” tasks
An issue the impact of which became obvious in the
preparation and during the execution of preliminary
experiments is the very limited ability of the AIBO robot tophysically manipulate its environment and to move precisely.
In our preliminary study [1], this issue was addressed byequipping the robot with a shovel to facilitate moving objects
around. However, it is still possible that the robot does notdetect detecting errors during task-execution, such as failingto pick up the correct object. This poses a risk formisinterpreting the current status of the task. Therefore we
decided to implement the training task in a way that the robot
can complete it without having to directly manipulate its
environment. When using a computer-based task, the current
situation of the robot can be assessed instantly and correctlyby the software at any time. During the experiments, the imageof the playfield is generated by a computer and projected fromthe back to the physical playfield, as seen in figure 1. The
robot visualizes its moves by motion and sounds and reacts tothe moves of its computer opponent by looking at theappropriate positions on the playfield.
By reducing the risk that either the robot or the user
misinterprets the situation, we ensure that the robot is able toanticipate the user’s next reward or instruction correctly. Thisis necessary for associating the observed behavior correctlywith a positive or negative reward.
IV. ASSUMPTIONS
Our further work on learning multimodal reward patterns
relies on various assumptions which were investigated upon in
the experiments, described in section V of this paper.
We assume that patterns of interaction between humans androbots range from rather universal ones, like pointing gestures,
which are roughly the same between different individuals tohighly individual patterns, like giving positive/negative
reward.
Patterns that are universal can be pre-trained and adapted toa certain user during task execution. Only patterns that vary
substantially between users need to be trained in a training
phase that precedes the actual use of the robot. In our work we

43

We conducted two experimental studies to analyze humanreward behavior during the interaction with a pet robot. Our
aim was to get a better understanding of how people naturally
employ different kinds of feedback and different modalities togive reward to a robot, which modalities are most importantand how the reward given, varies for different persons andthroughout different tasks.
In a preliminary study, which is described in more detail in[1] and [2] the participants were asked to instruct the robot tomove colored objects to their target places and give reward to
the robot for correct and incorrect actions. We imposed
different restrictions on the reward behaviors that theparticipants were allowed to use in order to find out, howrestrictions in allowed reward modalities affect the frequencyof reward given by the users. We confirmed that theparticipants reacted quite sensitive to restrictions in theallowed way of giving feedback and that they tended tounintentionally give additional feedback that did not conformto the given restrictions. We also recognized some issues,
such as the low ratio of actual interaction time to the time thatthe user was waiting for the robot to do something due to therobot’s slow movements and the not very enjoyable nature of
the service-task used in our preliminary experiments.
We used these findings from the preliminary study to
improve the design of the training task for our second study.
The participants interacted with an AIBO pet robot in four
different game-like tasks. Three of which were “virtual” tasks
as described in section III.B.
Audio and stereo video data as well as information
perceived through the sensors of the robot were recorded.
After the experiment and after each of the subtasks, we asked
the participants to answer a questionnaire about theirexperience.
A. Experimental setting and instruction
Ten persons participated in our study. All of them wereJapanese graduate students or employees at the National
Institute of Informatics in Tokyo. Five of them were females,
five males. Their ages ranged from 23 to 47. All participants
have experience in using computers. Two of them haveprevious experience in interacting with entertainment robots.
Interaction with the robot was done in Japanese. During theexperiment, we recorded roughly 5.5 hours of audio and video
data containing 2409 reward instances.

Based on this data we analyzed the usage of the different
modalities speech, gesture and touch for giving feedback to arobot. Moreover, we investigated on how the feedback, given
to a robot differs between different people and how it varies
for an individual person during one task and when training therobot in different tasks.

implemented in a way that the rate of correct choices and the
speed of finding the correct image increased over time.
b)

In the Pairs game, the robot played the children’s gamePairs, which can be seen in the second picture of figure 2. At
the beginning of a Pairs game, all cards are displayed upside
down on the playfield. The robot chooses two cards to turnaround by looking and pointing at them. In case, they show thesame image, the cards remain open on the playfield.
Otherwise, they are turned upside down again. The goal of thegame is, to find all pairs of cards with same images in as little
moves as possible. In this task, the participants were asked not
to give instruction to the robot, which card to chose but teachthe robot to play the game by only giving positive andnegative feedback for its moves. Waiting for and reacting to
feedback is realized in the same way as in the Same Images
task.

1) Experimental setting
After analyzing the videos and the questionnaire responsesof the preliminary study, the task for the next series ofexperiments was adapted according to our findings. The main
reasons for the modifications were to gather more user
feedback in a short time, to design the training task in a waythat it does not resemble as much to a simple service task as
the one used for the preliminary study, and to restrict theparticipants’ possible movements and postures in a
non-intrusive way. Moreover, the task should be feasible for
implementation without the need to remote control the robot.
We selected training tasks based on easy games.
In the experiments the participants were asked to teach therobot, how to correctly play these games by giving
instructions and positive/negative reward for the robot's
moves. They were instructed to interact with the robot freely
by gesture, voice and by touching the robots touch sensors.
They were told that the robot learns to play the different games
through their instructions. In order to endorse the impression,that the system actually processes and learns from gesturesand speech data a stereo camera was placed in 2.5 meters
distance, facing the participant, and a microphone was
attached to his/her clothes. The locations of the touch sensors
on the back and the head of the robot were explained to theparticipants.
In order to find out, whether certain features of the task likethe presence of an opponent in a game or the opportunity togive not only reward but also instructions have an influence on
the user behavior or the user’s evaluation of the task, we
decided to use four different training tasks with different
properties.
a)

Pairs

c)

Connect FourIn the Connect Four game, the robot plays the game
Connect Four against a computer player. Both players taketurns to insert one stone into one of the rows in the playfield,
which then drops to the lowest free space in that row. The goalof the game is, to align four stones of one’s own color either
vertically, horizontally or diagonally. As in the Pairs task, theparticipants were asked to not to give instructions to the robot
but provide feedback for good and bad moves in order tomake the robot learn how to win against the computer player.
d)

Dog training

In the Dog Training task, the participants were asked toteach the speech commands “forward” (mae), “back” (ushiro),“left” (hidari), “right” (migi), “sit down” (suwatte) and “standup” (tatte) to the robot. The “dog training” task is the only taskthat does not use the “virtual playfield”. In this task, weremote controlled the robot, so the participants could give
instructions in the order they liked without the restrictions for

Find same images

In the Find same images-Task, the users taught the robot tocompare six different pictures to a sample image in the middleof the screen and to choose the one that corresponds to thesample. This task is shown in figures 1 and 3. While playing,
the picture, that the robot is currently looking at, is marked
with a green frame to make it easier for the user to understandthe robot’s viewing direction. When AIBO chooses an image,
the robot points at it and a red frame is shown around it. By
waving its tail and moving its head, the robot indicates, it iswaiting for feedback from its user. Feedback is detected using
the touch sensors as well as voice activity detection. When
feedback has been detected or a timeout has exceeded therobot proceeds with the next image. The users were asked toprovide instruction as well as reward to the robot to make itlearn to perform the task correctly. The system was

Figure 2: Playfields for the different game tasks: 1) Find same images,
2) Pairs, 3) Connect four44

different verbal expressions to convey positive feedback and13.4 different expressions to express negative feedback.
However, this number varies strongly between individuals:
One person always used the same utterance for giving positive
feedback and a second utterance for giving negative feedback
while the person with the most variable feedback used 30
different expressions for giving positive and 28 different
expressions for giving negative feedback. 55.61% of allverbal feedback was given by the participants using theirpreferred feedback utterance. 88.73% of a user’s verbal
feedback was given using one of his/her six most frequently
used positive/negative utterances, so understanding a
relatively small number of different utterances suffices tocover most of a participant’s feedback. .
For positive feedback, four out of ten participants had one
preferred utterance which did not vary between the four
training tasks. In case of negative reward, this was true for fivepeople. For eight out of ten participants in case of positive
reward and six participants in case of negative reward, theiroverall most frequently used feedback utterance was among
the top three feedback utterances in each individual task.
In the cases, where the preferred feedback was not the samein all tasks, it typically differed for the “Connect Four” task,
while in the three other tasks, including the “Dog Training”
control task similar feedback was used as described above. As
in the “Connect Four” task it was difficult for the users tojudge, whether a move was good or bad in order to provide
immediate reward, feedback tended to be very sparse andtentative like “not really good” (amari yokunai), “Is this
good?” (ii kana?) or “good, isn’t it” (ii deshou).
In a questionnaire, we asked the participants to evaluate
their experience throughout the four different training tasks.
The participants could rate their agreement with thestatements, shown in table 1, on a scale from 1 to 5, where 1
was the best and 5 the lowest rating.

Figure 3: Scene from the video taken during the experiments.

suitable tasks defined in section III. It is the only task, where
the robot actually reacts to its user's commands. It is used as acontrol task to determine, whether the virtual playfield and thegame-like nature of the other tasks affect the participants' wayof giving reward in any way.
2) Results
As for the modalities used for giving reward, we found astrong preference for speech-based reward. Among 2409
rewards given, 1888 (78.37%) were given by speech, 504(20.92%) were given by touching the robot and 17 (0.71%)
were given by gestures. For the different users, the percentageof speech-based rewards ranged from 52.25% to 97.75%.
Gestures were frequently employed by the participants forgiving instructions, but we almost did not observe gestures
being used for giving positive or negative reward.
Typically, multiple rewards were given for a single positiveor negative behavior of the robot. Counting only the rewards
given during the time, when the robot signaled that it was
waiting for feedback after an action, 3.43 rewards were given
for one action on average, usually including one touch reward
and one to four utterances. One utterance was counted as one
reward. Repetitions of an utterance were counted as multiple
rewards. In case of touch reward, one or multiple contacts
with the robot’s touch sensors were counted as one reward, aslong as the participant kept his/her hand close to the sensor.
The favorite verbal feedback differed between the users
especially in case of positive reward. None of the utterances,used for positive feedback, appeared within the first six mostfrequently used utterances for all ten participants. On average,
each person shared his/her overall most frequently used
positive feedback with one other person. In case of negative
reward, the feedback, given by the participants was morehomogenous. The most frequently used feedback - “wrong”(chigau) - was preferred by eight out of ten persons. For thetwo remaining persons, it was the second and third mostfrequently used feedback utterance.
As for the variability of the feedback, given to the robot by
an individual user: On average, participants used 12.3

TABLE1: RESULTS OF THE QUESTIONNAIRE

Teaching the robot throughthe given task was enjoyable
The robot understood myfeedback
The robot learned through myfeedback
The robot adapted to my wayof teaching
I was able to teach the robot
in a natural way
I always knew, which
instruction or reward to giveto the robot

Same
1.81
1.04
1.27
0.4
1.36
0.59
1.45
0.66
2.18
0.96
2
0.72

Pairs
1.90
0.83
1.81
0.74
2.81
0.93
2.63
1.05
2.09
0.86
2.09
0.86

Four
1.81
0.89
2.90
0.85
3.45
0.95
3.45
1.04
2.54
1.12
2.90
1.02

Dog
1.63
0.81
1.81
0.30
1.54
0.69
1.64
0.58
1.64
0.69
1.91
0.83

(First value: average, second value: standard deviation)

The four tasks were considered almost equally enjoyable by
the participants. For the Find same Images task and the Dog
Training task, the participants' impression that the robot
actually learned through their feedback and adapted to their

45

way of teaching was strongest. Those two tasks allowed theparticipants to not only give feedback to the robot but also
provide instructions to it. Moreover, they were designed in a
way that the robot's performance improved over time.
In the Dog Training task, the robot was remote controlled
to react to the user's commands and feedback in a typicalWizard of OZ-Scenario. However, in the Same Image task,
the user's instructions and feedback were not actually
understood by the robot but anticipated from the state of the
training task which was designed using the restrictions definedin section III. This did not have a negative impact on theparticipants’ impression that the robot understood their
feedback, learned through it and adapted to their way ofteaching, compared to the Wizard of Oz scenario.
The lowest ratings were given for the "Connect Four" task.
As the robot's moves could not be evaluated as easily, as in theother tasks, the participants were unsure, which rewards togive and therefore did not experience an effective teachingsituation.

It focuses on positive and negative reward instead of
instructions or commands and it uses a robot as interaction
partner, not a GUI-based system.
Oviatt states in [6] that multimodal combining of speechand gesture is mainly used to describe spatial relations. In her
experiments, less than 1% of the participants expressed
themselves multimodally when performing actions without
any spatial component. This could be one explanation, why
we observed almost no gestures for giving positive andnegative feedback, while gestures could be observed when theparticipants gave instructions to the robot like, for instance,
pointing to an image to chose.
Our experiments have been conducted with a dog-shaped
AIBO pet robot. This might have persuaded the users tointeract with the robot in a similar way as with an actual pet.
Especially the frequent use of touch to express approval ordisapproval may be caused by the dog-like appearance of therobot. To ensure the transferability of our findings to generalhuman-robot-interaction tasks, experiments with other typesof robots would be required.

VI. DISCUSSION
REFERENCES

From our experiments, we found that for learning andunderstanding users’ positive/negative feedback, emphasisshould be placed on the modalities speech and touch. Gestureis mainly used for instruction and explanation.
Natural feedback given by different users can vary strongly,
especially in case of positive rewards. Therefore, learning tounderstand the feedback that a certain user employs instead ofusing hard-coded and potentially unintuitive commands,
which have to be learned by the user, helps to ensure naturalinteraction and a positive user experience.
Learning to understand feedback through a training task isonly feasible and useful, if the feedback given by one user issimilar within different tasks. The results from theexperiments suggest that this is actually the case and thattypically a limited number of utterances are used by anindividual to convey positive and negative reward.
However, there are cases where the contents of theutterances alone may not be correctly understood as a positiveor negative reward: For instance, some of the users
occasionally just repeated their previous command in a stricter
tone before or instead of giving other negative feedback to therobot. In these cases, analyzing and learning the prosody
which determines the sentence melody of typical positive andnegative feedback utterances can be expected to improve therecognition accuracy.
Problems arise, if the user is not exactly sure, how to judge
the robot's behavior, as in the "Connect Four" task. Therefore,
for automatically learning rewards, the task has to be designed
in a way that it is easy to understand for the user. Otherwise
the amount and reliability of the given reward, as well as theuser's motivation to complete the training, decrease.
Comparing the results of our study with existing work on
how people use multimodality is difficult because our study
differs from most existing work in two major aspects:[1]

A. Austermann, S. Yamada, "Learning Reward Modalities for
Human-Robot-Interaction in a Cooperative Training Task",
Proceedings of the IEEE International Symposium on Robot andHuman Interactive Communication 2007
[2] A. Austermann, S. Yamada, " Learning to Understand Multimodal
Rewards for Human-Robot-Interaction using Hidden Markov Models
and Classical Conditioning" To appear: Proceedings of the WCCI
2008
[3] C. Balkenius and J. Morn. "Computational models of classicalconditioning: a comparative study." Proceedings of the Fifth
International Conference on Simulation of Adaptive Behavior, 1998
[4] K.
Dautenhahn,“Methodology&
Themesof
Human-Robot-Interaction: A Growing Research Field”, International
Journal of Advanced Robotic Systems, Special Issue on
Human-Robot-Interaction,Vol. 4, No. 1, pp. 103-108 , 2007
[5] A. Haasch , S. Hohenner, S. Hüwel, M. Kleinehagenbrock, S. Lang, I.
Toptsis, G. A. Fink, J. Fritsch, B. Wrede, G. Sagerer, “BIRON - The
Bielefeld Robot Companion” (In E. Prassler, G. Lawitzky, P. Fiorini,and M. Hägele, editors), Proc. Int. Workshop on Advances in Service
Robotics, pages 27-32, Stuttgart, Germany, May 2004. Fraunhofer IRB
Verlag.
[6] S. Oviatt, “Ten myths of multimodal interaction” Communications of
the ACM, Vol. 42 , No. 11, pp. 74 – 81, 1999
[7] R. Stiefelhagen, C. Fugen, P. Gieselmann, H. Holzapfel, K. Nickel andA. Waibel "Natural Human-Robot Interaction using Speech, Head Poseand Gestures" Proceedings. 2004 IEEE/RSJ International Conferenceon Intelligent Robots and Systems, 2004. (IROS 2004). 2422- 2427
vol.3, 2004
[8] A. L. Thomaz, G. Hoffman, and C. Breazeal. "Reinforcement Learningwith Human Teachers: Understanding how people want to teachrobots." Proceedings of the 15th IEEE International Symposium on
Robot and Human Interactive Communication (RO-MAN), 2006.
[9] M. Ullerstam, M. Mizukawa, "Teaching robots behavior patterns byusing reinforcement learning: how to raise pet robots with a remotecontrol", SICE 2004 Annual Conference Volume 1, Issue , 4-6 Aug.
2004 Page(s): 143 - 146 vol. 1[10] J. Wainer, D. J. Feil-Seifer, D. A. Shell, M. J. Mataric "Embodimentand Human-Robot Interaction: A Task-Based Perspective", ,Proceedings of the IEEE International Symposium on Robot andHuman Interactive Communication 2007[11] S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, G.
Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, P.Woodland “TheHTK Book (for HTK Version 3.4)”, 2006

46