19th IEEE International Symposium on Robot andHuman Interactive Communication
Principe di Piemonte - Viareggio, Italy, Sept. 12-15, 2010

Does the Appearance of a Robot Affect Users’ Ways of Giving
Commands and Feedback?
Anja Austermann, Seiji Yamada, Kotaro Funakoshi and Mikio Nakano
Abstract— Our study compares users’ interaction with ahumanoid robot and a dog-shaped pet-robot. We conducteda user study in which the participants had to teach object
names as well as simple commands to either the humanoid or
the pet-robot and give feedback to the robot for correct andincorrect performance. While we found, that the way of utteringcommands rather depends on personal preference than on the
robots’ appearance, the way of giving positive and negative
feedback differed significantly between both robots: We foundthat for the pet-robot users gave reward in a similar way as
giving reward to a real dog by touching it and commenting
on its performance by uttering feedback like ”well done” or”that was right”. For the humanoid, users typically did not use
touch as a reward and rather used personal expressions like”thank you” to praise the robot. Our findings suggest that users
actually rely to some degree on the appearance of a robot as acue for deciding how to interact with it.

I. I NTRODUCTION
When humans interact with other humans or with theirpets they tend to adapt their way of speaking and interacting
to their interaction partner. For example, people talk to adults
in a more elaborated way than to small children, and they
pet their dog as a reward while they would rather say “thankyou” when their colleague has done them a favor. Moreover,they speak more slowly and clearly, when they assume theircommunication partner is not understanding them well.
We assume that similar mechanisms also affect how peopleinteract with robots. Especially the appearance of a robot
and its resemblance to familiar creatures or objects can be
an important factor which helps a human to anticipate thecapabilities of a robot and decide how to interact with it.
The results from our research can help inform the design
choices that roboticists make when considering what type ofinteraction they want with their robots.
II. R ELATED W ORK
In recent years, there have been various studies [8] [4] [5][3] investigating the effect of a robot’s appearance on theinteraction with a user. However, most studies concerning
the appearance of robots rather deal with the uncanny valley
effect [2] and users’ impression of robots than with the effectof a robot’s appearance on its user’s communicative behavior.
Kanda et al. [4] conducted a study with two different
humanoid robots and showed that different appearances of

the robots did not affect the participants’ verbal behavior
but did affect their non-verbal behavior such as distance anddelay of response. They explain the observed differences byimpressions, such as novelty, safety, familiarity and activity
as well as attributions, such as whether the robot is respected
as a conversation partner.
Kriz et al. [8] investigated users’ conceptualizations ofrobots by analyzing the way the users talked to the robot.
They compared features of robot-directed speech to howhumans talk to infants or adult non-native speakers. They
found that the participants spoke more loudly, raised theirpitch, and hyperarticulated when they spoke to the robot.
This behavior is typical when the conversation partner isassumed to have low linguistic competence. However, they
did not speak in easier sentences, which suggests, thatthey believed that the robot has almost humanlike cognitive
capabilities.
Goetz et al. [5] investigated users’ attribution of capabilities depending on the appearance of a robot. They createdimages of more or less human-like looking robots and hadparticipants judge their suitability for different tasks. They
found that people systematically preferred robots for jobs
when the robot’s human-likeness matched the sociability
required in those jobs. They also found in a second user study
with a humanoid robot, that playful or serious demeanorof the robot affects the compliance of the participants.
The participants performed a playful task longer, when
the instructing robot showed a playful demeanor while theparticipants performed a serious task longer, when the robot
behaved more seriously.
Similar results were obtained by Hegel et al. [3] who foundthat the appearance of robots affected users’ attribution of
possible applications. They conducted a user study in which
the participants were asked to match videos of twelve robots
to thirteen different categories of applications. Especiallythe perceived human-likeness or animal-likeness affectedwhich tasks the participants considered suitable for eachrobot. While the participants considered human-like robots
for fields like healthcare, personal assistance, security andbusiness, they considered animal-like robots as companions,entertainers, toys, and robotic pets.
III. O UTLINE OF THE S TUDY

A. Austermann is with the Graduate University for Advanced Studies,Tokyo, Japan anja@nii.ac.jp
Seiji Yamada is with the Graduate University for Advanced Studies andthe National Institute of Informatics, Tokyo, Japan seiji@nii.ac.jp
Kotaro Funakoshi and Mikio Nakano are with the Honda Research Institute Co., Ltd., Wako, Japan funakoshi@jp.honda-ri.co.jp,

nakano@jp.honda-ri.co.jp

978-1-4244-7989-4/10/$26.00 ©2010 IEEE

We conducted a user study on how participants givecommands and feedback to a pet-robot and a humanoid. Asa pet-robot, we used a dog-shaped robot, which has roughly
the size of a cat or a small dog. The humanoid robot is 1,20m tall, which is about the size of a 8-year-old child. Both254

TABLE I
C OMMANDS THAT W ERE U SED IN THE T RAINING TASK .
Command
move
bring
clean
switch on
switch off
callcharge battery
show status

Parameters
object, place
objectobjectobjectobjectobject-Example sentence
Put the ball into the box.
Bring me a coffee, please.
Please clean up the carpet.
Robot, switch on the light.
Switch off the radio.
Please make a phone call to Rita.
Recharge your battery.
What is your status?

robots are shown in Fig. 2. The goal of our study was tofind differences and similarities in user behavior when theparticipants give commands and feedback to the pet-robot
and the humanoid. The user study, described in this paper,
is a part of our work on learning commands and feedback
for human-robot interaction [1].
Each participant interacted with either the humanoid or
the pet-robot and instructed the robot to perform different
typical household tasks like bringing a coffee, switching on
the light or the TV, tidying up etc. and gave feedback to therobot for correct or incorrect performance.
A. The “Virtual Living Room”
In order to avoid time-consuming and error-prone task execution in the real world and because of the different physicalcapabilities of the two different robots, we implemented a“virtual living room”. The tasks as well as the actions of therobot were visualized on a large screen. The robot was placedin front of the screen and used motion and speech to informthe user which action it is currently performing in the virtualliving room. The robots’ actions and pointing direction werealso visualized in the living room scene with a hand or paw
icon and the scene changed in response to the actions of
the robot. Based on these cues the participants could easily
understand the relation between the robot’s motions and thechanges happening in the scene. While the robots differedin shape and size we kept all other parameters as similar
as possible, using the same synthesized speech utterances,similar gestures, same simulated learning rate, almost same
position of the robot in relation to the user etc.
During the training, the user could figure out by looking
at the scene, what command to give to the robot next. We
used a graphical representation of the scene without any text,
in order to avoid influencing the participants’ wording whengiving commands to the robot.
Table I shows the list of all commands, that were used in
the task and sample utterances for each command. The users
were not instructed in advance, which commands they had toteach to the robot but were asked to infer which commands
were appropriate by looking at the virtual living room scene.
B. The Training Phases
One experiment with one participant comprised two successive training phases. In the first phase, the user had toteach the names of eighteen different objects to the robot.
The robot pointed at objects on the screen and a spotlight
as well as a pointing arrow was shown in the living room
scene to make it easier for the user to understand the robot’s
pointing direction. The robot then asked “What is that”(“kore ha nan desu ka?”) to prompt the user for an object
name. After the user had uttered an object name, the robot
continued with the next object.
We asked the users to only utter the object names without
any additional words. This was a requirement for our learningalgorithm. Because of this restriction, the speech, recordedin the first training phase was not evaluated.
In the second phase, different scenes were shown on
the screen for learning commands. As described above,
each scene visualized a task, that had to be performed by
the robot, in a way that the user could understand which
instruction would be suitable. The robot looked at the user
while it was waiting for an instruction. After the user had
uttered an instruction, the robot either performed correctly or
incorrectly. For example when the user uttered a command
like “Can you bring me a coffee”, the robot would perform
correctly by pointing at the screen and making an appropriate
gesture. As a visualization, the robot hand icon on the screen
would move to the table to put the coffee cup there andthen the robot would say “Here you are” (“douzo”). For an
incorrect performance, the robot would, for example switch
off the light instead of bringing a coffee.
After executing the user’s command either correctly or
incorrectly, the robot looked at the user to wait for feedback. After receiving positive or negative feedback, therobot confirmed by either thanking for positive feedback
or confirming that it understood the negative feedback. As
the robot could not actually understand the given feedback
during the training task, it used the heuristics, that feedback,given after a correct performance, is positive feedback andfeedback, given after an incorrect performance is negative
feedback. This expectation almost always agreed with theactual feedback given by the user. After the robot had
acknowledged the user’s feedback, the next scene was shownon the screen, so that the user could continue teaching thenext command.
As the robot had direct access to the task server and could
request scenes for a certain command from the task server,
it was able to make the user give commands with a certain
meaning and also provoke positive or negative feedback
by correct or incorrect performance. Therefore it did notneed to apply speech recognition in order to understand theuser’s actual utterances and could run autonomously without
remote-control.
The robot was equipped with a voice activity detection, sothat it was able to react, when the user uttered a command or
feedback. Any speech utterance or touch, occurring after a
new scene had been shown, was assumed to be a command.
Every utterance or touch, that occurred after the robot had
executed a command, was expected to be feedback.
When executing any of the commands, the robot performed a specific gesture. The gestures were selected so that255

we could implement them in a similar way on the four-legged
pet-robot and the humanoid.
Sample virtual living room scenes for prompting the user
to give a command to the robot are shown in Fig. 1. In thefirst scene, the robot asks the user to name the “audioplayer”
object. In the second scene, the user is expected to tell therobot to switch the light on. In the third scene, the user
is expected to make the robot switch off the television. The
white texts in the images show our internal representations ofcommands and were not shown to the user during the trainingtask. Details on the implementation of the system and theuse of the training tasks for actually learning to understandcommands and feedback are given in [1].
IV. A SSUMPTIONS
Based on the schema theory [7] in psychology, whichsuggests that people use schemata of familiar objects andsituations to understand and handle unfamiliar situations, we
assumed that users are likely to interact with a pet-robot ina similar way as with a real dog, while interaction with ahumanoid was expected to resemble more to the interaction
with a human.
Moreover, we assumed that the participants were likely toconclude that the humanoid is more intelligent than the petrobot, based on its humanlike appearance. This might leadto higher expectations and to adaptations such as a moreelaborated speaking style, more politeness, more explanations etc. when interacting with the humanoid. Details on
our expectations as well as the actually observed interaction
are given in the results section.
V. E XPERIMENTAL S ETTING
We have conducted a user study with 16 participants
aged from 22 to 52. Ten participants (7 males, 3 females)interacted with the humanoid and six participants (4 males,
2 females) interacted with the pet-robot for roughly 45 minutes. The language, used in the experiments, was Japanese.
All participants were employees of the Honda ResearchInstitute Japan. 8 participants keep a pet or have kept a pet
and 5 participants have experience in keeping dogs.
Fig. 2 shows the experimental setting. The participants
were asked to sit at a table in order to avoid excessivechanges of position during the experiment. This was necessary because we also recorded video data for analysis andfor gesture recognition. The robot was placed to the right ofthe participant, close enough that all participants could easily
reach it with the hand to touch it. As the pet-robot was a lot
smaller than the humanoid, it was placed on the table, sothat the participants could reach it easily. The participants
were equipped with a headset microphone to record audio
data. Video data was recorded using a stereo camera which
was placed above the screen.
The participants were given explanations about the two
training phases. In the first phase, they were asked to namethe objects that the robot was pointing at. In the second
phase, they were instructed to give commands to the robot
and to give positive feedback if the robot reacted correctly

Fig. 1.

Sample Scenes from the “Virtual Living Room”

and negative feedback if the robot reacted incorrectly. They
were instructed to give commands and feedback in any way
they liked by speech, gesture and touch. The participants
had to teach each object name and each command ten times.
As the duration of an experiment was relatively long andthe users were required to talk a lot, there was five minutebreak between the training sessions for object names andcommands.

256

TABLE III
T YPES OF COMMANDS USED IN THE INTERACTION WITH THE
HUMANOID AND THE PET- ROBOTType
Plain commands
Polite commands
Questions in commands
Implicit commands
Parameters left out
Explanations in commands

Humanoid
75.01 (14.00)
9.86 (10.88)
10.23 (3.51)
3.40 (4.82)
6.78 (2.25)
1.81 (3.90)

Pet-Robot
60.83 (41.04)
26.23 (41.99)
8.34 (6.73)
4.10 (7.23)
4.13 (4.77)
0.95 (2.32)

All values in percent, value in brackets is the standard deviation

Fig. 2.

consider the robot less intelligent. However, we found, thatthe length of commands was almost the same for both robots.
An average command for the humanoid was 3.75 (sd=0.42)
words long, while an average command for the pet-robot
was 3.72 (sd=0.71) words long. The speaking speed was
also similar for the pet-robot with 0.45 (sd=0.09) seconds
per word, and the humanoid with 0.42 (sd=0.07) seconds
per word. This is in line with the participants’ subjective
evaluation of the robots’ intelligence, shown in Table II.

Experimental Setting.
TABLE IIU SERS ’ EVALUATION OF THE TRAINING TASKQuestion (5: fully agree - 1: do notagree)
I enjoyed teaching the robot
through the given task
The robot understood my feedback
The robot learned through my feedback
The robot adapted to my way ofteaching
I was able to instruct the robot ina natural way
The robot took too much time tolearn
The robot is intelligent
The robot behaves autonomously
The robot behaves cooperatively

HumanoidPet-robot

3.5 (0.8)

4 (0.8)3.6 (0.9)
3.2 (1.3)

4.3 (1.1)
4.3 (0.5)

3.2 (1.1)

3.8 (1.3)

3.6 (1.1)

3.5 (1.5)

3.6 (1.4)

2.7 (0.9)

2.7 (1.3)
2.7 (1.4)
3.7 (0.8)

2.8 (1.5)
2.8 (0.9)
3.3 (0.7)

C. Multimodality
During the interaction with both robots, we did not observe
pointing gestures from any of the users. A possible explanation is that all objects were very easy to distinguish verbally,so that pointing gestures would have been redundant. We
observed touch-based rewards for only one out of ten participants for the humanoid but for five out of the six participants
who interacted with the pet-robot. As touch is frequently
used with real dogs, we assume that users considered touch
to be appropriate for giving feedback to a pet-robot becauseof its dog-like appearance.

VI. R ESULTS
In our user study, we obtained two different kinds ofresults: We asked the participants to answer a questionnaire
about their subjective impression of the interaction and weannotated the data, which was recorded during the interaction to find objective similarities and differences in theparticipants’ behavior. We used the T-test to determine thestatistical significance of the observed differences.
A. Questionnaire results
From the results of the questionnaire, which are shown intable II we can see a slight tendency towards more positive
ratings for the interaction with the pet-robot. However, noneof the differences is statistically significant.
B. User behavior
We analyzed different aspects of the participants’ commands and feedback that we assumed to be related to theperceived intelligence and human-likeness of the robot. We
compared the speaking speed (in seconds per word) and thenumber of words per command/feedback, as we assumed
that people talk slower and in simpler sentences, when they

D. Verbal Commands
We analyzed how many commands had explanations orpolite expressions and how many commands were phrased
as a question. We estimated that users might be more polite,
explain more and use more questions when talking to ahumanoid robot, while they rather give plain commands
to a dog-like robot. We considered commands that containwords like “kudasai”, “kureru?”, “moraeru?” etc., which aresimilar to the English word “please” as polite commands. We
also analyzed, how many commands were implicit ones likesaying “it is too dark here” to make the robot switch the lighton, and in how many commands some expected parameters
were left out like in “put away the toy car” instead of “put the
toy car into the box”, because we assumed that this kind ofverbal behavior might be related to the perceived intelligenceof the robot.
The results can be found in Table III. The values do
not add up to 100% because not all types of commands
are mutually exclusive (e.g. a polite command can haveparameters left out).

257

TABLE IV
T YPES OF FEEDBACK USED IN THE INTERACTION WITH THE HUMANOIDAND THE PET- ROBOT
Type
Personal
Performance evaluation
Explanations

Humanoid
52.78 (17.99)
38.39 (18.28)
11.10 (14.29)

F. Behavior Changes over Time

Pet-robot
24.83 (27.41)
70.02 (28.16)
3.56 (3.90)

All values in percent, value in brackets is the standard deviation

Fig. 3.

Difference in Feedback for the Humanoid and the Pet-robot.

While we observed quite different utterances for different
users, the differences seemed to be rather caused by personal
preferences, than by the appearance of the robots. This
assumption is supported by the high standard deviations
between participants. None of the observed differences was
statistically significant.
E. Verbal Positive and Negative Feedback
We distinguished three different types of feedback: Personal rewards like “Thank you”, which emphasize, that the
robot has done something for the user, feedback which
directly comments on the performance of the robot, like
“Well done.” or “That was wrong.” and explanations used
as rewards like “That is not a toy car, it is a ball.” or“That is a toy car.”. The usage of different rewards forthe humanoid and the pet-robot is shown in table IV. We
found statistically significant differences for the usage of
personal rewards (df=14, t=2.48, p=0.026) and rewards,
which comment on the robots’ performance (df=14, t=2.75,
p=0.016). While the participants usually gave feedback like“well done (yoku dekimashita)” or “good (ii yo)” to the
pet-robot, they used more personal rewards like “Thank you(arigatou)” for the humanoid, especially for positive reward.
Fig. 3 shows the differences in user feedback given to thehumanoid and the pet-robot. While the participants gave
more explanations when talking to the humanoid, especiallyfor negative rewards, the difference between both robots was
not significant.

We also investigated the changes in user behavior overtime by comparing the commands and feedback, the participants gave in the first five minutes of the command learning
phase to the commands to the feedback given throughout
the whole experiment and to the last five minutes of theexperiment.
We did not find any significant changes in commands
given to both robots over time. We also did not observe significant changes in the feedback given to the pet-robot. However, we could observe two marginally significant changes
in the feedback given to the humanoid: The amount of
explanations for negative feedback was marginally significantly lower (p=0.071, t=2.06, df=9) at the beginning of theexperiments than it was throughout the whole experiment.
While at the beginning of the experiment only 26.98%(sd=32.32%) of the negative feedback for the humanoid
contained an explanation, it was an average of 34.57%(sd=35.87%) during the whole experiment and went up to
75.00% (sd=35.36%) at the end of the experiment.
We also observed a marginally significant increase(p=0.091, t=1.90, df=9) in personal feedback given to thehumanoid comparing the first five minutes of the commandlearning to the whole command learning phase. Overall,
the percentage of personal feedback increased from 34.85%(sd=22.62%) in the first five minutes to 61.92% (sd=24.60%)
in the last five minutes, while the average was 52.78%(sd=17.99%).
Similar trends toward more personal feedback and more
explanations for negative feedback were also found for the
pet-robot. However, the statistical significance of these trends
could not be confirmed. Fig. 4 compares the feedback given
during the whole task to the feedback given during the first
five and last five minutes of the task. For the interaction withthe pet-robot we did not observe any explanations within thelast five minutes of the training. This is because we looked
at explanations accompanying negative feedback when the
robot made mistakes. Due to the simulated learning, therobot made less mistakes towards the end of the training.
As the amount of explanations was generally lower for the
pet-robot, we could not observe any negative feedback with
explanations in the last five minutes of the experiments withthe pet-robot.
VII. D ISCUSSION AND C ONCLUSION
In our experiments, we observed less than expected differences in participants’ behavior toward the pet-robot and thehumanoid. While especially the way of uttering commands
seems to depend rather on the personal preferences of theuser, than on the appearance of the robot, we found robotdependent differences in the feedback, given by the participants. The most obvious one was the frequent use of touchfor giving feedback to the pet-robot, while touch was almost
not used for the humanoid. Moreover, we found, that users
tended to give personal feedback like ”Thank you” to thehumanoid, while they rather commented on the performancefor giving feedback to the pet-robot. These findings suggest

258

would be necessary to study more people including participants from outside Honda Research Institute to confirm thistrend scientifically. Moreover, further experiments would benecessary to confirm whether the trend that we found in our
experiments with one particular humanoid and one particular
pet-robot and a special training task can actually be generalized to other types of humanoids or pet-like robots andto more general tasks. As discussed in section II, previous
literature suggests, that depending on their appearance, userbehavior can vary for different types of humanoid robots andpresumably the same is true for different pet-robots.
In our future work, we are planning to further analyze
the variability and robot-dependence of given commands andfeedback. The results will be applied to improve our method
for learning to understand commands and feedback througha training task.
R EFERENCES

Fig. 4.

User Feedback Changes over Time.

that people actually use their experience with real dogs as a
guideline when giving feedback to the pet-robot.
When interacting longer with the humanoid, people started
to give more explanations when the robot performed incorrectly and also gave more personal reward. While the results
are only marginally significant and hard to interpret, one
explanation may be, that the perception of the humanoid
robot as an intelligent interaction partner increases when the
robot shows learning capabilities and improves its performance during the experiment. Similar tendencies could beobserved with the pet-robot. However, these tendencies were
not statistically significant.
The users’ subjective evaluation did not reveal significant
differences between the humanoid and the pet-robot. As both
robots were programmed to behave in the same way on
the same task, we assume that the users’ impression of therobots’ behavior on the given task depends rather on theiractual performance than on their appearance.
There are different possible explanations, why no significant differences were observed for giving commands. Oneof them is that both robots used speech to communicate
with the user. As speech is a typical human modality ofinteracting, differences might have been stronger, if the petrobot had communicated with the user in a more dog-like
non-verbal way. As there was no significant difference inusers’ evaluation of both robots’ intelligence, users may haveconsidered similar types of commands acceptable for both
robots.
While the initial T-test shows some interesting differences
between the feedbacks that were statistically significant it

259[1] A. Austermann, S. Yamada: “Learning to Understand ParameterizedCommands through a Human-Robot Training Task”, IEEE International Symposium on Robot and Human Interactive Communication(ROMAN’09), (2009), 757-762[2] K. F. Mac Dorman: “Androids as an experimental apparatus: Whyis there an uncanny valley and can we exploit it?” CogSci-2005
Workshop: Toward Social Mechanisms of Android Science, (2005),
106-118[3] F. Hegel, M. Lohse, B. Wrede: “Effects of Visual Appearance on the
Attribution of Applications in Social Robotics”, IEEE International
Symposium on Robot and Human Interactive Communication (ROMAN’09), (2009), 64-71[4] Takayuki Kanda, Takahiro Miyashita, Taku Osada, Yuji Haikawa, andHiroshi Ishiguro: “Analysis of Humanoid Appearances in HumanRobot Interaction”, IEEE TRANSACTIONS ON ROBOTICS, VOL.
24, NO. 3, (2008), 899 - 906[5] J. Goetz, S. Kiesler, and A. Powers, Matching robot appearance andbehavior to tasks to improve human-robot cooperation, IEEE Workshop on Robot and Human Interactive Communication (ROMAN’03),(2003), 55 - 60
[6] T. Komatsu, S. Yamada: “Effect of Agent Appearance on People’s Interpretation of Agent’s Attitude”, CHI-2008 Work-in-Progress, (2008),
2919 -2924[7] W. F. Brewer: “Bartlett’s Concept of the Schema and Its Impact on
Theories of Knowledge Representation in Contemporary Cognitive
Psychology.” Bartlett, Culture and Cognition, ed. Akiko Saito. Hove,Eng.: Psychology Press, (2000), 69-89[8] Sarah Kriz, Gregory Anderson, J. Gregory Trafton: “Robot-Directed
Speech: Using Language to Assess First-Time Users’ Conceptualizations of a Robot”, 5th ACM/IEEE International Conference on HumanRobot Interaction (HRI 2010), (2010), 267-275