How Appearance of Robotic Agents Affects How People
Interpret the Agents’ Attitudes
Takanori Komatsu

Seiji Yamada

Future University-Hakodate
116-2 Kamedanakano
Hakodate 041-8655, Japan+81-138-34-6220

National Institute of Informatics
2-1-2 Hitotsubashi
Tokyo 101-8430, Japan+81-3-4212-2562

komatsu@fun.ac.jp

seiji@nii.ac.jp

intended for dogs, such as “sit”, “lie down”, and “fetch”. However,they do not act this way toward a cat-like robot.

ABSTRACTAn experimental investigation of how the appearance of robotic
agents affects interpretations people make of the agents’ attitudes
is described. We conducted a psychological experiment whereparticipants were presented artificial sounds that can make people
estimate specific agents’ primitive attitudes from three kinds ofagents, e.g., a Mindstorms robot, AIBO robot, and normal laptop
PC. They were also asked to select the correct attitudes based on
the sounds expressed by these three agents. The results showedthat the participants had higher interpretation rates when a PC
presented the sounds, while they had lower rates when
Mindstorms and AIBO robots presented the sounds, even though
the artificial sounds expressed by these agents were completely
the same.

Several studies have focused on the effects of appearance ofagents on interactions with people [2,3,5]. Keisler [2] conducted a
psychological experiment where participants were asked to play aprisoner’s dilemma game with virtual characters (human and dog)that appeared on a computer display. The results showed thatparticipants who had some experience with owning dogs
interacted significantly more effectively with the dog-like virtual
agent, e.g., cooperating with this agent significantly. Goetz et al.
[5] investigated the effects on people’s impressions of different
designs of the head for humanoid robots. The results showed thatthe participants answered that the robots with human-like heads
are good at social tasks, while robots with machine-like heads aregood at industrial tasks.

Categories and Subject Descriptors

On the basis of the findings of these studies, we are currently
focusing on the issue of “What kinds of appearance should robotic
agents have in order to interact with people effectively? And
which kind of information should these agents express topeople?” As part of the first step to finding a comprehensive
solution to this issue, we investigated how people interpretpresented sounds to see if they could determine specific attitudes
from agents that have different appearance.

J.4 [Computer Applications]: Social and Behavioral Sciences –Psychology.

General Terms
Experimentation and Human Factors

Keywords
Appearance of agents, agents’ attitudes, subtle expressions,human-agent interaction

2. AGENTS’ ATTITUDES, APPEARANCE,AND EXPRESSED INFORMATION

1. INTRODUCTION

Several approaches can be used to tackle with this issue. In
this study, we experimentally investigated the effects of the basic
psychological relationship between the appearance of agents andthe information expressed on how people interpret the agents’attitudes.

Recently, one of the hottest topics in human-computerinteraction or human-agent interaction studies is “what kind of
appearance should robotic agents have in order to interact withpeople effectively.” People are said to determine agents’ behavior
model based on the appearance of agents. Therefore, the agents’appearance significantly affects the interaction with these people[1]. For example, when people encounter a dog-like robot, they
expect dog-like behaviors from this robot, and they would
naturally speak to it using commands and other utterances

We selected positive and negative attitudes corresponding tovalence values as the primitive attitudes that robotic agents should
express. Informing people of these two values is quite important if
the agents are to interact effectively with people. We selected
three artifacts for agents having different appearance: a
Mindstorms robot [7], AIBO robot [8], and a normal laptop PC(Let’s note W2, product of Panasonic Inc.). These artifacts
correspond to making mechanical impressions of people, familial
impressions, and non-agent-like impressions, respectively (Figure
2).

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.
ACE’07, June 13–15, 2007, Salzburg, Austria.
Copyright 2007 ACM 978-1-59593-640-0/07/0006…$5.00.

Concerning the effects of the relationship between theappearance of the agents and their expressed information and how

123they affect users’ interpretations of the agents’ attitudes, wehypothesized that agents with a lifelike appearance (quite similar
to people or pet animals) expressing true-to-life information(verbal information or animal-like behaviors) are actually more
confusing to users and are not really effective for interacting withpeople: Instead, agents without such a lifelike appearance
expressing simple but intuitive information (e.g., subtle
expressions [6]) are readily understood and are much more
effective for interaction (see Figure 1) [9]. If our hypothesis holds
true, we would be able to better facilitate people’s comprehensionof the agents’ primitive attitudes without having to develop
dexterous and complex robotic or computer graphic systems,
which would be quite expensive. Our hypothesis may eventuallycontribute to creating robotic agents more easily and toestablishing deeper and more natural interactions with people.
I want to talkwith you, sir.

Positive: the PC’s internal state appears to be good.
Negative: the PC’s internal state appears to be bad.
Undistinguishable: it is unclear whether the PC’sinternal state is positive or negative.
As part of the procedure, one randomly selected soundamong 44 prepared sounds was presented to the participants.
Afterwards, the participants were asked to select one of the three
aforementioned attitudes. Each participant heard all 44 preparedsounds. The order of the sounds was counterbalanced for all 10
participants. Actually, these sounds were presented by a normal
laptop PC (Let’s note W2, product of Panasonic Inc.).

Beep!
Beep!

Simple appearance
Human-like appearanceWith complex expressions But with intuitive expressions= much more effective= difficult to understand…

Figure 1. Comparison of human-like and simple robots andtheir impact on intuitive interactions.

3. PRELIMINARY EXPERIMENT

Figure 2. 11 different F0 contours (duration: 189 ms). For
example, “189u25” indicates that duration was 189 ms, and F0
transition range was 25 Hz with upward slope (increasing
intonation).

Before the actual experiment, we conducted a preliminary
experiment to determine what kinds of subtle expressions are
effective to evoke certain attitudes, positive or negative, frompeople. In this study, we focused on artificial sounds that acted as
subtle expressions in previous studies [4]. We then investigatedwhat kinds of sounds were interpreted as being positive or
negative attitudes.

3.3 Results
The results of this preliminary experiment show that all 10
participants believed the PC had positive attitudes for four sounds,
that is, 189 ms with an upward slope range of 125 Hz (189u125),418u125, 639u125, and 868u125. Also, all 10 participants
believed the PC had negative attitudes for four sounds, that is, 418
ms with a downward slope range of 25 Hz (418d25), 418d50,
418d75, and 639d50. Thus, sounds with increasing intonation
regardless of the duration were interpreted as being positive
attitudes, while sounds that have around 500 ms duration withslightly decreasing intonation were interpreted as being negative
attitudes. These eight sounds were then selected for agents withdifferent appearance in the next experiment.

We prepared 44 different types of triangle wave sounds withfour different durations and 11 different F0 (fundamental
frequency) contours. Specifically, the four durations were 189,
418, 639, and 819 ms. The 11 F0 contours were set so that thetransition range of F0 values between the onset and endpoint in
the sound stimuli were 0, ±25, ±50, ±75, ±100, or ±125, andthese were linearly downward or upward (Figure 2). All these 44
stimuli have the same F0 average of 131 Hz. And these sounds
have a tone that sounds like a computer’s beeping.

3.1 Participants

4. EXPERIMENTTen Japanese university students (6 men and 4 women; 19 –23 years old) participated.

The purpose of this experiment was to investigate theeffects of the agents’ appearance on how participants interpreted
the agents’ attitudes. Specifically, the participants were presented
the selected sounds used in the preliminary experiment by agentsthat have different appearances, the Mindstorms robot, AIBO
robot, and the normal laptop PC (see Figure 3). They were thenasked to select the correct attitudes (positive, negative, or
undistinguishable) based on the expressed sounds made by thesethree agents.

3.2 Procedure
First, an experimenter gave the instruction “please determine
the attitude of this laptop PC based on the sounds it makes.” They
were then asked to select one of the three attitudes “positive,”“negative,” or “undistinguishable” after presenting one of 44
prepared sounds. These three attitudes were described toparticipants as follows:124agent expressed certain information. Therefore, this procedure
was nearly the same as the one in the preliminary experiment. The
order of the experimental conditions and the order of eight sounds
or behaviors were counterbalanced for the participants.

4.1 Participants
Nine Japanese university students (8 men and 1 woman; 21– 24 years old) participated. These participants were not familiar
with robots or these toys and had not participated in the
preliminary experiment.

4.3 Results
We calculated the interpretation rates, which indicated how
many times the participants succeeded in correctly determining
the agents’ attitudes in all four experimental conditions. The
results were that the participants has interpretation rates of 3.33
for eight experimental stimuli in the MS-sound condition, 2.89 in
the AIBO-sound condition, 3.33 in the AIBO-motion condition
and 6.44 in the PC-sound condition (Figure 5). The results of an
ANOVA showed significant differences in these four
experimental conditions (F(3,24)=8.26, p<.01(**)), and a multiple
comparison using an LSD test showed significant differences
between the PC-sound condition and the other three conditions(Mse=2.9421, 5% level).

4.2 Procedure
First, the participants were told that the concrete task of thisexperiment was to select one of the three attitudes (positive,
negative, or undistinguishable) based on the sounds these agents
made.

Wireless LAN（Remote Access via Real VNC）

FM radio wave)))

FM radio tuner

Wireless LAN （motion command）

FM
Transmitter
AIBO

Mindstorms

PC

Figure 3. AIBO robot, Mindstorms robot, and the laptop PC(from left to right).
All participants experienced the following four conditions.

2.

3.

4.

Eight sounds expressed by Mindstorms (MS-soundcondition): the eight sounds came from an FM radio tuner
placed on the Mindstorms. This radio tuner received thetransmitted sounds from a sound expressing PC (Figure 4).

Place of agent

ParticipantExperimenter

Figure 4. Experimental Setting.

Eight sounds expressed by AIBO (AIBO-sound condition):
the sounds were presented using AIBO’s operating software“AIBO entertainment player” that was installed in an AIBO
operating PC.

7(* * )

6
Interpretation R ates（Tim es）

1.

Question
displaying
monitorSound
AIBO
Question
expressing operating expressing
ＰＣ
ＰＣ
ＰＣ

Eight of AIBO’s prepared behaviors (AIBO-motion
condition): The AIBO entertainment player has about 80
prepared behaviors, such as “good morning” or “delightful.”
We selected the following eight behaviors for expressing
primitive attitudes to the participants. The positive
behaviors were (four behaviors) “cheer1,” “cheer3,”“cheer4,” and “cheer5.” The negative behaviors were (fourbehaviors) “angry1,” “angry2,” “sad1,” and “sad2.” Thesebehaviors were selected based on the verbal labels of thesebehaviors.

5
4
3

M indstorm s-sound
A IB O -sound
A IB O -m otion
P C -sound2
1
0

Eight sounds expressed by the laptop PC (PC-soundcondition): The same conditions as those in the preliminary
experiment were used. This laptop PC was remotely
operated by a sound expressing PC.

Figure 5. Participants’ interpretation rates in four
experimental conditions.
These results showed that the participants’ interpretation
rates for the same sound stimuli differed based on each agent’s
appearance. Simply stated, the participants showed higher
interpretation rates in the PC-sound condition, while they showedsignificantly lower rates in the MS-sounds and AIBO-sounds
conditions, even though the same sounds were presented toparticipants in these three conditions. Moreover, the interpretation

First, participants experienced MS-sounds, AIBO–sounds,
and AIBO-motion conditions in a random order. Then, they heard
the PC-sound condition. In all four conditions, the eight sounds or
behaviors were randomly presented to the participants. The
participants were asked to select the correct attitudes from thethree attitudes (positive, negative, or undistinguishable) after the125rates observed in the AIBO-motion condition were alsosignificantly lower than the PC-sound condition, and theseinterpretation rates were nearly the same as those of the MSsound and AIBO-sound conditions, even though these motions
were prepared for a commercial product.

subtle expressions [6]) are readily understood and are much more
effective for interaction (see Figure 1) [9].” Of course, thebehaviors of AIBO were not designed for informing people of theprimitive attitudes we estimated. However, our results suggest adesign policy is needed to inform people of certain attitudes
effectively.

5. DISCUSSION AND CONCLUSIONS

We intend to pursue a series of follow-up studies based on
our results in this study and subsequently establish a design policy
for the most appropriate information based on the agents’appearance. For example, what information is appropriate for the
Mindstorms robot to inform people of its primitive attitudes? Are
Starwars’ R2D2 like behaviors appropriate? We expect that thesefollow-up studies will contribute to establishing a design policythat can clarify an effective coupling between the appearance of arobotic agent and the information it expresses so that interactive
agents can be created readily and easily.

Eight artificial sounds selected in a preliminary experimentwere presented to participants by a Mindstorms, AIBO, and PC.
The participants’ interpretation rates, indicating how many times
the participants selected the correct attitudes, were theninvestigated. The results were that the interpretation rates when a
PC expressed these sounds were significantly higher than the rates
when a Mindstorms and AIBO expressed them. This difference
revealed that the agents’ different appearances affected people’sinterpretations of the agents’ attitudes, even though these agents
expressed information that was completely the same.

6. REFERENCES

Let us consider why the PC-sound condition showed higher
rates compared with the other conditions. One reason is that theseeight sounds were selected in the preliminary experiment whenthese sounds were presented by the laptop PC, which was also
used in the PC-sound condition. Thus, these sounds may havebeen effective only for informing people of primitive attitudes
when they were presented by the laptop PC. This phenomenon
may be rooted in the fact that the PC expressing beep-like sounds
was very familiar for all participants.

[1] S. Yamada and T. Yamaguchi: Training AIBO like a Dog,the 13th International Workshop on Robot and Human
Interactive Communication, pp. 431-436 (2004).
[2] S. Kiesler, L. Sproull, & K. Waters. A Prisoner’s Dilemma
Experiment on Cooperation with People and Human-likeComputers. Journal of Personality and Social Psychology,
79(1), 47-65 (1995).
[3] S. Kiesler & L. Sproull. Social Responses to “Social”
Computers. In B. Friedman, Human values and the design oftechnology, CLSI Publications (1997).

In any case, these results indicate that robotic agents’
different appearances may have affected the users’ interpretationof the agents’ attitudes. Was this actually why the interpretation
rates in the PC-sound condition were higher than those in theother conditions? To answer this question, we need to conduct
other experiments to consider the relationship between
appearance and expressed information.

[4] T. Komatsu. Audio Subtle Expressions Affecting User’s
Perceptions, the 9th International Conference on Intelligent
User Interfaces, p. 306-308 (2006).
[5] J. Goetz, S. Kiesler, & A. Powers: Matching Robot
Appearance and Behavior to Tasks to Improve Human-Robot
Cooperation, 12th IEEE Workshop on Robot and Human
Interactive Communication (2003).

The results of this experiment revealed an interesting
phenomenon where the interpretation rates in the AIBO-motion
condition were lower than those in the PC-sound condition, just as
they were for the MS-sound and AIBO-sound conditions. This
indicates that the AIBO’s prepared behaviors were not really
efficient in informing the participants of its primitive attitudes,
positive or negative. This result would support our hypothesis
described in the Introduction: “agents with a lifelike appearance(quite similar to people or pet animals) expressing true-to-life
information (verbal information or animal-like behaviors) are
actually more confusing to users and are not really effective forinteracting with people: Instead, agents without such a lifelike
appearance expressing simple but intuitive information (e.g.,[6] K. Liu & W. R. Picard. Subtle expressivity in a Robotic
Computer, CHI2003 Workshop on Subtle Expressivity for
Characters and Robots (2003).
[7] MindStorms: http://mindstorms.lego.com/[8] AIBO: http://www.jp.aibo.com[9] S. Yamada & T. Komatsu. Designing Simple and Effective
Expressions of Robot’s Primitive Minds to a Human, 2006
IEEE/RSJ International Conference on Intelligent Robots
and Systems, pp. 2614-2619 (2006).

126