CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

How do Robotic Agents’ Appearances
Affect People’s Interpretations of theAgents’ Attitudes?
Takanori Komatsu

Abstract

Future University-Hakodate.

An experimental investigation of how the appearance ofrobotic agents affects interpretations people make ofthe agents’ attitudes is described. We conducted a
psychological experiment where participants werepresented artificial sounds that can make people
estimate specific agents’ primitive attitudes from threekinds of agents, e.g., a Mindstorms robot, AIBO robot,and normal laptop PC. They were also asked to selectthe correct attitudes based on the sounds expressed bythese three agents. The results showed that theparticipants had higher interpretation rates when a PC
presented the sounds, while they had lower rates when
Mindstorms and AIBO robots presented the sounds,
even though the artificial sounds expressed by theseagents were completely the same.

116-2 Kamedanakano.
Hakodate, 041-8655 JAPANkomatsu@fun.ac.jp
Seiji Yamada
National Institute of Informatics.
2-1-2 Hitotsubashi, Chiyoda.
Tokyo, 101-8430, JAPANseiji@nii.ac.jp

Keywords
Appearance of agents, agents’ attitudes, subtle
expressions, human-agent interaction

ACM Classification Keywords
Copyright is held by the author/owner(s).

J.4 [Computer Applications]: Social and Behavioral
Sciences–Psychology.

CHI 2007, April 28 – May 3, 2007, San Jose, California, USA
ACM 978-1-59593-642-4/07/0004.

2519

CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

Introduction
Recently, one of the hottest topics in human-computerinteraction or human-agent interaction studies is “whatkind of appearance should robotic agents have in order
to interact with people effectively.” People are said todetermine agents’ behavior model based on theappearance of agents. Therefore, the agents’appearance significantly affects the interaction withthese people [1]. For example, when people encountera dog-like robot, they expect dog-like behaviors fromthis robot, and they would naturally speak to it usingcommands and other utterances intended for dogs,
such as “sit”, “lie down”, and “fetch”. However, they do
not act this way toward a cat-like robot. Actually,
several studies have focused on the effects ofappearance of agents on interactions with people[2,3,5]. On the basis of the findings of these studies,
we are currently focusing on the issue of “What kinds ofappearance should robotic agents have in order tointeract with people effectively? And which kind of
information should these agents express to people?” As
part of the first step to finding a comprehensive
solution to this issue, we investigated how peopleinterpret presented sounds to see if they could
determine specific attitudes from agents that havedifferent appearance.

Agents’ attitudes, appearances, andexpressed information
In this study, we experimentally investigated theeffects of the basic psychological relationship between
the appearance of agents and the information
expressed on how people interpret the agents’ attitudes.
We selected positive and negative attitudes
corresponding to valence values as the primitive

attitudes that robotic agents should express. Informing
people of these two values is quite important if theagents are to interact effectively with people. We
selected three artifacts for agents having different
appearance: a Mindstorms robot [7], AIBO robot [8],and a normal laptop PC (Let’s note W2, product ofPanasonic Inc.).
I want to talkwith you, sir.

Beep!
Beep!

Simple appearance
Human-like appearanceWith complex expressions But with intuitive expressions= much more effective= difficult to understand…
figure 1. Conceptual Diagram of our hypothesis.

Concerning the effects of the relationship between theappearance of the agents and their expressed
information and how they affect users’ interpretationsof the agents’ attitudes, we hypothesized that agentswith a lifelike appearance (quite similar to people or petanimals) expressing true-to-life information (verbal
information or animal-like behaviors) are actually more
confusing to users and are not really effective forinteracting with people: Instead, agents without such alifelike appearance expressing simple but intuitive
information (e.g., subtle expressions [6]) are readily
understood and are much more effective for interaction(see Figure 1) [9]. If our hypothesis holds true, wewould be able to better facilitate people’scomprehension of the agents’ primitive attitudes
without having to develop dexterous and complex2520

CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

robotic or computer graphic systems, which would bequite expensive.

Preliminary Experiment
Before the actual experiment, we conducted apreliminary experiment to determine what kinds ofsubtle expressions are effective to evoke certain
attitudes, positive or negative, from people. In thisstudy, we focused on artificial sounds that acted as
subtle expressions in previous studies [4]. We theninvestigated what kinds of sounds were interpreted as
being positive or negative attitudes.
We prepared 44 different types of triangle wave soundswith four different durations and 11 different F0contours. Specifically, the four durations were 189, 418,639, and 819 ms. The 11 F0 contours were set so thatthe transition range of F0 values between the onset andendpoint in the sound stimuli were 0, ±25, ±50, ±75,±100, or ±125, and these were linearly downward orupward. All these 44 stimuli have the same F0 averageof 131 Hz. And these sounds have a tone that sounds
like a computer’s beeping. In this paper, sound stimulilabeled as “189u25” indicates that duration was 189 ms,
and F0 transition range was 25 Hz with upward slope(increasing intonation).
Participants and Procedure
Ten Japanese university students (6 men and 4women; 19 – 23 years old) participated. First, an
experimenter gave the instruction “please determine
the attitude of this laptop PC based on the sounds itmakes.” They were then asked to select one of thethree attitudes “positive,” “negative,” or“undistinguishable” after presenting one of 44 preparedsounds. These three attitudes were described toparticipants as follows: 1) Positive: the PC’s internal
state appears to be good, 2) Negative: the PC’s internal
state appears to be bad, and 3) Undistinguishable: it isunclear whether the PC’s internal state is positive ornegative.
As part of the procedure, one randomly selected soundamong 44 prepared sounds was presented to theparticipants. Afterwards, the participants were asked toselect one of the three aforementioned attitudes. Eachparticipant heard all 44 prepared sounds. The order ofthe sounds was counterbalanced for all 10 participants.
Actually, these sounds were presented by a normal
laptop PC.
Results
The results of this preliminary experiment show that all
10 participants believed the PC had positive attitudes
for four sounds: 189u125, 418u125, 639u125, and868u125. Also, all participants believed the PC had
negative attitudes for four sounds: 418d25, 418d50,418d75, and 639d50. These eight sounds were thenselected for agents with different appearance in thenext experiment.

Experiment
The purpose of this experiment was to investigate theeffects of the agents’ appearance on how participantsinterpreted the agents’ attitudes. Specifically, theparticipants were presented the selected sounds usedin the preliminary experiment by agents that havedifferent appearances, the Mindstorms robot, AIBO
robot, and the normal laptop PC (see Figure 2). They
were then asked to select the correct attitudes (positive,negative, or undistinguishable) based on the expressedsounds made by these three agents.

2521

CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

Participants and Procedure
Nine Japanese university students (8 men and 1woman; 21 – 24 years old) participated. Theseparticipants were not familiar with robots or these toys
and had not participated in the preliminary experiment.

AIBO’s operating software “AIBO entertainmentplayer” that was installed in an AIBO operating PC.
3.

Eight of AIBO’s prepared behaviors (AIBO-motioncondition): The AIBO entertainment player has
about 80 prepared behaviors, such as “goodmorning” or “delightful.” We selected the following
eight behaviors for expressing primitive attitudes
to the participants: Positive (four behaviors),“cheer1,” “cheer3,” “cheer4,” and “cheer5.”
Negative (four behaviors), “angry1,” “angry2,”“sad1,” and “sad2.” These behaviors were selectedbased on the verbal labels of these behaviors.

4.

Eight sounds expressed by the laptop PC (PCsound condition): This laptop PC was remotely
operated by a sound expressing PC.
Wireless LAN（Remote Access via Real VNC）

figure 2. AIBO robot, Mindstorms robot and the laptop PC
FM radio wave

(from left to right).

First, the participants were told that the concrete taskof this experiment was to select one of the three
attitudes (positive, negative, or undistinguishable)
based on the sounds these agents made. All
participants experienced the following four conditions.
1.

2.

Eight sounds expressed by Mindstorms (MSsound condition): the eight sounds came froman FM radio tuner placed on the Mindstorms. This
radio tuner received the transmitted sounds froma sound expressing PC (Figure 3).
Eight sounds expressed by AIBO (AIBO-soundcondition): the sounds were presented using)))

FM radio tuner

Wireless LAN （motion command）

FM
Transmitter
AIBO

Sound
AIBO
Question
expressing operating expressing
ＰＣＰＣＰＣ

Experimenter

Mindstorms

Question
displaying
monitor

PCPlace of agent

Participant

figure 3. Experimental Settings.

2522

CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

Results
We calculated the interpretation rates, which indicatedhow many times the participants succeeded in correctly
determining the agents’ attitudes in all four
experimental conditions. The results were that theparticipants has interpretation rates of 3.33 for eight
experimental stimuli in the MS-sound condition, 2.89 in
the AIBO-sound condition, 3.33 in the AIBO-motion
condition and 6.44 in the PC-sound condition (Figure 4).
The results of an ANOVA showed significant differencesin these four experimental conditions (F(3,24)=8.26,p<.01(**)), and a multiple comparison using an LSD
test showed significant differences between the PCsound condition and the other three conditions(Mse=2.9421, 5% level).
These results showed that the participants’interpretation rates for the same sound stimuli differedbased on each agent’s appearance. Simply stated, theparticipants showed higher interpretation rates in thePC-sound condition, while they showed significantly
lower rates in the MS-sounds and AIBO-soundsconditions, even though the same sounds werepresented to participants in these three conditions.

Moreover, the interpretation rates observed in theAIBO-motion condition were also significantly lower
than the PC-sound condition, and these interpretation
rates were nearly the same as those of the MS-soundand AIBO-sound conditions, even though these motions
were prepared for a commercial product.
7(* * )

6

Interpretation Rates（Times）

First, participants experienced MS-sounds, AIBO–
sounds, and AIBO-motion conditions in a random order.
Then, they did the PC-sound condition. In all four
conditions, the eight sounds or behaviors wererandomly presented to the participants. The
participants were asked to select the correct attitudes
from the three attitudes (positive, negative, orundistinguishable) after the agent expressed certain
information. The order of the experimental conditions
and the order of eight sounds or behaviors werecounterbalanced for the participants.

543

Mindstorms- sound
AIBO- sound
AIBO- motion
PC- sound2
1
0

figure 4. The participants’ interpretation rates in four
experimental conditions.

Discussion and Conclusions
Eight artificial sounds selected in a preliminary
experiment were presented to participants by aMindstorms, AIBO, and PC. The participants’interpretation rates, indicating how many times theparticipants selected the correct attitudes, were theninvestigated. The results were that the interpretation
rates when a PC expressed these sounds weresignificantly higher than the rates when a Mindstorms
and AIBO expressed them. This difference revealed thatthe agents’ different appearances affected people’sinterpretations of the agents’ attitudes, even though2523

CHI 2007
2006 •· Work-in-Progress

April 28-May 3, 2007 • San Jose, CA, USA

these agents expressed information that wascompletely the same.
Let us consider why the PC-sound condition showed
higher rates compared with the other conditions. One
reason is that these eight sounds were selected in the
preliminary experiment when these sounds werepresented by the laptop PC, which was also used in thePC-sound condition. Thus, these sounds may have beeneffective only for informing people of primitive attitudes
when they were presented by the laptop PC. Thisphenomenon may be rooted in the fact that the PC
expressing beep-like sounds was very familiar for allparticipants.
The results of this experiment revealed an interesting
phenomenon where the interpretation rates in theAIBO-motion condition were lower than those in thePC-sound condition, just as they were for the MS-soundand AIBO-sound conditions. This indicates that theAIBO’s prepared behaviors were not really efficient in
informing the participants of its primitive attitudes,positive or negative. This result would support ourhypothesis described in the Introduction: “agents
without a lifelike appearance expressing simple but
intuitive information are readily understood and aremuch more effective for interaction.” Of course, thebehaviors of AIBO were not designed for informing
people of the primitive attitudes we estimated.
However, our results suggest a design policy is needed
to inform people of certain attitudes effectively.
We intend to pursue a series of follow-up studies basedon our results in this study and subsequently establish
a design policy for the most appropriate information
based on the agents’ appearance. For example, what

information is appropriate for the Mindstorms robot toinform people of its primitive attitudes? Are Starwars’R2D2 like behaviors appropriate? We expect that thesefollow-up studies will contribute to establishing a design
policy that can clarify an effective coupling between theappearance of a robotic agent and the information it
expresses so that interactive agents can be createdreadily and easily.

References[1] S. Yamada and T. Yamaguchi: Training AIBO like aDog, the 13th International Workshop on Robot andHuman Interactive Communication, pp.431-436(2004).
[2] S. Kiesler., L. Sproull., & K. Waters. A Prisoner’s
Dilemma Experiment on Cooperation with People andHuman-like Computers. Journal of Personality andSocial Psychology, 79(1), 47-65 (1995).
[3] S. Kiesler., & L. Sproull. Social Responses to“Social” Computers. In B. Friedman, Human values andthe design of technology, CLSI, Publications (1997).
[4] T. Komatsu. Audio Subtle Expressions Affecting
User’s Perceptions, the 9th International Conference onIntelligent User Interface, p.306-308 (2006).
[5] J. Goetz., S. Kiesler., & A. Powers: Matching Robot
Appearance and Behavior to Tasks to Improve HumanRobot Cooperation, 12th IEEE Workshop on Robot andHuman Interactive Communication (2003).
[6] K. Liu., & W, R, Picard. Subtle expressivity in a
Robotic Computer, CHI2003 Workshop on Subtle
Expressivity for Characters and Robots (2003).
[7]

MindStorms: http://mindstorms.lego.com/[8]

AIBO: http://www.jp.aibo.com.

[9] S. Yamada, T. Komatsu. Designing Simple andEffective Expression of Robot’s Primitive Minds to a
Human, 2006 IEEE/RSJ International Conference onIntelligent Robots and Systems, pp.2614-2619 (2006)

2524