16th IEEE International Conference on Robot & Human Interactive Communication

MB4-3

August 26 ~ 29, 2007 / Jeju, Korea

Effects of robotic agents’ appearances on users’ interpretations of theagents’ attitudes: towards an expansion of “uncanny valley” assumption
Takanori Komatsu1, Member, IEEE, and Seiji Yamada2, Member, IEEE
1

Department of Media Architecture, Future University-Hakodate, Hokkaido, Japan, e-mail : komatsu@fun.ac.jp
2
National Institute of Informatics, Tokyo, Japan, e-mail:seiji@nii.ac.jp

Abstract— This paper describes experimental investigation
how agents’ appearances affect users’ interpretations ofagents’ attitudes. Specifically, we conducted a psychologicalexperiment that participants were presented artificial sounds
as subtle expressions that can make human estimate specificagents’ primitive attitudes from three kinds of different agents,
e.g., Mindstorms robot, AIBO robot, and normal laptop PC,
and they were asked to select the correct attitudes based on theexpressed sounds from these three agents. As the result, theparticipants showed the higher interpretation rates when thesounds were presented from PC, while they did the lower rates
from Mindstorms and AIBO robots, even though the artificial
sounds expressed from these agents were the completely same
sounds. Finally, the result was compared to the Mori’s assumption that is about the relationship between an agents’appearance (likeness) and users’ familiarity to the agents.

I. INTRODUCTION
Recently, one of the hottest topics in Human-Computer
Interaction or Human-Agent Interaction studies is “whichkinds of appearances should the agents have in order to interact with users effectively.” Because, it can be generally
said that users establish the agents’ behavior model in termsof their appearances so that it is expected that the agents’appearances significantly affect the interaction with theseusers [1]. For example, when users face the dog-like robot,
they expect the dog-like behaviors from this robot and theywould speak to this, like “sit, sit up, lie down” and so on. On
the other hand, they did not do this way toward the cat-likerobot.
Actually there are several studies that focus on the effectsof the agents’ appearances on the interactions with users[2,3,5]. Keisler [2] conducted a psychological experimentthat the participants were asked to play the prisoner’s dilemma game with virtual characters (human and dog) appearing on the computer display. The results showed that theparticipants who had some experiences in keeping dogs
cooperated with the dog-like virtual agent significantly.
Goetz et al [5] investigated the effects of the different designof head of humanoid robots on users’ impressions. The results showed that the participants answered that the human-like head robots are good at achieving social tasks,
while they did that the machine-like head ones are good atachieving industrial tasks.
Based on the findings of the above former studies, we are
currently focusing on the following issue; “which kinds ofappearance’s agents should express which kinds of information to users?” As the first step to realize this issue com-

978-1-4244-1635-6/07/$25.00

2007 IEEE.

pletely, we investigated how human users interpret the presented sounds that can make users estimate specific attitudes
from the agents having different appearances. We would
then discuss about the relationship between our results andMori’s assumption that is about the one between the agents’appearance (likeness) and the familiarities human users
would feel.
II. AGENTS’ ATTITUDES, APPEARANCE AND EXPRESSEDINFORMATION
There would be several approaches to tackle with theabove issue. In this study, we investigated the effects of thebasic relationship between the agents’ appearances and theexpressed information on users’ interpretation of agents’attitudes by means of a psychological experiment.
As the primitive attitudes that agent should express tousers, we selected the positive and negative attitudes corresponded to the valence values. It is assumed that informing
these two values to users are quite important for the agentsrequired in interacting and cooperating with users. And as
the agents having different appearances, we selected thefollowing three artifacts; Mindstorms robot [7], AIBO robot[8], and a normal laptop PC (Let’s note W2, product ofPanasonic Inc,). These artifacts were corresponded to making users mechanical impressions, familial impressions, andno-agent-like impressions, respectively.
I want to talkwith you, sir.

Beep!
Beep!

Simple appearance
Human-like appearanceWith complex expressions But with intuitive expressions= much more effective= difficult to understand…
Fig.1 Conceptual diagram of our hypothesis

Concerning the effects of the relationship between
agents’ appearances and expressed information on users’interpretation of the agents’ attitudes, we hypothesized thatthe agents with rich appearance (quite similar with actual
humans or pet animals) expressing the likely information(verbal information or animal like behaviors) are more
confusing to users and are not really effective for interactingwith users: Instead, the agents without rich appearance
expressing the simple but intuitive information (e.g., subtle
expressions [6]) are readily understood and are much more

380

effective for their interaction (see Figure 1) [9]. If our hypothesis is verified, it would become possible to make users
understand the agents’ primitive attitudes without developing the dexterous and complex robotic or computer graphic
systems consuming a bunch of costs. And then, eventually,
our hypothesis would contribute to easily realizing theagents that can establish the intimate and natural interaction
with users.
III. PRELIMINARY EXPERIMENT
Before the actual experiment, we conducted the preliminary experiment to confirm which kinds of subtle expressions are effective to evoke certain attitudes, positive or
negative, to humans. In this study, we focused on the artificial sounds as subtle expressions that were utilized in theformer studies [4]. We then investigated which kinds ofsounds were interpreted as positive or negative attitudes.

attitudes “positive,” “negative,” or “undistinguishable” after
presenting the one of the 44 prepared sounds. Specifically,
these three attitudes were described to participants as follows:z

Positive: felt that the PC’s internal state is good.

z

Negative: felt that the PC’s internal state is bad.

z

Undistinguishable: could not select positive or
negative.

As the concrete procedure, one randomly selected soundamong 44 prepared sounds was presented to the participants.
After presenting this sound, the participants were asked to
select the one of the three attitudes described in the above.
Each participant heard the all 44 prepared sounds. The orderof the presenting sounds was counterbalanced among all 10
participants. Actually, these sounds were presented from anormal laptop PC (Let’s note W2, product of PanasonicInc,).

Fig.2 11 different F0 contours (duration: 189 ms). For example, “189u025”
indicates that duration was 189 ms and F0 transition range was 25 Hz withupward slope (increasing intonation).

We prepared the 44 different types of triangle wave
sounds with four different durations and 11 different F0(fundamental frequency) contours. Specifically, the four
durations were 189, 418, 639, and 819 ms. The 11 F0 contours were set so that the transition range of F0 values between the onset and endpoint in the sound stimuli were 0, ±25, ±50, ±75, ±100, or ±125, and these were linearly
downward or upward (Figure 2). All these 44 stimuli havethe same F0 average 131 Hz. And these sounds’ tone sounds
like computer’s beep sounds.
A. Participants
10 Japanese university students (6 men and 4 women; 19 –23 years old) were participated.
B. Procedure
At first, an experimenter gave an instruction as “please estimate the attitudes of this laptop PC expressing somesounds.” Then, they were asked to select one of the three

Fig.3 Result of the preliminary experiment: This table indicates which kindof sounds were interpreted as positive (+), negative (-), or undistinguishable
(*).

381

B. Procedure
C. Results
The result of this preliminary experiment was summarized in
Figure 3. This figure showed which kinds of sounds wereinterpreted as “positive,” “negative,” or “undistinguishable.”
From this figure, there were four sounds that all 10 participants answered positive attitudes, that is, 189 ms with upward slope range 125 Hz (189u125), 418u125, 639u125, and868u125. And there were four sounds that all answered
negative, that is, 418 ms with downward slope range 25 Hz(418d25), 418d50, 418d75, and 639d50. Thus, the soundswith increasing intonation regardless of the duration wereinterpreted as positive attitudes, while the sounds that havearound 500 ms duration with slightly decreasing were interpreted as negative ones.
These eight sounds were then selected to express to users
from the agents having different appearances in the next
actual experiment.

At first, the participants were instructed that the concretetask of this experiment was to select one of the three attitudes (positive, negative or undistinguishable) based on theexpressed sounds from the three agents.
As concrete experimental conditions that all participants
experienced, the following four conditions were prepared.
1.

Expressing the eight sounds from Mindstorms(MS-sound condition): expressing the eight sounds toparticipants from the FM radio tuner loaded on the
Mindstorms. This radio tuner received the transmitted
sounds from the Sound expressing PC (Figure 4).

2.

Expressing the eight sounds from AIBO (AIBO-soundcondition): AIBO expressing the sounds by using the
AIBO operating software “AIBO entertainment
player” installed in the AIBO operating PC.

3.

Expressing the eight AIBO’s prepared behaviors(AIBO-motion condition): The AIBO entertainment
player has about 80 prepared behaviors, such as “goodmorning” or “delightful.” Among these behaviors, weselected the following eight behaviors to express to theparticipants as primitive attitudes. As positive behaviors (four behaviors): “cheer1,” “cheer3,” “cheer4,”and “cheer5.” As negative behaviors (four behaviors):“angry1,” “angry2,” “sad1,” and “sad2.” These behaviors were selected based on the verbal labels ofthese behaviors.

4.

Expressing the eight sounds from laptop PC(PC-sound condition): The same condition with thepreliminary experiment. This laptop PC was remotely
operated from the Sound expressing PC.

IV. EXPERIMENTThe purpose of this experiment is to investigate the effects of the agents’ appearance on participants’ interpretation of the agents’ attitudes. Specifically, the participants
were presented the selected sounds in the above preliminary
experiment from the agents having different appearances,e.g., Mindstorms robot, AIBO robot and normal laptop PC(see Figure 4). And they were asked to select the correct
attitudes (positive, negative or undistinguishable) based on
the expressed sounds from these three agents.

As the experiencing order of these conditions, at first,
participants experienced MS-sounds, AIBO-sounds andAIBO-motion conditions in the random order. And then,they did the PC-sound condition. In all four conditions, theeight sounds or behaviors were randomly presented to theparticipants. The participants were asked to select the correct
attitudes from the three attitudes (positive, negative, or undistinguishable) after the agent’s expressing certain information, so that this procedure was the nearly same with thepreliminary experiment. The experiencing order of the experimental conditions and the presenting order of eightsounds or behaviors were counterbalanced among the participants.
Fig.4 AIBO robot, Mindstorms robot and Laptop PC (from left to right).

A. Participants
9 Japanese university students (8 men and 1 woman; 21 – 24
years old) were participated. These participants were notfamiliar with robots or these toys, and did not participate in
the preliminary experiment.

382

Wireless LAN（Remote Access via Real VNC）

FM radio wave)))

FM radio tuner

Wireless LAN （motion command）

FM
Transmitter
AIBO

Mindstorms

Question
displaying
monitorSound
AIBO
Question
expressing operating expressing
ＰＣ
ＰＣ
ＰＣ

PC

Place of agent

ParticipantExperimenter

Fig.5 Experimental setting.

C. Results
We investigated the interpretation rates which indicatedhow many times the participants succeeded in estimating theagents’ attitudes correctly in all four experimental conditions.
As the results, the participants showed 3.33 times among
eight experimental stimuli in MS-sound condition, 2.89
times in AIBO-sound condition, 3.33 times in AIBO-motion
condition and 6.44 times in PC-sound condition (Figure 6).
The results of ANOVA showed that there were significant
differences among these four experimental conditions(F(3,24)=8.26, p<.01(**)), and multiple comparison by LSD
test showed that there were significant differences between
the PC-sound condition and the other three conditions(Mse=2.9421, 5% level).
7(**)

Interpretation Rates（Times）

6
5
4
3

Mindstorms-sound
AIBO-sound
AIBO-motion
PC-sound2
1
0

Fig.6 Participants’ interpretation rates in four experimental conditions.

This result showed that the participants’ interpretation
rates of the same sound stimuli differed based on the eachagent’s appearance. Simply says, the participants showed thehigher interpretation rates in PC-sound condition, while they
showed the significant lower rates in MS-sounds andAIBO-sounds conditions, even though the same sounds
were presented to participants in these three conditions.
Moreover, the interpretation rates observed in AIBO-motion
condition were also significantly lower than PC-sound condition, and these interpretation rates were the nearly samewith MS-sound and AIBO-sound conditions, even thoughtthese motions were prepared in commercial product.

D. Discussion
The eight artificial sounds selected in preliminary experiment were presented to the participants from Mindstorms,
AIBO and PC. The participants’ interpretation rates thatindicate how many times the participants select the correct
attitudes were then investigated. As the result, the interpretation rates when PC expressed these sounds were significantly higher than the rates when Mindstorms and AIBO
expressed these. This result reveled that the agents’ different
appearances affected the users’ interpretations of the agents’attitudes, even though these agents expressed the completely
same information.
Let us consider the reasons why PC-sound condition
showed the higher rates comparing with the other conditions.
One reason is that these eight sounds were selected in the
preliminary experiment when these sounds were presented
from the laptop PC which was also used in the PC-soundcondition. Thus, there were some possibilities that thesesounds were only effective for informing the primitive attitudes when these were presented from the laptop PC. Thisphenomenon would be rooted on the fact that the PC expressing beep like sounds was very natural for all participants, in other words, the relationship between the appearances of PC and expressing beep like sounds would havehigher affinities.
In any case, this result could reveal that agents’ different
appearances affected the users’ interpretation of the agents’attitudes, however, this did not the actual reason why theinterpretation rates in PC-sound condition were higher thanin the other conditions. To tackle with this issue, it would
require to carrying out the other experiments to consider theaffinity between the appearance and its expressing information.
From the results of this experiment, there was an interesting phenomenon that the interpretation rates in
AIBO-motion condition were lower than PC-sound condition like MS-sound and AIBO-sound conditions. From thisresult, it can be said that the AIBO’s prepared behaviors
were not really efficient to inform the participants theprimitive attitudes, such as positive or negative. This result
would support our hypothesis described in Introduction:“the agents with rich appearance (quite similar with actual
humans or pet animals) expressing the likely information(verbal information or animal like behaviors) are more
confusing to users and are not really effective for interactingwith users: Instead, the agents without rich appearance
expressing the simple but intuitive information (e.g., subtle
expressions) are readily understood and are much more
effective for their interaction (see Figure 1). Of course, thebehaviors of AIBO were not designed for informing theprimitive attitudes we estimated, however, this result would
suggest the design policy to inform users certain attitudes
effectively.

383

A. Likeness and Familiarity
So far we discussed about the relationship between theagent’s appearance and its expressed information for informing its attitudes for humans. On the other hand, about
the effects of the agents’ appearance (or human likeness) on
the familiarities human users would feel, Mori [10] proposed
the pioneering assumption “uncanny valley”; that is, theappearance of the agent is getting similar to ones of humanbeings, in some point, humans suddenly start feeling uncanny or losing the familiarities with this agent because thesubtle differences on the appearances between actual humanbeings and the agents are emphasized. Mori described thisrelationship between the agents’ likeness to human beingsand familiarity that human users would feel as the following
qualitative diagram shown in Figure 7.
Basically, we agree with this Mori’s assumption because
the relationship between the agents’ appearance and thefamiliarities seems to succeed in explaining our hypothesis
shown in Figure 1: the agents with rich appearance (quite
similar with actual humans or pet animals) expressing the
likely information (verbal information or animal like behaviors) are more confusing to users and are not really effective for interacting with users: Instead, the agents withoutrich appearance expressing the simple but intuitive information such as subtle expressions are readily understoodand are much more effective for their interaction.

Familiarity+

Uncanny Valley

B. Likeness and Implementability
In this paper, we assumed that the agent without rich appearance expressing the simple but intuitive information,e.g., subtle expression, is much more effective for humanusers to understand the agents’ internal states. One of thereasons to focus on utilizing “subtle expressions” is that it istechnically and economically easy to implement these kindsof expressions into the agent. However, this does not imply
that every kind of the agents should express the subtle expressions.
Corresponding to the results of this study, these subtle
expressions should be designed according to the appearanceof the agents, e.g., the beep sounds should be expressed fromPC, not from robotic agents. Therefore, each agent would
have each appropriate subtle expression, and then its “implementability” would be getting decreasing according withincreasing the life-likeness shown in Figure 8. Simply
saying, the agent without rich appearance (lower likeness)requires expressing rather simple expressions, while theagent with rich experience (higher likeness) does complex
expressions. We assumed that the implementability is constantly decreasing according to increment of the likeness.
And its value would become minus when the likeness value
exceeds around (P).

+Implementalibity

V. DISCUSSION AND CONCLUSIONS

0

B
0Life-Likeness

P

100%

Fig.7 The concept diagram of “Uncanny Valley” [10].

We assumed that the former case in the above our hypothesis (the agents with rich appearance expressing the
likely information) would correspond to the bottom (B) of
the uncanny valley, so that it is expected that users would
feel uncomfortable as shown in the left of Figure 1; On theother hand, we also assumed that the latter case (the agent
without rich appearances expressing subtle expressions)would correspond to the peak (P) just before starting “uncanny valley,” so that it is expected that users would feel
comfortable as shown in the right of the Figure 1.

Life-Likeness100%

Fig.8 The hypothesized implementability according to its likeness.

C. Familiarity, Implementability and Likeness
So far we discussed about the relationship between thelikeness of the agents and the familiarity human users would
feel, and the one between the likeness and the implementability of expressed information. It can be said that the former relationship “familiarity” is about between the agent andthe user, while the later one “implementability” is about
between the agent and the designer.
Familiarity=q(agent, user)(1)

Implementability=w(agent, designer)

(2)

384

Here, we assumed that the factor “feasibility” of the intimate interaction between users and the agents can be proposed by means of the two factors “familiarity” and “implementability” as the following equation (3) and (4):
Feasibility = Familiarity + Implementability=q(agent, user)+ w(agent, designer)

(3)
(4)

From the equation (4), this feasibility could include thethree factors, “user,” “agent” and “designer” that are theimportant factors to form the interaction design. Therefore,
we expect that this feasibility factor could literally indicate
whether the user and the agent could create the intimate interaction or not.
We hypothesized that it is possible to superpose the familiarity and the implementability according to the likeness.
We could then acquire the concept diagram of the “feasibility” shown in Figure 9. From this figure, at first its feasibility
values are getting increase around (P), however, this value
suddenly decrease, as if the familiarity value falls into theuncanny valley. In addition, even though the likeness value
is getting close to 100% likeness, the feasibility values could
not recover to the same level of 50% likeness.

our hypothesis shown in Figure 1, and accord the basic
concept of the Mori’s uncanny valley assumption.
This means that our feasibility concept mostly support
Mori’s uncanny valley assumption. However, there is one
significant difference Mori’s assumption: that is, once thefeasibility value falls into the uncanny valley, this value
never rises up even though its likeness value is getting close
to 100% likeness. We assumed that this phenomenon about
the feasibility diagram would support again our hypothesis
shown in Figure 1.
As the consecutive studies based on the results of thisstudy and our feasibility concept, we are planning to conduct
the other experiments to reveal what is the most appropriate
information according to the agents’ appearance. For example, what is the appropriate information expressing from
Mindstorms robot to inform its primitive attitudes? Are
Starwars’ R2D2 like behaviors appropriate? We expect thatthese consecutive studies would support our feasibility
concept that can clarify the effective coupling between theappearance and its expressing information for realizing interactive agents readily and easily.

+

VI. REFERENCES

Uncanny Valley0Life-Likeness

P

100%

[1]

S. Yamada and T. Yamaguchi.: Training AIBO like a Dog, the 13th
International Workshop on Robot and Human Interactive Communication, pp.431-436 (2004).

[2]

S. Kiesler., L. Sproull., & K. Waters.: A Prisoner’s Dilemma Experiment on Cooperation with People and Human-like Computers.
Journal of Personality and Social Psychology, 79(1), 47-65 (1995).

[3]

S. Kiesler., & L. Sproull.: Social Responses to “Social” Computers. In
B. Friedman, Human values and the design of technology, CLSI,
Publications (1997).

[4]

T. Komatsu.: Audio Subtle Expressions Affecting User’s Perceptions,the 9th International Conference on Intelligent User Interface,p.306-308 (2006).

[5]

J. Goetz., S. Kiesler., & A. Powers.: Matching Robot Appearance andBehavior to Tasks to Improve Human-Robot Cooperation, 12th IEEE
Workshop on Robot and Human Interactive Communication (2003).

[6]

K. Liu., & W, R, Picard.: Subtle expressivity in a Robotic Computer,
CHI2003 Workshop on Subtle Expressivity for Characters and Robots(2003).

[7]

MindStorms: http://mindstorms.lego.com/[8]

AIBO: http://www.jp.aibo.com.

[9]S. Yamada, T. Komatsu: Designing Simple and Effective Expressionof Robot’s Primitive Minds to a Human, 2006 IEEE/RSJ International
Conference on Intelligent Robots and Systems, pp.2614-2619 (2006).

Fig.9 The hypothesized “feasibility” according to its likeness.

Thus, it can be said that this figure shows that the agent
with higher likeness would have the lower feasibilities tocreate an intimate relationship between the users and theagents, while the agent with lower likeness would have thehigher feasibilities. This feasibility concept would support[10] M. Mori.: Bukimi no tani (The Uncanny Valley). Energy, 7(4), 33–35(1970). (Originally in Japanese)385