How does Difference between Users’ Expectations andPerceptions about a Robotic Agent (Adaptation Gap)
Affect Their Behaviors?
Takanori Komatsu

Rie Kurosawa

Seiji Yamada

Shinshu University
3-15-1 Tokida
Ueda 386-8567, Japan+81-268-21-5588

Shinshu University
3-15-1 Tokida
Ueda 386-8567, Japan+81-268-21-5588

NII/Sokendai
2-1-2 Hitotsubashi
Tokyo 101-8430, Japan+81-3-4212-2000

tkomat@shinshu-u.ac.jp

f077005@shinshu-u.ac.jp

seiji@nii.ac.jp

ABSTRACTWe describe how the notion of “adaptation gap” can be used todescribe the differences between the functions of a robotic agent
that the users are expecting from it before starting theirinteractions and the functions they perceive after their interactionsin this paper. We investigated the effect of this adaptation gap on
the users’ behaviors toward a robotic agent. The results show thatthe positive or negative signs of this adaptation gap significantly
affect the users’ behaviors towards the agents.

Categories and Subject Descriptors
H5.2 User Interfaces: Evaluation/methodology; J.4 Social andbehavioral sciences: Psychology.

General Terms
Experimentation, Human Factors.

Keywords
Adaptation gap, users’ expectations and perceptions, users’
behaviors toward agents.

1. INTRODUCTION
Various interactive agents such as robotic agents [1] andembedded conversational agents (ECA) [2,3] have beendeveloped to assist us with our daily tasks. In particular,
researchers in the human-computer interaction and human-robot
interaction communities are working hard to create suchinteractive agents. In these fields, the issue “how the users’
mental models of an agent formed before the interactions affecttheir interaction with it” is keenly focused on. Since users
supposedly base their mental models about an agent on its
appearance, its behaviors, and their preferences for the agent, thePermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.
Conference’10, Month 1–2, 2010, City, State, Country.
Copyright 2010 ACM 1-58113-000-0/00/0010…$10.00.

users’ mental model significantly affects their interaction [4]. For
example, when a user encounters a dog-like robot, s/he expects a
dog-like behavior from it, and s/he naturally speaks to it usingcommands and other utterances intended for a real dog, such as“sit,” “lie down,” and “fetch.” However, s/he does not act thisway toward a cat-like robot.
Several studies have focused on the effects of the users’ mental
models about an agent on their interactions. Matsumoto et al. [5]proposed a “Minimal Design Policy” for designing interactive
agents and concluded that the agent’s appearance should be
minimized in its use of anthropomorphic features so that the users
do not overestimate or underestimate the agents’ competences. In
fact, they applied this minimal design policy to developing Muu,
their interactive robot [6] and Talking Eye, a life-like agent [7].
Kiesler [8] argued that the design of an agent should include a
process that anticipates a user’s mental model about the agent on
the basis of the theory of common ground [9]; that is, individuals
engaged in conversation must share knowledge (so-called,common ground) in order to be understood and have a meaningfulconversation. In particular, she stated that the agents should bedesigned in such a way that a user could easily estimate her/his
common ground (shared knowledge) with them. We believe that
this design approach would be quite effective for users, especiallyat the beginning of an interaction, because it may determine
whether or not the user would actually start interaction with agiven agent.

2. ADAPTATION GAP BETWEEN A
HUMAN AND AN AGENT
However, approaches like Matsumoto et al.’s [5] or Kiesler’s [8]have a serious problem when the agent expresses behaviors thatcompletely deviate from the users’ mental model. Imagine that auser meets a human-like robot that looks very much like a realhuman being. This user would intuitively form a mental model ofthe robot, expecting fluent human-like speech, dialogue, anddexterous limb motions. However, if this particular robot could
only express machine-like speech and halting limb motions thatcompletely deviate from her/his mental model, s/he would be
immediately disappointed with this robot because of its
unexpected behaviors. The user would then stop interacting with
it. To solve this problem, we need to carefully design the users’
expectations and perceptions of the agents during theirinteractions, because such expectations and perceptions would
assist users in determining whether this agent is worth interactingwith or not.

significantly influenced by AG (=Fafter - Fbefore), we can conclude
that the properties of AG are verified.

3. EXPERIMENT
3.1 Overview
We conducted an experiment to investigate how the positive or
negative signs of AG affected the users’ behaviors towards anagent. This experiment consisted of two phases. The first phase
was to measure the sign of the AG as an independent variable(exploration phase), while the second phase was to measure theusers’ behaviors as a dependent variable (exploitation phase).

Figure 1. Intuitive Concept of Adaptation Gap
In this study, we focus on the difference between the users’
expectations regarding the function of the agent and the users’
actual perceived function, which is one of the factors that affects
the users’ impressions. We called this difference the adaptationgap (AG). In particular, AG can be defined as AG = Fafter - Fbefore.
Here, Fafter is the function that a user actually perceives of theagent, and Fbefore is the users’ expected function of the agent. We
assume that this AG would have the following three properties[10,11].
AG < 0 (Fafter< Fbefore): When the users’ expected function
exceeds their perceived function, there is a negative
adaptation gap. In this case, most people would be
disappointed by the agent, would not believe the robot’s
outputs, and stop interacting with it.

AG > 0 (Fafter> Fbefore): When the users’ perceived function
exceeds their expected function, there is a positive
adaptation gap. In this case, most people would not bedisappointed by the agent, would believe the robot’s outputs,
and continue interacting with it.

AG = 0 (Fafter= Fbefore): When the perceived function equals
the expected function, there is no adaptation gap. In thiscase, the agent would be regarded as just an instrument forusers.

For example, when Fbefore is larger than Fafter (say, when a user
meets the human-like robot on the left in Figure 1), AG would
have a negative value (AG<0), and the user would most likely bedisappointed. However, when Fafter is larger than Fbefore (say,when a user meets the machine-like robot on the right in Figure 1),
AG would have a positive value (AG>0), and the user would beinterested in interacting with this agent.
In particular, we assume that the sign of AG value strongly affects
the user’s behaviors toward the agents. Therefore, we investigated
the relationship between the signs of AG and the user’s actual
behavior toward the agents, e.g., whether the users accept theagents’ suggestions or not, in this study. Therefore, theindependent variable in this study is the sign of AG while thedependent variable is the users’ behaviors. We assumed that thisinvestigation would lead to verification of the above threeproperties concerning AG. Namely, if the users’ behaviors are

Figure 2. Treasure Hunting Video Game

Figure 3. MindStroms Robotic Agent
We chose a “treasure hunting” video game (Figure 2) as theexperimental environment for observing the interaction between auser and an agent in both phases. In this game, a character on acomputer monitor operated by a user walks on a straight road,with three tiny hills appearing along the way. A gold coin is
inside one of the three hills, while the other two hills have nothing.
In the exploration phase, the game ends after the character meets
40 sets of hills and the approximate duration of the game is about
3 minutes, while in the exploitation phase, the game ends after 20
sets of hills. The goal of this game is to get as many gold coins as
possible. A robotic agent (MS; MindStorms, LEGO Corporation,
Figure 3), which was placed next to the user, told the participant
where it expected the coin would be each time. MS told the userthe expected position by beeping the number, e.g., one beep
meant the first hill, two beeps meant the second hill (middle), andthree beeps meant the third hill. The participant could freely
accept or reject the agents’ suggestions. The participants wereallowed to know whether the robot’s suggestion was right or notin each trial in the exploration phase, while they were not allowed
to know whether the given suggestion was right or not in the
exploitation phase (actually, the selected hill just showed aquestion mark and a closed treasure box, see Figure 2). Note that
this experimental setting was introduced because we needed theparticipant to estimate the robotic agent's function and the sign of
AG was determined only in an exploration phase, not in an
exploitation phase.
The participants were informed that 1 point was equivalent to 10
Japanese yen (about 10 US cents) and that, after the experiment,
they could purchase some stationery (e.g., file holders or USB
flash memories) of equivalent value with their points. The
position of the coin in the three hills was randomly assigned.

3.2 Participants
Thirty Japanese university students (22 men and 8 women; 19 25 years old) participated. These participants were randomly
divided into the following two groups in terms of their
expectations of the robot’s ability before the experiment.


Lower Expectation Group (15 participants): Before theexploration phase, an experimenter gave the following
instructions to these participants, “The rate at which thisrobot succeeded in detecting the position of a coin was10%.” Therefore, their expectations (Fbefore) were forced set
at 10%.
Higher Expectation Group (15 participants): Before theexploration phase, the experimenter gave these instructions
to them, “The rate at which this robot succeeded in
detecting the position of a coin was 90%.” Therefore, their
expectations (Fbefore) were forced set at 90%.

coin in the exploration phase was set at 33%. This 33% shouldhave became Fafter for all the participants in both groups, so thevalues (and sign) of AG would be automatically determined; thatis, the ideal values of AG in the Lower Expectation Group should
be around +23 (i.e., Fafter - Fbefore = 33% - 10%), and the ideal AG
in the Higher Expectation Group should be around - 67 (Fafter Fbefore = 33% - 90%).
The speech sounds of the robotic agent were remotely operated by
an experimenter in the next room performing in the Wizard of Oz(WOZ) manner via an FM transmitter and radio tuner loaded on
the MS. The treasure hunting video game was projected on a 46inch LCD screen in front of the participants (Figure 4). The orderof the beeping sounds from the robotic agent was counterbalanced
across the participants.

3.3 Analysis
We investigated the effect of the signs of AG on the users’
behaviors towards the robotic agents. Therefore, the independent
variable was the sign of AG and the dependent variable was theparticipants’ behaviors. The sign of AG would be automaticallydetermined in the exploration phase; that is, the participants in theLower Expectation Group would show the positive sign of AG
while the ones in the Higher Expectation Group gave the negative
sign of AG. Also, in order to acquire the users’ behaviors as
dependent variables, we then calculated the acceptance rate,
indicating how many of the agent’s suggestions the participants
accepted in the exploitation phase; because the 20 sets of hills
appear in the exploitation phase, the maximum acceptance rate
was 20.
The purpose of this experiment was to compare the participants’
acceptance rates among the two experimental groups. If we could
observe the phenomenon in which the participants in the Lower
Expectation Group showed higher acceptance rates than the onesin the Higher Expectation Group, we would have concluded thatthe signs of AG significantly affected the users’ behaviors
towards the agents in the way we expected. Moreover, we could
argue that the properties of AG were verified.

3.4 Results

Figure 4. Experimental Setting.
We conducted a manipulation check in both groups just before theexperiment to ask them, “What rate will this robot succeed in
detecting the position of a coin?” However, no participants were
eliminated because no one answered the totally deviated rates(e.g., “100%” in Lower Expectation Group). Actually, the rate at
which the robotic agent succeeded in detecting the position of theAt first we checked whether the acquired independent variables(e.g., signs of AG) were appropriately set in the exploration
phase; specifically, whether the participants in the Lower
Expectation Group showed the positive signs of AG and also theones in the Higher Expectation Group gave the negative sign of
AG. For the 15 participants in the Lower Expectation Group, theaverage value of AG was +6.0 (SD=8.79), and for the 15
participants in the Higher Expectation Group, the average valuesof AG was -43.9 (SD=25.2). Although these values were notreally similar to our ideal AG values (i.e., +23 in Lower
Expectation Group and -67 in Higher Expectation Group), there
was a significant difference between the two values(F(1,28)=49.12, p<.01 (**)). Therefore, we confirmed that theindependent variables were appropriately set.
We then calculated the acceptance rate as a dependent variable.
For the 15 participants in the Lower Expectation Group, theaverage acceptance rate of the robot’s 20 suggestions was 9.40
(SD=4.33), and for the 15 participants in the Higher Expectation
Group, the average rate was 5.13 (SD=4.47, see Figure 5). The
acceptance rates for both experimental groups were then analyzed

using a one-way analysis of variance (ANOVA) (between-subject
design; independent variables: signs of AG, positive or negative,
dependent variable: acceptance rates). The result of the ANOVA
showed a significant difference between the two experimentalgroups (F(1,28)=6.58, p<.05 (*)); that is, the participants in theLower Expectation Group showed a significantly higher
acceptance rate compared to the ones in the Higher Expectation
Group. Therefore, we could conclude that the signs of AG
significantly affected the participants’ behaviors towards theagent, and moreover, we also confirmed that the properties of AG
was clearly verified; when the users’ expected the function to
exceed their perceived function, most of the people would be
disappointed with the agent and would stop interacting with it,
while when the users’ perceived function exceeded their expectedfunction, most people would not be disappointed with the agent
and would continue interacting with it.

now planning to tackle the issue of “how to handle the values of
AG” in collaboration with product designers and social
psychologists. We believe such collaborations would control thevalues of AG in a more elegant manner and would lead tocontributing to a much more sophisticated concept of AG.

5. ACKNOWLEDGMENT
This study was partially supported by Special Coordination Funds
for Promoting Science and Technology from the Ministry of
Education, Culture, Sports, Science and Technology (MEXT),
Japan.

6. REFERENCES[1] M. Imai, T. Ono, & H. Ishiguro, “Robovie: Communication
Technologies for a Social Robot.” International Journal of
Artificial Life and Robotics, 2003, pp. 73-77.
[2] H. Prendinger, & M. Ishizuka, Life-Like Characters.
Springer, 2004.
[3] J. Cassell, T. Stocky, T. Bickmore, Y. Gao, Y. Nakano, K.
Ryokai, D. Tversky, C. Vaucelle, & H. Vilhjalmsson,“MACK: Media lab Autonomous Conversational Kiosk.”
Proceedings of Imagina02, 2002.
[4] T. Komatsu, & S. Yamada, “How do robotics agents’appearances affect people’s interpretations of the agents’attitudes?” Extended Abstract of ACM-CHI2007 (work-inprogress), 2007, pp. 2519-2525.

Figure 5. Acceptance Rate in Two Groups.

4. DISCUSSION AND CONCLUSIONS
From the results of our experiment, we observed that theparticipants with a positive sign of AG showed a higher
acceptance rate compared to the ones who showed a negative signof AG. Therefore, we confirmed that the signs of AG significantly
affected the users’ behaviors towards the robotic agents. These
results clearly supported the properties of AG mentioned in the
Section 2, so they will contribute in proposing a novel interaction
design strategy, e.g., “the agents that evoke higher expectations
compared to the actual functions should not be used for theinteraction task with users.”
At a glance, these results seem to recommend that a specificdesign strategy like “Fbefore should be set as low as possible to
make the signs of AG positive.” However, such a lower Fbefore
would have some possibilities to make users deeply disappointed
with the agent before the interaction, and eventually they would
not start the interaction. Therefore, clarifying the appropriate
range of Fbefore would be a significant issue for utilizing this AG
for actual interaction design strategy.
In this study, we did not focus the values of AG, but on the signsof AG, since it was quite difficult to precisely comprehend or
measure the users’ digitized, expected, and perceived functions of
the agents. Moreover, it is assumed that such digitized values forthe agents’ functions would be affected by various aspects, e.g.,gender, educational level, religious belief, or preferences. We are[5] N. Matsumoto, H. Fujii, M. Goan, & M. Okada, “Minimal
design strategy for embodied communication agents.”
Proceedings of the 14th International Symposium on Robotand Human Interactive Communication (RO-MAN2005),
2005, pp. 335-340.
[6] M. Okada, S. Sakamoto, & N. Suzuki, “Muu: Artificial
Creatures as an Embodied Interface.” Proceedings of 27th
International Conference on Computer Graphics andInteractive Techniques (SIGGRAPH 2000), 2000, p. 91.
[7] N. Suzuki, Y. Takeuchi, K. Ishii, & M. Okada, “Talking
Eye: Autonomous Creatures for Augmented Chatting.”
Robotics and Autonomous Systems, Vol. 31, 2000, pp. 171184.
[8] S. Kiesler, “Fostering common ground in human-robot
interaction.” Proceedings of the 14th International
Symposium on Robot and Human Interactive
Communication (RO-MAN2005), 2005, pp. 729-734.
[9] H. H. Clark, Arenas of language use, University of Chicago
Press, Chicago, USA, 1992.
[10] S. Yamada, & T. Komatsu, “Designing Simple and Effective
Expressions of Robot’s Primitive Minds to a Human.”
Proceedings of 2006 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS-2006), 2006, pp.
2614-2619.
[11] T. Komatsu, & S. Yamada, “Effect of agent appearance on
people’s interpretation of agent’s attitudes.” Extended
Abstract of ACM-CHI-2008 (work-in-progress), 2008, pp.
2929-2934.