Session: Understanding Gamers

CHI 2012, May 5–10, 2012, Austin, Texas, USA

Experimental Investigation of Human Adaptation to Change
in Agent’s Strategy through a Competitive Two-Player Game
Kazunori Terada
Gifu University
Yanagido 1-1, Gifu, Japan
501-1193
terada@gifu-u.ac.jp

Seiji Yamada
National Institute ofInformatics / SOKENDAI
2-1-2 Hitotsubashi, Chiyoda,
Tokyo, Japan 101-8430
seiji@nii.ac.jp

ABSTRACT

We conducted an experimental investigation on human adaptation to change in an agent’s strategy through a competitive
two-player game. Modeling the process of human adaptation to agents is important for designing intelligent interface
agents and adaptive user interfaces that learn a user’s preferences and behavior strategy. However, few studies on human adaptation to such an agent have been done. We proposea human adaptation model for a two-player game. We prepared an on-line experimental system in which a participant
and an agent play a repeated penny-matching game with abonus round. We then conducted experiments in which different opponent agents (human or robot) change their strategy
during the game. The experimental results indicated that, as
expected, there is an adaptation phase when a human is confronted with a change in the opponent agent’s strategy, andadaptation is faster when a human is competing with robot
than with another human.
Author Keywords

Human-Agent Interaction; human adaptation to an agent;
two player game; appearance; intentional stance.
ACM Classification Keywords

H.5.2 [Information Interfaces And Presentation]: User Interfaces - Theory and methods;
General Terms

Experimentation; Human Factors.
INTRODUCTION

Human-Agent Interaction, Human-Robot Interaction and embodied conversational agents have been extensively studied
in the HCI community. It is important to experimentally investigate the human adaptation process to construct an agent
model to use when a user confronts an agent. Such a model
enables us to adequately design an agent’s appearance andPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.
CHI’12, May 5–10, 2012, Austin, Texas, USA.
Copyright 2012 ACM 978-1-4503-1015-4/12/05...$10.00.

Akira Ito
Gifu University
Yanagido 1-1, Gifu, Japan
501-1193
ai@gifu-u.ac.jp

behavior based on its influence on user adaptation [6, 8]. Although there have been many studies that have focused on theinfluence of an agent’s appearance and behavior on humans[7, 5, 3], few have investigated human adaptation to an agent.
We believe human adaptation to change in an agent’s behavior strategy is especially significant in various human adaptations. Because intelligent agents that learn a user’s preference and behaviors have been developed, and most constantly
change their behaviors through learning. To maintain successful interaction, a user needs to recognize and adapt to theagent’s change in strategy.
We experimentally investigated human adaptation to change
in an agent’s behavior strategy through a simple two-player
game. We prepared a modified matching-pennies game. A
participant and an agent play the game through the on-line experimental system. The independent variable is the opponent
agent’s appearance including a bear-like robot and a human.
The opponent agent changes its behavior strategy during thegame. We carefully observed the behaviors of the participant
to the change in the agent’s strategy and investigated how different agent appearances influence human adaptation by using two different opponent agents.
Komatsu and Yamada proposed the Adaptation Gap[4],which describes the difference between an agent model and auser model before/after actual interaction, and conducted various experiments to understand human adaptation. Their focused sign (+, 0, −) of adaptation gap’s value AG, and madethe following assumptions, “AG < 0: most people would be
disappointed by the agent and would stop interacting with it
and AG > 0: most users would continue interacting with it”.
Gajos et al. [2] also studied human adaptability to an adaptive user interface. They prepared a simple adaptive user interface equipped with strategies, and conducted experiments
to determine the relationship between a user’s predictability
on behaviors of the adaptive user interface and its accuracy insuch an environment. They observed that improvement in thesystem’s accuracy had a stronger effect on user’s performanceand satisfaction than the improvement in predictability.
MODEL OF HUMAN ADAPTATION TO AGENT
Model of human adaptation phase

We modeled the process of human adaptation to an agent’s
change in strategy in terms of exploitation and adaptation

2807

CHI 2012, May 5–10, 2012, Austin, Texas, USA

Performance of human participant

Session: Understanding Gamers

design stance). Various factors, such as human-like appearance and life-like autonomy, lead a human to an intentional
stance. Thus, we assigned a human opponent player anda robot player as independent variables for our experiments
and investigated the change in human adaptation dependingon whether the opponent is human or a robot. We expecteda participant to construct a simpler and deterministic strategy
model for a robot opponent and a more complicated and probabilistic model for a human one. Hence, we also expected
human adaptation to a human opponent to be faster than to arobot opponent. As described in the next section, this expectation is hypothesized and verified by observing adaptation
speed in an experiment with human participants.

Time
Exploitation
phase

Adaptation phase

Exploitation
phase

Figure 1. Player’s performance in adaptation and exploitation phases.

phases. We focused on the adaptation phase in a competitive two-player game because adaptation to an agent’s change
in strategy is critical in a competitive situation. Figure 1
shows a simplified performance profile of a player in this twophase framework. In a competitive two-player game, a player
needs to identify the opponent’s behavior strategy and decide
his/her behavior strategy based on it. However, estimating the
opponent’s strategy based only on observed behavior is an illposed inverse problem because different strategies sometimes
produce the same behavior. Therefore, probing an agent’s
strategy by actively changing player’s strategy is more effective for quickly estimating than just waiting without changing
strategy: the adaptation phase.
The player’s performance in the adaptation phase increases
according to improvement in estimation accuracy; therefore,
performance in this phase can be approximately modeled asa linear function of time not as a step function. Once theplayer identified the opponent’s strategy, he/she purely executes actions according to the current strategy and exploits
the opponent as long as the opponent’s strategy is stable intime: the exploitation phase. These two phases are repeatedalternately.
The length of the adaptation phase differs depending on thecomplexity of the opponent’s strategy. A more complex strategy requires a longer adaptation period because more probing
steps are needed to identify the opponent’s strategy. The gradient of the adaptation phase in Figure 1 indicates adaptation
speed.
Different adaptation to different opponents

The appearance of an agent affects its modeling of cognitive
ability and behavior. Dennett [1] proposed three stances, “intentional”, “design”, and “physical”, as a criterion for an observer to model an object. Physical stance is the most concrete and concerned with the domain of physics. At this level,
we are concerned with such things as mass, energy, and velocity. More abstract is the design stance, which is the domainof biology and engineering. At this level, we are concernedwith such things as purpose, function, and design. Intentional
stance is the most abstract and concerned with the domain ofhuman/agent’s minds. At this level, we recognize objects interms of the belief, thinking, and intention.
With an artificial agent, we often recognize it as having intention (an intentional stance) as well as just being a machine (a2808

EXPERIMENTS

The above-mentioned model of human adaptation to an agent
leads to the following hypotheses:
H1: An adaptation phase exists when a human is confronted
with a change in the opponent’s strategy.
H2: Adaptation is faster when a human is competing with arobot than with a human.
The two hypotheses were tested through an experiment withhuman participants.
Penny-matching game with bonus roundA repeated penny-matching game with a bonus round wascreated for validating the hypotheses. The penny-matchinggame is a zero-sum game and is played between two players,
players A and B. Each player has a penny and must secretly
turn the penny to heads or tails. The players then reveal theirown choices simultaneously. If the pennies match (both heads
or both tails) player A keeps both pennies and gets to keep
player B’s penny (+1 for A, -1 for B). If the pennies do notmatch (one heads and one tails), player B keeps both pennies(-1 for A, +1 for B). While this game has no pure strategy
Nash equilibrium, the unique Nash equilibrium of this gameis in mixed strategies: each player chooses heads or tails withequal probability.
We modified the game rules so that players are able to use
a deceptive strategy. The penny-matching game was played
repeatedly. One game consists of six rounds, and ten games
were played in the experiment. The payoff of the sixth roundin every game is increased twenty times: the bonus round. A
reasonable strategy for this game is to strive to win the bonus
round and abandon the other five rounds because of the large
payoff gap. A player is able to trap the opponent by making aseries of choices during the normal five rounds so that the opponent’s prediction of the player’s sixth choice will be wrong.
Note that the unique Nash equilibrium of the penny-matchinggame with a bonus round is still a mixed (random) one eventhough the bonus round is added.
The agents used as opponents in our experiment used only
the two strategies (straightforward and deceptive) listed inTable 1. The two strategies are realized with two series of
uniform and alternate choices. Uniform and alternate choicesin the first five rounds were suggestive of the opponent; i.e.,
those exposed to the obvious trapping choices are forced toSession: Understanding Gamers

deceptive

1
1

•
•
•
•

uniform
alternate
uniform
alternate

2
1

3
1

•
◦
•
◦

•
•
•
•

4
1

•
◦
•
◦

5
1

•
•
•
•

100

6
20

90

•
◦
◦
•

80

Table 1. Two strategies (straightforward and deceptive) in pennymatching game with bonus round used in experiment. Black and whitecircles represent heads and tails, respectively. Only difference between
straightforward and deceptive strategy is choice in sixth round.

Wining percentage (%)

round
payoffstraightforward

CHI 2012, May 5–10, 2012, Austin, Texas, USA

70
60
50
40
30
20

Against robot
Against human

100Robot condion

Human condion

1st game䐠(Total 5 games)

Opponent’s
Score
1

䐣
䐟

Your
Score
3

Heads

Tails

䐢

1

2

3

4

5
6
Game no.

7

8

9

10

Figure 3. Percentage of participants who won sixth round of ten games.

䐥A single-factor two-level between-subject experimental design was used. Twenty-eight graduate and undergraduate students participated in the study. Participants were randomly
assigned to either a robot or human opponent condition andwere seated in front of a desktop computer. Participants were
informed that the point of the experiment was to access theusability of an online game system.

䐤Choose a sidefor 5th round.
5 seconds remaining䐡

Figure 2. Interface of on-line experimental system: 1) participant’s face,2) opponent agent’s appearance, 3) remaining time, 4) choice buttons, 5)histories of both players’ choices, 6) scores for both players, and 7) game
number.

anticipate that the opponent’s sixth choice will be either deceptive (violating regularity) or straightforward (keeping regularity).
Experimental setup and measurement

The game was implemented with JavaScript and HTML andplayed in a Web browser (Firefox). Figure 2 shows the gameinterface. A Flash video of the opponent (robot or human)is displayed at the top of the interface. The bear-like robot
moves its head and arms randomly in the video. The behavior of the human was recorded when he played the game inadvance. The agents’ behaviors, such as choosing a side of
the coin, are automatically controlled by JavaScript program.
Participants were told that the opponent is online. To makethe participants believe the game is online, the participants’faces captured using a web camera mounted on the monitor
were displayed at the bottom of the interface. Participants
were instructed to click the button corresponding to his/her
choice within 10 seconds for every round. Scores for bothplayers are shown in the interface. The choices of both players remain displayed so that the participant is able to recognize the opponent’s strategy. Before the main experiment, allparticipants played five games only for training.

2809

A total of eight series of choices including another four series beginning from tails equivalent to the four series shownin Table 1 were used. The agents used a straightforward strategy, one of the four series of straightforward strategy was randomly selected, in the first three games and a deceptive one
for the rest. This is the change in agent’s behavior strategy
during the game, and observing human adaptation to it is ouraim of the experiment.
The outcomes of the sixth round for all ten games wererecorded because the participants’ expectation of their opponents’ strategies, i.e., which strategy, straightforward or deceptive, did they expect, was of interest. Winning the sixthround against an agent indicates that the participants expectation of the agent’s strategy is correct.
RESULTS

The percentages of the participants who won the sixth roundfor all ten games are shown in Figure 3. Almost all the participants lost the fourth game because the agents changed theirstrategy from straightforward to deceptive. After the fourth
game, the winning percentage with both opponent agents
gradually recovered to the level of the third game. A chisquare test was conducted to investigate whether the winning
percentages of the fourth to tenth games were different fromthat of third one (see Table 2). The results indicate that at least
two games were required to adapt to the change in strategy ofboth agents. The gradual increase in the winning percentage
after the opponent’s strategy changed indicates the existenceof the adaptation phase, confirming hypothesis H1.
The differences between recovery speeds of the winning percentage between those playing against the robot and thoseagainst the human agent (plotted on the graph in Figure 3)

Session: Understanding Gamers

Game no.
Against robot condition
Against human condition

4
24.26, p<.01
14.29, p<.01

CHI 2012, May 5–10, 2012, Austin, Texas, USA

519.33, p<.01
7.34, p<.01

62.15, n.s.
5.60, p<.05

7
1.04, n.s.
0.85, n.s.

83.36, n.s.
2.80, n.s.

94.67, p<.05
2.80, n.s.

10
6.09, p<.05
0.85, n.s.

Table 2. Chi-Square values (df=1) and significance levels for the winning percentages of fourth to tenth games compared with that of the third one.

were statistically confirmed. Table 2 indicates that those who
played against a human agent required one more game foradapting to their opponent’s change in strategy than those
who played against a robot. This implies that participant’s
adaptation speed against a human is significantly slower thanagainst a robot, confirming hypothesis H2.
DISCUSSION

The participants did not equate the robot to the human interms of adaptation speed against the opponent’s change instrategy. While the strategy equally changed in the fourthgame for both opponents, what caused the inequality in participant’s adaptation speed is the difference in the video theparticipants watched. Slow adaptation of the participants whoplayed against the human indicates that they were cautious
but not optimistic against the opponent’s strategies after the
strategy change in the fourth game. This is because the humanappearance made the participants anticipate another change
in the human opponent’s strategy, and the robot appearance
gave the participants the impression that its strategy was stable. The behavior of the designed artifacts, including robots
and computers, is governed by laws such as mechanical, algorithmic, or structural constraints; therefore, the input-output
relations of the artifacts are stable. This prototypical concept
of artifact’s regulated behavior might make the participants
expect the simpler and less complicated strategy of the robot.
There are two causes for the gradual increase in the winning percentage in the adaptation phase: 1) Each participantchanged his/her strategy just once in the adaptation phase(during games 5 to 7) and the timings of the changes varied
across the participants. 2) Each participant changed his/her
strategy plural times to find the most suitable strategy. To investigate these causes, we compared the number of strategy
changes in the adaptation phase in two conditions. The average number of strategy changes by the participants was 1.07
under an ”against robot” condition and 1.57 under an ”against
human” condition was 1.57, so there was a marginally significant difference (t(26) = 1.919, p = 0.066). This suggeststhat strategy changes by the participants were one-shot deterministic under an ”against robot” condition bat indecisive andexploratory under an ”against human” condition.
We consider the drop of winning percentage in the eighth
game could be accounted for by meta-adaptation. Our gamedesign requires at least two levels to be adapted. The first is
adaptation for regularity of the agent’s choice in the first fiverounds. The second is adaptation for strategy change in the
fourth game. The third, which was beyond our expectations,is an adaptation for periodic change in an agent’s strategy.
Participants might think the agent’s strategy changes once every three games and so expect to change again in the seventhgame.
Not only the appearance but also the behavior of the agents2810

differed between the two videos. We did not separate the appearance and motion factors in our experiment. Investigating
which factor contributes to the change in human adaptation
speed to an agent is for future work.
CONCLUSION

We theoretically modeled and experimentally investigatedhuman adaptation to an agent. We conducted a pennymatching game with a bonus round in a web browser for investigating how humans adapt to an opponent agent’s change
in strategy. The experimental results indicated there is anadaptation phase when a human is confronted with a change
in the opponent agent’s strategy, and adaptation is faster whena human is competing with robot than with another human.
Although this experiment might be just a preliminary stage,
we strongly believe this work provided a significant casestudy and will be inspiration for practical design of an agent
and its interaction with a human.
REFERENCES

1. Dennett, D. C. The Intentional Stance. Cambridge, Mass,
Bradford Books/MIT Press, 1987.
2. Gajos, K. Z., Everitt, K., Tan, D. S., Czerwinski, M., andWeld, D. S. Predictability and accuracy in adaptive userinterfaces. In CHI’08 (2008), 1271–1274.
3. Kiesler, S. Fostering common ground in human-robot
interaction. In RO-MAN’05 (2005), 729–734.
4. Komatsu, T., Kurosawa, R., and Yamada, S. How does thedifference between users’ expectations and perceptions –about a robotic agent affect their behavior? Int. Journal ofSocial Robotics (2011), 1–8.
5. McQuiggan, S. W., Rowe, J. P., and Lester, J. C. The
effects of empathetic virtual characters on presence innarrative-centered learning environments. In CHI’08(2008), 1511–1520.
6. Tinwell, A., and Grimshaw, M. Bridging the uncanny: an
impossible traverse? In Proceedings of the 13th
International MindTrek Conference: Everyday Life in the
Ubiquitous Era, MindTrek ’09 (2009), 66–73.
7. Vugt, H. C. V., Bailenson, J. N., Hoorn, J. F., and Konijn,
E. A. Effects of facial similarity on user responses toembodied agents. ACM Tran. on Computer-Human
Interaction 17 (May 2008), 7:1–7:27.
8. Yee, N., Bailenson, J. N., and Rickertsen, K. A
meta-analysis of the impact of the inclusion and realismof human-like faces on user experiences in interfaces. In
CHI’07 (2007), 1–10.