Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

Two-Dimensional Mind Perception Model of Humanoid
Virtual Agent
Tetsuya Matsui
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japan
tmatsui@nii.ac.jp

Seiji Yamada
National Institute of Informatics/Sokendai/
Tokyo Institute of Technology
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japanseiji@nii.ac.jp

ABSTRACTIn this paper, we verified two kinds of two-dimensional mind
perception models of humanoid virtual agents and investigate
the relationship between the models and effect of emotional
contagion. To verify the two kinds of dimensional models, weused questionnaires from prior works and our own questionnaire. From these questionnaires, we constructed an “agency”“experience” model and “familiarity”-“’reality” model from
EFA. These two models are valid for distinguishing humanoid
agents and predicting the effect of emotional contagion. The
factor scores of “experience” and “familiarity” have a highcorrelation coefficient with the effect of emotional contagion.
This result suggests a method for designing humanoid agentsthat have a high emotional contagion ability.

In this paper, we verify Gray’s two-dimensional model withhumanoid agents and suggest a novel two dimensional model.
With humanoid agents, there are many factors, for example,
gender, race and ethnic groups, and abstraction.
Gender is a basic factor for distinguishing humanoid agents.
Kramer et al. showed that when a virtual agent’s gender was
opposite that of a participant, the participant formed rapport
with the agent [10]. However, Guadagno et al. showed thatparticipants change their attitude positively when an agent’s
gender matches that of themselves [6]. Kim et al. showed thatstudents feel that a teaching agent is more “human-like” when
the agent is male rather than female [9]. These impressions
seem to be derived by an agent’s appearance and participant’sculture.
Payne et al. showed that female participants tended to selectfemale virtual agent assistants: however, male participants
selected female and male assistants to the same degree[15].
This study showed that the effect of an agent’s gender depends
on the participant’s gender.

ACM Classification Keywords

H.5.2. User Interfaces
Author Keywords

virtual agent; anthropomorphic agent: emotional contagion:appearance; mind perception; agency

These prior works showed that the gender of an agent affects
users.

INTRODUCTION

We live surrounded by many kinds of virtual agents, and theseagents have many kinds of appearances. Gray et al. suggested a two-dimensional model derived from the impressionof agents [4]. In an experiment, the authors used many kindsof agents including animals, male and female human, robots,and gods. The authors found an “agency” axis and “experience” axis. Gray et al. showed that this model was valid forrevealing the mind perception of psychopathology patients [5].
Terada et al. used this model to investigate the effect of theappearance of virtual agents [19]. The model revealed how
users perceived agents; however, 5 out of the 13 agents usedwere humanoid: male, girl, female, persistent vegetative state
male, and “you” [4]. There were no elderly female and healthy
old male, and race and ethnic groups were excluded.

An agent’s race and ethnic group can be expressed throughappearance. Rossen et al. investigated the effect of the skincolor of humanoid agents [17]. They showed that users ofhumanoid agent are affected by real world skin-tone biases.
Peck et al. showed that using a black skin color for an avatar
reduces racial bias [16]. Racial bias is a large social problem;
thus, the effect of an agent’s race and ethnic group is importantin the HAI field.
A lot of prior works revealed that the abstraction of agentsaffected a user’s perception. Geven et al. showed that the abstraction of an agent affected how a user perceives the agent’s
intelligence and trustworthiness [3]. They showed that a low
abstraction brought about a positive effect for humanoid agents.
Mori et al. suggested the “uncanny valley’ ’[14]. This theory
states that excessive reality reduces a robot’s familiarity.
McCloud suggested a triangular design space for distinguishing a cartoon character’s face [13]. He defined three angles ofthis triangle as “realistic object,” “iconic object,” and “abstract.”
He also defined “abstract” as low realism. Duffy applied thistriangle model to distinguishing robot faces [2]. Also, Blowet al. used this model to design robots [1]. From these prior

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.
HAI 2017, October 17–20, 2017, Bielefeld, Germany..
Copyright © 2017 ACM ISBN 978-1-4503-5113-3/17/10 ...$15.00.
http://dx.doi.org/10.1145/3125739.3125761

311

Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

works, abstraction is an important parameter for designinghumanoid agents.

• Self control: How capable of feeling self control do you
think this agent is?

These variations in humanoid agents seem to affect the effectiveness of the practical functions of the agents. In thispaper, we focused on the emotional contagion function. Emotional contagion (EC) is a phenomenon in which one’s emotion
infects a partner [7]. Tsai et al. showed that EC occurred between virtual agents and users [21]. Howard and Gengler
showed that EC can transit the customer’s attitude for products[8]. In HAI field, we showed that EC from the virtual
agent to the user increase the agent’s trustworthy[12]. These
researches showed that EC was important to using the virtual
agents in real world. However, it is not clear which parametersof virtual agents affect EC. We aimed to reveal the factor important to EC by constructing two kinds of two-dimensionalmodel.

• Pleasure: How capable of feeling pleasure do you think thisagent is?

Gray et al. suggested two axes of mind perception, “agency”
and “experience” [4]. We used this model in this paper. In
addition, we aimed to construct new axes. Takahashi et al.
suggested “emotion” and “intelligence” axises of mind perception, however, 2 out of the 7 agents used were humanoid in thisresearch[18]. We hypothesized that “familiarity” and “reality”
axes can be used to construct two kinds of two-dimensionalmodel. Familiarity was reported to be an important factor increating rapport [11]. In the uncanny valley theory, increasing reality reduces familiarity [14]. This theory suggests that“familiarity” and “reality” are dependent of each other. We
aimed to investigate this hypothesis.

• Memory: How capable of remembering do you think thisagent is?
• Hunger: How capable of feeling hunger do you think thisagent is?
• Moral: How capable of acting morally do you think thisagent is?
Gray et al. [5] defined fear, pleasure, and hunger as factorsof agency and self control, memory, and moral as factors ofexperience.
The other set was constructed of six of our own questions.
These questions aimed to verify the hypothesis that humanoid
agents contained a “reality” dimension and “familiarity” dimension. The questions are as follows.
• Familiarity: How capable of feeling familiarity do you think
this agent is?
• Warmth: How capable of feeling warmth do you think thisagent is?
• Communication: How capable of feeling communication
possibilities do you think this agent is?
• Alive: How capable of feeling aliveness do you think thisagent is?

EXPERIMENT

We conducted an experiment on the web. We used Yahoo
crowd sourcing1 to recruit the participants and conducted theexperiment. We recruited 200 Japanese participants, 122 males
and 78 females. The age distribution was 2 people under 19
years of age, 16 people aged 20-29, 68 aged 30-39, 70 aged
40-49, 37 aged 50-59, and 7 over the age of 60.

• Human-likeness: How capable of feeling human-likenessdo you think this agent is?
• Reality: How capable of feeling reality do you think thisagent is?

We used eight virtual agents. Caucasian agents were chosenfrom FaceRig2 , which is software that mimics a user’s facial
expression and performs lip-synching. Asian agents werechosen from FaceRig+Live2D Module3 , which is a modulethat adds Japanese agents to FaceRig. We accommodated threefactors: gender, race and ethnic group, and abstraction. For
each factor, we defined binary states, that is, male or female,
Caucasian or Asian, and low abstraction or high abstraction.
All agents are shown in Table 1.

We hypothesized that familiarity, warmth, and communication
are factors of familiarity and alive, human-likeness, and reality
are factors of reality.
We conducted an explanatory factor analysis (EFA) to derive
the two kinds of two-dimensional model. EFA is widely used
to analyze variables observed from latent variables [20]. We
conducted EFA with the maximum likelihood method andvarimax rotation. We set the number of factors to two because
our aim was to construct two kinds of two-dimensional model.
In addition to these questions, we asked about the effect ofEC.

The participants were asked to answer all questionnaire inexperiment with seven point Likert scale; 1 was “Strongly
disagree” and 7 was “Strongly agree”. We used two sets ofquestionnaire. One set was cited from Gray et al. [5]. The
questions are as follows.

• Do you feel happy when you see this agent?
The average of this question is the EC score. Facial recognitionand physiological signals may be better; however, Tsai et al.
[21] used only questions to detect EC.

• Fear: How capable of feeling fear do you think this agent
is?

RESULT
1 https://crowdsourcing.yahoo.co.jp/special/owner

We described humanoid agents by using <gender, racial andethnic group, and abstraction>, for example, <male, Asian,
low abstraction>.

2 http://store.steampowered.com/app/274920/FaceRig/?l=japanese
3 http://staff.live2d.com/archives/55477552.html

312Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

genderrace and ethnic group

abstraction

Caucasian

low

male

female

high

Asian

low

high

Table 1. Agents used in experimentquestion
hungerpleasurefear
memorymoral
self control

factor 1 (agency)
0.84310.82200.74680.68210.6517
0.5214questionfamiliarity
warmthcommunication
alive
human-likenessrealityfactor 2 (experience)
0.5336
0.56370.62870.71510.73150.8533

Table 2. Factor loading of “agency”-“experience” questions

factoragencyexperience

eigenvalue
3.10602.7723factor 1 (familiarity)
0.93590.87020.84150.55640.45020.3498factor 2 (reality)
0.35240.48250.53500.83090.86050.9066

Table 4. Factor loading of “familiarity”-“reality”

contribution rate
51.77
46.21

Figure 1 shows a two-dimensional model of “agency” and“experience.” Each plot represents each agent, and the numbers
mean the order of EC scores. The x-axis means the “agency”
factor score, and the y-axis means the “experience” factor
score.

Table 3. Eigenvalue and contribution rate of “agency”-“experience”

Experience-Agency dimension

Reality-Familiarity dimension

Table 2 shows the result of factor loading. We defined factor 1
as “agency” and factor 2 as “experience.” This result complies
with that of Gary et al. [4]. Table 3 shows the eigenvalue
and contribution rate. Thus, we suggest that the “agency” and“experience” axes existed in this experiment.

Table 2 shows the result of factor loading. We defined factor
1 as “familiarity” and factor 2 as “reality.” The result complies with our hypothesis. Table 5 shows the eigenvalue andcontribution rate. Thus, we suggest that the “familiarity” and“reality” axis existed in this experiment.

313

Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

Figure 1. “Agency”-“experience” model and order of EC scores

factorfamiliarity
realityeigenvalue
2.97592.8959contribution rate
49.60
48.26

factoragencyexperiencefamiliarity
reality

Table 5. Eigenvalue and contribution rate of “familiarity”-“reality”

Figure 2 shows a two-dimensional model of “familiarity” and“reality.” Each plot represents each agent, and the numbers
means the order of EC scores. The x-axis means the “familiarity” factor score, and the y-axis means the “reality” factor
score.
Emotional contagion

Figure 3 shows the EC score of each agent. We conducteda one-way ANOVA for this result. There were significant differences (p < 0.01). From sub-effect tests using
Bonferroni method, there were significant differences between <male, Asian, low abstraction> and <male, Asian,high abstraction> (p < 0.01), <female, Caucasian, lowabstraction> and <female, Caucasian, high abstraction> (p< 0.01), <female, Asian, low abstraction> and <female,
Asian, high abstraction> (p < 0.01), <male, Caucasian,
low abstraction> and <male, Asian, low abstraction> (p <0.01), <female, Caucasian, low abstraction> and <female,
Asian, low abstraction> (p < 0.01), <female, Caucasian,high abstraction> and <female, Asian, high abstraction> (p< 0.01), <male, Caucasian, low abstraction> and <female,
Caucasian, low abstraction> (p < 0.01), <male, Asian, lowabstraction> and <female, Asian, low abstraction> (p <0.01), and <male, Asian, high abstraction> and <female,
Asian, high abstraction> (p < 0.01)
Table 6 shows the correlation between EC scores and factor
scores.

314

correlation coefficient
0.55880.73490.92690.3383p−valuep>0.05p<0.05p<0.01p>0.05

Table 6. Correlation coefficient between EC score and factor score foreach factor

Discussion
Two-dimensional models

From figure 1, we reinforce Gray’s two-dimensional model.
This model can be used to classify humanoid agents. From
figure 1, we suggest that the abstraction of agents affects both“agency” and “experience.”
Figure 2 reinforces our hypothesis that agents have the factorsof “familiarity” and “reality.” Table 4 shows that our questions
fulfilled our aim. Figure 2 shows a distribution different fromthat of figure 1. This means our questions can reveal an agent’s
factors that cannot be revealed by Gray’s questions.
Two-dimensional models and emotional contagions

Figure 3 shows that the effect of EC depends on an agent’s
appearance. Table 6 shows that a particular factor affects theEC effect. In the “agency”−”experience” model, the experience factor affects EC. Among all agents, the <female, Asian,
low abstraction> agent had the highest EC score. In figure 1,
the <female, Asian, low abstraction> agent had both a high“agency” factor score and “experience” factor score. In contrast, the <male, Caucasian, low abstraction> agent had ahigher “agency” factor score than did the <female, Asian, lowabstraction> agent; however, he had a low “experience” factor
score and EC score.

Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

Figure 2. “Familiarity”-“reality” model and order of EC scores

In the “familiarity”−”reality” model, the familiarity factor
affected EC, as shown in table 6. In figure 2, we can see thatthe “familiarity” axis directly affected the EC score.
From these results, we suggest a method for designing agents.
If we aim to bring EC from agents to users, we had better
design agents that have “experience” and “familiarity.” From
this experiment, <female, Asian, low abstraction> is the best.
In particular, abstraction is the most important factor for EC.
However, this experiment had structural problems. We experimented with only Japanese participants. This may have biased
how they perceived the variation in agents, especially the raceand ethnic groups. Conducting cross-cultural experiments is
our future work.
CONCLUSION

In this paper, we verified two kinds of two-dimensional mind
perception models. The first model was suggested by Grayet al. [4]. We aimed to apply this model to distinguishinghumanoid agents. The second model was our original model.
This model focused on “familiarity” and “reality.” As a result,
we showed that these two models were effective for distinguishing humanoid agents.

Figure 3. Average of EC(emotional contagion) score for each agent. Error bar means dispersion.

In addition, we verified the relationship between the effectof emotional contagion with these models. As a result, we
showed that “experience” and “familiarity” factors are important to emotional contagion.
These results suggest ideas for designing agents and reveal ourmechanism of mind perception toward humanoids.

315

Session 13: Virtual Agent Applications

HAI 2017, October 17–20, 2017, Bielefeld, Germany

ACKNOWLEDGMENTS

This study was partially supported by JSPS KAKENHI “Cognitive Interaction Design" (No. 26118005).
REFERENCES

1. Mike Blow, Kerstin Dautenhahn, Andrew Appleby,
Chrystopher L Nehaniv, and David Lee. 2006. The art ofdesigning robot faces: Dimensions for human-robot
interaction. In Proceedings of the 1st ACM
SIGCHI/SIGART conference on Human-robot interaction.
331–332.

11. Gerrard Macintosh. 2009. The role of rapport inprofessional services: antecedents and outcomes. Journalof Services Marketing 23, 2 (2009), 70–78.
12. Tetsuya Matsui and Seiji Yamada. 2016. Building Trustin PRVAs by User Inner State Transition through AgentState Transition. In Proceedings of the FourthInternational Conference on Human Agent Interaction.
111–114.
13. Scott McCloud. 1993. Understanding Comics. 1993. New
York (1993).

2. Brian R Duffy. 2003. Anthropomorphism and the social
robot. Robotics and autonomous systems 42, 3 (2003),
177–190.

14. Masahiro Mori, Karl F MacDorman, and Norri Kageki.
2012. The uncanny valley [from the field]. IEEE Robotics& Automation Magazine 19, 2 (2012), 98–100.

3. Arjan Geven, Johann Schrammel, and Manfred Tscheligi.
2006. Interacting with embodied agents that can see:How vision-enabled agents can assist in spatial tasks. In
Proceedings of the 4th Nordic Conference on
Human-Computer Interaction. 135–144.

15. Jeunese Payne, Andrea Szymkowiak, Paul Robertson,
and Graham Johnson. 2013. Gendering the machine:
Preferred virtual assistant gender and realism in
self-service. In International Workshop on Intelligent
Virtual Agents. 106–115.

4. Heather M Gray, Kurt Gray, and Daniel M Wegner. 2007.
Dimensions of mind perception. Science 315, 5812
(2007), 619–619.

16. Tabitha C Peck, Sofia Seinfeld, Salvatore M Aglioti, and
Mel Slater. 2013. Putting yourself in the skin of a black
avatar reduces implicit racial bias. Consciousness andcognition 22, 3 (2013), 779–787.

5. Kurt Gray, Adrianna C Jenkins, Andrea S Heberlein, andDaniel M Wegner. 2011. Distortions of mind perception
in psychopathology. Proceedings of the National
Academy of Sciences 108, 2 (2011), 477–479.
6. Rosanna E Guadagno, Jim Blascovich, Jeremy N
Bailenson, and Cade Mccall. 2007. Virtual humans andpersuasion: The effects of agency and behavioral realism.
Media Psychology 10, 1 (2007), 1–22.
7. Elaine Hatfield, John T Cacioppo, and Richard L Rapson.
1994. Emotional Contagion. Cambridge University Press.
8. Daniel J Howard and Charles Gengler. 2001. Emotional
contagion effects on product attitudes. Journal ofConsumer research 28, 2 (2001), 189–201.
9. Yanghee Kim, Amy L Baylor, and E Shen. 2007.
Pedagogical agents as learning companions: the impactof agent emotion and gender. Journal of Computer
Assisted Learning 23, 3 (2007), 220–234.

17. Brent Rossen, Kyle Johnsen, Adeline Deladisma, Scott
Lind, and Benjamin Lok. 2008. Virtual humans elicitskin-tone bias consistent with real-world skin-tone biases.
In Intelligent virtual agents. 237–244.
18. Hideyuki Takahashi, Midori Ban, and Minoru Asada.
2016. Semantic Differential Scale Method Can Reveal
Multi-Dimensional Aspects of Mind Perception.
Frontiers in Psychology 7 (2016).
19. Kazunori Terada, Liang Jing, and Seiji Yamada. 2015.
Effects of Agent Appearance on Customer Buying
Motivations on Online Shopping Sites. In Proceedings ofthe 33rd Annual ACM Conference Extended Abstracts on
Human Factors in Computing Systems. 929–934.
20. Bruce Thompson. 2004. Exploratory and confirmatory
factor analysis: Understanding concepts andapplications. American Psychological Association.

10. Nicole C Krämer, Bilge Karacora, Gale Lucas, Morteza
Dehghani, Gina Rüther, and Jonathan Gratch. 2016.
Closing the gender gap in STEM with friendly male
instructors? On the effects of rapport behavior and genderof a virtual agent in an instructional interaction.
Computers & Education 99 (2016), 1–13.

21. Jason Tsai, Emma Bowring, Stacy Marsella, Wendy
Wood, and Milind Tambe. 2012. A Study of Emotional
Contagion with Virtual Characters. In Proceedings of
12th International Conference on Intelligent Virtual
Agents. 81–88.

316