25th IEEE International Symposium on
Robot and Human Interactive Communication (RO-MAN)
August 26-31, 2016. Columbia University, NY, USA

Emotional Contagion between User and Product Recommendation
Virtual Agent
Tetsuya Matsui1 and Seiji Yamada1,2,3
Abstract— The notion of Emotional contagion is a phenomenon in which a human emotion infects others. Various
studies have been done to cause emotional contagion between
a human and a robot or an anthropomorphic agent in HRI
research ﬁelds. However, few studies have been done to compare
different kinds of agents in order to ﬁnd important properties
in emotional contagion in human-agent interaction (HAI). In
this research, we conducted an experiment to determine which
properties cause this phenomenon between anthropomorphic
agents and users. We prepared two kinds of agents. One isa cartoon-like agent, and the other is a concrete agent. The
cartoon-like agent smiled exaggeratedly, and the concrete agent
smiled modestly. As a result, we found that the concrete agent
was more effective than the cartoon-like agent at emotional
contagion. This result suggests a model for designing moretrustworthy and familiar agents and robots.

I. INTRODUCTION
Emotional contagion is a phenomenon in which aspeaker’s emotion infects his or her partner during conversation [1]. Hatﬁeld et al. showed that this phenomenon iscaused not only by a partner’s utterances but also by their
non-verbal cues [1]. Kim et al. showed that positive emotions
have the tendency to infect partners even in character-based
conversations on Twitter [2]. This effect must be caused byconversational text. However, in face-to-face communication,
non-verbal cues are more important [1]. Many cues were
investigated after the report by Hatﬁeld et al., for example,
facial expression mimicry [3], vocal expression of voice [4],and whole body movement [5]. Through these studies, it
was revealed that these factors are important for emotional
contagion.
In the research areas of HAI and HRI, emotional contagion between a user and an agent or a robot was studied
through experiments with participants. Tsai at al. showedthat 3D visual game characters can infect users with theiremotion through facial expression [6]. In the area of HRI,
Xu et al. showed the humanoid robot NAO can express andinfect users with his mood [7]. From these studies, agents,
including anthropomorphic agents and physical robots, caninfect users with their positive emotions, which is useful for
constructing a positive interaction between users and agents.
Dunn and Schweizer showed that people trust unfamiliar
persons when they expressed a positive emotion [8], andDruckman and McDermott showed that people judged risky
1 National Institute of Informatics 2-1-2 Hitotsubashi Tokyo 101-8430,
Japan {tmatsui,seiji}@nii.ac.jp
2 The Graduate University for Advanced Studies (SOKENDAI) 2-1-2
Hitotsubashi, Tokyo 101-8430, Japan
3 Tokyo Institute of Technology 2-12-1 Ookayama Meguro-ku, Tokyo,
Japan

978-1-5090-3928-9/16/$31.00 ©2016 IEEE

1172

choices by their positive and negative valences [9]. Barsade
showed a positive emotional contagion among group members was useful for group work [10]. Pugh showed that apositive emotion contagion between customers and employees causes customers to evaluate service quality highly [11].
In this paper, we investigate emotional contagion between auser and product recommendation virtual agent (PRVA).
Our ﬁnal goal is to make users evaluate the PRVA’s
service as having high quality. Terada et al. showed that ayoung woman agent is one of the most effective agents forincreasing a user’s buying motivation [12]. Thus, we used
two young women agents and compared a concrete agent
and a cartoon agent. Geven at al. showed that interaction
with a concrete anthropomorphic agent is perceived to be
more concrete than that with a cartoon-like agent [13].
It has not been clariﬁed which properties affect emotional
contagion from PRVAs to users in on-line shopping. Thus,
we aimed to experiment to show that concreteness is an
important property for the emotional contagion of PRVAs.
II. E XPERIMENT: M ATERIALS AND M ETHOD
We recruited 30 Japanese participants for this experiment.
The participants were 15 males and 15 females, and they
were between the ages of 20 and 39, for an average of 30.0
(S.D. = 6.3). They were recruited for this experiment. We
arranged participants into two groups. Each group contained
15 participants. Group 1 contained seven males and eightfemales, and they were between the ages of 21 and 39, foran average of 30.7 (S.D. = 5.6). Group 2 contained eightmales and seven females, and they were between the ages of
20 and 39, for an average of 29.3 (S.D. = 6.9).
We designed two series of recommendation movies foreach groups. Both series were contained 10 successive
recommendations. All recommendations were presented byvirtual agents executed by MMDAgent1 , distributed by the
Nagoya Institute of Technology.
In the movie series shown for group 1, the PRVA had long
black hair and wore a black suit. This model2 and suit3 is
distributed by one of the general users. The model is called“Mobuko-san” by users. The model is shown on the left ofFig. 1 and is a cartoon-like agent. She has big eyes and cutesmile but looks to be constructed with only a few polygons.
She does not appear very concrete and has a high level ofabstraction.
1 http://www.mmdagent.jp/
2 https://bowlroll.net/ﬁle/3706
3 http://bowlroll.net/up/dl43967

TABLE I
R ECOMMENDATION TEXT
Text

Fig. 1.

Well, I recommend that you take a trip to
Gihu castle.
Gihu castle stands in Gihu city, Gihu Prefecture.
This castle stands on a mountain, so you can geta nice view from the castle tower.
In the Warring State Period, the Saito family lived in
the castle and reigned over the prefecture.
In particular, Saito Dozan is well-known forbeing Oda Nobunaga’s father-in-law
We are currently selling a package tour to thiscastle for only 21,060 yen, which covers theadmission fee to the castle.
The package tour also contains
travel expenses and lunch.
If you miss this chance, you will surely regret it.

“Mobuko-san” and “Mei”

For group 2, the PRVA was “Mei,” distributed by the
Nagoya Institute of Technology. She has short hair and wears
a warm colored suit. She looks like a 3D game character.
She looks to be constructed elaborately and moves smoothly.
This model is shown on the right of Fig. 1. She appears
more concrete and has a low level of abstraction. Also,
we use VOCELOID+ Yuduki Yulari EX4 , which is text tospeech software, for both agents in order to execute smoothutterances.
The PRVAs recommended the same ten package tours
to Japanese castles at random. All castles are tourist destinations in Japan. For the recommendations, they did notshow any facial expressions and gestures from the ﬁrst
recommendation to the ﬁfth. We deﬁned these PRVA states
as the neutral states. From the sixth recommendation to thetenth, the agents smiled and made cute gestures. We deﬁnedthese PRVA states as happy states. Fig. 1 shows the neutral
states of the two models. In Fig. 2 and Fig. 3, both models
are in happy states. The left-most pictures of Fig. 2 andFig. 3 show the agents smiling. The middle pictures show theagents introducing the tours. The right-most pictures show
the agents are making cute gestures. We executed different
gestures for the two agents because more overstated gestures
seemed to have more of an effect on users than did small
gestures. In Fig 2, the model moves her shoulder. In Fig 3,
the model strikes a victory pose with both arms. Although
the recommendation utterances are slightly different between
the two PRVAs, the utterances of both contained historicalepisodes of each castles. This means both agents expressed
the same intelligence and familiarity in the utterances. Noneof the movies were more than one minute. Fig. 4 shows one
screenshot of a recommendation made by Mei. Table I shows
the recommendation text spoken by Mei.
The participants were required to watch the movie series
and answer two questions after watching each movie:
Q1: Do you feel happy?
Q2: Do you feel at home?

Fig. 2.

PRVR, “Mobuko-san,” shown to group 1

Fig. 3.

Fig. 4.

PRVR, “Mei,” shown to group 1

Screenshot of “Mei” recommending package tour

III. R ESULTS

The participants were required to answer these questions
with “YES” or “NO.” In addition to this, before all therecommendations, the participants were required to watch ashort movie showing the agent only standing without making
any smiles, gestures, and utterances and answer the same twoquestions. We called these “pre-recommendation questions.”
4 http://www.ah-soft.com/voiceroid/yukari/1173

We indicated the calculation of the t-test between the ﬁrst
ﬁve recommendations and the latter ﬁve recommendations.
We compared number of participants that answered “Yes”
after each recommendations. Fig. 5, Fig. 6, Fig. 7 andFig. 8 shows these results. The error bars in these graphsmeans standard deviation. In Fig. 6, Fig. 7 and Fig. 8 shows
signiﬁcant differences (p<0.01).
In the upper graphs of Fig. 9, Fig. 10, Fig. 11, and Fig. 12,
the x-axis indicates pre-recommendation and R-n (the nth

Fig. 5. Average of number of participants in group 1 answering “YES” to
Q1 for each recommendations

Fig. 8. Average of number of participants in group 2 answering “YES” to
Q2 for each recommendations

Fig. 6. Average of number of participants in group 1 answering “YES” to
Q2 for each recommendations

recommendation), and the y-axis indicates the frequency
ratio of the participants who answered “Yes.” We analyzed
these results by applying a chi-squared test to two ratios of“’Yes” answers for every two consecutive questions.
The lower graphs of these ﬁgures show the participant’s
inner state transitions. We deﬁned that an inner state astransitioning if there is a signiﬁcant difference between thetwo consecutive questions obtained by applying the chisquare test.
Fig. 9 and Fig. 10 show the result of Q1. Fig. 11 andFig. 12 show the result of Q2.
In Fig. 10, there is a signiﬁcant difference between R5

Fig. 7. Average of number of participants in group 2 answering “YES” to
Q1 for each recommendations

1174

Fig. 9. Frequency ratio of participants in group 1 answering “YES” to Q1

and R6 for group 1 (p<0.05). Thus, we determined that theparticipants inner states transitioned between R5 and R6.
In Fig. 9 Fig. 11, and Fig. 12, there was no signiﬁcantdifference.
IV. D ISCUSSION
The result of t-test shows whether user inner state transition happened or not. From Fig. 6, Fig. 7 and Fig. 8, user
inner state transitions happened through 10 recommendations. These result may be caused by emotional contagion,
however, we can’t know when state transitions happened.
Thus, we judged whether transitions caused by transition
operators or not by the result of chi-squared test.
From Fig. 10, we can see that the agent’s positive emotion
infected only participants in group 2. This effect must becaused from the agent’s smile and cute gestures because
the contagion occurred between R5 and R6 when the agent
started to smile and make gestures. However this effect did
not occur in Fig. 9 although the same actions occurred forthis group. This difference is considered to be caused by
the difference in the level of abstraction between these two
agents.

Fig. 10.
Q1Frequency ratio of participants in group 2 answering “YES” to

Fig. 12.
Q2

Frequency ratio of participants in group 2 answering “YES” toof participants in each groups had no differenced in prerecommendation phase.
A. Uncanny Valley

Fig. 11.
Q2

Frequency ratio of participants in group 1 answering “YES” to

Fig. 11 and Fig. 12 show that both agents had no effectthat made participants feel at home. This result shows thatthe signiﬁcant difference that occurred in Fig. 10 was notcaused from feeling at home. This effect must be caused byemotional contagion. The signiﬁcant differences in Fig. 6 andFig. 8 may be caused by only listening recommendations, notemotional contagion. From these results, Mobuko-san failed
to infect positive emotion to users. We proved that the levelof abstraction affect emotional contagion.
However, other interpretations may be happened. Thatis, “the appearances of Mobuko-san caused positive emotion in pre-recommendation phase, thus the effect of emotional contagion was buried”. This hypothesis is rejectedby the ratios of pre-recommendation in Fig. 9 and Fig. 10.
There are almost no differences between the ratios of prerecommendation in Fig. 9 and Fig. 10. Thus, the inner state

1175

In the research ﬁeld of HAI, the uncanny valley hypothesis has been researched for a long time. This hypothesis
states that robots have familiarity as they come to resemble
humans, but those that appear too similar to humans cause
people to feel an uncanny emotion [14]. This hypothesis
has been veriﬁed for a long time. MacDorman shows thatparticipants feel more eerie toward images of a robot morphing into a human than a robot or a human; however, video
clips of anthropomorphic androids do not have this effect[15]. MacDorman insisted that human likeness is not the
only factor for familiarity or eeriness. In our research, Mei
has more of a human likeness than Mobuko-san. The resultthat she was more effective at positive emotion contagion
shows that having a human likeness makes users feel morefamiliarity than eeriness.
For the ﬁrst reason explaining this effect, Mei’s smilingand gesturing was appropriate for the recommendations. She
made an introduction gesture by only moving her right handand an emphatic gesture by striking a victory pose. If we
execute many more and overdone facial expressions andgestures, we cause participants to feel uncomfortable andeerie feelings. For the second reason, Mei’s design seem tobe the halfway point between cartoon characters and concrete
humans or androids. Mei is more of a concrete agent thana cartoon character. However, she has less concreteness thanthe CGI characters in games and movies.
Both of the two agents we used in this experiment smiledduring the recommendations. However, these smiles weredifferent as shown in Fig. 2 and Fig. 3. Mobuko-san smiledwith scrunched eyes. Mei only smiled softly. Mei’s smile hasless distinction from her neutral face than Mobuko-san’s, but
Mei’s smile was more effective than Mobuko-san’s.
This result may be based on the difﬁculty of mimicry.
Hatﬁeld et al. suggested that emotional contagion primarily

stems from facial expression mimicry [1]. We assume thatthe smile of the cartoon-like agent was difﬁcult to mimic
because we cannot scrunch up our eyes like Mobuko-san
did in Fig. 2 . Mei’s smile can be mimicked. Thus, Mei’ssmile was more effective than Mobuko-san’s.
Also, these results may be caused by the difference of thecolor of suits, the color of hair, and the length of hair. One
female participant wrote in a free description column, “Thisagent seemed to be untidy because she has long hair.” Weguess she means long and unbraided hair appears untidy on
clerks. We do not know if this impression is something felt
by the general public or not and whether the color of suits
or other dresses have the same effect. This is our future
research.
B. Implementing robots that can infect humans with emotion
From this experiment, the concreteness, smiling, and thecute gestures were determined to be useful factors for emotional contagion. We suggest implementing these factors inrobots in order to spread positive emotions between robots
and users. We discovered that a more human-like simple
smile is more effective than an exaggerated smile. This ispractical for robots. Also, the gestures are needed only alittle. Only occasionally expressing two kinds of gestures caninfect users with a positive emotion. If we aim to constructrapport between a robot and user, we may need to executemany kinds of facial expression and gestures frequently andsatisfy many other conditions. For example, Tichle-Degnen
et al. suggested that mutual attentiveness, positivity, andcoordination are the three essential factors for constructing
rapport [16], and Huang et al. applied these three factors totheir Rapport Agent [17]. However, infecting only a positive
emotion is very easy. This can occur with only a few patterns
of facial expressions and gestures.
V. CONCLUSION
In this experiment, we investigated the important factors
for emotional contagion between an agent and a user. We
experimented with two kinds of agents that have a differencein concreteness and level of abstraction. As a result of theexperiment, the concrete agent was found to be able to infectparticipants with a positive emotion better than the cartoonlike agent.
The result shows that the effect of emotional contagion
from an anthropomorphic agent depends on the design of
the agent. A concrete agent can infect a positive emotion
better than a cartoon-like agent. Also, our future work is to
discover the factors that are perceived by users when theyjudge how concrete an agent is. In this experiment, the two
agents had differences in the roughness of their surfaces,
gestures, hairstyles, and color of hair and suits. One of them
may be a critical factor. However, these factors express thelevel of concreteness when combined together. Investigating
the relationships between each factor and effect can help
suggest a methodology for designing agents.

1176

R EFERENCES[1] E. Hatﬁeld, J. T. Cacioppo, and R. L. Rapson, Emotional contagion.
Cambridge university press, 1994.
[2] S. Kim, J. Bak, and A. Oh, “Discovering emotion inﬂuence patterns
in online social network conversations by suin kim, jinyeong bak, andalice oh, with ching-man au yeung as coordinator,” ACM SIGWEB
Newsletter, no. Autumn, p. 3, 2012.
[3] U. Hess and S. Blairy, “Facial mimicry and emotional contagion to
dynamic emotional facial expressions and their inﬂuence on decoding
accuracy,” International journal of psychophysiology, vol. 40, no. 2,
pp. 129–141, 2001.
[4] R. Neumann and F. Strack, “” mood contagion”: the automatictransfer of mood between persons.,” Journal of personality and social
psychology, vol. 79, no. 2, p. 211, 2000.
[5] B. De Gelder, J. Snyder, D. Greve, G. Gerard, and N. Hadjikhani,“Fear fosters ﬂight: a mechanism for fear contagion when perceiving
emotion expressed by a whole body,” Proceedings of the National
Academy of Sciences of the United States of America, vol. 101, no. 47,
pp. 16701–16706, 2004.
[6] J. Tsai, E. Bowring, S. Marsella, W. Wood, and M. Tambe, “A studyof emotional contagion with virtual characters,” in Intelligent Virtual
Agents, pp. 81–88, Springer, 2012.
[7] J. Xu, J. Broekens, K. Hindriks, and M. A. Neerincx, “Robot moodis contagious: effects of robot body language in the imitation game,”in Proceedings of the 2014 international conference on Autonomous
agents and multi-agent systems, pp. 973–980, International Foundation
for Autonomous Agents and Multiagent Systems, 2014.
[8] J. R. Dunn and M. E. Schweitzer, “Feeling and believing: the inﬂuenceof emotion on trust.,” Journal of personality and social psychology,
vol. 88, no. 5, p. 736, 2005.
[9] J. N. Druckman and R. McDermott, “Emotion and the framing of risky
choice,” Political Behavior, vol. 30, no. 3, pp. 297–321, 2008.
[10] S. G. Barsade, “The ripple effect: Emotional contagion and its inﬂuence on group behavior,” Administrative Science Quarterly, vol. 47,
no. 4, pp. 644–675, 2002.
[11] S. D. Pugh, “Service with a smile: Emotional contagion in the service
encounter,” Academy of management journal, vol. 44, no. 5, pp. 1018–
1027, 2001.
[12] K. Terada, L. Jing, and S. Yamada, “Effects of agent appearance on
customer buying motivations on online shopping sites,” in Proceedingsof the 33rd Annual ACM Conference Extended Abstracts on Human
Factors in Computing Systems, pp. 929–934, ACM, 2015.
[13] A. Geven, J. Schrammel, and M. Tscheligi, “Interacting with embodied
agents that can see: how vision-enabled agents can assist in spatial
tasks,” in Proceedings of the 4th Nordic conference on Humancomputer interaction: changing roles, pp. 135–144, ACM, 2006.
[14] M. Mori, K. F. MacDorman, and N. Kageki, “The uncanny valley[from the ﬁeld],” Robotics & Automation Magazine, IEEE, vol. 19,
no. 2, pp. 98–100, 2012.
[15] K. F. MacDorman, “Subjective ratings of robot video clips for human
likeness, familiarity, and eeriness: An exploration of the uncanny
valley,” in ICCS/CogSci-2006 long symposium: Toward social mechanisms of android science, pp. 26–29, 2006.
[16] L. Tickle-Degnen and R. Rosenthal, “The nature of rapport and its
nonverbal correlates,” Psychological inquiry, vol. 1, no. 4, pp. 285–
293, 1990.
[17] L. Huang, L.-P. Morency, and J. Gratch, “Virtual rapport 2.0,” inIntelligent virtual agents, pp. 68–79, Springer, 2011.