Building Trust in PRVAs by User Inner State Transition
through Agent State Transition
Tetsuya Matsui
National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japan
tmatsui@nii.ac.jp

Seiji Yamada
National Institute of Informatics/Sokendai/
Tokyo Institute of Technology
2-1-2 Hitotsubashi, Chiyoda, Tokyo
101-8430, Japanseiji@nii.ac.jp
Moon’s study focused on the trust that was constructed within
the interchange between users and computers. It takes a longtime to construct this trust. In this paper, we aimed for users’
trust to be constructed by only perceiving product informationthat PRVAs gave. This trust needs little time and simple technology. We aimed for user trust to be constructed by PRVAs
expressing agent motion and utterance.

ABSTRACTIn this research, we aim to suggest a method for designing
trustworthy PRVAs (product recommendation virtual agents).
We define an agent’s trustworthiness as being operated by user
emotion and knowledgeableness perceived by humans. Also,
we suggest a user inner state transition model for increasing
trust. To increase trust, we aim to cause user emotion to transition to positive by using emotional contagion and to cause user
knowledgeableness perceived to become higher by increasing
an agent’s knowledge. We carried out two experiments toinspect this model. In experiment 1, the PRVAs recommended
package tours and became highly knowledgeable in the latter
half of ten recommendations. In experiment 2, the PRVAs recommended the same package tours and expressed a positive
emotion in the latter half. As a result, participants’ inner statestransitioned as we expected, and it was proved that this model
was valuable for PRVA recommendation.

USER TRUST STATE TRANSITION MODEL AND USERSTATE TRANSITION OPERATORS
User Trust State Transition Model

We describe a user trust state as the user emotion state andknowledgeableness perceived state. Our basic hypothesis isthat “positive emotion and high knowledgeableness perceived
brings high trust."
Dunn and Schweizer showed that how much people trust
their partners depended on familiarity and emotion [?]. They
showed that positive people tended to trust partners whenpartners were unfamiliar with the truster. Myers and Tingley
studied negative emotion [?] . They showed negative people trust their partners less than positive people. From thesestudies, the emotion state seemed to affect the trust state.

ACM Classification Keywords

H.5.2. User Interfaces
Author Keywords

product recommendation virtual agent, user inner state,emotional contagion, trustworthy, anthropomorphic agent

We aimed to cause a user’s emotion state to transition by using
emotional contagion. Emotional contagion is a phenomenon
in which a speaker’s emotion is spread to a partner [?]. Tsai etal. showed that emotional contagion can be caused between
users and virtual anthropomorphic agents [?].

INTRODUCTION

PRVAs, product recommendation virtual agents, are agentsthat take the role of clerks and advisers in online stores. In
this paper, we experimented to design trustworthy PRVAs to
increase users’ buying motivation. Terada et al. showed thatthe appearance of PRVAs affected the recommendation result[?]. Kamei et al. experimented with robot clerks and showedthat customers feel more familiarity with robots who navigated them toward the store rather than robots that only recommended lunch [?]. Moon et al. showed that self-disclosure,exchanging information with each other, induced users to give
their information to computers [?].

“Knowledgeable" is one aspect of intelligence. Geven et al.
showed that users perceived more intelligence and trustworthiness with real agents rather than cartoon-like agents [?]. Also,
Mimoun et al. indicated that a lack of intelligence was one of
the most crucial problems with anthropomorphic agents [?].
In our experiment, we focused on knowledgeableness, one
aspect of intelligence. From these prior pieces of research,
it is clear that positive emotion and high knowledgeablenessperceived brings high trust.

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.

In our model, we described a user’s emotion state and knowledgeableness perceived state as { +} and { − }. State { +}means a positive or high state, and state { − } contains anegative, low, and neutral state. We defined a user’s basic
state to be { − − }. The left value means the emotion state,
and the right value means the knowledgeableness perceived
state. Thus, state { − − } means a negative or neutral emotion

HAI ’16, October 04-07, 2016, Biopolis, Singapore© 2016 ACM. ISBN 978-1-4503-4508-8/16/10. . . $15.00
DOI: http://dx.doi.org/10.1145/2974804.2974816

111

Figure 1. User inner state transition model

Figure 2. Agent with positive emotion transition operators and agent
without operators (neutral emotion)and low or neutral knowledgeableness perceived. If a user’s
emotion transitions to positive, the user state transitions to {+ − }. If a user’s knowledgeableness perceived transitions tohigh, the user state transitions to { − + }.

Also, we executed smooth utterance with text to speech software, VOCELOID+ Yuzuki Yukari EX2 . Figure ?? shows theagent with and without emotion transition operators.
The PRVAs recommended package tours to Japanese Middle
Ages castles. They recommended ten tours by taking turns,
and the destination changed for each recommendation. The
order of destinations was random for each participant. We
defined each recommendation as Rn, and n means the recommendation number (1 to 10). Each recommendation took nolonger than one minute.

Our goal state was { + + }, that is, positive emotion and highknowledgeableness perceived. Users having this state seem totrust PRVAs more than when having the other three states. We
show this model in Figure ??.
Positive emotion and high knowledgeableness are instinctively
effective for PRVAs or any agent; however, it is not clear
whether positive emotion and high knowledgeableness work
together. In this paper, we inspected this point.

From R1 to R5, the PRVAs executed only positive emotion
operators. The PRVAs made recommendations with smiles
and cute gestures; however, their utterances contained only
location and expense information. Also, from R6 to R7, thePRVAs executed positive emotion transition operators and highknowledgeableness perceived transition operators. They maderecommendations with smiles, cute gestures, and knowledge
on location, expense, history, and architecture. Thus, fromour hypothesis, from R1 to R5, the participants’ state was { +− }, positive emotion and low knowledgeableness perceived.
Also, from R6 to R7, the participants’ state was { + + }. Thus,
knowledgeableness perceived transitioned to high.

User State Transition Operators

The aim of this research was to increase users’ trust by causing
only the agent state to transition. Thus, positive emotion andhigh knowledgeableness perceived were executed with only
the agent state. Positive emotion was executed by “agent’s
smile" and “cute gesture" in order to cause positive emotional
contagion [?]. High knowledgeableness perceived was executed by “much product knowledge."
We carried out two experiments to prove that these transition
operators were effective in user inner state transition and that
our transition model was valuable for designing PRVAs.

After watching each recommendation, participants were asked
to answer some questions. In this paper, we focus on the result
for two of the questions as follows.

EXPERIMENT 1: EXECUTING HIGH KNOWLEDGEABLENESS PERCEIVED• Q1: Did you feel happy when you watched the movie?

Experiment 1: Materials and Method
Participants• Q2: Did you feel that the agent has correct knowledge?

Fifteen Japanese participants were recruited for experiment
1. They were aged between 20 and 39, for an average of 29.0
and SD was 5.7. There were seven males and eight females;
however, one female’s data was removed from analysis because of machine trouble. Each participant spent about 20 to
30 minutes for the whole experiment.

• Q3: Did you feel that the agent have considerable persuasive
power?
Question Q1 was the indicator of the user’s emotion, and Q2
was the indicator of the user’s knowledgeableness perceived.
Q3 was the indicator of the agent’s trustworthy perceived byuser. All participants were asked to answer “Yes" or “No" forthese questions. Also, all participants were asked to answer
the same questions before recommendations after watching
PRVAs standing without any motion and utterance.

Task

The PRVAs and recommendation systems were constructedwith MMDAgent 1 , the free dialogue agent “Mei," and a platform that is distributed by the Nagoya Institute of Technology.
1 http://www.mmdagent.jp/

2 http://www.ah−soft.com/voiceroid/yukari/

112

Figure 3. Ratio of participants’ answers for Q1 and Q2 in experiment 1

Figure 4. Ratio of participants’ answers for Q1 and Q2 in experiment 2

Experiment 1: Result

perceived. Also, from R6 to R7, the participants’ state was { ++ }. Thus, emotion transitioned to positive.

We conducted a statistical analysis based on the ratio of thenumber of participants who answered “Yes" for each question. The x−axis indicates each questions. We conducted achi-square test for every successive recommendations. If there
were significant differences, the number of users whose innerstate transitioned increased significantly between those tworecommendations. For the result of Q1, there were no significant differences. For the result of Q2, there was a significant
difference between R5 and R6 (p<0.05).

Experiment 2: Result

We conducted the same analysis as in experiment 1. For the
result of Q1, there was a significant difference between R5 andR6 (p<0.05). For the result of Q2, there were no significant
differences.
The left graph of Figure ?? means the ratio of the numberof participants who answered “Yes" for Q1 after R5 and R6.
The right graph of Figure ?? means the ratio of the number ofparticipants who answered “Yes" for Q2 after R5 and R6.

The left graph of Figure ?? means the ratio of the numberof participants who answered “Yes" for Q1 after R5 and R6.
The right graph of Figure ?? means the ratio of the number ofparticipants who answered “Yes" for Q2 after R5 and R6.

Regarding Q3, we conducted a chi-square test between the
average of the ratio of the participants who answered “Yes" inthe first five recommendations and the ratio of the participants
who answered “Yes" in the latter five recommendations. As aresult, there was a significant difference between R5 and R6(p<0.05).

Regarding Q3, we conducted a chi-square test between the
average of the ratio of the participants who answered “Yes" inthe first five recommendations and the ratio of the participants
who answered “Yes" in the latter five recommendations. As aresult, there was a significant difference between R5 and R6(p<0.01).

DISCUSSION

The result shows that our hypothesis and model is proper. The
right graph of Figure ?? shows that our knowledgeablenessperceived transition operators worked according to our assumption. This graph shows that many participants felt thatthe agents imparted historical and architectural knowledge wasmore knowledgeable than agents who imparted only locationand expense information. This result seems to be axiomatic;
however, historical and architectural knowledge seemed to
not be more important information than location and expense
information for many users. It was possible that participants
judged the knowledgeable PRVAs as being redundant. This
result contradicted this expectation. Also, this result seems
to suggest that an “informative" agent is perceived knowledgeable/intelligent. However, knowledgeable PRVAs may
not only impart many pieces of knowledge but also proper
information to users.

EXPERIMENT 2: EXECUTING POSITIVE EMOTION
Experiment 2: Materials and Method
Participants

Fifteen Japanese participants were recruited for experiment 2.
They were aged between 20 and 39, for an average of 29.3 andSD is 6.9. There were eight males and seven females. Eachparticipant spent about 30 minutes for the whole experiment.
Task

The PRVAs, recommendation products, recommendation format, and questions for experiment 2 were the same as in experiment 1. All participants watched the recommendations for
package tours to Japanese castles. The difference was what
transition operators were executed for each recommendation.
From R1 to R5, the PRVAs executed only high knowledgeableness perceived operators. The PRVAs’ recommendations
contained historical and architectural knowledge; however,they stayed expressionless without making any gestures. Also,from R6 to R7, the PRVAs executed high knowledgeablenessperceived transition operators and positive emotion transition
operators. They made recommendations with historical andarchitectural knowledge, smiles, and cute gestures. Thus, fromour hypothesis, from R1 to R5, the participants’ state was { −+ }, negative or neutral emotion and high knowledgeablenessThe left graph of Figure ?? shows that our emotion transition
operators worked according to our assumption. This graphshows that smiles and cute gestures were effective for emotional contagion between agents and users. This effect was
already reported for 3D game characters and users [?] androbots and users [?]. However, there are scarcely any studies
about emotional contagion between PRVAs and users. This
result suggests that emotional contagion is valid for PRVAs to
increase their own trustworthyness.

113

CONCLUSION

In this research, we suggested a model of user inner state transition regarding trustworthiness and experimented to inspect
this model. This model supposes that the user inner state canbe operated by only changing the agent state. The result ofexperiments showed that this hypothesis was proper and that
we can cause user the inner state to transition in an arbitrary
phase. The emotion transition operators and the knowledgeableness perceived transition operators immediately worked
after executing operators. Also, trustworthiness can be increased by only an agent’s positive emotion and knowledge
without complex interactions.
This result suggests a valuable new design method for PRVAs.
This method does not need any particular equipment and prerecommendation phase. Also, this result suggests that complex
social relationships can be constructed without bidirectional
communications.

Figure 5. Observed transition paths in two experiments

ACKNOWLEDGMENT

This study was partially supported by JSPS KAKENHI "Cognitive Interaction Design" (No.26118005).
The left graph of Figure ?? and the right graph of Figure ??
show no significant differences. However, in both figures, theratio of R5 showed high values. The participants’ emotion
state kept at { + } between R5 and R6 in experiment 1, andtheir knowledgeableness perceived state kept at { + } between
R5 and R6 in experiment 2.

REFERENCES
1. Jennifer R Dunn and Maurice E Schweitzer. 2005. Feeling and believing:
the influence of emotion on trust. Journal of personality and social
psychology 88, 5 (2005), 736.
2. Arjan Geven, Johann Schrammel, and Manfred Tscheligi. 2006.
Interacting with embodied agents that can see: how vision-enabled
agents can assist in spatial tasks. In Proceedings of NordiCHI 2006.
135–144.

The result of Q3 shows the emotion state and knowledgeableness perceived state transitions effected to trustworthy state.
When emotion state and knowledgeableness perceived statetransited to { + }, trustworthy state also transited to positive.

3. Elaine Hatfield, John T Cacioppo, and Richard L Rapson. 1994.
Emotional contagion. Cambridge university press.
4. Koji Kamei, Tetsushi Ikeda, Masahiro Shiomi, Hiroyuki Kidokoro,
Akira Utsumi, Kazuhiko Shinozawa, Takahiro Miyashita, and Norihiro
Hagita. 2012. Cooperative customer navigation between robots outside
and inside a retail shop?an implementation on the ubiquitous market
platform. annals of telecommunications-annales des télécommunications
67, 7-8 (2012), 329–340.

From these result, we constructed two different participants’transition paths as shown in Figure ??. From this, we suggested that the transition operators worked immediately foruser inner state transition. In general, long and complex interaction (for example, self-disclosure [?]) was regarded as theessential phase constructing trust or other social relationships
between an agent and user. Our result suggested that trust
could be constructed by only expressing the agent state. Thismethod takes little time and can be executed on finite media,
for example, small screens or simple structure machines.

5. Mohammed Slim Ben Mimoun, Ingrid Poncin, and Marion Garnier.
2012. Case study?Embodied virtual agents: An analysis on reasons for
failure. Journal of Retailing and Consumer services 19, 6 (2012),
605–612.
6. Youngme Moon. 2000. Intimate exchanges: Using computers to elicit
self-disclosure from consumers. Journal of consumer research 26, 4(2000), 323–339.
7. Dan Myers and Dustin Tingley. 2011. The influence of emotion on trust.
Unpublished PhD thesis.) Princeton, NJ: Princeton University (2011).

Also, these results suggested that emotion and knowledgeableness perceived can be operated independently. In bothexperiments 1 and 2, participants’ emotion and knowledgeableness perceived transitioned independently. This is valuable
for constructing agents for varying aims.

8. Hiroki Ozono, Motoki Watabe, Sakiko Yoshikawa, Satoshi Nakashima,
Nicholas O Rule, Nalini Ambady, Reginald B Adams Jr, and others.
2010. What’s in a Smile? Cultural Differences in the Effects of Smilingon Judgments of Trustworthiness. Letters on Evolutionary Behavioral
Science 1, 1 (2010), 15–18.

This research contained some limitations. First, we did notuse biological signals (for example, brain waves, perspiration,and heartbeat) to measure the user inner state. This is our future works. Second, although PRVAs can work passing acrossborders, the notion of trust depends on cultures. Ozono etal. showed that both Japanese and American people judged
partners as being trustworthy by observing their smiles; however, they focused on the other part of smiling faces [?]. This
study suggests that we need to customize an agent’s smilingdepending on the culture.

9. Kazunori Terada, Liang Jing, and Seiji Yamada. 2015. Effects of Agent
Appearance on Customer Buying Motivations on Online Shopping Sites.
In Proceedings of CHI ’15. 929–934.
10. Jason Tsai, Emma Bowring, Stacy Marsella, Wendy Wood, and Milind
Tambe. 2012. A study of emotional contagion with virtual characters. InIntelligent Virtual Agents. Springer, 81–88.
11. Junchao Xu, Joost Broekens, Koen Hindriks, and Mark A Neerincx.
2015. Mood contagion of robot body language in human robot
interaction. Autonomous Agents and Multi-Agent Systems 29, 6 (2015),
1216–1248.

114