情報処理学会研究報告
IPSJ SIG Technical Report

Vol.2018-MUS-118 No.2
Vol.2018-SLP-120 No.2
2018/2/20

Positive Emotion Elicitation in
an Example-Based Dialogue System
Nurul Lubis, Sakriani Sakti, Koichiro Yoshino, and Satoshi Nakamura
Nara Institute of Science and Technology
Abstract: An emotion-sensitive dialogue system is highly potential in addressing user emotional
needs through HCI. For example, by providing a general emotional support through positive emotion elicitation. To date, works on emotion elicitation have only focused on the intention ofelicitation itself, but are yet to examine the process that gives rise to the change of emotion in thefirst place. In this paper, we utilize examples of spontaneous human emotional responses to elicit
a positive emotional impact through dialogue system interaction. Text-based human subjective
evaluation with crowdsourcing shows that the proposed dialogue system elicits an overall morepositive emotional impact.

1

Introduction

Recently, there has been an increasing interest ineliciting user’s emotional response [1, 2]. However,emotion appraisal that gives rise to the elicited emotion is not yet studied for this task. That is, the relationship between an utterance, which acts as emotional trigger, and the resulting emotional response.
By examining this, it would be possible to reverse theprocess and determine the appropriate trigger to adesired emotional response.
In this paper, we attempt to elicit a positive emotional change through computer interaction by exploiting examples of emotion appraisal. We collecteddialogue sequences containing emotional triggers andresponses as examples in a dialogue system. Subsequently, we augment the traditional response selection
criterion with emotional parameters: 1) user’s emotional state, and 2) expected future emotional impactof the candidate responses.

2Emotion Definition

In this work, we define the emotion scope based on
the circumplex model of aﬀect [3]. Two dimensionsof emotion are defined: valence and arousal. Valencemeasures the positivity or negativity of emotion; e.g.
the feeling of joy is indicated by positive valence whilefear is negative. On the other hand, arousal measures the activity of emotion; e.g. depression is low inarousal (passive), while rage is high (active). Readersⓒ 2018 Information Processing Society of Japanare referred to [3] for further details of this emotion
model.
Figure 1 illustrates the valence-arousal dimension
in respect to a number of common emotion terms.
High arousal
Tense
NervousStressed

Alert

ExcitedElated
Happy

Upset

Negativevalence Sad

Content

Depressed

Positivevalence

SereneRelaxedLethargic
FatiguedCalm

Low arousal

Figure 1: Emotion dimensions and common terms.

3

Example-based Dialogue Modeling (EBDM)EBDM is a data-driven approach of dialogue modeling that uses a semantically indexed corpus of queryresponse pair examples instead of handcrafted rules orprobabilistic models [4]. At a given time, the system
will return a response of the best example according
to a semantic constraint between the query and example queries.
Lasguido et al. have previously examined the utilization of cosine similarity for response retrieval in
an example-based dialogue system [5]. In their approach, the similarity is computed between TF-IDF
weighted term vectors of the query and the examples.

1

情報処理学会研究報告
IPSJ SIG Technical Report

Given a query, the cosine similarity scores between its
term vector and each of the example queries’ in thedatabase are calculated, and treated as the example
scores. The response of the example with the highest score is then returned to the user as the system’sresponse.
This approach has a number of benefits. First,
The TF-IDF weighting allows emphasis of importantwords. Such quality is desirable in considering emotion in spoken utterances. Second, as this approach
does not rely on explicit domain knowledge, it is practically suited for adaptation into an aﬀective dialoguesystem. Third, the approach is straightforward andhighly reproducible. On that account, it serves as thebaseline in this study.

4

Proposed Dialogue SystemWe make use of tri-turn units in the selection process in place of the query-response pairs in the traditional EBDM approach. A tri-turn consists of threeconsecutive dialogue turns that are in response to eachother. In this work, we exploit the tri-turn format toobserve emotional triggers and responses in a conversation. Within this work, the first, second, and third
turns in a tri-turn are referred to as query, response,and future, respectively. The change of emotion observed from query to future can be regarded as theimpact of response.
In designing our method, we consider an analogy
in human-human communication: emotional impactof a response is heavily dependent on semantic andemotional context of the conversation. For example,
an apology could have contrasting outcomes depending on what happened preceding the apology as wellas the emotional state of the listener. This suggeststhat the emotional impact in our examples are specificto the tri-turn context. In other words, semantic andemotional similarity is pre-requisite for the response
to yield consistent impact in real interaction.
In addition to semantic constraint as described inSection 3, we formulate two types of emotional constraints: (1) emotion similarity between the query andthe example queries, and (2) expected emotional impact of the candidate responses.
We measure emotion similarity by computing thePearson’s correlation coeﬃcient of the emotion vector between the query and the example queries. Secondly, we measure the expected emotional impact ofⓒ 2018 Information Processing Society of Japan

Vol.2018-MUS-118 No.2
Vol.2018-SLP-120 No.2
2018/2/20

the candidate responses by calculating the diﬀerenceof emotional states between each of their respective
query and future.
ExampleDatabase

Query

text
Semantic
similarity
scoring

emotional changeemotionm-bestEmotioncorrelation
scoring

n-bestEmotional
impactscoring

best

best

Response(baseline)

Response(proposed)

Figure 2: Steps of response selection.
Figure 2 illustrates the steps of response selectionof the baseline and proposed systems. We performthe selection in three steps based on the defined constraints. For each step, a new score is calculated andre-ranking is performed only with the new score, i.e.
no fusion with the previous score is performed.

5Experimental Set Up

We utilize The SEMAINE database, consisting of
dialogues between a user and an agent in a Wizardof-Oz fashion [6]. There are 5 agent characters withdistinct personalities; cheerful Poppy, angry Spike,sad Obadiah, and sensible Prudence. Poppy and Prudence tend to draw the user into the positive-valence
region of emotion compared to Spike and Obadiah.
To promote positive emotion, we exclusively use sessions of Poppy and Prudence to construct the example database. The training set and test set comprise
29 (15 Poppy, 14 Prudence) and 4 (2 Poppy, 2 Prudence) sessions, respectively. We construct the example database exclusively from the training set, containing 1105 tri-turns.
We utilize the transcription and emotion annotation provided from the corpus as information of thetri-turns to isolate any recognition errors. We sample
the emotion annotation of every dialogue turn into
100-length vectors. For the n-best filtering, we chose
10 for the semantic similarity constraint and 3 for theemotion.

6

Human Subjective Evaluation

A total of 50 queries are evaluated through crowdsourcing. The queries are presented in form of text,2

情報処理学会研究報告
IPSJ SIG Technical Report

Vol.2018-MUS-118 No.2
Vol.2018-SLP-120 No.2
2018/2/20

along with the responses from the baseline and proposed systems. The evaluators are asked to selectthe better response in terms of naturalness, potential
emotion connection, and positiveness of elicited emotion. 50 judgements are collected per query. The finaljudgement of each query for each question is based
weighted majority voting of the judgements.
0%

25%

50%

coherence

0.34

0.66

connection

0.3

0.7

impact0.34

0.66
baseline

75%

100%

proposed

Figure 3: Subjective evaluation result.
Figure 3 visualizes the evaluation result. It is shownthat in comparison to the baseline system, the proposed system is perceived as more coherent (66% ofthe time), having more potential in building emotional connection (70%), and giving a more positive
emotional impact (66%). Furthermore, we observethat the queries where the proposed system wins havefar stronger agreement than that where the baseline
system wins (Fleiss’ Kappa of 0.35 vs. 0.16). Higher
agreement level suggests a stronger win, where bigger
majority of the evaluators are voting for the winner.

7Conclusions

We presented a novel attempt in eliciting postive
emotional impact in dialogue response selection byutilizing examples of human appraisal in spoken dialogue. We augment the response selection criteria totake into account emotion similarity between queryand the example query, as well as the expected future
impact of the candidate response. Human subjective
evaluation showed that the proposed system can elicit
a more positive emotional impact in the user, as wellas achieve higher coherence and potential emotional
connection.

References[1] Marcin Skowron, Mathias Theunis, SebastianRank, and Arvid Kappas, “Aﬀect and social processes in online communication–experiments with
an aﬀective dialog system,” Transactions on Affective Computing, vol. 4, no. 3, pp. 267–279, 2013.
[2] Takayuki Hasegawa, Nobuhiro Kaji, Naoki Yoshinaga, and Masashi Toyoda, “Predicting and eliciting addressee’s emotion in online dialogue.,” inProceedings of Association for Computational Linguistics (1), 2013, pp. 964–972.
[3] James A Russell, “A circumplex model of aﬀect.,”
Journal of personality and social psychology, vol.
39, no. 6, pp. 1161, 1980.
[4] Cheongjae Lee, Sangkeun Jung, Seokhwan Kim,and Gary Geunbae Lee, “Example-based dialog
modeling for practical multi-domain dialog system,” Speech Communication, vol. 51, no. 5, pp.
466–484, 2009.
[5] Nio Lasguido, Sakriani Sakti, Graham Neubig,
Tomoki Toda, and Satoshi Nakamura, “Utilizing human-to-human conversation examples fora multi domain chat-oriented dialog system,”Transactions on Information and Systems, vol. 97,no. 6, pp. 1497–1505, 2014.
[6] Gary McKeown, Michel Valstar, Roddy Cowie,
Maja Pantic, and Marc Schroder, “The SEMAINE database: Annotated multimodal recordsof emotionally colored conversations between aperson and a limited agent,” Transactions on Affective Computing, vol. 3, no. 1, pp. 5–17, 2012.

Acknowledgement
This research and development work was supportedby the MIC/SCOPE #152307004.

ⓒ 2018 Information Processing Society of Japan3