言語処理学会 第25回年次大会 発表論文集 (2019年3月)

Aﬀect-sensitive Dialogue Response Generation for
Positive Emotion Elicitation
Nurul Lubis1

Sakriani Sakti1,21Satoshi Nakamura1,2

Koichiro Yoshino1,2,3

Nara Institute of Science and Technology
2
RIKEN AIP
2
JST PRESTO{nurul.lubis.na4, ssakti, koichiro, s-nakamura}@is.naist.jp

1Introduction

tory throughout the dialogue. Figure 1 shows aschematic view of this architecture.

An emotionally-competent computer agent could
oﬀer valuable assistance in various aﬀective tasks.
For example caring for the elderly, low-cost ubiquitous chat therapy, and providing emotional support
in general. In this paper, we build an end-to-end
chat-oriented dialogue system that can dynamically
mimic aﬀective human interactions by utilizing a hierarchical neural network architecture. End-to-end
approaches have been reported to show promising
results for non-goal oriented dialogue systems. However, application of this approach towards incorporating emotion in the dialogue is still very lacking.
Our contributions in this paper are 1) a neuralnetwork based chat-oriented dialogue system that
captures user’s emotional state and considers it ingenerating a dialogue response, trained to elicit positive emotion through the interaction, 2) a dialogue
corpus that reflects a positive emotion elicitation
strategy for model training to influence its aﬀective
tendency. This allows positive emotion elicitation
without any elaborate dialogue strategy.

2

Proposed Model

We build our model extending the recently proposed hierarchical recurrent encoder-decoder architecture [4]. We propose to incorporate an emotion
encoder into the architecture. The emotion encoderis placed in the same hierarchy as the dialogue encoder, capturing emotion information at dialogueturn level and maintaining the emotion context his-… 𝑤2,𝑁2

𝑤2,1ℎ𝑑𝑙𝑔

𝑤3,1… 𝑤3,𝑁3ℎ𝑒𝑚𝑜ℎ𝑢𝑡𝑡

𝑤1,1… 𝑤1,𝑁1

𝑤2,1… 𝑤2,𝑁2

Figure 1: Proposed architecture
After reading the input sequence, the dialogue turnis encoded into utterance representation hutt . hutt isthen fed into the recurrent dialogue encoder to capture the information up to the current dialogue turn,into dialogue context hdlg . In Emo-HRED, the hdlg
is then fed into the emotion encoder, which models
the emotion context hemo . The response generation
process is conditioned by the concatenation of thedialogue and emotion contexts.
The training cost of the Emo-HRED is a linear
interpolation between the response generation error costutt (i.e. negative log-likelihood of the generated response) and the emotion label prediction
error costemo by the emotion encoder. The final costis then propagated to the network and the parameters are optimized as usual.
We propose to pre-train the Emo-HRED with alarge scale conversational corpus to infer content andsyntactic knowledge prior to training its emotion-― 1030 ―Copyright(C) 2019 The Association for Natural Language Processing.
All Rights Reserved.

Table 1: Model perplexity on test set. In subjective evaluation, ∗ denotes significant diﬀerence (p < 0.05).
Model
Baseline HRED
Proposed Emo-HRED

Parameter update
standard
selectiveselectivePerplexity
121.44100.94
42.26

Naturalness
3.26
3.27

Emotional Impact
n/a
3.22
3.39 *

related parameters due to the limited amount of emo-

HRED as baseline model and Emo-HRED as the pro-tion rich data. We further propose selective fine-posed model.Second, we consider two diﬀerent pa-tuning, limiting the parameter updates to the emo-

rameter update schemes: standard and selective.

tion encoder and utterance decoder only. As emotion

In the standard scheme, we fine-tune all the param-

is not yet involved during encoding, we hypothesize

eters of the model. In the selective scheme, wethat the pre-trained encoders can be used for thefix the utterance and dialogue encoders parameters.

aﬀect-sensitive response generation task as is.

The fine-tuning is done with the constructed positive data We hypothesize that the positive corpus
will cause the model to elicit more positive emotion.

3Constructing Positive Data

Table 1 shows the evaluation result of the diﬀer-

We aim to use emotion information to elicit positive emotion through dialogue. We realize this implicitly by relying on the training data.

We en-

hance the existing emotion-rich SEMAINE corpus[3] to contain positive-emotion eliciting responses.
First, for all triples extracted from the data, i.e.
three consecutive dialogue turns with agent-useragent speaker order, we obtain new responses thatelicit positive emotion using an EBDM with modified selection criteria [2]. We ask human judges todecide which response elicits a more positive emotional impact, the default or the system generated

ent set ups. In objective evaluation, we measure themodel perplexity on a held out test set. We observe significant improvements when selective update scheme is employed in place of the standard
scheme. With identical starting model and fine-tune
set up, the Emo-HRED architecture converges tosignificantly better models compared to the HRED.
In subsequent subjective evaluation through crowdsourcing, we found that the responses generated by
the proposed model are perceived as 1) more natural,and 2) elicit a significantly more positive emotional
response (p < 0.05) compared to the baseline.

one. For each triple, we obtain at least 3 humanjudgements, or more when ties occur. The final response is obtained by majority voting.

5Conclusion
We propose a dialogue response generator archi-tecture for eliciting positive emotion through dia-

4

Experiment and Evaluation

We utilize the HRED trained on the SubTle corpus[1] as our starting model. The data pre-processing
steps are performed as in [4]. This dataset is fed
sequentially into the network until it converges. Inaddition to the model parameters, we also learn theword embeddings of the tokens. The model is trained
to optimize the log-likelihood of the training triples

logue interactions. Both the subjective and objective evaluation show consistent incremental improvements when each of the proposed set ups is applied tothe dialogue system. They also show that a system
performs best when all of the proposed set ups areapplied at the same time. In the future we hope tofurther improve the proposed system, both in termsof response quality and user emotional experience.

using the Adam optimizer.
Two aspects are considered in fine-tuning theabove model. First, we consider two diﬀerent models:― 1031 ―Copyright(C) 2019 The Association for Natural Language Processing.
All Rights Reserved.

Acknowledgements
Part of this work was supported by JSPS KAKENHI
Grant Numbers JP17H06101 and JP17K00237.

References[1] David Ameixa, Luisa Coheur, Pedro Fialho, andPaulo Quaresma. Luke, I am your father: dealing
with out-of-domain requests by using movies subtitles. In International Conference on Intelligent
Virtual Agents, pp. 13–21. Springer, 2014.
[2] Nurul Lubis, Sakriani Sakti, Koichiro Yoshino,and Satoshi Nakamura. Eliciting positive emotional impact in dialogue response selection. InProceedings of International Workshop on Spoken Dialogue Systems Technology, 2017.
[3] Gary McKeown, Michel Valstar, Roddy Cowie,
Maja Pantic, and Marc Schroder. The SEMAINE
database: Annotated multimodal records of emotionally colored conversations between a person
and a limited agent. Transactions on Aﬀective
Computing, Vol. 3, No. 1, pp. 5–17, 2012.
[4] Iulian V Serban, Alessandro Sordoni, Yoshua
Bengio, Aaron Courville, and Joelle Pineau.
Building end-to-end dialogue systems using generative hierarchical neural network models. In
Thirtieth AAAI Conference on Artificial Intelligence, 2016.

― 1032 ―Copyright(C) 2019 The Association for Natural Language Processing.
All Rights Reserved.