Aﬀect-sensitive Dialogue Response Generation for
Positive Emotion Elicitation
Nurul Lubis1

Sakriani Sakti1,21Satoshi Nakamura1,2

Koichiro Yoshino1,2,3

Nara Institute of Science and Technology
2
RIKEN AIP
2JST PRESTO{nurul.lubis.na4, ssakti, koichiro, s-nakamura}@is.naist.jp

1

Introduction

tory throughout the dialogue.

Figure 1 shows aschematic view of this architecture.
An emotionally-competent computer agent could… 𝑤2,𝑁2

𝑤2,1oﬀer valuable assistance in various aﬀective tasks.

𝑤3,1… 𝑤3,𝑁3

For example caring for the elderly, low-cost ubiquitous chat therapy, and providing emotional support
in general. In this paper, we build an end-to-end

ℎ𝑑𝑙𝑔

ℎ𝑒𝑚𝑜

chat-oriented dialogue system that can dynamically
mimic aﬀective human interactions by utilizing a hi-ℎ𝑢𝑡𝑡

erarchical neural network architecture. End-to-end
approaches have been reported to show promising
results for non-goal oriented dialogue systems. However, application of this approach towards incorporating emotion in the dialogue is still very lacking.
Our contributions in this paper are 1) a neuralnetwork based chat-oriented dialogue system that
captures user’s emotional state and considers it ingenerating a dialogue response, trained to elicit positive emotion through the interaction, 2) a dialogue
corpus that reflects a positive emotion elicitation
strategy for model training to influence its aﬀectivetendency. This allows positive emotion elicitation
without any elaborate dialogue strategy.

𝑤1,1… 𝑤1,𝑁1

𝑤2,1… 𝑤2,𝑁2

Figure 1: Proposed architecture
After reading the input sequence, the dialogue turnis encoded into utterance representation hutt . hutt isthen fed into the recurrent dialogue encoder to capture the information up to the current dialogue turn,into dialogue context hdlg . In Emo-HRED, the hdlg
is then fed into the emotion encoder, which models
the emotion context hemo . The response generation
process is conditioned by the concatenation of the
dialogue and emotion contexts.
The training cost of the Emo-HRED is a linear

2

Proposed Modelinterpolation between the response generation error costutt (i.e. negative log-likelihood of the gen-We build our model extending the recently pro-erated response) and the emotion label prediction

posed hierarchical recurrent encoder-decoder archi-

error costemo by the emotion encoder. The final costtecture [4]. We propose to incorporate an emotionis then propagated to the network and the parame-

encoder into the architecture. The emotion encoder

ters are optimized as usual.

is placed in the same hierarchy as the dialogue en-

We propose to pre-train the Emo-HRED with acoder, capturing emotion information at dialogue-

large scale conversational corpus to infer content andturn level and maintaining the emotion context his-syntactic knowledge prior to training its emotion-

Table 1: Model perplexity on test set. In subjective evaluation, ∗ denotes significant diﬀerence (p < 0.05).
ModelBaseline HRED
Proposed Emo-HRED

Parameter update
standard
selective
selectivePerplexity
121.44100.94
42.26

Naturalness
3.26
3.27

Emotional Impact
n/a
3.22
3.39 *

related parameters due to the limited amount of emo-

HRED as baseline model and Emo-HRED as the pro-tion rich data. We further propose selective fine-posed model.Second, we consider two diﬀerent pa-tuning, limiting the parameter updates to the emo-rameter update schemes: standard and selective.

tion encoder and utterance decoder only. As emotion

In the standard scheme, we fine-tune all the param-

is not yet involved during encoding, we hypothesize

eters of the model. In the selective scheme, wethat the pre-trained encoders can be used for thefix the utterance and dialogue encoders parameters.

aﬀect-sensitive response generation task as is.

The fine-tuning is done with the constructed positive data We hypothesize that the positive corpus
will cause the model to elicit more positive emotion.

3

Constructing Positive Data
We aim to use emotion information to elicit pos-

itive emotion through dialogue. We realize this implicitly by relying on the training data.

We en-hance the existing emotion-rich SEMAINE corpus[3] to contain positive-emotion eliciting responses.
First, for all triples extracted from the data, i.e.
three consecutive dialogue turns with agent-useragent speaker order, we obtain new responses thatelicit positive emotion using an EBDM with modified selection criteria [2]. We ask human judges todecide which response elicits a more positive emotional impact, the default or the system generated

Table 1 shows the evaluation result of the diﬀerent set ups. In objective evaluation, we measure themodel perplexity on a held out test set. We observe significant improvements when selective update scheme is employed in place of the standardscheme. With identical starting model and fine-tune
set up, the Emo-HRED architecture converges tosignificantly better models compared to the HRED.
In subsequent subjective evaluation through crowdsourcing, we found that the responses generated by
the proposed model are perceived as 1) more natural,and 2) elicit a significantly more positive emotional
response (p < 0.05) compared to the baseline.

one. For each triple, we obtain at least 3 humanjudgements, or more when ties occur. The final response is obtained by majority voting.

5Conclusion
We propose a dialogue response generator archi-

tecture for eliciting positive emotion through dia-

4

Experiment and Evaluation

We utilize the HRED trained on the SubTle corpus[1] as our starting model. The data pre-processing
steps are performed as in [4]. This dataset is fed
sequentially into the network until it converges. Inaddition to the model parameters, we also learn theword embeddings of the tokens. The model is trained
to optimize the log-likelihood of the training triples
using the Adam optimizer.
Two aspects are considered in fine-tuning theabove model. First, we consider two diﬀerent models:logue interactions. Both the subjective and objective evaluation show consistent incremental improvements when each of the proposed set ups is applied tothe dialogue system. They also show that a system
performs best when all of the proposed set ups areapplied at the same time. In the future we hope tofurther improve the proposed system, both in termsof response quality and user emotional experience.

Acknowledgements
Part of this work was supported by JSPS KAKENHI
Grant Numbers JP17H06101 and JP17K00237.

References[1] David Ameixa, Luisa Coheur, Pedro Fialho, andPaulo Quaresma. Luke, I am your father: dealing
with out-of-domain requests by using movies subtitles. In International Conference on Intelligent
Virtual Agents, pp. 13–21. Springer, 2014.
[2] Nurul Lubis, Sakriani Sakti, Koichiro Yoshino,
and Satoshi Nakamura. Eliciting positive emotional impact in dialogue response selection. InProceedings of International Workshop on Spoken Dialogue Systems Technology, 2017.
[3] Gary McKeown, Michel Valstar, Roddy Cowie,
Maja Pantic, and Marc Schroder. The SEMAINE
database: Annotated multimodal records of emotionally colored conversations between a person
and a limited agent. Transactions on Aﬀective
Computing, Vol. 3, No. 1, pp. 5–17, 2012.
[4] Iulian V Serban, Alessandro Sordoni, Yoshua
Bengio, Aaron Courville, and Joelle Pineau.
Building end-to-end dialogue systems using generative hierarchical neural network models. InThirtieth AAAI Conference on Artificial Intelligence, 2016.