DIALOGUE MODEL AND RESPONSE GENERATIONFOR EMOTION IMPROVEMENT ELICITATION
1,21,31,3,41,3
Nurul Lubis , Sakriani Sakti , Koichiro Yoshino, Satoshi Nakamura
1 NAIST, Japan, 2 HHU Düsseldorf, Germany, 3 RIKEN, AIP, Japan, 4 PRESTO, JST, Japan

2. CORPUS CONSTRUCTION

1. OVERVIEW
Chat-based dialogue system has huge
potential in affect-sensitive tasks
Goal: User emotional improvement
Task: Response generation in end-to-end
dialogue systems• Processing negative emotion throughhuman-computer interaction
Problem: existing approaches only focus
on short-term emotion elicitation• Only considers next dialogue turn
• No awareness of emotional processes
Example:System Hey, what’s up?
UserI just got a horrible news. Resp. 1 “Want to tell me more about it?”Resp. 2 “Don’t worry, it will be fine!”

Which response is better long-term
for emotion improvement?

1. Induce

participant

expertA multimodal database that shows• Everyday negative emotion in dialogue• How an expert responds to negative emo.

1 expert, 30 participants, 60 dialogues,20 emotion inducers, 23:41 hours

3. DIALOGUE MODELING AND RESPONSE GENERATION
Dialogue model of emotion processing• Human assessment of dialogue data based on literature on
counseling skills and techniques• Guided by counselor’s questions and assessment in dialogue• Counselor’s turns are manually annotated with phase-action labels
Opening• Small talkUnderstanding• Emotion• Event• Experience• Strategy

𝑤𝑡+1,1

Response generation for emotion processing• Applying the dialogue model on response generation• Considers short- and long-term emotion improvement• Dialogue level (long-term)• N-gram simulator; N=1, 2, 3
• Trained on counselor dialogue act• Utterance level (short-term)• Pre-training: HRED using SubTle corpus• Selective fine-tuning: MC-HRED with counselingdata, utt. and dlg. encoders are fixed• Testing: MC-HRED using action from N-gram… 𝑤𝑡+1,𝑁𝑡+1ℎ𝑑𝑙𝑔

𝑎𝑡+1N-gram
simulatorℎ𝑎𝑐𝑡

GENERATION EXAMPLES
𝑤𝑡,1…

𝑤𝑡,𝑁𝑡

ℎ𝑒𝑚𝑜

Closing• GoodbyeResolution• Brain-storming• Distancing• Positive asset
search (PAS)

Dialogue level:
N-gram counselorsimulatorℎ𝑢𝑡𝑡...or whether it’s his impression that hehas to work that long.
P
mm that’s true.
Cbut somehow I guess I’m sure the society
probably has what the company has
created that impression.
P
yes yes.
Base. it’s quite interesting to understand that.
Prop. yes. Do you think there’s something thatyou can do about it?

Speech transcription• Paid commercial ASR• Corrected by profess.
transcribers

2. Interact

Our contributions:
1. Dialogue corpus highlighting negative
emotion processing
2. Dialogue model of emotion processing
3. Response generator combining shortand long-term emotion improvement
elicitation

In dialogue

Emotion annotation• Valence and Arousal• Gtrace [Cowie+, 2000]

emotion inducer

𝑎𝑡Word-level:
MC-HRED
responsegenerator

C

Different action labels
OP

Smalltalk
UND Emo.
Eve.
Exp.
RES

CLS

oh thank you. so are you having agood week?
it’s kind of sad.
yes it’s quite a heavy video.
so you feel like it’s so important toyou.
Brains do you think that’s something thattrm. you can do about it?
Dist. so you just came here two weeks
ago?
PAS
it’s definitely a way to understand.
Good anyway thank you for telling mebye
about your opinions.

4. EVALUATION
Objective: Perplexity on test set
Subjective: Human evaluation• Likert scale 1 to 5 (higher is better)• The response is natural• The response elicits a positive
emotional impact• 100 dialogue with 4 turns as context• Longer context for better
understanding• 20 human judgements for eachBaseline: Emo-HRED (without dlg. model)
Proposed: MC-HRED using N-gram sim.

Model
Baseline
Emo-HRED

Simulator

Ppx.

Nat.

Emo.

n/a

42,60

3,56

3,26

ProposedHybrid MCHREDunigram
bigramtrigram

49,74
49,62
49,78

n/a
3,77

3,51

The proposed model...
• Has slightly higher perplexity• No significant perplexity difference across sims.
• Is more natural and emotionally positive• Elicits diff. phases of emotion processing

5. CONCLUSION AND FUTURE WORKS• Emotion processing in dialogue follows acertain underlying structure• Enforcing structure of emotion processing in
response generation improves humansubjective perception of the system• Model is able to simulate different phases ofemotion processing at appropriate times

Future works• Comparison with different
combination approaches• WoZ study to investigate differences
between HCI and human interactions• Real user interaction for evaluation