Modeling Situations in Neural Chat Bots
Shoetsu Sato1, Naoki Yoshinaga2, Masashi Toyoda2, and Masaru Kitsuregawa2,3{shoetsu, ynaga, toyoda, kitsure}@tkl.iis.u-tokyo.ac.jp
1 The University of Tokyo, 2 IIS, the University of Tokyo, 3 NII, Japan

Motivation

Our approach

In chat-dialogue modeling…

Baseline• Seq2seq-based neural conversational model [Vinyal+, 15]

existing data-driven models hesitate whenchoosing from various possible responses•Idea:• Improve neural conversational model to handle
conversational situations represented as discrete variables

They tend to generate typical,
frequent responses [Li+, ’15]

•It’s time to go to bed.

I feel so sleepy.

Targeted situation-types and how to obtain them:Were you up all night?

Time (season) : split conversation data into 4 season types
depending on timestamps
Utterance : cluster utterances by their topics or speaking-styles
and regard their belonging cluster’s id as a situation
Speaker / Addressee (Profile) :In the same way as utterance, cluster speaker’s / addressee’s profiles•If you fall asleep, we’ll die.

•The appropriate response varies depending onconversational situations•Proposal: Situation-aware Neural Conversational Models
Local / Global Seq2Seq

Seq2seq with situation embeddings
Encoder

Decoder
Utterance / Response
Situation-ID

RNNL0

RNNL0

…

RNN

Local-RNNs…

RNN

Local-RNNs

RNNLk0x0 x1

Utterance
Situation

Utterance

xI

sk00

y0 y1

Response
Situation

…

yJ
x0 x1

ResponsexI

Utterance

Situations as additional features:• Prepend an Utterance / Response situation embedding
to the encoder’s / decoder’s input sequence…

sk 0

…

RNNLk00

…

…

sk00

sk 0

Weighted
sum

RNNLk

RNNLk

RNNG

RNNG

Global-RNN

Weightedsum

Global-RNN…
y0 y1yJ

Response

Situations as independent tasks:• Train multiple local-RNNs by the given situations andone global-RNN to avoid data sparsenessExperiments
Dataset: Japanese Twitter archive••

A tweet and a mention to it are considered as anutterance-response pair
About 23M pairs for training and 6K pairs for test

Evaluation: Response selection task•
•Scoring utterance-candidate pairs by cross entropy
Metric: 1 in t P@k
the percentage of correct utterances in top k responses
chosen from t candidates(t-1 random dummy responses, 1 actually replied)Hyperparameters••••

RNN: 3-layer LSTM [Zaremba+, ‘14]
optimizer: Adam [Kingma+, ‘15] (learning rate: 1e-4)
hidden/embedding layer, and RNN: 100 dims
vocabulary size : 100,000 words

Examples:Situation: season (summer)

Model

1 in 2P@1

1 in 5P@1

1 in 5P@2

Baseline

64.5%

33.9%

56.6%

Seq2Seq emb

67.3%

37.6%

60.7%

L/G Seq2Seq65.9%

35.8%

58.1%

Seq2Seq emb

65.6%

35.4%

58.2%

L/G Seq2Seq

68.5%

38.2%

62.1%

Situation: time (season)

Situation: utterance

Situation: speaker / addressee (profiles)

Seq2Seq emb

67.8%

37.5%

61.1%

L/G Seq2Seq

66.4%

36.4%

59.2%

The situation-aware conversational models…
• are better at selecting ground-truth responses forsituation-specific conversations• avoid typical responses such as “Yes !” or “You’ve gotta
be tired.”
Situation: utterance (opinions, questions)Input

7⽉になって、流⽯にパーカーは暑くなってきた(July is too warm to wear a hoodie.)Input

ちょっと最近BOTのフォロー多いんですけど(I’ve recently been followed by many bot accounts.)

Baseline

そうなんです︕ (Yes!)

Baseline

お疲れ様やで (You’ve gotta be tired.)

Seq2Seq emb

まだ着てたの!? (Do you s_ll wear one?)L/G Seq2Seq

ブロックしちゃいましょう (Let’s block them. )