3Rin2-27
The 33rd Annual Conference of the Japanese Society for Artificial Intelligence, 2019ର࿩ߦҝΛ༻੍͍ͨ‫ޚ‬Մೳͳχϡʔϥϧର࿩Ϟσϧͷ‫ݕ‬౼
An Investigation of Controllable Neural Conversation Model with Dialogue Acts∗1Տ໺ ੣໵ ∗1٢໺ ޾Ұ࿠ ∗1∗2

தଜ ఩ ∗1

Seiya Kawano

Koichiro Yoshino

Satoshi Nakamura∗2ಸྑઌ୺Պֶٕज़େֶӃେֶ

Nara Institute of Science and TechnologyՊֶٕज़ৼ‫ߏػڵ‬Japan Science and Technology Agency

Dialogue act is known as an essential component of the dialogue system, which captures the user’s intention
and produces the appropriate response. In this paper, we propose a controllable response generation model given
dialogue acts. Recent neural conversation models are based on the end-to-end approach that learns a mapping
a mapping between dialogue histories and response utterances. However, it was diﬃcult to control the contentsof the response generated by the model. Several models tackled the problem of generating responses under thespeciﬁed dialogue acts as a condition; however, these models still have problems on conditioned generations. In thispaper, we introduced an extended framework of the generative adversarial network that optimizes both conditioned
generator and discriminator which explicitly classiﬁes dialogue act classes. Experimental results showed that ourconditional response generation model improved both the response quality and controllability of neural conversation
generation.

1.

͸͡ΊʹϓϯυϝΠϯͷඇλεΫࢤ޲ର࿩γεςϜʹ͓͍ͯద༻͠Α͏ͱͨ͠৔߹ɼର࿩ߦҝͱͦͷεϩοτ৘ใͷ྆ํΛϑϨʔϜͱͯ͠อ࣋͢Δ͜ͱ͸‫࣮ݱ‬తͰ͸ͳ͍ɽ·ͨɼZhao Β͸ NCM ͕ա౓ʹҰൠԽ͞ΕͨԠ౴Λฦ͢ͱ͍͏໰୊ [Li 16a] ʹରͯ͠ɼର࿩ߦҝͷ৘ใΛ༻͍Δ͜ͱͰΑΓଟ༷͔ͭߴ඼࣭ͳԠ౴ੜ੒Λߦ͏ KgCVAE (Knowledgeguided Conditional Variational Autoencoder) ΛఏҊ͍ͯ͠
ΔɽKgCVAE Ͱ͸ɼEncoder ͷग़ྗ݁Ռ͔ΒԠ౴ͷର࿩ߦҝΛ༧ଌ͠ɼ͜ΕΛ Decoder ΁ͷ৚݅ͱͯ͠࢖༻͍ͯ͠Δɽ·ͨɼಉ༷ͷΞϓϩʔνͱͯ͠ɼNCM ʹ͓͚Δ Decoder ʹ‫ײ‬৘ϥϕϧ΍࿩ऀϥϕϧΛ৚݅ͱͯ͠ར༻͠ɼNCM ʹ͓͚ΔԠ౴಺༰Λ੍‫͠ޚ‬Α͏ͱ͢ΔϞσϧ΋ఏҊ͞Ε͍ͯΔ [Zhou 18, Li 16b]ɽ
͜ΕΒͷ NCM ͸ɼSCE (Softmax Cross Entropy Loss) ʹ‫ ͍ͯͮج‬Decoder ͷ֤୯‫ޠ‬ੜ੒εςοϓʹ͓͚Δ୯‫ޠ‬༧ଌͷ࠷దԽΛߦ͏ɽ͔͜͠͠ΕΒͷϞσϧ͸ඞͣ͠΋ੜ੒͞ΕͨԠ౴͕ࢦఆͨ͠ϥϕϧʹ‫͍ͨͮج‬ద੾ͳԠ౴Ͱ͋Δ͜ͱΛอূ͠ͳ͍ɽ͜ͷ܏޲͸ಛʹɼ௿ස౓ͷϥϕϧͰ‫ݦ‬ஶͰ͋Δɽ͜Ε͸ɼ‫܇‬࿅σʔλʹ͓͚Δϥϕϧͷ෼෍ͷภΓʹ‫ى‬Ҽ͢Δ [Zhou 18]ɽҰํͰɼNCM ʹఢରతੜ੒ωοτϫʔΫ (Generative Adversarial Net; GAN) Λಋೖ͢Δ͜ͱͰɼΑΓଟ༷͔ͭߴ඼࣭ͳԠ౴ੜ੒Ϟσϧͷߏங͕‫ݕ‬౼͞Ε͍ͯΔ [Li 17a]ɽ۩ମతʹ͸ɼԠ౴ൃ࿩Λੜ੒͢Δ Generator ͱɼ༩͑ΒΕͨԠ౴ֶ͕
शσʔλ͔ Generator ͷͲͪΒʹ༝དྷ͢Δ΋ͷ͔Λࣝผ͢ΔDiscriminator Λߏங͢Δɽ͜ΕΒͷ 2 ͭͷϞσϧΛަ‫ʹޓ‬ఢରతʹ‫܇‬࿅͢Δ͜ͱͰɼΑΓࣗવͳൃ࿩Λੜ੒͠Α͏ͱ͢ΔࢼΈͰ͋Δɽ͜ͷΑ͏ͳɼֶशͷ࿮૊Έ͕ैདྷͷ NCM ͱൺֱͯ͠ߴ඼࣭͔ͭଟ༷ͳԠ౴Λੜ੒ͨ͜͠ͱ͕ใࠂ͞Ε͍ͯΔɽຊ‫ڀݚ‬͸͜͏ͨ͠ GAN ʹ‫ֶͮ͘ج‬शͷ࿮૊ΈΛɼࢦఆͨ͠ର࿩ߦҝΒ͍͠Ԡ౴Λੜ੒͢Δ͜ͱΛ໨తͱͯ͠ಋೖ͢Δɽର࿩ߦҝͱ͸ɼ࿩ऀ͕ൃ࿩ʹ͓͍ͯ࣋ͭԿ͔͠Βͷʮҙਤʯ͋Δ͍͸ൃ࿩ʹ͓͚Δʮ‫ػ‬ೳʯͰ͋Γɼͦͷҙਤ΍‫ػ‬ೳͷछྨͱͯ͠ର࿩ߦҝλά͕ఆٛ͞ΕΔ [Boyer 10, Տ໺ 18]ɽର࿩
ߦҝ͸ɼର࿩Ϟσϧʹ͓͚Δ‫ج‬ຊ୯ҐͷҰͭͱͯ͠ར༻͞Ε͖͓ͯͯΓɼಛʹۙ೥ɼର࿩ʹ͓͚Δൃ࿩ؒͷ૬‫༻࡞ޓ‬ΛϞσϧԽ͢Δ্Ͱ༗༻Ͱ͋Δ͜ͱ͕஌ΒΕ͍ͯΔ [Yoshino 15]ɽ͠
͔͠ɼۙ೥޿͘༻͍ΒΕ͍ͯΔχϡʔϥϧର࿩Ϟσϧ (NeuralConversation Model; NCM) Ͱ͸ɼ͜͏ͨ͠ର࿩ߦҝʹΑͬͯγεςϜൃ࿩Λ໌ࣔతʹૢ࡞͢Δ͜ͱ͕ࠔ೉Ͱ͋Δɽͦ͜Ͱຊ‫Ͱڀݚ‬͸ɼγεςϜԠ౴͕࣋ͭର࿩ߦҝͷ৘ใΛ৚݅ͱͯ͠༻͍ͨఢରతੜ੒ֶशͷ࿮૊ΈΛ NCM ʹಋೖ͢Δɽ۩ମతʹ͸ɼ༩͑ΒΕͨର࿩ߦҝʹ‫͍ͯͮج‬Ԡ౴Λੜ੒͢Δ Generator
ͱɼGenerator ͕ੜ੒ͨ͠Ԡ౴͕ࢦఆͨ͠ର࿩ߦҝʹ‫͍ͨͮج‬ద੾ͳ΋ͷͰ͋Δ͔Λ൑ผ͢Δ Discriminator Λߏங͠ɼ͜ΕΒͷ 2 ͭͷϞσϧΛަ‫ʹޓ‬ఢରతʹ‫܇‬࿅͢Δɽ͜ͷΑ͏ͳֶशͷ࿮૊ΈͷಋೖʹΑΓɼNCM ͕೚ҙͷର࿩ߦҝʹ‫͍ͨͮج‬ద੾ͳԠ౴Λੜ੒͢Δ͜ͱ͕Մೳ͔Ͳ͏͔ͷධՁΛߦͬͨɽ2.

ؔ࿈‫ڀݚ‬ର࿩ߦҝͷ৘ใΛ༻͍ͯൃ࿩ੜ੒Λߦͬͨ‫ͯ͠ͱڀݚ‬ɼWen΍ Zhao Βͷ‫͕͋ڀݚ‬Δ [Wen 15, Zhao 17]ɽWen Β͸ɼϨετϥϯҊ಺ର࿩ͷυϝΠϯʹ͓͍ͯɼ೚ҙͷର࿩ߦҝ(e.g. Inform) ͱͦͷεϩοτ৘ใ (e.g. food=Chinese,name=Seven Days) Λ༻͍ͯൃ࿩ੜ੒ (e.g. “Seven Daysserves Chinese.”) Λߦ͏χϡʔϥϧ‫ޠݴ‬Ϟσϧ (SemanticallyConditioned LSTM; SC-LSTM) ΛఏҊ͍ͯ͠ΔɽSC-LSTMͰ͸ɼLSTM ‫ޠݴ‬ϞσϧʹʮಡΈࠐΈήʔτʯͱ‫ݺ‬͹ΕΔήʔςΟϯά‫ߏػ‬Λ৽ͨʹಋೖ͢Δ͜ͱͰɼ୯‫ޠ‬ͷੜ੒࣌ʹɼࢦఆͨ͠ର࿩ߦҝͱεϩοτ৘ใͷ͏ͪͲͷ৘ใΛ࢖͏͔ͷ੍‫ޚ‬Λߦ͏ɽ͔͠͠ͳ͕ΒɼSC-LSTM ͸‫ݶ‬ΒΕͨυϝΠϯͷλεΫୡ੒Λର৅ͱ͍ͯ͠ΔͨΊɼऔΓѻ͏ൃ࿩ͷൣғ͕๲େͳΦʔ3.

ఏҊख๏ຊ‫Ͱڀݚ‬͸ɼલઅͰड़΂ͨ NCM ʹ͓͚Δ GAN ʹ‫ൃͮ͘ج‬࿩ੜ੒ͷ࿮૊ΈΛ֦ு͠ɼԠ౴ͷର࿩ߦҝΛ৚݅ͱͯ͠༻͍Δ
NCM (৚݅෇͖ NCM) ʹԠ༻͢Δɽ۩ମతʹ͸ɼ༩͑ΒΕͨର࿩ߦҝʹ‫͍ͯͮج‬Ԡ౴Λੜ੒͢Δ Generator (G) ͱɼG ͕

࿈བྷઌ: Տ໺੣໵ɼಸྑઌ୺Պֶٕज़େֶӃେֶɼ˟ 630-0101ɼಸྑ‫ݝ‬ੜۨࢢߴࢁொ 8916 ൪஍ͷ 5ɼkawano.seiya.kjo@is.naist.jp

1

3Rin2-27
The 33rd Annual Conference of the Japanese Society for Artificial Intelligence, 2019ੜ੒ͨ͠Ԡ౴͕ࢦఆͨ͠ର࿩ߦҝʹ‫͍ͨͮج‬ద੾ͳ΋ͷͰ͋Δ͔Λࣝผ͢Δ Discriminator (D) Λߏங͠ɼ͜ΕΒͷ 2 ͭͷϞσϧΛަ‫ʹޓ‬ఢରతʹ‫܇‬࿅͢Δɽຊख๏͸ɼG ʹΑΔԠ౴
ੜ੒ͱ D ʹΑΔԠ౴ൃ࿩ͷධՁͷ࿮૊ΈʹɼԠ౴ͷର࿩ߦҝͷ৘ใΛ໌ࣔతʹಋೖ͍ͯ͠Δ఺Ͱैདྷख๏ [Li 17a] ͱҟͳΔɽຊ‫Ͱڀݚ‬͸ɼ͜ͷΑ͏ͳԠ౴ͷର࿩ߦҝͷ৘ใΛར༻ͨ͠
GAN ͷ࿮૊ΈͷಋೖʹΑΓɼNCM ͕೚ҙͷର࿩ߦҝʹ‫ͮج‬͍ͨద੾ͳԠ౴Λੜ੒͢Δ͜ͱ͕Մೳ͔Ͳ͏͔ͷධՁΛߦ͏ɽ

3.1ͳ‫ྻܥ‬σʔλʹ GAN Λద༻Ͱ͖Δ͜ͱΛ͍ࣔͯ͠ΔɽҎԼʹ
GAN ͕࣋ͭ Generator, Discriminator ͷߏ੒Λࣔ͢ɽ

3.2.1 Generator (G)Generator Ͱ͸ɼԠ౴ੜ੒ʹ͓͚Δ୯‫ޠ‬બ୒ͷϓϩηεΛϚϧίϑܾఆաఔͱΈͳ͠ɼ֤εςοϓʹ͓͚Δ୯‫ޠ‬ੜ੒͕͋Δ੓ࡦʹ‫͍ͯͮج‬બ୒͞ΕͨߦಈʹΑܾͬͯఆ͞ΕΔͱߟ͑Δɽ
SeqGAN Ͱ͸ɼDiscriminator ʹΑͬͯਪఆ͞ΕͨԠ౴จͷଥ౰ੑʹ͍ͭͯͷείΞ Q(M, R, d) Λ Generator ͷใुͱͯ͠
ར༻͠ɼ͜ͷ‫ظ‬଴ใुΛ࠷େԽ͢ΔΑ͏ʹ Generator Λํࡦޯ഑๏ʹΑΓߋ৽͢Δ [Williams 92]ɽຊ‫͚͓ʹڀݚ‬Δର࿩ߦҝʹΑΔ৚݅෇͖Ԡ౴ੜ੒ʹ͓͍ͯ͸ɼͦͷ໨తؔ਺ J ͱޯ഑Λ࣍ࣜͰఆٛ͢Δ ∗2 ɽ͜͜Ͱɼπ ͸Ԡ౴ R ͷੜ੒֬཰Ͱ͋
Δɽຊ‫Ͱڀݚ‬͸ɼલઅͰड़΂ͨର࿩ߦҝʹΑΔ৚݅෇͖ NCM
Λ Generator ͱͯ͠༻͍Δɽର࿩ߦҝʹΑΔ৚݅෇͖ NCM

ର࿩ߦҝʹΑΔ৚݅෇͖Ԡ౴ੜ੒Ͱ͸ɼର࿩ཤྺ M ={Mi−1 , Mi−2 , . . . , Mi−n } ͱԠ౴ͷର࿩ߦҝ di Λ༻͍ͯɼԠ
౴ൃ࿩ Ri = {w1 , w2 , · · · , wI } Λੜ੒͢Δɽ͜͜Ͱɼn ͸ର࿩௕ɼJ ͸ൃ࿩௕Ͱ͋Δɽຊ‫Ͱڀݚ‬͸ɼ࠷΋ૉ๿ͳ NCM ͷ֦ுͱͯ͠ਤ 1 ͷΑ͏ʹɼ֊૚తͳ Encoder-decoder Ϟσϧʹ‫ج‬ͮ͘ NCM[Tian 17] ʹରͯ͠ɼൃ࿩ͷσίʔυ࣌ʹ໌ࣔతʹର࿩ߦҝͷ৘ใΛ༩͑ΔϞσϧΛߏஙͨ͠ɽ௨ৗͷ Encoder-Decoder ϞσϧͰ͸ɼDecoder ʹ͓͚Δ֤୯‫ޠ‬ग़ྗεςοϓ t ʹ͓͍ͯɼ௚લͷӅΕঢ়ଶ ht−1 ͱ‫ࡏݱ‬ͷ୯‫ ޠ‬wt ͷϕΫτϧද‫ ݱ‬xt ͕ೖྗͱͯ͠࢖༻͞ΕΔɽҰํͰɼຊϞσϧͰ͸ɼ௚લͷӅΕঢ়ଶ ht−1 ʹՃ͑ͯɼର࿩ߦҝ di ͷϕΫτϧද‫ ݱ‬e ͱ xt ͷ࿈݁ϕΫτϧ vt = e ⊕ xt Λೖྗͱ͠
ͯ࢖༻͢Δ͜ͱͰɼࢦఆͨ͠ର࿩ߦҝʹ‫͍ͨͮج‬Ԡ౴Λ NCM
͕ੜ੒͢Δ͜ͱΛ‫ظ‬଴͢Δ ∗1 ɽRNN ʹ͸ɼLSTM ΍ GRU ͱൺֱͯ͠ߴ଎Ͱ‫܇‬࿅͕ՄೳͰ͋Δ͜ͱ͕஌ΒΕ͍ͯΔ SRU (Simple Recurrent Unit)[Lei 17] Λ༻͍ͨɽධՁ࣮‫Ͱݧ‬͸ɼEncoder ‫ ͼٴ‬Decoder Ͱ࢖༻͢Δ SRU ͷ૚਺͸ 2ɼதؒ૚ͷ࣍‫ݩ‬͸ 1024ɼ୯‫ޠ‬ຒΊࠐΈϕΫτϧͷ࣍‫ݩ‬͸ 256ɼର࿩ߦҝຒΊࠐΈϕΫτϧͷ࣍‫ݩ‬͸
100 ͱͨ͠ɽ·ͨɼEncoder ʹ͓͚Δ࠷େର࿩௕ n ͸ 5 ʹઃఆͨ͠ɽϞσϧͷ‫܇‬࿅͸ SCE ʹΑΓߦ͍ɼ։ൃηοτʹ͓͚
Δ Perplexity ͕࠷΋Լ͕ͬͨϞσϧΛධՁʹ࢖༻͢Δɽ

(1)
(2)
(3)

• Implicit-Discriminator; ର࿩ཤྺ M ɼԠ౴ൃ࿩ Rɼର࿩ߦҝd Λೖྗͱͯ͠ɼ༩͑ΒΕͨ R ͕ຊ෺ (ֶशσʔλ༝དྷͷ΋ͷ)ِ͔෺ (Generator ༝དྷͷ΋ͷ) Ͱ͋Δ͔ͷ 2 Ϋϥε෼ྨΛߦ͏ɽGenerator ʹΑͬͯੜ੒͞Εͨ R ͕ຊ෺Ͱ͋Δͱਪఆ͞ΕΔ֬཰Λ Q(M, R, d) ͱͯ͠ར༻͢Δɽ͜Ε͸ɼର࿩ߦҝλάΛૉ๿ʹ Discriminator ͷೖྗͱͯ͠༻͍ɼੜ੒݁Ռ੍͕‫͞ޚ‬ΕΔ͜ͱΛ‫ظ‬଴͢Δ΋ͷͰ͋Δɽ
• Explicit-Discriminator; ର࿩ཤྺ M ɼର࿩ߦҝ d ʹରԠ͢Δ
Ԡ౴ൃ࿩ R Λೖྗͱͯ͠ɼR ͕ Generator ༝དྷͷ΋ͷͰ͋Δͱِ͖෺ɼֶशσʔλ༝དྷͷ΋ͷͰ͋Δͱ͖ͦͷର࿩ߦҝΛ෼ྨ͢ΔΑ͏ʹ Discriminator Λ‫܇‬࿅͢Δɽͭ·Γɼର࿩ߦҝͷछྨ਺Λ N ͱͨ͠ͱ͖ɼN + 1 ΫϥεͷଟΫϥε෼ྨΛߦ͏ɽ͜
͜ͰɼQ(M, R, d) ͸ɼGenerator ʹΑͬͯੜ੒͞Εͨ R ֶ͕शσʔλ༝དྷͷର࿩ߦҝ d Ͱ͋Δͱਪఆ͞ΕΔ֬཰Ͱ͋Δɽੜ੒จ͕֤ର࿩ߦҝͷಛ௃Λ൓ө͍ͯ͠Δ͔Λ Discriminator ͕໌ࣔతʹ༧ଌ͢Δ͜ͱʹΑΓɼ֤ର࿩ߦҝͷಛ௃͕ੜ੒จʹ൓ө͞ΕΔ͜ͱΛ‫ظ‬଴͢Δɽ

Discriminator ʹ͸ɼਤ 2 ʹࣔ͢Α͏ͳ Utterance Encoderͱ Dialogue Encoder Ͱߏ੒͞ΕΔ֊૚‫ ܕ‬RNN Λ Discriminator ͱͯ͠༻͍Δɽ·ͨɼGenerator ͷ৔߹ͱಉ༷ʹɼRNN
ʹ͸ SRU Λ Discriminator ͷ‫܇‬࿅ͷߴ଎ԽͷͨΊʹ࠾༻͢ΔɽධՁ࣮‫͍͓ͯʹݧ‬͸ɼSRU ͷ૚਺͸ 1ɼதؒ૚ͷ࣍‫ݩ‬͸ 1024ɼ୯‫ޠ‬ຒΊࠐΈϕΫτϧͷ࣍‫ݩ‬͸ 256ɼର࿩ߦҝຒΊࠐΈϕΫτϧͷ࣍‫ݩ‬͸ 100ɼશ݁߹૚ʹ͓͚Δதؒ૚ͷ࣍‫ݩ‬͸ 512 ʹઃఆͨ͠ɽ·ͨɼDiscriminator ͷߋ৽ʹ͓͚Δଛࣦؔ਺ʹ͸ SCEΛ༻͍Δɽର࿩ߦҝʹΑΔ৚݅෇͖ఢରతੜ੒ωοτϫʔΫຊ‫Ͱڀݚ‬͸ɼఢରతੜ੒ֶश (GAN) ͷख๏ͱͯ͠ SeqGAN(Sequence Generative Adversarial Net) [Yu 17, Li 17a] ͷ࿮૊ΈΛ࠾༻͠ɼର࿩ߦҝʹΑΔ৚݅෇͖Ԡ౴ੜ੒΁Ԡ༻͢
Δɽൃ࿩ͷΑ͏ͳ཭ࢄ‫ྻܥ‬σʔλͷੜ੒ʹ GAN Λద༻͢Δ
৔߹ɼGenerator ͷग़ྗ͸୯‫ޠ‬ͷ཭ࢄ஋ͷ‫ྻܥ‬Λલఏͱ͢Δͨ
ΊɼDisciriminator ͔Β Generator ΁ͷޯ഑ͷߋ৽Λߦ͏͜ͱ͕೉͍͠ɽ͜ͷ໰୊ʹରͯ͠ɼSeqGAN Ͱ͸ɼ‫ڧ‬ԽֶशʹΑΔΞϓϩʔνΛ Generator ͷߋ৽ʹ༻͍Δ͜ͱͰɼ཭ࢄత
∗1= ER∼p(R|M,d) (Q(M, R, d)|θ)≈ Q(M, R, d)∇ log π(R|M, d)
= Q(M, R, d)∇ t log π(wt |M, d, w1:t−1 )3.2.2 Discriminatorຊ‫Ͱڀݚ‬͸ɼҎԼͷ 2 छྨͷ Discriminator ʹ͍ͭͯ‫ݕ‬౼Λߦ͏ɽਤ 1: ର࿩ߦҝʹΑΔ৚݅෇͖ NCM ͷ֓ཁ

3.2J(θ)∇J(θ)

4.

ධՁ࣮‫ݧ‬

4.1σʔληοτର࿩ߦҝʹΑΔ৚݅෇͖ NCM ͷ‫܇‬࿅ͱධՁʹ͸ DailyDialog Corpus[Li 17b] Λ༻͍ͨɽຊίʔύεʹ͸ɼ13,118 ର࿩ʹ͓͚Δൃ࿩ʹɼInform (4,6532 ൃ࿩) ɼQuestions (29,428ൃ࿩)ɼDirectives (17,29 ൃ࿩)ɼCommisive (9,724 ൃ࿩) ͷ∗2ೖྗ୯‫ ޠ‬wt ‫ͼٴ‬ର࿩ߦҝ d ͸ຒΊࠐΈ૚Λ༻͍ͯີϕΫτϧද‫ ݱ‬xt , e ΁ͱม‫͞׵‬ΕΔɽ2ຊ‫Ͱڀݚ‬͸؆୯ԽͷͨΊʹจશମͰ୯ҰͷใुΛ࢖༻͍ͯ͠Δ఺ʹཹҙ͞Ε͍ͨɽ

3Rin2-27
The 33rd Annual Conference of the Japanese Society for Artificial Intelligence, 2019

ද 1: ର࿩ߦҝʹΑΔ৚݅෇͖ NCM ͷධՁϞσϧ

Perplexity

Average

GreedyExtremeDistinct-1

Distinct-2

Accuracy৚݅෇͖ NCM (greedy)
Adversarial-Implicit (greedy)
Adversarial-Explicit (greedy)৚݅෇͖ NCM (sampling)
Adversarial-Implicit (sampling)
Adversarial-Explicit (sampling)৚݅෇͖ NCM (beam)
Adversarial-Implicit (beam)
Adversarial-Explicit (beam)

36.661439.2864
39.699336.661439.2864
39.699336.661439.2864
39.6993

0.7912
0.78820.78670.79170.78700.78650.78470.77930.77965.50755.62395.66615.17855.25665.27825.56805.66685.71670.52980.53940.53950.49050.50350.5041
0.5381
0.5411
0.54310.03360.02970.03110.07670.05940.05830.03250.02670.02740.01600.00690.00100.26500.20930.11860.00060.00310.00170.86440.85850.88430.81760.80620.85730.87070.86150.8865• ଟ༷ੑ (Diversity); ର࿩γεςϜͷ෼໺Ͱ޿͘࠾༻͞Ε͍ͯ
Δ Distinct-1 ‫ ͼٴ‬Distinct-2 ʹΑΓධՁΛߦ͏ɽDistinct-1(Distinct-2) ͸ɼςετσʔλʹରͯ͠ੜ੒͞ΕͨԠ౴ൃ࿩ू߹ʹ͓͚ΔϢχʔΫͳ unigram (bigram) ͷ਺Λɼੜ੒͞Εͨ͢΂ͯͷ unigram (bigram) ͷ਺ͰεέʔϦϯάͨ͠ࢦඪͰ͋
Δ [Li 16a]ɽ
• ੍‫ޚ‬ੑ (Contorollability); ೖྗͱͯ͠࢖༻ͨ͠ର࿩ߦҝͱɼ࣮ࡍʹੜ੒͞ΕͨԠ౴จʹ͓͚Δର࿩ߦҝͷਪఆ݁ՌͷҰக཰
(AccuracyɼPrecisionɼRecallɼF1) ʹ‫͍ͯͮج‬ධՁ͢ΔɽԠ౴จͷର࿩ߦҝʹ͍ͭͯ͸ɼࣄલ‫܇‬࿅ͨ͠ର࿩ߦҝ෼ྨ‫∗ ث‬5 ʹ
ΑΔਪఆ݁Ռ (ධՁ༻σʔλʹ͓͚Δਪఆਫ਼౓͸ɼ0.8303) Λਖ਼ղͱͯ͠༻͍Δɽ

5.

ਤ 2: Implicit-DiscriminatorɼExplicit-Discriminator ͷ֓ཁද 1 ʹର࿩ߦҝʹΑΔ৚݅෇͖ NCM ͷࣗಈධՁ݁Ռʹ͍ͭͯࣔ͢ɽ͜͜Ͱɼ৚݅෇͖ NCM ͸ SCE ʹΑΓ‫܇‬࿅͞ΕͨϞσϧͰ͋ΓɼAdversarial-Implict ϞσϧɼAdversarialExplicit Ϟσϧ͸ͦΕͧΕɼImplicit-DiscriminatorɼExplicitDiscriminator Λ࢖༻ͨ͠ GAN ʹΑΓ‫܇‬࿅͞ΕͨϞσϧͰ͋ΔɽԠ౴จͷੜ੒ʹ͓͍ͯ͸ɼශཉతʹग़ྗ୯‫ޠ‬Λબ୒͢Δ৔߹ (greedy) ͱଟ߲෼෍ʹ‫͍ͯͮج‬αϯϓϦϯάΛߦ͏৔߹(sampling)ɼbeam search (ϏʔϜ෯ 5) Λߦ͏৔߹ (beam) ͷ3 ௨ΓΛࢼͨ͠ɽ4 ͭͷର࿩ߦҝλά͕Ξϊςʔγϣϯ͞Ε͍ͯΔɽຊ‫Ͱڀݚ‬͸ɼ͜ͷίʔύεΛ‫܇‬࿅ (11,118 ର࿩)ɼ։ൃ (1,000 ର࿩)ɼධՁ༻σʔλ (1,000 ର࿩) ʹ෼ׂ͠ɼର࿩ߦҝʹΑΔ৚݅෇͖ NCM ͷ‫܇‬࿅ͱධՁʹ༻͍ͨɽ‫ޠ‬ኮαΠζ͸ 2,5000 ʹઃఆ͠ɼະ஌‫ޠ‬͸ಛघ‫“ ߸ه‬UNK” ʹஔ͖‫܇Ͱ͑͏ͨ͑׵‬࿅Λߦ͏ɽର࿩ߦҝʹΑΔ৚݅෇͖ఢରతੜ੒ωοτϫʔΫͷ‫܇‬࿅ํ๏ʹ͍ͭͯ͸ɼLi Βͷख๏ [Li 17a] ΛࢀߟʹҎԼͷखॱͰߦ͏ɽD-stepɼG-step ʹ͓͚Δεςοϓ਺͸ͦΕͧΕɼ4ɼ20 ʹઃఆ͠ɼbatch size ͸ 32 ͱͨ͠ɽ·ͨɼDisciriminato ͷߋ৽ʹ͸ SGD (ֶश཰ 1e-3)ɼGenerator ͷߋ৽ʹ͸ Adam (ֶश཰ 1e-5) Λ༻͍ͨ ∗3 ɽ5.1Ԡ౴ੜ੒݁ՌͷධՁGAN Λಋೖͨ͠ Adversarial-Implicit ϞσϧͰ͸ɼ͍ͣΕͷ decoding ख๏Λ༻͍ͨ৔߹ʹ͓͍ͯ΋ɼGAN Λ༻͍ͳ͍৚݅෇͖ NCM ͱൺֱͯ͠ɼؔ࿈ੑई౓ͷ Vector Extreme ͕޲্͍ͯ͠Δ͜ͱ͕֬ೝͰ͖ΔɽҰํͰɼଟ༷ੑɾ੍‫ޚ‬ੑʹ͍ͭͯ͸վળ͕֬ೝͰ͖ͳ͔ͬͨɽ·ͨɼsampling Λ࢖༻͢Δ৔߹Ͱ͸ɼgreedy ʹԠ౴Λੜ੒͢Δ৔߹ͱൺֱͯ͠ɼଟ༷ੑ͸վળ͢ΔҰํͰɼ੍‫ޚ‬ੑ͸ 0.0523 ϙΠϯτ΋ͷ௿Լ͕֬ೝ
Ͱ͖ͨɽ͔͠͠ͳ͕ΒɼAdversarial-Explicit ϞσϧͰ͸ɼ͍ͣΕͷ deocding ख๏Λ༻͍ͨ৔߹Ͱ΋ɼGAN Λ༻͍ͳ͍৚
݅෇͖ NCM ͱൺֱͯؔ͠࿈ੑ (Greedy Matching & VectorExtreme) ͱ੍‫ޚ‬ੑ͕Ұ؏ͯ͠վળ͍ͯ͠Δɽಛʹɼ੍‫ޚ‬ੑʹ͓͍ͯ͸ɼgreedy ʹԠ౴Λੜ੒͢Δ৔߹͸ 0.0199 ϙΠϯτɼ
sampling Λ࢖༻͢Δ৔߹Ͱ͸ 0.0396 ϙΠϯτɼbeam searchΛ࢖༻͢Δ৔߹Ͱ͸ 0.0157 ϙΠϯτ΋ͷվળ͕֬ೝͰ͖ɼຊ
Ϟσϧͷ༗ޮੑ͕ࣔࠦ͞ΕΔ݁Ռͱͳͬͨɽ
Adversarial-Explicit ϞσϧͰ͸ɼԠ౴จ͕ Generator ֶ͔शσʔλͷͲͪΒʹ༝དྷ͢Δ΋ͷ͔Λ൑ผ͢Δͱಉ࣌ʹɼ໌ࣔతʹର࿩ߦҝͷਪఆΛߦ͍ͬͯΔɽ͜Εʹରͯ͠ɼAdversarialImplicit ϞσϧͰ͸ɼੜ੒͞ΕͨԠ౴ͷର࿩ߦҝΛ໌ࣔతʹ

Algorithm 1 ৚݅෇͖ఢରతੜ੒ωοτϫʔΫͷ‫܇‬࿅
1: for number of iteration do2:
for number of D-step do3:
sample (M, R, d) from training data
4:
generate response R̂ using G on (M, d)
5:update D using (M, R̂, d) and (M, R, d)
6:
for number of G-step do
7:
sample (M, R, d) from training data
8:
generate response R̂ using G on (M, d)
9:compute reward r for (M, R̂, d) using D
10:
update G on(M, R̂, d) using r

4.2

࣮‫݁ݧ‬Ռର࿩ߦҝʹΑΔ৚݅෇͖Ԡ౴ੜ੒ͷࣗಈධՁର࿩ߦҝʹΑΔ৚݅෇͖Ԡ౴ੜ੒ͷධՁʹ͍ͭͯ͸ɼؔ࿈ੑ(Relevance)ɼଟ༷ੑ (Diversity)ɼ੍‫ޚ‬ੑ (Contorollability)ͷ 3 ͭͷ‫͔఺؍‬Βଟ໘తʹධՁΛߦ͏ɽ• ؔ࿈ੑ (Relevance); ੜ੒͞ΕͨԠ౴ൃ࿩ͱϨϑΝϨϯεͷԠ౴ൃ࿩ʹ͓͚Δ୯‫ޠ‬෼ࢄද‫ݱ‬ͷྨࣅ౓ʹΑΓධՁΛߦ͏ɽྨࣅ౓ͷࢉग़ํ๏ʹ͸ Embedding AverageɼGreedy Matchingɼ
Vector Extreme ͷࡾͭͷධՁࢦඪ [Liu 16] Λ࠾༻ͨ͠ ∗4 ɽͰ͸ɼwikiepedia σʔλͰ‫܇‬࿅͞Εͨ fastText ʹΑΔ୯‫ޠ‬෼ࢄද‫( ݱ‬300 ࣍‫ )ݩ‬Λ༻͍ͨɽ∗5 Explicit-Discriminator ͱಉ༷ͷΞʔΩςΫνϟΛ࠾༻ͨ͠ɽ

∗3 800 iteration ࣮ߦͨ͠‫ޙ‬ͷϞσϧΛධՁʹ࢖༻ͨ͠∗4 ͜ΕΒͷৄࡉʹ͍ͭͯ͸‫ݪ‬࿦จ [Liu 16] Λࢀর͞Ε͍ͨɽຊ‫ڀݚ‬3

3Rin2-27
The 33rd Annual Conference of the Japanese Society for Artificial Intelligence, 2019ਪఆ͓ͯ͠Β੍ͣ‫ޚ‬ੑͷείΞվળʹͭͳ͕Βͳ͔ͬͨͱߟ͑ΒΕΔɽ·ͨɼGAN Λಋೖͨ͠ 2 ͭͷϞσϧͰ͸ɼGANΛ༻͍ͳ͍৚݅෇͖ NCM ͱൺֱͯ͠ Perplexity ͷվળ͕֬ೝͰ͖ͳ͔ͬͨɽGAN ʹΑΔςΩετੜ੒Ϟσϧ͕ඞͣ͠΋
Peprlexity Λվળ͠ͳ͍͜ͱ͸ઌߦ‫[ ڀݚ‬Tevet 18] ʹ͓͍ͯ΋ࢦఠ͞Ε͍ͯΔɽ͜Ε͸ɼఢରతੜ੒ֶश͸୯‫ޠ‬༧ଌͷ࠷దԽΛ໨తͱ͠ͳ͍ͨΊͰ͋Δɽ

5.2ධՁͷΈʹͱͲ·Γɼੜ੒จͷ඼࣭ʹ͍ͭͯΑΓ‫ʹີݫ‬ධՁ͢ΔͨΊʹ͸ਓखʹΑΔධՁΛߦ͏͜ͱ͕ඞཁͰ͋Δͱߟ͑ΒΕΔɽ͞ΒʹɼGenerator ͷߋ৽ͷࡍʹ࢖༻͢Δใु஋ͷ‫ࢉܭ‬ʹଟ༷ੑ΍ൃ࿩௕ͳͲͷ৘ใΛಋೖ͢Δ͜ͱͳͲɼఢରతੜ੒ωοτϫʔΫͷ‫܇‬࿅ํ๏ͦͷ΋ͷͷվળΛߦ͏͜ͱ΋ॏཁͰ͋Δͱߟ͑ΒΕΔɽ·ͨɼࠓ‫ޙ‬͸ఏҊख๏ͷଞλεΫ (‫ײ‬৘΍ϖϧιφΛߟྀͨ͠Ԡ౴ੜ੒ͳͲ) ʹ͓͚Δ༗ޮੑͷ‫ݕ‬౼΋ਐΊΔ༧ఆͰ͋Δɽ֤ର࿩ߦҝλάຖʹ͓͚Δ੍‫ޚ‬ੑͷධՁද 2ɼද 3 ͸ͦΕͧΕ Adversarial-Explicit Ϟσϧʹ͓͍ͯɼbeam search ͱ sampling Λ࢖༻ͯ͠Ԡ౴ੜ੒Λߦͬͨ৔߹ʹ͓͚Δɼ֤ର࿩ߦҝλάຖͷ੍‫ޚ‬ੑͷධՁ݁Ռʹ͍͍ͭͯࣔͯ͠Δɽ‫಺ހׅ‬͸ɼGAN Λ༻͍ͳ͍৚݅෇͖ NCM ʹ͓͚ΔείΞͱͷࠩ෼Ͱ͋Δɽ͜͜Ͱɼςετηοτʹ͓͚Δ֤ର࿩ߦҝλάͷग़‫਺ݱ‬͸ɼInform ͸ 2639ɼQuestions ͸ 1679ɼ
Directives ͸ 938ɼCommisive ͸ 484 Ͱ͋Δɽँࣙຊ‫ڀݚ‬͸ɼJST ͖͕͚͞ (JPMJPR165B) ͓Αͼ JST CREST(JPMJCR1513) ͷࢧԉΛड͚ͨɽࢀߟจ‫ݙ‬[Boyer 10] Boyer, K. E., Ha, E. Y., Phillips, R., Wallis, M. D.,Vouk, M. A., and Lester, J. C.: Dialogue act modeling in acomplex task-oriented domain, in Proc. of SIGDIAL, pp. 297–305Association for Computational Linguistics (2010)ද 2: ֤ର࿩ߦҝλάຖʹ͓͚Δ੍‫ޚ‬ੑ (beam search)ର࿩ߦҝλά

Precision

RecallF1

InformQuestions
Directives
CommisiveMacro Avg
Weighted Avg

0.88550.96990.92630.59760.84480.8926

0.9231
0.9976
0.66950.72110.82780.88640.9039 (+0.0125)
0.9836 (+0.0060)
0.7772 (+0.0622)
0.6536 (+0.0187)
0.8296 (+0.0249)
0.8854 (+0.0111)[Lei 17] Lei, T., Zhang, Y., and Artzi, Y.: Training rnns as fast ascnns, arXiv preprint arXiv:1709.02755 (2017)[Li 16a] Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B.: A
Diversity-Promoting Objective Function for Neural Conversation
Models, in Proc. of NAACL-HLT, pp. 110–119 (2016)[Li 16b] Li, J., Galley, M., Brockett, C., Spithourakis, G. P., Gao, J.,
and Dolan, W. B.: A Persona-Based Neural Conversation Model,CoRR, Vol. abs/1603.06155, (2016)ද 3: ֤ର࿩ߦҝλάຖʹ͓͚Δ੍‫ޚ‬ੑ (sampling)ର࿩ߦҝλά

Precision

RecallF1

InformQuestions
Directives
Commisive
Macro Avg
Weighted Avg

0.87660.9086
0.79630.63640.80450.85260.90750.9774
0.63750.59300.77890.85730.8918 (+0.0256)
0.9418 (+0.0214)
0.7081 (+0.1126)
0.6139 (+0.0587)
0.7889 (+0.0546)
0.8530 (+0.0293)[Li 17a] Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., and Jurafsky, D.: Adversarial Learning for Neural Dialogue Generation, in
Proc. of EMNLP, pp. 2157–2169 (2017)[Li 17b] Li, Y., Su, H., Shen, X., Li, W., Cao, Z., and Niu, S.: DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset, arXiv
preprint arXiv:1710.03957 (2017)[Liu 16] Liu, C.-W., Lowe, R., Serban, I., Noseworthy, M., Charlin, L., and Pineau, J.: How NOT To Evaluate Your Dialogue
System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation, in Proc. of EMNLP, pp.
2122–2132 (2016)ද 2ɼද 3 ΑΓɼGAN Λ༻͍ͳ͍৚݅෇͖ NCM ͱൺֱ͠
ͯɼAdversarial-Explicit ϞσϧͰ͸ɼ͢΂ͯͷର࿩ߦҝλάͰ F ஋͕վળ͍ͯ͠Δ͜ͱ͕֬ೝͰ͖ͨɽ·ͨɼ֤ର࿩ߦҝλάʹ͓͚Δ F ஋ͷϚΫϩฏ‫ۉ‬ɼՃॏฏ‫ۉ‬΋ಉ༷ʹվળ͍ͯ͠
Δ͜ͱ͕֬ೝͰ͖ͨɽಛʹɼsampling Λ࢖༻ͨ͠৔߹ʹ͓͍ͯ͸ɼ֤ࢦඪͷΑΓ‫ݦ‬ஶͳվળ͕֬ೝͰ͖Δɽ͔͠͠ͳ͕Βɼ
Quenstions λάʹ͓͍ͯ͸ɼ֤ର࿩ߦҝλάͷதͰ࠷΋খ͞ͳ F ஋ͷվળʹͱͲ·ͬͨɽ͜Ε͸ɼQuestions λά͕ٙ໰ූ΍ 5W1H ͷΑ͏ͳ୯‫ޠ‬Λ༻͍ͨ͋Δఔ౓ύλʔϯԽ͞Εͨจߏ଄Λ࣋ͭͨΊɼ໰୊͕؆୯ (‫ʹط‬ϕʔεϥΠϯ͕ൺֱతߴ੍͍‫ޚ‬ੑΛ࣮‫͍ͯ͠ݱ‬Δ) Ͱ͋Δ͜ͱ͕‫ݪ‬Ҽͩͱਪ࡯͞ΕΔɽҰํͰɼDirectives λάʹ͍ͭͯ͸ɼbeam search Λ࢖༻͢Δ৔߹Ͱ͸ 0.0622 ϙΠϯτɼsampling Λ࢖༻͢Δ৔߹Ͱ͸ 0.1126
ϙΠϯτͷ F ஋ͷ‫ݦ‬ஶͳվળ͕֬ೝͰ͖ΔɽCommisive λάʹ͓͍ͯ΋ಉ༷ʹɼbeam search Λ࢖༻͢Δ৔߹Ͱ͸ 0.0187
ϙΠϯτɼsampling Λ࢖༻͢Δ৔߹Ͱ͸ 0.0587 ϙΠϯτͷ F஋ͷվળ͕֬ೝͰ͖ͨɽ6.

[Tevet 18] Tevet, G., Habib, G., Shwartz, V., and Berant, J.:
Evaluating Text GANs as Language Models, arXiv preprint
arXiv:1810.12686 (2018)[Tian 17] Tian, Z., Yan, R., Mou, L., Song, Y., Feng, Y., andZhao, D.: How to make context more useful? an empirical study
on context-aware neural conversational models, in Proc. of ACL,Vol. 2, pp. 231–236 (2017)[Wen 15] Wen, T.-H., Gasic, M., Mrksic, N., Su, P.-H., Vandyke, D.,
and Young, S.: Semantically conditioned lstm-based natural language generation for spoken dialogue systems, arXiv preprint
arXiv:1508.01745 (2015)[Williams 92] Williams, R. J.: Simple statistical gradient-following
algorithms for connectionist reinforcement learning, Machinelearning, Vol. 8, No. 3-4, pp. 229–256 (1992)[Yoshino 15] Yoshino, K. and Kawahara, T.: Conversational system for information navigation based on POMDP with user focus tracking, Computer Speech & Language, Vol. 34, No. 1, pp.
275–291 (2015)[Yu 17] Yu, L., Zhang, W., Wang, J., and Yu, Y.: SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient., inAAAI, pp. 2852–2858 (2017)[Zhao 17] Zhao, T., Zhao, R., and Eskenazi, M.: Learning Discourselevel Diversity for Neural Dialog Models using Conditional Variational Autoencoders, in Proc. of ACL, Vol. 1, pp. 654–664 (2017)

͓ΘΓʹ[Zhou 18] Zhou, H., Huang, M., Zhang, T., Zhu, X., and Liu, B.:
Emotional chatting machine: Emotional conversation generation
with internal and external memory, in Thirty-Second AAAI Conference on Artiﬁcial Intelligence (2018)ຊ‫Ͱڀݚ‬͸ɼ೚ҙͷର࿩ߦҝʹ‫͍ͯͮج‬Ԡ౴ൃ࿩ͷੜ੒Λߦ͏৚݅෇͖ఢରతੜ੒ωοτϫʔΫΛఏҊ͠ɼͦͷධՁΛߦͬͨɽͦͷ݁ՌɼఏҊϞσϧ͸ϕʔεϥΠϯϞσϧͱൺֱͯ͠ɼؔ࿈ੑɼ੍‫ޚ‬ੑʹ͓͚Δ٬‫؍‬ධՁई౓͕վળ͞ΕΔ͜ͱΛࣔͨ͠ɽ͔͠͠ͳ͕Βɼຊ‫Ͱڀݚ‬͸٬‫؍‬ධՁࢦඪʹΑΔఏҊख๏ͷ[Տ໺ 18] Տ໺੣໵, ٢໺޾Ұ࿠, தଜ఩ɿ৚݅෇͖ఢରతੜ੒ωοτϫʔΫΛ༻͍ͨσʔλ֦ுʹΑΔର࿩ߦҝ෼ྨ๏ͷ‫ݕ‬౼, ৘ใॲཧֶձ‫ڀݚ‬ใࠂԻ੠‫ޠݴ‬৘ใॲཧ (SLP), No. 9 (2018-SLP-125), pp. 1–6 (2018)4