Diverse dialogue generation with context dependent dynamic loss function (2020)

Abstract: Dialogue systems using deep learning have achieved generation of fluent response sentences to user utterances. Nevertheless, they tend to produce responses that are not diverse and which are less context-dependent. To address these shortcomings, we propose a new loss function, an Inverse N-gram loss (INF), which incorporates contextual fluency and diversity at the same time by a simple formula. Our INF loss can adjust its loss dynamically by a weight using the inverse frequency of the tokens’ n-gram applied to Softmax Cross-Entropy loss, so that rare tokens appear more likely while retaining the fluency of the generated sentences. We trained Transformer using English and Japanese Twitter replies as single-turn dialogues using different loss functions. Our INF loss model outperformed the baselines of SCE loss and ITF loss models in automatic evaluations such as DIST-N and ROUGE, and also achieved higher scores on our human evaluations of coherence and richness.

## introduction recently, many reports have described studies using deep learning for dialogue systems that have achieved good performance. they can generate fluent sentences based on a user's utterances @xcite @xcite . nevertheless, such neural dialogue systems tend to generate phrases such as "yes" and "i do not know" frequently in non-task-oriented dialog systems, referred to as the low diversity issue and the generic response issue. after training by a loss function of similarity with gold standard reference sentences, frequent phrases are more likely to be assigned a large occurrence probability than rare phrases are. @xcite proposed an inverse token frequency (itf) loss, which multiplies the softmax cross-entropy (sce) loss by weights based on the inverse of the frequency of tokens. this itf loss incorporates the frequency distribution of token classes so that rare tokens become more likely to appear. however, sentence diversity is based not only on individual tokens, but also on the token sequence. we are able to compute weights for loss functions dynamically, depending on the context, while retaining the fluency of generated sentences. we propose such a loss function, inverse n-gram frequency (inf) loss, which uses the inverse of the frequency of the n-gram of the tokens, rather than the token frequency. we built a neural dialogue system trained by inf loss using huge amounts of dialogue data extracted from twitter. after comparing models using the sce loss, the itf loss, and the inf loss, we evaluated their diversity and fluency. results show that our proposed inf loss model outperformed the sce loss and itf loss models for most automatic assessment measures such as dist-n @xcite and rouge @xcite . our inf loss model also achieved higher scores on our human evaluations of coherence and richness. ## related work the diversity of neural dialogue generation has been studied actively. @xcite first addressed this problem using maximum mutual information (mmi) as the objective function of the neural model. takayama and arase (2019 in the dialogue corpus that were likely to appear both in response utterances and their input utterances. @xcite proposed a model that uses topic words extracted from conversations to simulate human prior knowledge, generating informative and interesting responses. in addition, variational au-toencoder (vae) and generative adversarial network (gan), which were proposed originally for image generation, have also been applied to text and dialogue generation @xcite @xcite . although gan helps to reduce response text ambiguity, their primary purpose was not diversity. @xcite proposed and demonstrated the effectiveness of adversarial information maximization (aim) as a new method for generating informative and diverse conversational responses. their work also resolved instability that arose when training the gan model. ## experiments the sce model and the itf model were used as baseline models. experiment settings are the same among these models and our proposed inf model, except for loss functions. ## discussion results show that inf produced superior perplexity values compared to sce and itf. therefore, inf produces more fluent sentences than the baselines. bleu and rouge often showed good results in sce and itf. bleu calculates n-gram precision between generated and target sentences, which does not incorporate recall. therefore, sce and itf can obtain better performance because their generated sentence lengths were shorter. they are not directly comparable with the inf of the longer sentences. also, inf presents excellent values for the diversity in both dist-1 and dist-2. particularly, the inf of dist-2 for the japanese model and the japanese dataset was 15.285, which is much higher than sce and itf. this result demonstrates that inf can generate responses that are more diverse, which also incorporate sequential information of tokens. overall, all three models generated semantically relevant responses, but inf generates a wider variety of responses with longer sentences. ## conclusion we proposed a weighted inf loss based on the inverse of the frequency of token n-grams, which can generate diverse responses while retaining fluency. comparison to baseline methods revealed that our automatic and human evaluation scores obtained using our proposed method generated more diverse responses with improved contextual consistency. future works are expected to include a tri-gram version of our inf model and the use of deep learning models other than transformer.