SMRT Chatbots: I mproving Non-Task-Oriented Dialog with S imulated M ultiple R eference T raining (2020)

Abstract: Non-task-oriented dialog models suffer from poor quality and non-diverse responses. To overcome limited conversational data, we apply Simulated Multiple Reference Training (SMRT; Khayrallah et al., 2020), and use a paraphraser to simulate multiple responses per training prompt. We find SMRT improves over a strong Transformer baseline as measured by human and automatic quality scores and lexical diversity. We also find SMRT is comparable to pretraining in human evaluation quality, and outperforms pretraining on automatic quality and lexical diversity, without requiring related-domain dialog data.

## introduction non-task-oriented dialog is a low-resource nlp task. while large and noisy related corpora exist (e.g. movie subtitles, social media, and irclogs; @xcite , the publicly-released curated corpora are small. serban et al. note that smaller corpora have lower lexical diversity and topic coverage, leading to models with poor quality non-diverse responses. pretraining on larger data may improve performance, but requires a large dialog corpus in the right language and related domain. we leverage simulated multiple reference training (smrt; @xcite to overcome sparse dialog data. smrt uses a wordlevel knowledge distillation-inspired objective and a paraphraser to simulate multiple references per training example. @xcite introduce smrt for machine translation (mt) and simulate training on all translations for a source sentence, assuming: (1) all paraphrases of a target are translations of the source; and (2) all translations of the source are paraphrases of the target. (1) is true for dialog, but (2) is not-valid chatbot responses vary in meaning. smrt captures syntactic diversity though it cannot represent all semantic variations. the tree of paraphrases includes some . . . . . . . . possible . . . . . . . . . . . . paraphrases of the original promt, a sampled path and some of the other tokens also considered in the training objective. we apply smrt to chatbots and find that it: ## method we model the non-task-oriented dialog system (chatbot) task as conditional language modeling. these models are typically trained using negative log likelihood (nll) with respect to a single reference. an alternative approach is knowledge distillation @xcite which assumes access to a teacher distribution (q(y | x)) and minimizes the cross entropy with the teacher's probability distribution. ## results ## analysis smrt outperforms a strong baseline; here we analyze it in additional settings: pretraining and mmi. ## related work paraphrasing neural paraphrasing is actively improving @xcite @xcite @xcite ; we expect future improved paraphrasers will improve smrt. ## conclusion smrt improves upon a strong transformer baseline in quality and diversity. it also has human evaluation quality comparable to pretraining, with better automatic quality and lexical diversity. this method, which works even in settings where pretraining is impractical due to a lack of in-domain same language dialog data, has a high potential for impact in creating chatbots for more languages.