Diversifying Dialog Generation via Adaptive Label Smoothing (2021)

Abstract: Neural dialogue generation models trained with the one-hot target distribution suffer from the over-confidence issue, which leads to poor generation diversity as widely reported in the literature. Although existing approaches such as label smoothing can alleviate this issue, they fail to adapt to diverse dialog contexts. In this paper, we propose an Adaptive Label Smoothing (AdaLabel) approach that can adaptively estimate a target label distribution at each time step for different contexts. The maximum probability in the predicted distribution is used to modify the soft target distribution produced by a novel light-weight bi-directional decoder module. The resulting target distribution is aware of both previous and future contexts and is adjusted to avoid over-training the dialogue model. Our model can be trained in an endto-end manner. Extensive experiments on two benchmark datasets show that our approach outperforms various competitive baselines in producing diverse responses.

## introduction the success of neural models has greatly advanced the research of dialog generation @xcite @xcite . however, most of these models suffer from a lowdiversity issue where models tend to generate bland and generic responses such as i don't know or i'm ok @xcite . although various approaches have been proposed to tackle this issue @xcite @xcite @xcite , there are still remarkable gaps between responses generated by neural models and those from humans @xcite . further, some existing methods may even harm the fluency or coherence when improving the diversity of generated 0.01 0.0 0.10 ugly 0.01 0.0 0.08 dull 0.01 0.0 0.11 fun … hard target (one hot) label smoothing adalabel (ours) figure 1: a dialogue sampled from the opensubtitles dataset. we demonstrate the hard target, label smoothing, and adaptive label smoothing approach when learning to predict the next word ("human"). unfortunately, the ideal soft target is challenging to obtain. early works try to tackle this issue using label smoothing @xcite , i.e., a small probability is uniformly assigned to nontarget words. however, the target distribution constructed in this way is far from ideal: first, the probability of the target word is chosen manually and fixed, which cannot adapt to different contexts. however, as @xcite demonstrated, human text distribution exhibits remarkable fluctuations in the per-token perplexity. we argue that different target probabilities should be used for different contexts. second, the uniform assignment of the probability mass on non-target words ignores the semantic relationship between the context and each word. ideally, a word should receive more probability mass if it is more relevant to the context. for the example shown in figure 1 , word "fun" is more likely to appear behind the context "i make the robots seem more " than word "bank". to address the above issue, we propose an adaptive label smoothing (adalabel) method that can dynamically estimate a soft target distribution at each time step for different contexts. specifically, for each target word y t in the training data, the probability distribution predicted by the current model is first obtained. the maximum probability p max in this distribution measures the confidence of the current prediction, i.e., a higher p max means higher confidence for the current prediction. to avoid over-confidence, we use p max as the supervision signal for the target word y t in the training process so that the model will not be optimized towards y t when it correctly predicts y t . a word-level factor is also introduced to facilitate the learning of low-frequency words. moreover, we introduce a novel auxiliary decoder module d a to produce the supervision signals for these non-target words in each training step. d a only contains one transformer block, and it is optimized to predict words based on bi-directional contexts. a novel target-mask attention scheme is devised to prevent d a from seeing the target word in the training process. this scheme also enables parallel training and inference of d a . we perform extensive experiments on two benchmark datasets: dailydialog and opensubtitles. our method outperforms various competitive baselines and significantly improves the diversity of generated responses while ensuring fluency and coherency. our major contributions are summarized: ## conclusion we address the low-diversity issue of neural dialogue models by introducing an adaptive label smoothing approach, adalabel. in our method, the probability of each target word is estimated based on the current dialogue model's prediction, and the probabilities for these non-target words are calculated using a novel auxiliary decoder d a . a target-masked attention scheme is introduced in d a to help capture forward and backward contexts. we evaluate our method on two benchmark datasets: dailydialog and opensubtitles. extensive experiments show that our method effectively alleviates the over-confidence issue and improves the diversity of the generated responses. as future work, we believe this method is extensible to other text generation tasks. yizhe zhang, michel galley, jianfeng gao, zhe gan, xiujun li, chris brockett, and bill dolan. 2018. generating informative and diverse conversational responses via adversarial information maximization. in advances in neural information processing systems, pages 1810-1820. tiancheng zhao, ran zhao, and maxine eskenazi. 2017. learning discourse-level diversity for neural dialog models using conditional variational autoencoders. in proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pages 654-664, vancouver, canada. association for computational linguistics. yinhe zheng, zikai chen, rongsheng zhang, shilei huang, xiaoxi mao, and minlie huang. 2020a. stylized dialogue response generation using stylized unpaired texts. in aaai. yinhe zheng, rongsheng zhang, minlie huang, and xiaoxi mao. 2020b. a pre-training based personalized dialogue generation model with persona-sparse data. in proceedings of the aaai conference on artificial intelligence, volume 34, pages 9693-9700. ganbin zhou, ping luo, yijun xiao, fen lin, bo chen, and qing he. 2018. elastic responding machine for dialog generation with dynamically mechanism selecting. in thirty-second aaai conference on artificial intelligence.