DLGN et: A Transformer-based Model for Dialogue Response Generation (2020)

Abstract: Neural dialogue models, despite their successes, still suffer from lack of relevance, diversity, and in many cases coherence in their generated responses. On the other hand, transformer-based models such as GPT-2 have demonstrated an excellent ability to capture long-range structures in language modeling tasks. In this paper, we present DLGNet, a transformer-based model for dialogue modeling. We specifically examine the use of DLGNet for multi-turn dialogue response generation. In our experiments, we evaluate DLGNet on the open-domain Movie Triples dataset and the closed-domain Ubuntu Dialogue dataset. DLGNet models, although trained with only the maximum likelihood objective, achieve significant improvements over state-of-the-art multi-turn dialogue models. They also produce best performance to date on the two datasets based on several metrics, including BLEU, ROUGE, and distinct n-gram. Our analysis shows that the performance improvement is mostly due to the combination of (1) the long-range transformer architecture with (2) the injection of random informative paddings. Other contributing factors include the joint modeling of dialogue context and response, and the 100% tokenization coverage from the byte pair encoding (BPE).

## introduction recent successes of pretrained transformer-based language models, such as bert @xcite , gpt(-2) @xcite , transformer-xl @xcite , xlnet @xcite , and ernie(2.0) @xcite , have led to state-of-the-art performance on many natural language understanding (nlu) tasks including sentence classification, named entity recognition, sentence similarity, and question answering. the exceptional performance of transformer-based language models is due to their ability to capture long-term temporal dependencies in the input sequence. this attribute should be very beneficial to dialogue modeling, especially in multi-turn scenarios. most of the existing neural dialogue response generation models are based on recurrent neural networks @xcite @xcite @xcite @xcite @xcite . these models have yielded promising results by generating mostly coherent responses given the dialogue context. however, most of them, including the state-of-the-art models trained with naturalistic dialogue data, still perform well below the human level. generated responses tend to be either generic, out-of-context, or disproportionately short. previous work points to some causes of these limitations: in this paper, we propose dlgnet, a transformer-based model for multi-turn dialogue modeling that addresses some of the highlighted problems above. the use of a transformer architecture allows dlgnet to capture long-term temporal dependencies in the dialogue data better than the existing rnn-based architectures @xcite . however, applying a vanilla seq2seq transformer @xcite and its multi-turn variants, such as recosa @xcite , for dialogue modeling does not work well because of the semantic redundancy in dialogue data. to overcome this, dlgnet models the joint distribution of the context and response instead of the conditional distribution of the response given the context, usually employed in seq2seq frameworks @xcite @xcite . dlgnet also addresses the syntactic redundancy in dialogue data by appending random paddings before and after the input data. this helps to break down the learning barrier from the concave entropy profile of human conversation data, as shown in fig. 1 . the flattening of the entropy profile also provides regularization during training, and reduces even the extent of the exposure bias problem. finally, to avoid the out-of-vocabulary problem, dlgnet uses byte pair encoding (bpe) similar to gpt-2 @xcite to provide 100% coverage for any unicode input and output texts. given all these proposed changes, we train dlgnet models using only the maximum likelihood objective. dlgnet models, <@xmath0. ## dlgnet model description in order to address the semantic redundancy, we propose to jointly model both the context and the response as an alternative to the mutual information objective @xcite . the resulting distribution and the objective function can then be respectively expressed as while (3) addresses the semantic redundancy, it does not address the syntactic redundancy coming from the concave positional entropy profile of dialogue data. to circumvent this, we append random informative paddings (sampled from the dataset) before (x b i ) and after (x a i ), the dialogue example of interest, leading to dlgnet adopts gpt-2's autoregressive transformer architecture @xcite using only the decoder part of the original transformer architecture @xcite since there is no need for a separate encoder network (see fig. 2 ). autoregressive transformer models use multiple layers of masked multi-head self-attention to map a sequence of input tokens to a sequence of output tokens (i.e., the input sequence token shifted one position to the right). during inference, at each step, the model is autoregressive, consuming the previously generated token as additional input when generating the next. there are some basic conceptual differences between autoregressive architectures based on transformers and those based on recurrent neural networks (rnns). for instance, while the output of an rnn layer depends on only the immediate previous output, a transformer layer output consists of attention over all previous outputs. due to this lack of ordering in transformer architectures, the position representation is usually passed along with the input tokens into the model @xcite . in order to take advantage and evaluate the impact of pretrained parameters, we use two model configurations i.e., (i) dlgnet-117m -with 117m parameters, 12 attention layers, and a hidden state size of 767, and (ii) dlgnet-345m -with 345m parameters, 24 attention layers, and a hidden state size of 1024; similar to the publicly available gpt-2 models @xcite . ## model training we trained the small dlgnet-117m and the medium dlgnet-345m models on multi-turn dialogue datasets initialized with either random noise or pretrained language model parameters. the models are trained end-to-end using the adaptive moment estimation (adam) stochastic gradient descent algorithm with a learning rate of 0.001. the maximum sequence length is 1024. due to gpu memory limitations, we use a batch size of 2 and accumulate gradients over 5 iterations, making the effective batch size 10. both models are trained until the training perplexity on the dialogue datasets reaches a steady state. finally, the models are implemented, trained, and evaluated using python and the tensorflow deep learning framework. ## ablation studies on dlgnet models with random informative padding in this section, we carry out a more detailed analysis and discussion of different configurations of dl-gnet models as well as their performance across datasets, using the evaluation results in ## further ablation studies on dlgnet models we also set out to analyze the features of dlgnet that make it suitable for multi-turn dialogue modeling. we train both dlgnet-117m and dlgnet-345m models on both datasets, but replace the random informative paddings with static paddings using a pad token. below are the definitions of the model configuration factors considered: 1.) multi-turn data (m): training data is variable-length multi-turn data padded to a fixed length. this helps to evaluate the effect of using random informative padding. ## conclusion in this paper, we have proposed dlgnet, an extension of autoregressive transformer models such as gpt-2 for multi-turn dialogue modeling. our experiments show that dlgnet models perform better than existing state-of-the-art multi-turn dialogue models. they also achieve the best performance to date on open-domain movie and closeddomain ubuntu datasets based on bleu, rouge and distinct n-gram scores. our experiments reveal that the combination of (i) the transformer architecture with (ii) the injection of random paddings exploiting the large maximum input sequence is responsible for the performance improvement over existing methods. other contributing factors include joint modeling of dialogue context and response, and the 100% tokenization coverage from the byte pair encoding (bpe). our analysis also reveals some tradeoffs between response relevance and response length, and we showed how different sampling strategies can be used to make an informed decision about such response relevancediversity compromises. in our future work, we plan to investigate how to improve on the length of the generated responses without necessarily sacrificing their coherence and their relevance to the dialogue context.