EM Pre-training for Multi-party Dialogue Response Generation (2023)

Abstract: Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but leaving a great gap for multi-party dialogues at the same time. Different from two-party dialogues where each response is a direct reply to its previous utterance, the addressee of a response utterance should be specified before it is generated in the multi-party scenario. Thanks to the huge amount of two-party conversational data, various pre-trained language models for two-party dialogue response generation have been proposed. However, due to the lack of annotated addressee labels in multi-party dialogue datasets, it is hard to use them to pre-train a response generation model for multi-party dialogues. To tackle this obstacle, we propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. Theoretical analyses and extensive experiments have justified the feasibility and effectiveness of our proposed method. The official implementation of this paper is available at https://github.com/EricLee8/MPDRG .

## introduction inspired by the tremendous success in pre-training large language models (plms) in general domains @xcite @xcite , efforts have been made to train plms for dialogue response generation @xcite @xcite . however, they constrain the dialogues to be either two-party, or sequential structured (i.e. each utterance replies directly to its previous utterance). different from them, a multi-party dialogue can involve multiple interlocutors, where each interlocutor can reply to any preceding utterances, making the response relations of the dialogue tree-structured and much more complicated @xcite @xcite . besides, the speaker and addressee of a response utterance should be specified before it is generated in multi-party scenario, making the annotated data for multi-party dialogue response generation (mp-drg) less available. figure 1 illustrates an example of mpdrg task taken from the ubuntu irc benchmark @xcite . the upper part shows the tree-structured addressee relations of the dialogue, where the arrows point from addressees to speakers, and different colors represent different interlocutors. the middle part displays the content of the dialogue history, where u 7 is the response to be generated. the addressee (u 6 ) and the speaker (#4) of it are given, and the content of this response is the target of our model. the lower part gives the human response, which is also called the ground truth reference. previous works on mpdrg fine-tune generative plms on small multi-party dialogue datasets with explicit addressee annotations. they utilize the response annotations to form a tree-structured response graph, then encode the dialogue history using either homogeneous or heterogeneous graph neural networks (gnns) @xcite . nevertheless, none of them make attempts to pre-train a response generation model for multiparty dialogues due to the lack of large-scale corpora with annotated addressee labels. to solve the aforementioned problem of data scarcity, we propose an em approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. specifically, we treat the addressee of each utterance in the dialogue history as a discrete latent variable z. during the e-steps, given the current dialogue history c t and the the response utterance r t , we model the distribution of the current addressee z t as p(z t |c t , r t ; θ), where θ is the current model parameters. during the m-steps, we sample (c t , r t , z t ) triplets from distribution p(z t |c t , r t ; θ) and optimize the generative model p(r t |c t , z t ; θ) on these samples. with the iteration number increasing, the accuracy of latent variable prediction and the quality of generated responses will grow together. it is worth noting that during these iterations, annotated addressee labels are not required, which makes it possible to leverage the huge amount of multi-party dialogue corpora without addressee labels. we provide theoretical analyses to prove the feasibility of our em method, and conduct experiments on the ubuntu irc benchmark, which is used in previous works @xcite . the contributions of our work can be summarized as the following three folds: • to the best of our knowledge, we are the first to study the pre-training of multi-party dialogue response generation, which is much more challenging and complicated than two-party dialogues. ## methodology to design a model for multi-party dialogue response generation and make it compatible with the em training algorithm, there are two important things to consider: how to model p(r t |c t , z t ; θ) in the maximization step, and how to compute p(z t |c t , r t ; θ) in the expectation step. in this section, we will first address these two problems, then mathematically derive the feasibility of our em pre-training algorithm. ## experiments in this section, we first introduce the datasets to pre-train and evaluate our model, then present the experimental results and comparisons with previous methods. ## analysis in order to get more insights into the proposed em pre-training method, we dive deeper into it by conducting extensive analyses.