MADN et: Maximizing Addressee Deduction Expectation for Multi-Party Conversation Generation (2023)

Abstract: Modeling multi-party conversations (MPCs) with graph neural networks has been proven effective at capturing complicated and graphical information flows. However, existing methods rely heavily on the necessary addressee labels and can only be applied to an ideal setting where each utterance must be tagged with an “@” or other equivalent addressee label. To study the scarcity of addressee labels which is a common issue in MPCs, we propose MADNet that maximizes addressee deduction expectation in heterogeneous graph neural networks for MPC generation. Given an MPC with a few addressee labels missing, existing methods fail to build a consecutively connected conversation graph, but only a few separate conversation fragments instead. To ensure message passing between these conversation fragments, four additional types of latent edges are designed to complete a fully-connected graph. Besides, to optimize the edge-type-dependent message passing for those utterances without addressee labels, an Expectation-Maximization-based method that iteratively generates silver addressee labels (E step), and optimizes the quality of generated responses (M step), is designed. Experimental results on two Ubuntu IRC channel benchmarks show that MADNet outperforms various baseline models on the task of MPC generation, especially under the more common and challenging setting where part of addressee labels are missing.

## introduction the development of intelligent dialogue systems that are able to engage in conversations with humans, has been one of the longest running goals in artificial intelligence @xcite @xcite ., * equal contribution. † corresponding author. @xcite where a few addressee labels, i.e., "@", are missing. (b) illustration of the graphical information flow and conversation fragments of the instance above established in hetermpc @xcite . here, the bidirectional edges are merged for simplicity. 2020). thanks to breakthroughs in sequence modeling @xcite and pre-trained language models (plms) @xcite @xcite , researchers have proposed various effective models for conversations between two participants @xcite @xcite . recently, researchers have paid more attention to a more practical and challenging scenario involving more than two participants, which is well known as multi-party conversations (mpcs) @xcite @xcite @xcite @xcite . unlike two-party conversations, utterances in an mpc can be spoken by anyone and address anyone else in this conversation, constituting a graphical information flow. encoding mpc contexts with either homogeneous @xcite or heterogeneous @xcite graph neural networks (gnns) has been proven effective at modeling graphical information flows. these methods rely heavily on the necessary addressee labels and can only be applied to an ideal setting where each utterance must be tagged with an "@" or other equivalent addressee label, to establish a consecutively connected graph. however, interlocutors in mpcs do not always strictly obey the talking rule of specifying their addressees in each utterance, as shown by a randomly sampled mpc instance in figure 1(a) . statistics show that addressees of 55% of the utterances in the ubuntu irc dataset @xcite are not specified. in this common case, the expected conversation graph in previous work became fragmented @xcite as shown in figure 1 @xcite . therefore, nodes without direct connections cannot exchange information between each other through one-hop message passing. despite disconnected nodes can instead be accessed indirectly via other detours through multi-hop passing, inevitable information loss and passing latency will affect generation performance significantly. but this common issue has not been studied in previous work. in light of the above issues, we propose madnet that maximizes addressee deduction expectation in heterogeneous graph neural networks to mitigate performance degradation and to enhance model robustness, for mpc generation conditioned on incomplete addressee labels. given an mpc with a few addressee labels missing, existing methods fail to build a consecutively connected conversation graph, but only a few separate conversation fragments instead @xcite . to ensure message passing between these conversation fragments, four additional types of latent edges are designed to complete a fully-connected graph. in this way, nodes without direct connections can also directly interact with each other, and be distinguished from existing edges by parameterization. furthermore, edge-type-dependent message passing has been verified effective at mpc modeling @xcite . in order to optimize edge characterization and message passing for utterances without addressee labels, a hard expectation-maximization-based approach @xcite @xcite is designed for addressee deduction. on the one hand, the expectation steps iteratively generate silver addressee labels by considering the addressee of an utterance as a discrete latent variable. on the other hand, the maximization steps selects the addressee with the highest probability from the addressee distribution and optimize the generative dialogue model. as the number of em iterations increases, the accuracy of the latent addressee distribution as well as the quality of generated responses can be improved simultaneously. compared with previous methods, madnet can be applied to more common and challenging mpc scenarios, indicating its generalization and robustness. to measure the effectiveness of the proposed method, we evaluate the performance on two benchmarks based on ubuntu irc channel. one was released by @xcite where a few addressee labels were missing. the other was released by @xcite where addressee labels were provided for each utterance. experimental results show that madnet outperforms previous methods by significant margins, achieving a new state-of-the-art performance for mpc generation especially in the more common and challenging setting where a few addressee labels are missing. in summary, our contributions in this paper are three-fold: 1) to the best of our knowledge, this paper makes the first attempt to explore the issue of missing addressee labels and to target on more common mpc scenarios. 2) a fully-connected heterogeneous graph architecture along with em training is designed to help deduce the addressee of an utterance for improving mpc generation. ## preliminaries problem formulation the task of response generation in mpcs is to generate an appropriate response r given the conversation history, the speaker of a response, and which utterance the response is going to reply to. this can be formulated as: here, g is a heterogeneous graph, c is the context of dialogue history, θ is the model parameters. the speaker and addressee of the response are known and its contents are masked. the response tokens are generated in an auto-regressive way. r t and @xmath0. |r| is the length of r. next, we briefly present the key process of the hetermpc baseline @xcite to avoid lengthy method descriptions, which shares the gnn backbone with our method. readers can also refer to @xcite for more details. ## methodology in this section, we first describe how to construct a fully-connected conversation graph to ensure message passing between conversation fragments. then, the expectation and maximization steps for addressee deduction are defined. finally, an addressee initialization method is designed for better convergence of the em algorithm. ## conclusion we present madnet to maximize addressee deduction expectation to study the issue of scarcity of addressee labels in multi-party conversations. four types of latent edges are designed to model interactions between indirectly related utterances, and those between utterances and interlocutors for conversation contextualization. furthermore, an em-based approach is designed to deduce silver addressee labels and optimize the quality of generated responses. experimental results show that the proposed madnet outperforms previous methods by significant margins on two benchmarks of mpc generation. it especially shows better generalization and robustness in the more common and challenging setting where a few addressee labels are missing.