Online Conversation Disentanglement with Pointer Networks (2020)

Abstract: Huge amounts of textual conversations occur online every day, where multiple conversations take place concurrently. Interleaved conversations lead to difficulties in not only following the ongoing discussions but also extracting relevant information from simultaneous messages. Conversation disentanglement aims to separate intermingled messages into detached conversations. However, existing disentanglement methods rely mostly on handcrafted features that are dataset specific, which hinders generalization and adaptability. In this work, we propose an end-to-end online framework for conversation disentanglement that avoids time-consuming domain-specific feature engineering. We design a novel way to embed the whole utterance that comprises timestamp, speaker, and message text, and propose a custom attention mechanism that models disentanglement as a pointing problem while effectively capturing inter-utterance interactions in an end-to-end fashion. We also introduce a joint-learning objective to better capture contextual information. Our experiments on the Ubuntu IRC dataset show that our method achieves state-of-the-art performance in both link and conversation prediction tasks.

## introduction with the fast growth of internet and mobile devices, people now commonly communicate in the virtual world to discuss events, issues, tasks, and personal experiences. among the various methods of communication, text-based conversational media, such as internet relay chat (irc), facebook messenger, whatsapp, and slack, has been and remains one of the most popular choices. multiple ongoing conversations seem to occur naturally in such social and organizational interactions, especially when the conversation involves more than two participants @xcite . for example, time sp message text 02:26 @xmath0:26 zelot hi, where can i get some help in regards to issues with mount? 02:26 tuxthepenguin after taking it out 02:26 hannasanarion tuxthepenguin, try booting with monitors connected to motherboard 02:26 pnunn tuxtheponguin, sounds like there is on board graphics as well, so try that without the card 02:26 pnunn yeh, just one monitor though 02:27 hannasanarion pnunn, right 02:27 tuxthepenguin makes sense to me :) 02:27 pnunn process of elimination. 02:27 tuxthepenguin along with occam's razor 02:27 bashing-om zelot: if you are on a supported release of 'buntu, this is a good place to ask. 02:27 tuxthepenguin any solution is most likely the simplest one 02:28 wllrt i'm a emacs newb and looking to prevent rsi. figure 1 : an excerpt of a conversation from the ubuntu irc corpus (best viewed in color). same color reflects same conversation. mentions of names are highlighted. however, often there are no explicit structures or metadata to separate out the individual conversations. naive heuristics to disentanglement often lead to sub optimal results as @xcite found that only 10.8% of the conversations in the widely used ubuntu irc dialog corpus were extracted correctly by the heuristics employed by @xcite . previous studies have therefore investigated traditional machine learning methods with statistical and linguistic features for conversation disentanglement, e.g., @xcite , @xcite , @xcite , to name a few. the task is generally solved by first finding links between utterances, and then grouping them into a set of distinct conversations. recent work by @xcite and @xcite adopt deep learning approaches to learn abstract linguistic features and compute message pair similarity. however, these methods heavily rely on hand-engineered features that are often too specific to the particular datasets (or domains) on which the model is trained and evaluated. for example, many of the features used in @xcite are only applicable to the ubuntu irc dataset. this hinders the model's generalization and adaptability to other domains. in this work, we propose a more general framework for conversation disentanglement while avoiding time-consuming and domain-specific feature engineering. in particular, we cast link prediction as a pointing problem, where the model learns to point to the parent of a given utterance (figure 2 ). each pointing operation is modeled as a multinomial distribution over the set of previous utterances. a neural encoder is used to encode each utterance text along with its speaker and timestamp. the pointing function implements a custom attention mechanism that models different interactions between two utterances. this results in an end-toend neural framework that can be optimized with a simple cross entropy loss. during training, we jointly model the reply-to relationship and pairwise relationship (whether two utterances in the same conversation) under the same framework, so that more contextual and structural information can be learned by our model to further improve the disentangling performance. furthermore, the framework supports online decoding, which is naturally provided by the pointer network framework, disentangles a conversation as it unfolds, and can provide real-time help to participants in contributing to the right conversations. we performed extensive experiments on the recently released ubuntu irc dataset @xcite and demonstrate that our approach out-performs previous methods for both link prediction and conversation prediction tasks. foot_0 ablation studies reveal the importance of different components of the model and special handling of self-links. our framework is generic and can be applied to chat conversations from other domains. ## background in this section, we give a brief overview of previous work on conversation disentanglement and the generic pointer network model. ## our disentanglement model given a sequence of streaming utterances @xmath1, u 2 , . . . , u i , . . .}, our task in link prediction ( §3.1) is to find the parent utterances u p i ⊂ @xmath2. here, @xmath3, that is, an utterance can reply to itself (i.e., @xmath4, for example, the initial message in a conversation or a system message. besides, one utterance may have multiple parents, and one parent can be replied to by multiple (children) messages. for example, in figure 1 , both pnunn and hannasanarion reply to tuxtheponguin's message. the case of one message replying to multiple parents is very rare in our corpus (see after link prediction, we employ a decoding algorithm ( §3.3) to construct the individual threads. ## experiment in this section, we present our experiments -the dataset used, the evaluation metrics, experimental setup, and the results with analysis. ## conclusion in this paper, we have proposed a novel online framework for disentangling multi-party conversa-tions. in contrast to previous work, our method reduces the effort of complicated feature engineering by proposing an utterance encoder and a pointer module that models inter-utterance interactions. moreover, we propose a joint-training framework that enables the pointer network to learn more contextual information. link prediction in our framework is modeled as a pointing function with a multinomial distribution over previous utterances. we also show that our framework supports online decoding. extensive experiments have been conducted on the #ubuntu dataset, which show that our method achieves state-of-the-art performance on both link and conversation prediction tasks without using any handcrafted features. there are some possible future directions from our work. we have shown in our experiments that self-link predictions have a significant impact on clustering results. this reminds us that neither our and most of the existing methods took good advantage of graph information in disentangling conversations. this can be done in two ways, encoding, and decoding. from the encoding side, it would be ideal to encode an utterance within its context. one challenge for this problem is that conversations are tangled, so sequential encoding methods like the one of @xcite would not be appropriate. from the decoding side, a promising direction would be to make global inference in a more efficient way.