Disentangling Online Chats with DAG -structured LSTM s (2021)

Abstract: Many modern messaging systems allow fast and synchronous textual communication among many users. The resulting sequence of messages hides a more complicated structure in which independent sub-conversations are interwoven with one another. This poses a challenge for any task aiming to understand the content of the chat logs or gather information from them. The ability to disentangle these conversations is then tantamount to the success of many downstream tasks such as summarization and question answering. Structured information accompanying the text such as user turn, user mentions, timestamps, is used as a cue by the participants themselves who need to follow the conversation and has been shown to be important for disentanglement. DAG-LSTMs, a generalization of Tree-LSTMs that can handle directed acyclic dependencies, are a natural way to incorporate such information and its non-sequential nature. In this paper, we apply DAG-LSTMs to the conversation disentanglement task. We perform our experiments on the Ubuntu IRC dataset. We show that the novel model we propose achieves state of the art status on the task of recovering reply-to relations and it is competitive on other disentanglement metrics.

## introduction online chat and text messaging systems like facebook messenger, slack, wechat, whatsapp, are common tools used by people to communicate in groups and in real time. in these venues multiple independent conversations often occur simultaneously with their individual utterances interspersed. it is reasonable to assume the existence of an underlying thread structure partitioning the full conversation into disjoint sets of utterances, which ideally represent independent sub-conversations. modeling work on conversation disentanglement spans more than a decade. @xcite use feature based linear models to find pairs of utterances belonging to the same thread and heuristic global algorithms to assign posts to threads. @xcite and @xcite , while also adopting similar heuristics, use features extracted through neural models, lstmss @xcite and siamese cnns @xcite respectively. wang et al. (2011) follow a different approach by modeling the interactions between the predicted reply-to relations as a conditional random field. one challenge in building automatic systems that perform disentanglement is the scarcity of large annotated datasets to be used to train expressive models. a remarkable effort in this direction is the work of @xcite and the release of a dataset containing more that 77k utterances from the irc #ubuntu channel with annotated reply-to structure. in the same paper, it is shown how a set of simple handcrafted features, pooling of utterances glove embeddings @xcite , and a feed-forward classifier can achieve good performances on the disentanglement task. most of the follow-up work on the dataset relies on bert @xcite embeddings to generate utterance representations @xcite @xcite . @xcite use an additional transformer module to contextualize these representations, while @xcite ; @xcite use an lstm. two exceptions are @xcite , which models thread membership in an online fashion and discards reply-to relationships, and the recent yu and joty (2020a) which uses pointer networks @xcite . in this short paper, we use dag-structured lstms @xcite to study disentanglement. as a generalization of tree-lstms @xcite , dag-lstms allow to faithfully represent the structure of a conversation, which is more properly described as a directed acyclic graph (dag) than a sequence. furthermore, dag-lstms allow for the systematic inclusion of structured information like user turn and mentions in the learned representation of the conversation context. we enrich the representation learned by the dag-lstm by concatenating to it a representation of the thread to which the utterance belongs. this thread encoding is obtained by means of a gru unit @xcite and captures thread specific features like style, topic, or persona. finally we manually construct new features to improve username matching, which is crucial for detecting user mentions, one of the most important features for disentanglement. our results are summarized in ## conclusions in this paper we apply, for the first time, dag-lstms to the disentanglement task; they provide a flexible architecture that allows to incorporate into the learned neural representations the structured information which comes alongside multi-turn dialogue. we propose thread encoding and a new set of features to aid identification of user mentions. there are possible directions left to explore. we modeled the reply-to relationships in a conversation by making an assumption of conditional independence of reply-to assignments. this is possibly a poor approximation and it would be interesting to lift it. a challenge with this approach is the computational complexity resulting from the large dimension of the output space of the reply-to classifier. we notice that thread encoding allows a non-greedy decoding strategy through beam search which would be interesting to further explore.