Structural Characterization for Dialogue Disentanglement (2022)

Abstract: Tangled multi-party dialogue contexts lead to challenges for dialogue reading comprehension, where multiple dialogue threads flow simultaneously within a common dialogue record, increasing difficulties in understanding the dialogue history for both human and machine. Previous studies mainly focus on utterance encoding methods with carefully designed features but pay inadequate attention to characteristic features of the structure of dialogues. We specially take structure factors into account and design a novel model for dialogue disentangling. Based on the fact that dialogues are constructed on successive participation and interactions between speakers, we model structural information of dialogues in two aspects: 1)speaker property that indicates whom a message is from, and 2) reference dependency that shows whom a message may refer to. The proposed method achieves new state-of-the-art on the Ubuntu IRC benchmark dataset and contributes to dialogue-related comprehension.

## introduction communication between multiple parties happens anytime and anywhere, especially as the booming social network services hugely facilitate open conversations, such as group chatting and forum discussion, producing various tangled dialogue logs @xcite @xcite @xcite . whereas, it can be challenging for a new participant to understand the previous chatting log since multi-party dialogues always exhibit disorder and complication @xcite @xcite . in fact, it is because of the distributed and random organization, multi-party dialogues are i cant ssh into it until i log in as a user. do the ubuntu lts versions able to run as a live cd? regum: because the root accoutn is disabled in ubuntu, by default. figure 1: here is an example piece of multi-party chatting logs from ubuntu irc @xcite . jancoow figures out conversation threads, understand the context and reply to the related message (1003 from daftykins), and the dialogue develops. as the example shown in figure 1 , the development of a multi-party dialogue has the following characteristics: 1) random users successively participate in the dialogue and follow specific topics that they are interested in, motivating the development of those topics. 2) users reply to former related utterances and mention involved users, forming dependencies among utterances. as a result, multiple ongoing conversation threads grow as the dialogue proceeds, which breaks the consistency and hinders both humans and machines from understanding the context, let alone giving a proper response @xcite @xcite . in a word, the behavior of speakers determines the structure of a dialogue passage. and the structure causes problems of reading comprehension. hence, for better understanding, structural features of dialogue context deserve special attention. disentanglement is worthy of study. decoupling messages or clustering conversation threads help with screening concerned parts among contexts, therefore it may be naturally required by passage comprehension, and related downstream dialogue tasks @xcite @xcite , such as response selection, question-answering, etc. nevertheless, existing works on dialogue disentanglement @xcite @xcite generally ignore or pay little attention to characters of dialogues. earlier works mainly depend on feature engineering @xcite @xcite , and use well-constructed handcrafted features to train a naive classifier @xcite or linear feed-forward network @xcite . recent works are mostly based on two strategies: 1) two-step @xcite @xcite @xcite and 2) end-to-end @xcite . in terms of the two-step method, the disentanglement task is divided into matching and clustering. it means firstly matching utterance pairs to detect reply-to relations and then clustering utterances according to the matching score. in the end-toend strategy, alternatively, for each conversation thread, the state of dialogue is modeled, and is mapped with a subsequent utterance to update. at the same time, the subsequent utterance is judged to belong to the best-matched thread. nonetheless, the essence of both strategies is to model the relations of utterance pairs. recently, pre-trained language models (prlms) @xcite @xcite have brought prosperity to numbers of natural language processing tasks by providing contextualized backbones. various works have reported substantial performance gains with the contextualized information from prlms @xcite @xcite @xcite . studies on dialogue disentanglement also get benefit from prlms @xcite , whereas, there is still room for improvement due to their insufficient enhancement of dialogue structure information. so as to enhance characteristic structural features of tangled multi-party dialogues, we design a new model as a better solution for dialogue disentanglement. structure of a multi-party dialogue is based on the actions of speakers according to the natural development of dialogues. hence, we model two structural features to help with the detection of reply-to relationships: 1)user identities of messages, referred to as speaker property; and 2) mention of users in messages, called reference dependency. with the two features enhanced between encoding and prediction, the model makes progress on dialogue disentanglement. evaluation is conducted on dstc-8 ubuntu irc dataset @xcite , where our proposed model achieves new state-of-the-art. further analyses and applications illustrate the advantages and scalability additionally. our source code is available foot_0 . ## methodology the definition of the dialogue disentanglement task and details of our model are sequentially presented in this section, illustrating how we make efforts for disentanglement with dialogue structural features. ## experiments our proposed model is evaluated on a largescale multi-party dialogue log dataset ubuntu irc @xcite , which is also used as a dataset of dstc-8 track2 task4. the results show that our model surpasses the baseline significantly and achieves a new state-of-the-art. ## applications empirically, it is consistent with our intuition that clarifying the structure of a passage helps with reading comprehension. this section studies the potential of dialogue disentanglement by conducting experiments on different tasks and domains. ## conclusion in this paper, we study disentanglement on long multi-party dialogue records and propose a new model by paying close attention to the characteristics of dialogue structure, i.e., the speaker property and reference dependency. our model is evaluated on the largest and latest benchmark dataset ubuntu irc, where experimental results show a new sota performance and advancement compared to previous work. in addition, we analyze the contribution of each structure-related feature by ablation study and the effect of the different model architecture. our work discloses that speaker and dependency-aware structural characters are significant and deserve studies in multi-turn dialogue modeling. recently study made efforts to analyze measures @xcite , where human satisfaction measures are applied on metrics: normalized mutual information (nmi), adjusted rand index (ari), shen-f, and f1. results show that f1 is the most similar to human satisfaction scores, while ari, nmi, and shen-f tend to overrate disentanglement results but f1 underrates. here we present a scatterplot 4 based on our experimental results.