Speaker Clustering in Textual Dialogue with Pairwise Utterance Relation and Cross-corpus Dialogue Act Supervision (2022)

Abstract: We propose a speaker clustering model for textual dialogues, which groups the utterances of a multi-party dialogue without speaker annotations, so that the actual speakers are identical inside each cluster. We find that, without knowing the speakers, the interactions between utterances are still implied in the text, which suggest the relations between speakers. In this work, we model the semantic content of utterance with a pre-trained language model, and the relations between speakers with an utterance-level pairwise matrix. The semantic content representation can be further instructed by cross-corpus dialogue act modeling. The speaker labels are finally generated by spectral clustering. Experiments show that our model outperforms the sequence classification baseline, and benefits from the auxiliary dialogue act classification task. We also discuss the detail of determining the number of speakers (clusters), eliminating the interference caused by semantic similarity, and the impact of utterance distance.

## introduction processing dialogues is a classical linguistic task. with the development of pre-trained language models in recent years, studies on dialogues have made great progress @xcite @xcite . in general, these training processes, especially pre-training, need a large amount of data. meanwhile, most of dialogue models are designed to input speaker information, for example, applying trainable speaker embeddings, or just assuming the dialogue is composed of two speakers involved turn by turn, to introduce dialogue structure information into the models. but for common researchers, dialogue data is hard to collect. datasets like subtitles @xcite contain a lot of dialogue data of daily communication, but lack of speaker annotation. some re-* corresponding author. in order to reconstruct the speaker labels in the dialogue, this work is dedicated to the method of speaker clustering. different from previous researches on speaker identification @xcite @xcite , which aim at selecting the most similar speaker from the pre-modeled candidates, the speaker clustering task aims at grouping the utterances into speakerspecific clusters without any preset candidates @xcite . it is more useful because it works on open corpus where the speakers cannot be modeled in advance. speaker clustering is relevant to dialogue structure, because the process of turns follows certain patterns. these patterns include the semantic content and the communicative functions of utterance, and can be specifically represented as the dialogue act (da) of utterance and associations between dialogue acts respectively @xcite . the associations between dialogue acts include questionanswer, request-response, offer-acceptance, etc., which are closely related to alternation of speakers. conversely, the relations between speakers will be predicable if these patterns are available from textual utterances. in this work, the speaker relations are inferred from the communicative functions by using an utterance-level pairwise matrix. the speaker relations have only two possible values, either same or different. the relations among the whole dia-logue form this matrix, which is regarded as the similarity matrix of the ground speakers. the matrix can reconstruct the clusters of speakers with a density-based clustering method. the most popular algorithms of density-based clustering are spectral clustering @xcite and dbscan @xcite . in this work, we use spectral clustering as the implementation, because it is less sensitive to sparse points, which follows this task that each utterance must be in a cluster. based on the above analysis, we build a model that models the semantic content of utterances with multi-task cross-corpus da supervision, calculates the speaker relations with the form of bilinear, and generates the cluster labels with the method of spectral clustering. the main contributions of this paper are summarized as follows. ## related work this work targets for speaker clustering, and is based on the theories of dialogue structure. ## model the three main processes of this model are getting representation of utterances, cooperating with cross-corpus da supervision, and calculating the similarity score. therefore, as the overall structure shown in figure 1 , the model is divided into three parts in general: the utterance embedding part (blue), the speaker clustering part (yellow), and the set-specific da classification part (red). during training process, each data batch consists of b dialogues. to simplify the expression, we will omit the term of batch averaging in the following formulas. in a dialogue of the batch, there are t turns. the speaker of the i-th turn is s i . the utterance of the i-th turn is u i . the objective of the model follows multi-task learning framework. the loss function of each data batch l is a combination of the binary cross entropy loss of the pairwise matrix l mat and the cross entropy loss of the da prediction l da . we use a hyperparameter λ to moderate the association between the two objectives. formally, the objective and structure will be described in detail in the following sections. ## discussion in this section, we discuss whether the substructures of the model work accurately, and whether the model is disturbed by some possible factors (semantic similarity and utterance distance). ## conclusion we propose a text-based dialogue speaker clustering model. based on the theory of the dialogue structure, the model holds the semantic content and the communicative functions explicitly with the bert layer and the similarity matrix. the model is enhanced by the idea of cross-corpus supervision with the auxiliary set-specific dialogue act classification task. it finally generates the cluster labels of speakers with spectral clustering. our model outperforms the sequence classification baseline and the non-da ablation on almost all tests. additional discussion illustrates the accuracy in predicting the number of speakers (clusters) and the ability to distinguish between speaker relation and semantic similarity of our model. we also show that the precision of speaker similarity prediction varies with utterance distance. in future research, it is worth trying further pretraining the model on dialogue data, which will likely help to perceive dialogue turns and extract better utterance embeddings. we will also explore for a method to make the similarity calculation between long-distance utterance pairs more accurate.