With a Little Help from my (Linguistic) F riends: Topic segmentation of multi-party casual conversations (2024)

Abstract: Topics play an important role in the global organisation of a conversation as what is currently discussed constrains the possible contributions of the participant. Understanding the way topics are organised in interaction would provide insight on the structure of dialogue beyond the sequence of utterances. However, studying this high-level structure is a complex task that we try to approach by first segmenting dialogues into smaller topically coherent sets of utterances. Understanding the interactions between these segments would then enable us to propose a model of topic organisation at a dialogue level. In this paper we work with open-domain conversations and try to reach a comparable level of accuracy as recent machine learning based topic segmentation models but with a formal approach. The features we identify as meaningful for this task help us understand better the topical structure of a conversation.

## introduction topics play a crucial role in understanding and interpreting conversations. when participants have a wrong understanding of the current topic, their contributions can become irrelevant @xcite or even incoherent, leading to confusion among the addressees. similarly, misinterpreting the topic can hinder a participant's ability to understand others' interventions accurately. while topics are more constrained and easily identifiable in controlled settings, such as formal work meetings, open-domain casual conversations have a greater flexibility, allowing participants to switch topics with minimal indication and still be followed by others in the conversation. the larger the number of participants, the more challenging it becomes to maintain control, as everyone contributes to the context. understanding how topics interact in dialogue is thus essential when it comes to modelling dialogue structure beyond the sequence of utterances. however, analysing this structure requires insight on the topics themselves. being able to segment a dialogue into topically coherent segments seems to be a first step towards modelling high level dialogue structure. the segments could later be linked inside a structure that describes the interactions between them. this task, called dialogue topic segmentation (dts), finds utility in dialogue generation @xcite and summarising @xcite , among other applications. dts has received less attention compared to monologue or written text topic segmentation, primarily due to the scarcity of annotated data but some dts approaches get good results on taskoriented dialogues @xcite or conversations with a restricted set of possible topics such as meeting minutes @xcite . @xcite suggest another method to tackle more varied dialogues. they use the texttiling algorithm @xcite , that relies on a similarity metric between subsequent blocks of text to identify topic boundaries, and enhance it with a learned utterance-pair coherence scoring model based on bert @xcite as similarity metric. they obtain good results in english and chinese when evaluating their model on three datasets: dialseg_711 @xcite , doc2dial @xcite , and zys @xcite . even though these datasets cover different domains, they all contain task-oriented conversations. evaluating this model on more open-domain dialogues would provide insight on the limits of its generalisation capability. in this paper we present an improved version of the original texttiling algorithm foot_0 , where we use linguistics properties of dialogue to identify the topic shifts. our aim is to reach a comparable level of accuracy as the model proposed by @xcite but with a formal approach. since we are interested in the structure of topical interactions, an explainable model would help us better understand what features play a role in perceiving topic shifts. a rule-based approach also has the advantage of minimising the amount of computation required by our model, which is more sustainable. following our goal to build a general model of interaction, we work with multi-party casual conversations, characterised by their more chaotic nature. to summarise our contributions in this work: we (1) reproduced @xcite 's model; (2) trained a bert-based model to improve the text-tiling algorithm; (3) improved the texttiling algorithm based on linguistic properties; (4) evaluated topic segmentation in multiparty casual conversations using the friends corpus. ## improving the original texttiling algorithm with linguistic features in parallel to the experiments with the bertbased model, we worked on enhancing the original texttiling algorithm with more linguistic features. such a model has the advantage of being explainable, as opposed the bert-based one. two basic ideas are discussed in the literature when it comes to identifying changes in topics @xcite . the first one is that a change in topic implies an important change in terms of content. for example, it corresponds to the introduction of a new vocabulary @xcite which is more or less constant inside a topic @xcite . additionally in a dialogue, the most active participants can change based on the topic. the second insight is that there exist distinctive topic boundary features such as discourse markers or aspects of the prosody. questions can also indicate a continuity of the current topic. we decided to include both approaches in our version of texttiling. our idea was to complexify the similarity metric by taking more features of the text into account. ## final comparisons and conclusion 6.1 comparison of all the models as we expected, using the friends dataset for training gives significantly better results than a less relevant dataset, as the one used originally by @xcite . nevertheless, we can note that in terms of f1-score, our feature-based model and the ml-based model trained on the original data are equivalent. as we have explained earlier, the f1-score is not the most meaningful measurement for the topic segmentation task but this result still shows a certain generalisation capacity from xing and carenini (2021)'s model. we also see a clear improvement between the original texttiling algorithm and our enhanced version, especially for the p k , which shows that the linguistic properties we considered and described in section 5.2 are relevant for our task. the best model is the ml-based model trained on friends when we look at the f1 and the f k1 . however, our feature-based model is better in terms of p k and f k2 . this shows that for the topic segmentation task a feature-based model can compete with language models on certain types of dialogues. moreover, the bert-based model is not very stable on our dataset, which we believe is due to the complexity of multi-party casual conversations as opposed to the more controlled dialogues usually used in topic segmentation. our approach based on linguistic features provides an explainable baseline.