An Empirical Study of Topic Transition in Dialogue (2022)

Abstract: Although topic transition has been studied in dialogue for decades, only a handful of corpora based quantitative studies have been conducted to investigate the nature of topic transitions. Towards this end, this study annotates 215 conversations from the switchboard corpus, perform quantitative analysis and finds that 1) longer conversations consists of more topic transitions, 2) topic transition are usually lead by one participant and 3) we found no pattern in time series progression of topic transition. We also model topic transition with a precision of 91%.

## introduction human conversation consists of multiple natural topic transitions, from introductions, to topics of interest, and on to leave talking, and thus relies on topic change and shading mechanisms to allow participants to maintain and change topics 1 . an example of topic transition can be seen in figure 1 , participants first begin by talking about each others age, then move on to the places they want to visit and finally move on to talking about the state of arizona in the usa. although topic transition has been studied in linguistics for decades @xcite @xcite , there are only a few corpora based studies investigating the nature of topic change. this is because of the labour intensive task of manually annotating datasets. even though the task of annotation is labour intensive and manual, it is necessary to empirically understand how human participants engage in topic transition in a conversation. towards this end, this work annotates 215 conversations from the switchboard @xcite corpus and studies different aspects 1 our annotated dataset and models do not differentiate between the types of topic transition (change, shift, shading, fading etc.) depicted in gardner's model @xcite . for simplicity, this paper uses 'topic transition' to describe all forms. where necessary, it uses specific terms to differentiate. ## background theory definitions of topic in the literature fall into two categories; sentence level @xcite and discourse level @xcite . @xcite emphasizes the presence and identification of a topic to be a intuitive phenomenon answering the question of 'being about'. multiple sentence-level topics about the same thing may consist of a discourselevel topic @xcite . as this study is concerned with discourse level topic annotation, we adopt the definition of @xcite which maintains that topic at a discourse level is the "segments of the discourse sharing coherent information (about the same thing)". topic transition has been categorized by @xcite , whose model of topic development in spoken interaction details the multiple means by which humans introduce, maintain, and change topics. two areas which have received particular attention in the literature are topic change and topic shift. they have been defined as the point between two pieces of discourse which are considered to have different topics. @xcite differentiates between topic change and topic shift as having low and high degrees of connectivity respectively to the previous topic. topic shift includes both topic shading and topic fading @xcite @xcite . topic change includes reintroduction and full blown change. we annotate all such topic transitions under one common label. ## related work related work in the literature is primarily found in the domains of manual topic annotation and automatic topic segmentation. ## the annotation framework we annotate 215 conversations from the switchboard-1 release 2 corpus @xcite . annotations are based on previous studies demonstrating that naive annotators are capable of annotating topic transition with success. @xcite @xcite . switchboard corpus the switchboard-1 release 2 corpus consists of recordings of about 2400 telephone conversations between 543 distinct speakers who did not know each other @xcite . all interlocutors spoke american english. they choose a topic from a list of about 70 topics and were connected to another interlocutor by a switchboard robot. about 50 of the 70 topics were chosen regularly. the conversation is not limited to the initial topic and participants could transition topics at any time. the individual conversation transcripts have been transcribed and annotated to the utterance level and include conversation ids, time stamps, and label for speakers identity. annotation framework 215 conversations were drawn at random from the switchboard corpus are annotated at sentence-level. the annotation were performed for start (s) and end (e) of the conversation, greeting and leave taking (gil), topic, topic transition (c), and failed topic transition (x). detailed annotation guidelines can be seen in appendix d. this manually annotated corpus consists of 20, 566 turns from 215 conversation. the average number of turns per conversation is 96 with the shortest conversation lasting 33 turns and the longest conversation lasting 242 turns. mean turns per conversation were found to be 8 and mean turns per topic were observed to be 12. the conversations were annotated by two annotators. the inter-annotator agreement (cohen 's kappa) obtained on a sample of five conversations is 0.64, signifying substantial agreement. ## empirical studies of topic transitions having obtained an annotated corpus of 215 corpus, we conducted quantitative analysis on some aspects nature of topic change. the empirical findings are discussed in the subsections below. length of a conversation and number of topic transitions longer conversation are a sign of successful and engaging conversation. we wanted to examine if longer conversation consist of more topic transitions than shorter conversation or the number of topic transitions remains similar and some topics are conversed for more turns than others. towards investigating this relationship, we cal-culate number of topic transitions per conversation and plot it in figure 2 . the value of pearson correlation coefficient is found to be 0.74, indicating a positive correlation between length of a conversation and number of topic transitions. we also plot a linear regression line and observe a r 2 value of 0.55 (@xmath0.001). figure 2 further highlights that number of topic transitions increase as length of a conversation increases. most conversations consist of five to thirteen topic transitions. thus, it is observed that longer conversations have more topic transitions. ## modelling topic transition in addition to the empirical studies performed, we also modelled topic transition on the manually annotated switchboard corpus, described in section 4. before describing the modelling in detail, we briefly describe the approaches to model topic transition in literature. approaches to topical segmentation in dialogue include unsupervised and supervised methods. unsupervised algorithm work on finding similarity or dissimilarity between segments of text, texttiling (hearst, 1997) is a seminal work in unsupervised topic segmentation. supervised approaches work with hand-crafted features or deep learning based methods such as used @xcite @xcite . following the related research work, we formulate topic transition turn detection as a binary classification problem. we implement texttiling hearst (1997) as a baseline and then proceed to implement classical machine learning as well as deep learning based classification algorithms since, dialogue is inherently context based i.e. the next utterance is influenced by previous utterances and a topic can span across multiple turns, consecutive utterances are grouped by speaker and termed turn. texttiling @xcite is implemented (employing the code from nltk @xcite ). turns are formatted as paragraphs separated by two line breaks (\n\n) as required by texttiling which works with lexical cohesion. the last turn of a paragraph, obtained from texttiling, is labelled as topic transition turn and all other turns are labelled as topic continuation turns. additionally, as classic machine learning classifiers, naive bayes and lightgbm are implemented. finally, utilizing modern deep-learning based classification algorithms, xlnet @xcite is implemented using hugging face's transformers @xcite . ## conclusion empirical study on how participants engage in topic transitions in a dyad is presented. it is observed that longer conversations have more topic transitions, topic transition is generally carried out more by one participant and there is no particular pattern observed in time series of topic transition. this study was also able to detect topic transition in dialogue with 91% precision.