Filtering conversations through dialogue acts labels for improving corpus-based convergence studies (2020)

Abstract: Cognitive models of conversation and research on user-adaptation in dialogue systems involves a better understanding of speakers convergence in conversation. Convergence effects have been established on controlled data sets, for various acoustic and linguistic variables. Tracking interpersonal dynamics on generic corpora has provided positive but more contrasted outcomes. We propose here to enrich large conversational corpora with dialogue act (DA) information. We use DA-labels as filters in order to create data sub sets featuring homogeneous conversational activity. Those data sets allow a more precise comparison between speakers’ speech variables. Our experiences consist of comparing convergence on low level variables (Energy, Pitch, Speech Rate) measured on raw data sets, with human and automatically DA-labelled data sets. We found that such filtering does help in observing convergence suggesting that studies on interpersonal dynamics should consider such high level dialogue activity types and their related NLP topics as important ingredients of their toolboxes.

## introduction the way participants engaged in a conversation speak tends to vary depending on their interlocutor's speech. the tendency to co-adjust speaking styles in response to the partner speaking style is known as convergence. convergence is presented in general and influential models of communication such as accommodation theory @xcite or interactive alignment @xcite . this variation due to the other party has been studied for many levels of speech and language, for example in phonology @xcite or in prosody @xcite @xcite . our approach aims at deepening and generalizing the investigation of convergence and related effects in real-life corpora. considered from the angle of speech and linguistic variables, an essential aspect of conversations is their extreme variability. this is due to a large extent to different conversational activities speakers can participate in. for instance, they can enter in a storytelling sequence in which one interlocutor starts to produce mostly back-channels @xcite while the main speaker develops lengthy monologues. this variability makes comparison of values across participants very problematic. we propose to create subsets of similar dialogues acts (da) (e.g. 'statements' and 'back-channels', see our work concerns more specifically acoustic convergence. our definition of convergence comes from several studies @xcite @xcite , and consists of comparing distance between speakers in different parts of a conversation (see section 3). we do not claim it is the best measure to approach inter-personal dynamics (see (priva and sanker, 2019)) but it is an interesting way to assess convergence within a conversation and it allows to test whether our da based approach can help this domain. ## related work convergence has been approached at different granularity levels and for a large range of variables. in terms of granularity, studies can be interconversation comparisons or intra-conversation (focusing on the dynamics within conversations). inter-conversation comparisons range from simple inter-speaker correlation studies @xcite or, when the data allows, comparison between speech values of a speaker and his conversational partners vs. a speaker and all other non-partner corpus participants @xcite . intra-conversation studies vary a lot in terms of approaches ranging from "differencein-difference" convergence @xcite , @xcite , @xcite approaches consisting of comparing distances between speakers in different intervals to finer grained synchrony methods typically using sliding windows to compare local speaker similarities @xcite . while a large body of carefully controlled experiments on lab speech provided results on convergence, the results on real corpora (from the studies listed in the previous paragraph) provide a more complex picture, with a relative fragility of the effects @xcite and raised methodological comments (see @xcite and @xcite ). more precisely, for intra-conversation studies, @xcite found that participants tend to be more similar (in terms of gaps and pauses duration) to their partners than chance would predict. however, the absence of significant results in comparing the inter-speaker distance in the first and second halves of the conversations makes the authors conclude that convergence cannot be captured with such an approach. @xcite conducted a similar experiment (on intensity and pitch) on english map-task @xcite ) with partial positive results. the dynamic nature of the phenomenon as well as the social factors render such studies difficult to be performed. these two studies were grounded on conversational corpora that are sizeable but not huge (6 x 20 minutes for the @xcite ; and about 60 maptasks dialogues for (truong and heylen, 2012)). @xcite used the much bigger switchboard corpus but use only an inter-conversation approach. our hypothesis is that automatic entrainment and strategic adaptation are blending in to produce convergence and synchrony phenomena. our hypothesis is that low-level variables (such as intensity) are be more directly related to automatic entrainment, while higher-level variables (such as lexical or syntactic choices) are more prone to strategic adap-tation. this could explain why more and firmer results seem to be obtained on low-level variables @xcite . to summarize, convergence dynamic can be difficult to track in real conversations. our approach combines three ingredients that, to our best knowledge, were not yet brought together. first, we consider that a major reason for this difficulty comes from the heterogeneity of speech behaviors within the time-frame of a conversation. we propose to use da to filter conversational activities from large corpora. second, to account for strategic adaptation one must take precise care of speaker profiles. our approach therefore focuses on relatively low level variables to avoid as much as possible the "adaptation" part of the interpersonal dynamics. third, similarly to @xcite our approach is based on a large conversational corpus with the intention of overcoming noise and effect small magnitude by increasing the amount of data considered. ## dialogue act filtering and data sets switchboard (swbd) @xcite is a corpus of telephonic conversations between randomly assigned speakers 1 of american english discussing a preassigned topic. the corpus consists of 2430 conversations (of an average duration of 6 minutes) for a total of 260 hours, involving 543 speakers. the corpus has audio, time aligned transcripts and a segmentation into utterances. nxt data set @xcite . 2 the datagged set has been simplified to 42 tags but a few of them are dominating the distribution (statement: 36%, acknowledgment: 19%, opinion: 13%), illustrated in ## results our question is whether we can observe more reliably interpersonal dynamics in raw, manually datagged (small) or automatically da-tagged (large) data sets. an underlying question is whether the noise introduced by the da-tagging uncertainty and / or the data size reduction is compensated by the gain in homogeneity between the material that is compared. ## discussion we scrutinized convergence during the course of a conversation and in a real world setting (switchboard corpus). the positive results in our experiments complement the picture provided by the literature by showing that convergence effects do happen in the time course of conversation of generic corpus. moreover, we open up the possibility of a range of new studies taking advantage on arbitrary large corpora partially controlled a posteriori thanks to automatic dialogue act tagging. a appendices