Using Interaction Style Dimensions to Characterize Spoken Dialog Corpora (2022)

Abstract: The construction of spoken dialog systems today relies heavily on appropriate corpora, but corpus selection is more an art than a science. As interaction style properties govern many aspects of dialog, they have the potential to be useful for relating and comparing corpora. This paper overviews a recently-developed model of interaction styles and shows how it can be used to identify relevant corpus differences, estimate corpus similarity, and flag likely outlier dialogs.

## motivation today the process of selecting corpora for dialog systems training or tuning is rarely systematic. this is a problem because dialog systems developers rely heavily on machine learning from corpora to acquire the various knowledge and parameters needed for effective systems. models for predicting likely corpus suitability would therefore be very useful, but existing methods for corpus comparison rely mostly on lexical and topic overlap, e.g. @xcite , making it hard to predict how well other types of knowledge will transfer. the scientific investigation of dialog behaviors is similarly impeded by corpus choice issues. different research teams choose corpora to study for all sorts of reasons, leading to a healthy diversity, but also to many contradictory findings @xcite @xcite . methods for systematically describing corpus properties could help resolve these, potentially enabling the field of computational pragmatics to clearly describe the realm of validity of each generalization. this paper focuses on interaction style, as this is an essential issue in providing high quality user experiences @xcite . this is, moreover, no longer a distant goal, as core speech components have advanced to the point where it is becoming possible to implement situation-appropriate turn taking, politeness behaviors, rapport building strategies, and so on @xcite . because our fundamental knowledge in these areas are still spotty, developers rely on discovery or learning from corpora. indeed, it is still common for a new development project to start with the collection of a new corpus, specific to the task, domain, user demographic, system persona and so on. instead, we would like to be able to better exploit existing resources @xcite . one recent success was a socially well-behaved recommendation system for movies, created by discovering behaviors from a suitable subset of switchboard data @xcite . selection of this subset was easy because switchboard was designed around topics, and in particular the "movies" tag was available. however, we would like to be able to more precisely delineate relevant corpus subsets, and to do so even when annotations are lacking. this paper introduces three ways to characterize spoken dialog corpora and their subsets. ## precursor work biber, in his landmark contribution to style description, investigated what he termed "conversation text types" @xcite . using transcripts from various corpora as data and a text-based feature set, he used principal component analysis to derive three dimensions of variation, and showed how different conversations could be automatically located in this space. this method has been very influential in the comparison of diverse text corpora, and also occasionally for speech corpora @xcite . however these models generally seem to have low explanatory power; for example, biber's three dimensions accounted for only 36% of the variance. further, although acoustic-prosodic features potentially provide much more information than text, these have been used in corpus selection so far only by @xcite , who demonstrated their value, but only for the narrow problem of training emotion recognizers. overall, work in this tradition appears not to have found practical use. in contrast to text-based models @xcite , styles in spoken dialog, and in particular interaction styles, have been less studied. much work in this area has built on tannen's seminal observations on "conversational styles" @xcite . importantly, these are not fixed properties of speakers, and frequently vary even in the course of a conversation @xcite . more recently, computational models have been developed to study style in dialog @xcite @xcite . these works have variously used features of turn-taking and prosodic and other behaviors to derive models of style. however these models have previously been applied only to questions of how individuals vary in style, not to corpus characterization. ## model properties the explorations reported in this paper build on our own model of interaction style variation @xcite , because it is the most comprehensive and because the code is available. the purpose of this section is only to explain the model briefly while clarifying the aspects not clear in @xcite but relevant for the current exploration. for current purposes, the model serves to take as input one or more 30-second fragments of american english conversation, and to output a representation of its style as a vector of length 8: that is, it maps dialogs into a vector space representation of interaction styles. while for current purposes this is used as a black box, it may be worth overviewing the steps of the process. ## use 1: corpus characterization this model supports visualization of corpus differences. as an example, if we view switchboard @xcite as a collection of subcorpora, one per topic, we can map them out, for example by plotting the average interaction style of all fragments within that topic. figure 1 shows this for dimensions 1 and 3. (projections onto other dimensions are available at the companion website.) to avoid clutter, the figure show only topics for which there was ample data (225 minutes or more) or which were among the most distinctive topics, in terms of distance on these two dimensions from the global average style. the positions of the topics in the figure suggest that the model is at least picking up something meaningful. it is informative to consider further some of the topics that appear, at first glance, to be misplaced. for example, it may seem strange that the model characterizes conversations on the topic of "metric system" as positive in style, but listening to examples shows that these conversations are mostly by engineers, who indeed discussed it positively. it may also seem strange that "woodworking" and "painting" are placed differently, as both can be at-home hobbies and projects. according to the model, their interaction styles are very different, as seen in in general, diagrams like these may help researchers and developers understand the diversity within and between corpora. ## use 2: similarity estimation such similarity estimates could be used to support targeted data augmentation. considering again the scenario of seeking data to train a movie recommendation system, the subcorpora closest to "movies" were "tv programs," "clothing and dress," "football," and "baseball," indicating that these would be likely be most compatible as supplemen-tary data. ## use 3: identifying outliers the similarity metric could also be used in support of data cleaning. for example, many conversations in switchboard have the "movies" tag, but not all fragments are good exemplars of the typical style for talking about movies. the model can help identify these, as fragments distant from the average interaction style for this topic. for the movies topic, examination of the five most distant fragments revealed that these were indeed mostly atypicaltwo involved strong moral judgments, and one was mostly about audience behavior -and would be good candidates for exclusion from the training set for a movie recommending system with a normal, upbeat style. ## prospects spoken data is fundamentally richer than text data, and recent work is exploiting this to create more informative models of corpus similarities and differences. this brief report has proposed new ways to exploit one such model, involving interaction style. eventually, direct quantitative evaluation of this method should be done. one way would be to examine the correspondences to human judgments of interaction styles and style similarities. this would be a long-term project, but potentially of great benefit for systematizing the scientific study of dialog phenomena. in the short term, we think the value of these methods will instead be shown by their practical value: their ability to support the creation of bettertailored dialog systems, and to reduce the datacollection efforts required to develop them. more specifically, in addition to the three ways illustrated above, we conjecture that the model will be useful in at least three other ways. 1) it could suport quality control and consistency control during corpus collection. 2) it could support attempts to collect corpora with a sweet-spot style that is simultaneously natural for humans and implementable with current technology @xcite , by identifying the dimensions in which such corpora most resemble both humanhuman dialogs and technically-realizable dialogs. to support such uses, the code is available at @xcite .