Dramatic Conversation Disentanglement (2023)

Abstract: We present a new dataset for studying conversation disentanglement in movies and TV series. While previous work has focused on conversation disentanglement in IRC chatroom dialogues, movies and TV shows provide a space for studying complex pragmatic patterns of floor and topic change in face-to-face multi-party interactions. In this work, we draw on theoretical research in sociolinguistics, sociology, and film studies to operationalize a conversational thread (including the notion of a floor change) in dramatic texts, and use that definition to annotate a dataset of 10,033 dialogue turns (comprising 2,209 threads) from 831 movies. We compare the performance of several disentanglement models on this dramatic dataset, and apply the best-performing model to disentangle 808 movies. We see that, contrary to expectation, average thread lengths do not decrease significantly over the past 40 years, and characters portrayed by actors who are women, while underrepresented, initiate more new conversational threads relative to their speaking time.

## introduction movie and tv dialogues, or dramatic dialogues more generally, have offered linguists a wealth of resources to study conversational behaviors @xcite @xcite , including within nlp @xcite @xcite . while dramatic dialogues do not necessarily mimic conversations in real life, they present complex pragmatic and sociolinguistic phenomena that warrant study; given the widespread viewership of movies and tv, what appears on screen-both visually and in dialogue-can have a real social impact in the world @xcite @xcite . an important feature of such dialogues is that they are entangled. in his work on dramatic dialogues, @xcite 3) articulates a speech act view: in screenplays, "all talk responds to a need, engages a purpose, and performs an action." in any given scene in a movie or tv show, then, we can often see multiple needs expressed by different characters in one sequence of conversation. consider this scene from young sheldon (fig. 1 ): missy relays a message, sheldon wants to know where his bow tie is, and georgie seeks to avoid showing up with sheldon at school-each of them starts a new conversational thread (or, subconversation) with their speech act that reflects those different intents. in the screenplay, there is no explicit structure that indicates where each subconversation starts and ends. if, however, we could disentangle dramatic conversations, we could ask such questions as: what kind of characters get to start a thread? how long do conversations tend to last? answers to those questions can enhance our understanding of cultural representations on screen. much of the work on conversation disentanglement in nlp has studied internet relay chat (irc) logs, most notably #linux @xcite and #ubuntu @xcite . irc logs are in a different domain from dramatic conversations, and some salient features in irc, such as invoking a username to indicate replies to that user, are not found in screenplays. conversely, there is no equivalent for "off-screen" speakers in irc. given the face-to-face nature of conversations in drama, movie and tv characters can start new conversational threads by entering the scene. those major differences mean chat logs may be insufficient to train models that disentangle drama. to bridge the gap, we present in this work a new annotated dataset to support the study of conversation disentanglement in the domain of movies and tv shows. we draw heavily on the theoretical resources found in film studies, sociology, and linguistics as we design our annotation framework, with particular attention paid to the semantic and pragmatic signals of the start of a new thread. in this work we make the following contributions: in terms of modeling, there are two popular approaches for this task @xcite : two steps (models link individual utterances first, and then we recover thread membership) or end-to-end (models predict thread membership directly). our work adopts the two-step method: we first calculate the similarity score to identify the reply-to relations between two utterances (the link prediction task), and apply a greedy clustering algorithm to put utterances that reply to one another into threads (the clustering task). for the two-step method, there have been attempts to adopt a multi-task learning setup: at training time, when gold cluster information is available, this auxiliary task calculates another loss function dedicated to thread prediction, which can be used to improve the performance of link prediction, the main task, with which we also experimented. datasets for conversation disentanglement are currently limited. in @xcite comprehensive survey on multi-party dialogue understanding, most datasets are not curated with this purpose in mind. @xcite 's corpus, built on annotations of irc chat logs, has been the standard benchmark dataset for this task. @xcite released a dataset of movie dialogues, where they "collect 869 movie scripts that explicitly indicate the plot changing" and "extract 56,562 sessions from the scripts and manually intermingle these sessions to construct a synthetic dataset." in this work, we present a new annotated dataset built on movies and tv shows, adding a spoken, scripted corpus to facilitate this line of work. instead of inferring from the narrative structure and threading conversations through a synthetic process, we developed an annotation framework (described in §4). theoretical approaches to conversation. conversation has been extensively theorized and studied in sociology, linguistics, and film studies, which this work draws on. for our annotation scheme, @xcite idea that conversation is a form of focused interaction and mckee's (2016) speech act view on dramatic dialogues provide us with the theoretical foundation. the ideas related to focus and topic are further explored in the following: ervin-tripp (1964), who considers the surface and semantic features of sustained attention in conversational organization; @xcite , who expounds on "topic under discussion" in pragmatics; @xcite , who see topic change and floor claiming from the perspective of power dynamics between speakers involved. more broadly, @xcite ; @xcite detail the organizational and pragmatic principles for conversation, and he and herman (1998) consider them specifically in the context of drama. this work is further motivated by the social implications of conversations taking place on screen, and we find the following particularly relevant: @xcite carried out a sociolinguistic study on tv dialogues; @xcite ; @xcite ; @xcite highlights related issues as they pertain to gender and race. ## data to study conversation disentanglement in drama, we consider 831 titles: 340 movies and 491 tv series, foot_1 randomly sampling one scene from each title for annotation. movies are taken from scriptbase-j (gorinski and lapata, 2018), based on imsdb, foot_2 because of its extensive coverage of genres and temporal span, along with rich metadata that can adequately support nlp research related to movies. since tv dialogues have different linguistic styles from movie dialogues (nelmes, 2010), we curated a new dataset, tvpilots, using teleplays of pilot episodes made available by tv writing. 4 all screenplays and teleplays from these sources come in the standard format (fig. 2 ): they have distinct scene headers, speaker labels, and other typographical features for annotators to distinguish between an action statement and a dialogue line. since those features are consistent, we have reliable scene markers and speaker labels, and annotators can refer back to the original screenplays when necessary. action statements also helped annotators better understand the scene. ## experiments we compare seven models to see how well existing model architectures that have been proposed for conversation disentanglement perform in the domain of dramatic texts. we would also like to know if models trained on @xcite data, but evaluated on our dramatic data, can leverage its size (seven times larger than ours) to compensate for the striking difference in domain. ## analysis to illustrate the usefulness of conversation disentanglement in drama, we disentangled 808 movies from scriptbase-j and carried out two analyses enabled by this work to explore two questions that engage previous work in film studies: 8 are conversational threads in movies getting shorter over the years? in his analysis on visual style in contemporary films, film historian bordwell (2002, p. 16) observed, "for many of us, today's popular american cinema is always fast": the average shot length is decreasing and cuts and camera movements have become more rapid over the course of the twentieth century, leading to an impression of "intensified continuity." a similar observation is made in cutting et al.'s (2010) empirical work, which relates this trend to our natural fluctuation of attention. notably, such work emphasizes the visual aspect of films, which reinforces the established hierarchy in film studies: the film is a visual medium, and image is more important than sound. this hierarchy is critiqued in studies on film dialogues in particular @xcite , since characters converse with one another only after the advent of sound films. it then leads us to ask: are conversational threads in films also getting shorter over the years? since we have disentangled movie conversations into threads, we can calculate the average number of utterances there are in a thread in a given movie and in a given year. our movie data, while spanning from 1930s to 2010s, is not evenly distributed. as a result, for this analysis, we aggregated movies in a 5-year range. while we would expect thread lengths to also decrease, fig. 3 tells a different story. we can see the average thread length decreasing (although not statistically significantly so) until 1970, and the trend is flat since. film dialogues seem to be resisting the broader trend associated with visual styles. what is the pattern of floor claiming between men and women in movies? it has been pointed out that the film industry became dominated increasingly by men over the twentieth century @xcite . the bechdel test @xcite is a popular and well-known measure for the representation of women in films, often used for advocating that women on screen should "speak up" (o'meara, 2016) to encourage more diverse representation. through this work, we would like to add an additional dimension to it: how often do characters who are women start a conversation in films? in the tradition of continental philosophy, to initiate a conversation-or, to become a speaking subject-is a socially and ethically significant act @xcite . this influenced much of feminist film studies that considers the presence and absence of women's voices in films @xcite @xcite . this work inspires us to frame and measure the agency of characters who are women in relation to the frequency with which they get to start a new conversational thread and claim the floor. we start by using tmdb's api to look up the gender of the actor portraying the character. 9 for this analysis, we only consider movies released after 1979, after which point we have at least five movies each year.in our data, 30.4% of threads are started by women. this is aligned with the oftstated observation that men talk more than women in films @xcite , and the trend has not significantly changed for decades. however, as we see in fig. 4 , when we subtract the percentage of speaking time by women (the number of lines they have) from that of threads started by women (e.g., in 2011, 32.7% of threads were started by women, and 30.8% of lines were spoken by women, so we see an absolute difference of +1.9), we see that women generally start more threads relative to their speaking time, and that is also relatively constant over time. in the figure, any year in which the 95% confidence interval does not overlap with 0 is significant at that level; while this is not significant for many individual years (given the limited number of movies per year), it is significant over all years (+1.0, [0.07, 0.14]). this finding is surprising because it suggests that despite their under-representation, women characters are written to initiate conversations more than their male counterparts. ## conclusion we present in this work a new dataset for studying conversation disentanglement in movies and tv shows in order to enrich the landscape of this line of work in nlp. movie and tv dialogues offer pragmatic patterns and interaction structures different from chat logs, on which standard benchmarks for this task are built. to ensure high quality of this dataset, we digitized teleplays written for tv pilots, so we have more screenplays in the standard format, which we find most useful for annotation. in addition, we draw on theoretical resources from sociolinguistics, sociology, and film studies to create a robust annotation scheme that considers topic and floor changes specifically in the context of drama, which we believe speaks to the needs of the wider scholarly community. to the best of our knowledge, no previous technical or theoretical work has offered a working operationalization of conversational thread in the context of dramatic (scripted, spoken, face-to-face) conversations, or examined the significance of initiating a conversation or gaining the floor in this domain. while we do not claim that the results from our analysis are definitive, our work has demonstrated a new method to further investigate the sound-image hierarchy, gendered power dynamics in films, and communication behaviors in cultural representations on screen. we hope this will encourage and facilitate future research on drama and conversation in nlp, film studies, and the computational humanities.