Stage Direction Classification in F rench Theater: Transfer Learning Experiments (2024)

Abstract: The automatic classification of stage directions is a little explored topic in computational drama analysis, in spite of their relevance for plays’ structural and stylistic analysis. With a view to start assessing good practices for the automatic annotation of this textual element, we developed a 13-class stage direction typology, based on annotations in the FreDraCor corpus (French-language plays), but abstracting away from their huge variability while still providing classes useful for literary research. We fine-tuned transformers-based models to classify against the typology, gradually decreasing the corpus size used for fine tuning, to compare model efficiency with reduced training data. A result comparison speaks in favour of distilled monolingual models for this task, and, unlike earlier research on German, shows no negative effects of model case-sensitivity. The results have practical relevance for computational literary studies, as comparing classification results with complementary stage direction typologies, limiting the amount of manual annotation needed to apply them, would be helpful towards a systematic study of this important textual element.

## introduction machine learning methods have brought important contributions to computational literary studies (cls). to name just one monograph-length work, @xcite used such methods to provide insights on complex issues like the longterm evolution of genres and of literary prestige criteria, focusing mainly on fiction and poetry. drama has also benefited from such approaches. the recent computational drama analysis workshop 1 featured work on the automatic classification of dramatic situations, character types, and emotions in drama. in this paper, we approach a little explored dramatic analysis topic: the automatic classification of stage directions, using a french theater corpus. stage directions introduce indications about performance, decoration or other information to 1 https://page.hn/anuvah complement character speech, but can sometimes be largely independent from it @xcite . several typologies have been proposed for them in literary studies (see 3.1). however, their automatic classification has scarcely been studied and poses challenges, given types hard to distinguish from each other. stage directions on characters' entrance and exit indicate changes to character copresence on stage and are thus tied to play structure and dramatic technique. the frequency and length of stage directions and their types can be stylistic parameters related to author groups or subgenres. automatically classifying stage directions facilitates large-scale quantitative analyses of this element's structural and stylistic role. for french theater, the fredracor corpus @xcite , based on @xcite and covering mostly the 16 th to 20 th centuries, offers over 38,000 annotated stage directions. given the large number of categories (over 5,000), exploiting these annotations for supervised learning is a challenge, that we address in the paper. pre-trained language models were a game changer in nlp, allowing for transfer learning (e.g. @xcite ) that yields viable classifiers even with a reduced number of examples, or, in the case of larger language models @xcite , in-context learning from (almost) zero examples. in our study, we work with specialized categories for which we could develop annotations, and we opted for transfer learning. we examine the extent to which we can reduce manually annotated training data for the supervised classification of stage directions, in french. as producing manual annotations can be costly, and exploring literary questions may require comparing the results of classifying against several, complementary typologies, the question addressed has practical relevance for cls. the paper's contributions are: an overall goal is to start a reflection on good practices to develop methods to classify this textual element, across languages. the paper is structured as follows: section 2 reviews related work. section 3 describes our typology and classification workflow. section 4 presents results, and section 5 outlines future perspectives. ## related work heterogeneous criteria have been used to design stage direction typologies @xcite @xcite @xcite . among other aspects, taxonomies pay attention to whether stage directions refer to verbal/speech-related or visual information, to characters or setting, whether they describe movements (including entrance and exit) or character interactions. another feature used is their narrative vs. descriptive nature, their relatedness with or independence from spoken text, or their impact on the play's plot. the text encoding initiative (tei) guidelines (tei consortium, 2023) reflect this heterogeneity in their description of possible @type attribute values for the @xmath0, used for stage directions. @xcite attempts to reconcile inconsistencies in earlier typologies, systematizing types via a set of values for the @ana attribute @xmath1. automatic stage direction classification was performed in german by dennerlein (2016), with four classes (exit, entrance, dead and aside); per-class results ranged between 0.75 and 0.88 f1 using random forests. the typology is less complex than our 13-class typology and the classes are more distinct. @xcite , working with russian, developed a model to classify against the tei guidelines' 9 categories, trained on 6,569 manually annotated examples and reaching ca. 0.75 f1. @xcite performed a related but not equivalent task. in their study on predicting german plays' structural elements in tei, one of the five classes was stage directions, besides act and scene divisions, speaker names and speeches. they thus classified stage directions vs. other structural elements, reporting that a binary classification between stage directions and character speech was not trivial (0.81 f1). stage directions were also the worst performing class in the 5-way classification (0.84 f1, while other classes were above 0.9). to assess the role for language-specific knowledge, they fine-tuned both english and german bert cased and uncased models, with best results for the german uncased model. their experimental setup informed our own (3.2). ## results and discussion as however, an iterative workflow may be attempted with manual correction of the model outputs and fine-tuning of a better model while keeping the number of manual annotations low. manual corrections could focus on the worst performing categories. a possible use of the best models would be to automatically annotate fredracor stage directions beyond the most frequent 87 single-type labels, which we did not handle here (section 3.1) and account for ca. 30% of the corpus, to assess whether the model's predictions could be a viable way to simplify the corpus' wide variety of labels. the same could be done to corpus examples that bear mixed-type labels. confusion matrices for distilled camembert fine-tuned on 100% and 10% of the data are in figure 2 , 3 and per-category results in ## conclusion and outlook the results are encouraging towards automatic large-scale stage-direction classification: 0.7 f1 with a 13-class typology using less than 1,200 manually annotated examples; a more costly set of ca. 5,900 annotations allowed for 0.81 f1. a distilled monolingual model was the best choice, offering satisfactory results and faster fine-tuning than the full model. besides comparing with large language models and in-context learning, a relevant future task is the detection and classification of internal stage directions, implicit from character speech @xcite . reliable multilingual stage direction classification would open the door to large-scale quantitative comparative and diachronic work on an important element of dramatic texts.