Measuring the Effects of Bias in Training Data for Literary Classification (2020)

Abstract: Downstream effects of biased training data have become a major concern of the NLP community. How this may impact the automated curation and annotation of cultural heritage material is currently not well known. In this work, we create an experimental framework to measure the effects of different types of stylistic and social bias within training data for the purposes of literary classification, as one important subclass of cultural material. Because historical collections are often sparsely annotated, much like our knowledge of history is incomplete, researchers often cannot know the underlying distributions of different document types and their various sub-classes. This means that bias is likely to be an intrinsic feature of training data when it comes to cultural heritage material. Our aim in this study is to investigate which classification methods may help mitigate the effects of different types of bias within curated samples of training data. We find that machine learning techniques such as BERT or SVM are robust against reproducing the different kinds of bias within our test data, except in the most extreme cases. We hope that this work will spur further research into the potential effects of bias within training data for other cultural heritage material beyond the study of literature.

## introduction one of the challenges facing researchers working with cultural heritage data is the difficulty of producing historically representative samples of data @xcite . while we have access to very large collections of digitized material (e.g. hathi trust, gale), we often lack knowledge about the distributions of different types of documents and their stylistic qualities within these collections (not to mention within the broader sweep of history more generally). researchers aiming to build collections for historical study using automated methods are thus faced with a two-part challenge: first, the collection of reliable training data given the absence of annotated data within larger collections; and second, the mitigation of potentially unknown biases within such training data when scaling to the classification of larger historical collections. in this work, we attempt to measure the effects of such potential unknown biases within training data for the purpose of literary classification by testing cases of known bias. in essence, we want to simulate the following scenario. a researcher wishes to construct a large sample of historical documents from within a given heritage repository using automated methods. because there is no consistently annotated data for her purposes, she constructs a small training data sample by hand based on her domain expertise, either by randomly sampling from the larger collection or building around some prior disciplinary consensus. as she moves to implement a process to automatically classify documents based on her training data, she is left with a fundamental uncertainty: because the underlying distribution of different stylistic and social features of the data within the larger collection are unknown, and given that her sample represents a tiny fraction of all documents, how confident can the researcher be that whatever biases may be present in the training data will (not) be reproduced in the subsequent automated annotations? this work is licensed under a creative commons attribution 4.0 international licence. licence details: http: //creativecommons.org/licenses/by/4.0/ to model this scenario, we work with a collection of data in which the underlying distributions of stylistic and social features are known and then test a variety of cases of increasing bias within the training data to measure its effects on the test data. we assess three separate forms of stylistic and social bias, which include genre, dialogue, and authorial gender, as well as a variety of classification techniques, including the use of data augmentation, to identify conditions under which the reproduction of bias is minimized. in all cases, the goal of our classification task is the detection of fiction from a larger collection of documents. the identification of fictional or "literary" documents within large historical collections is a pressing need for the field of literary studies and has been taken up in several cases @xcite @xcite . as yet, however, no assessment has been made of the potential effects of bias within the training data used for such annotation exercises. as we discuss in detail in section 5, we surprisingly find that current state-of-the-art techniques in nlp such as bert @xcite (or even svms) are robust against reproducing all three kinds of bias within our test data, except in the most extreme cases. one bias in particular, authorial gender, appears to exhibit no effect at all in classification tasks, even under the most extreme circumstances. ## methodology the goal of our experiments is to test the effects of bias in training data on the classification of test data belonging to the domain of literature. for our purposes, following the work of @xcite , we attempt to predict whether a document is a work of "fiction" within a binary classification task (fiction/non-fiction) and modulate different stylistic features, which we describe in section 4. in this section, we describe the dataset followed by the classification algorithms used along with the data augmentation techniques applied to the training data. foot_0 ## experiment design in this section, we introduce our experimental setup to test the robustness of the classifiers when presented with different forms of bias in the training data, grouped into the following three categories: genre, dialogue, and gender. our research question is which classification techniques (if any) mitigate different types of known biases in the training data when it comes to accurately classifying test documents? in all cases, we hold our test set constant and then manipulate our training data according to different kinds of stylistic and social features described below, and evaluate each model's performance. we begin by establishing a baseline where the distribution of the feature of interest (e.g. genre, dialogue, gender) is the same for both the train and test sets. we then gradually distort the training data and measure observed declines in model performance using the evaluation metrics described below. the size of the training data remains constant throughout: 200 fiction and 200 non-fiction documents. 4 a "document" refers to a 500-word passage randomly sampled from a single work. when sampling passages, we condition on the middle 40% of the work in order to avoid paratext at the beginning or end of a work. additionally, we experiment with using longer passages from these volumes (up to 10,000 words) as described in section 5. finally, we sample no more than 1-3 passages per work for each scenario. ## conclusion in this paper, we have tested different classifiers, different data augmentation techniques, and different forms of training data bias to assess their effects on the task of literary classification. overall, we have found that bert is the best-performing classifier with svms comparable with text-passages above 5,000 words in length. data augmentation as we have implemented it provides little performance gain. finally, the stylistic and social biases tested here exhibit little effect except in the most extreme cases (> 90% bias for a given category) suggesting that at least for the purposes of literary text classification, underlying biases in training data are not as impactful as researchers have initially hypothesized. nevertheless, our work is limited in its historical scope (different historical periods may exhibit different effects), cultural specificity (our effects have only been observed on english-language documents), classification task (other types of classification may perform differently), and stylistic breadth (stylistic features important to other domains or research questions may behave differently). it is also important to emphasize that while classifiers can mitigate the propagation of bias within training data scenarios (up to a point and under certain conditions), they cannot address biases built into the underlying digital collections from which new collections are created @xcite . we hope that these experiments provide a useful framework for further refining our understanding of the effects of bias on multiple forms of cultural classification. future work will want to test different classification scenarios, types of stylistic or social bias, multiple linguistic contexts, as well as further historical document types to better understand how unknown biases in training data may impact our representation of the past using digital collections.