Detecting Gender Discrimination on Actor Level Using Linguistic Discourse Analysis (2024)

Abstract: With the usage of tremendous amounts of text data for training powerful large language models such as ChatGPT, the issue of analysing and securing data quality has become more pressing than ever. Any biases, stereotypes and discriminatory patterns that exist in the training data can be reproduced, reinforced or broadly disseminated by the models in production. Therefore, it is crucial to carefully select and monitor the text data that is used as input to train the model. Due to the vast amount of training data, this process needs to be (at least partially) automated. In this work, we introduce a novel approach for automatically detecting gender discrimination in text data on the actor level based on linguistic discourse analysis. Specifically, we combine existing information extraction (IE) techniques to partly automate the qualitative research done in linguistic discourse analysis. We focus on two important steps: Identifying the respectiveperson-named-entity (an actor) and all forms it is referred to (Nomination), and detecting the characteristics it is ascribed (Predication). Asa proof of concept, we integrate these two steps into a pipeline for automated text analysis. The separate building blocks of the pipeline could be flexibly adapted, extended, and scaled for bigger datasets to accommodate a wide range of usage scenarios and specific ML tasks or help social scientists with analysis tasks. We showcase and evaluate our approach on several real and simulated exemplary texts.

## introduction ethical considerations as, e.g., formulated in the unesco's recommendations on the ethics of artificial intelligence, as well as emerging legislation such as the eu ai act, require that any ai system adheres to fundamental values such as "the inviolable and inherent dignity of every human" (un-esco, 2022). specifically, this demand also holds true for systems based on large language models (llms). this implies that systems based on llms must carefully ensure that they do not reproduce, reinforce or broadly disseminate any existing biases, stereotypes or other discriminatory patterns, as this would violate the inherent human dignity. however, llms are trained on existing data. if this input data is pervaded by stereotypes, biases and discrimination (as is often the case), the resulting model will reflect these discriminatory patterns. thus, if developers need to ensure that an llmbased system adheres to the ethical standards mentioned above, they can take one of two approaches: filter the llm's output downstream to ensure that it is free from discrimination -or purge the input data from any discriminatory patterns, to ensure that the llm itself will be free from discrimination in the first place. research on downstream gender bias mitigation in word embeddings by @xcite shows that downstream mitigation only hides bias and does not remove it. thus, the effective alternative is to address bias upstream by selecting unbiased training data. as the training corpora for llms need to be very extensive, it is impossible to ensure their quality manually. therefore, technical means need to be developed that automatically detect discrimination in vast amounts of natural language texts. what we read and see in media shapes our reality @xcite . if we are surrounded by bias and discrimination, we are likely to include these in our reality and act on them. that explains why media, notably text, plays an important role in the striving for equality for all genders. by detecting bias and especially discrimination against particular genders, it is possible to be wary of these texts and not distribute them. this is particularly important when choosing training data for natural language processing (nlp) tasks. the term gender has at least three different notations: the linguistic gender, sex, and the social gender. the linguistic or grammatical gender can be defined as follows: "[...] grammatical gender in the narrow sense, which involves a more or less explicit correlation between nominal classes and biological gender (sex)." @xcite . for example, in german, nouns could be female, male, or neutral. the sex, however, refers to a "biological" notion of gender that is "binary, immutable and physiological" @xcite . this notion is flawed because intersex humans do exist, as well as trans-persons, thus refuting the binary and immutable part of this notion. for our work, we use the third notion, the social gender. this notion defines gender as a social construct represented by a person's intentional and unintentional actions to represent their gender and the reception of these actions. therefore, the social gender is non-binary, flexible, and constructed by the person themselves and the persons perceiving them @xcite . we use the terms woman for persons who can be read as female-identifying, men for persons who can be read as male-identifying, and non-binary for persons who do not adhere to the before mentioned. bias against a particular gender entails discriminating against this gender. while bias contains all notions and beliefs towards a person/group (mateo and williams, 2020), (social) discrimination is a more intentional act: an offender treats someone or a group of people differently in a negative way, based on a specific feature of this person/group @xcite . textual discrimination is a special kind of (social) discrimination because the offender is not always apparent. linguistics and sociology have studied discrimination for over eighty years, mainly focusing on racism in the early research @xcite @xcite . during this period, different definitions of discrimination were defined, leading to different approaches for detecting it. one of these approaches is linguistic discourse analysis (lingda), which inspects discourse to identify discriminating tendencies by combining research from sociology and linguistics @xcite . computational linguistics integrates lingda and computer science into computational discourse analysis. so far, this discipline concentrates on the quantitative parts of lingda, mostly focusing on coherence and cohesion @xcite . we concentrate on the qualitative parts of lingda and partly automate the discrimination detection within the text. ## problem formulation and goals existing approaches for automatic discrimination detection often focus on identifying drastic wording, which is relatively easy to detect by simple comparison with a database of discriminatory terms. however, in many cases, textual discrimination manifests more subtly, requiring a more semantic approach to detect it. to achieve our goal of automatically identifying discrimination and biases in text, we seek to enhance computational discourse analysis (compda) by integrating two fundamental, qualitative strategies from linguistic discourse analysis for detecting gender discrimination on the actor level: identifying the respective person named entity (an actor) and all forms in which it is referred to (nomination), and then detecting the traits, characteristics, qualities, and features that are ascribed to this actor (predication). by focusing on actors, we aim to reveal even subtle gender-specific discrimination. furthermore, we can analyse the text's meaning on a deeper level. to automatically process large amounts of input text data, we implement a pipeline for automated text analysis that integrates nomination and predication by using ie techniques (cf. figure 1 ). specifically, as a first step, we identify nominations by extracting the actors and detecting their pronouns. second, we extract the predication of these actors and finally use the extracted information to analyse the whole text for discrimination. by ensuring a modular structure built from exchangeable components, we aim to make our pipeline flexibly adaptable, accommodating a wide range of usage scenarios and specific ml tasks. for example, the pipeline should be able to scale from single texts to a whole corpus, process different languages, and focus on different criteria, thus reflecting cultural differences. finally, we evaluate our approach and implementation by analysing several sample texts, two realworld examples, and three generated texts, and discuss the discrimination markers identified in these samples. ## background this work combines qualitative research on lingda with ie, thus enhancing quantitative com-pda methods for detecting gender discrimination in text. discrimination is a form of bias. we define discrimination and its relation to bias. ## methodical approach our analysis pipeline can be subdivided into four consecutive steps that build on each other (cf. figure 1 ): the first task is to extract the actors, fol-lowed by a gender approximation for each actor. in these steps, we save the nomination of each actor in our knowledge base. the third step expands the knowledge base with the predication of each actor detected in step one. as the fourth and final step of the pipeline, we analyse the extracted information for potential discrimination. ## implementation and validation as mentioned in section 4, we start by collecting the nomination of actors and subsequently enhance our knowledge base with the predication of the actors. the content of the knowledge base is subsequently analysed for discrimination and biases 3 . the code for our pipeline can be found on github foot_0 . ## discussion our method shows promising first results, even on our limited test data. ## conclusion and future work in this work, we build a flexible pipeline to analyse newspaper articles and blog posts about people. we use linguistic methods to detect how actors are described within a text. in contrast to common discrimination detection methods, we do not treat the whole text as one object. by focusing on actors and the gender of the actors, we can do more nuanced text analyses that can detect subtle discrimination on a gender basis. first, limited tests on newspaper articles show that we can detect how actors are treated differently, depending on their gender. the first proof-of-concept pipeline implementation has some limitations that will be addressed in future work. other future work includes using the pipeline in different languages, such as german. furthermore, instead of analysing one text at a time, we will scale the input to several documents, analysing complete corpora. we will also experiment with different pipeline components, for example, exchanging the simplistic abusive language detection with a sophisticated hate-speech detection or coreferencing detected actors with real-world actors to detect their pronouns. as today's discourse is not only written, analysis of multi-modal data might also be an interesting endeavour.