Personal noun detection for G erman (2023)

Abstract: Personal nouns, i.e. common nouns denoting human beings, play an important role in manifesting gender and gender stereotypes in texts, especially for languages with grammatical gender like German. Automatically detecting and extracting personal nouns can thus be of interest to a myriad of different tasks such as minimizing gender bias in language models and researching gender stereotypes or gender-fair language, but is complicated by the morphological heterogeneity and homonymy of personal and non-personal nouns, which restrict lexicon-based approaches. In this paper, we introduce a classifier created by fine-tuning a transformer model that detects personal nouns in German. Although some phenomena like homonymy and metalinguistic uses are still problematic, the model is able to classify personal nouns with robust accuracy (f1-score: 0.94).

## motivation following @xcite , personal nouns are common nouns denoting humans such as kinship terms (daughter) or occupational titles (teacher). they form a segment of the animacy hierarchy @xcite , which is widely used in language typology, see figure 1 . personal nouns correspond to the segment characterized as @xcite , between proper names and common nouns denoting non-human living beings. identifying personal nouns is not only motivated by typological interests. in german, a language with a tri-partite grammatical gender system (masculine, feminine, neuter), there are morphological means to express the gender of the persons referred to, which leads to a congruent interpretation of grammatical form and human gender (such figure 1 : personal nouns form the segment @xcite . the animacy hierarchy of @xcite was originally introduced for typological analyses of 'accusative' vs. 'ergative' case-marking splits. as mother and father or actor and actress in english). in recent years, there has been a vigorous debate in germany whether to consequently disambiguate personal nouns in concordance with the gender of their referents @xcite . the actual implementation in texts varies between using masculine forms as the traditional 'general' expression (e.g. die zuschauer ['the spectators, masculine']), explicit markings of feminized (e.g. die zuschauerinnen ['the spectators, feminine']) and gender-diverse forms with a special character and feminine suffix (e.g. die zuschauer:innen ['the spectators, gender-diverse']), or using neutral forms (e.g. die zuschauenden ['the spectators, neutralized for gender']). as a result, personal nouns are a crucial part of expressing gender in german texts, and thus also a crucial part of manifesting gender stereotypes in texts. the detection of personal nouns is useful for analyzing these stereotypes from various perspectives. gender bias in large language models or their training data has become an active research field in nlp. 1 there are methods of detecting gender bias in word embeddings such as the word embedding association test (weat) @xcite . one method of balancing gender in the training data, for example, is 'counterfactual data augmentation' @xcite which is based on adding synthetic sentences to the training corpus that are created by means of a bidirectional lexicon of gendered words such as actor:actress. in languages like german, such a lexicon would need to include all personal nouns, because german uses lexical and morphological means very productively to create their feminized or neuter forms. from a linguistic standpoint, recent developments of gender-fair language in german (kunkel-razum, 2020) have led to increasing interest in the forms and use of personal nouns, e.g. regarding frequencies of feminized forms (student ['student, masculine'] > studentin ['student, feminine']) or neutral forms derived from a verbal participle form (studenten ['students, masculine'] > studierende ['people who study, plural, neutralized for gender']). newer overtly gender-inclusive forms employ e.g. an asterisk (wähler*innen ['voters, female plural suffix']) or a colon (bürger:innen ['citizens, female plural suffix']) to explicitly include not only women but people of all non-binary genders. the problem with researching these phenomena in a quantitative way is that it has not been possible to gauge the basic population of personal nouns in a given corpus in order to put frequencies of e.g. forms with an asterisk into perspective, for instance to approximate whether such forms are getting more frequent. this is due to personal nouns being a heterogeneous class in german that includes the products of many different word formation processes. derivational suffixes for personal nouns, for example, include -er (lehrer 'teacher'), -ung (leitung 'leader/manager') and -ling (lehrling 'apprentice'). this heterogeneity is further complicated by some personal nouns being ambiguous with non-personal nouns, e.g. leitung 'leader, manager' vs. leitung 'wire, pipeline', restricting the use of word-list based approaches like @xcite for swedish vocational terms. furthermore, other nouns that do not refer to a human contain these suffixes as well (e.g. gräber 'graves', fälschung 'forgery', frischling 'shoat'), leading to false positives when querying a corpus for these word formation patterns. thus, it is not possible to identify all personal nouns in a corpus with a regular expression without extensive manual correction. instead, machine learning-based token classification could be the way to go. to test the feasibility of such a semantic annotation, we have fine-tuned a pre-trained language model on manually annotated data to automatically identify personal nouns in a corpus. we discuss problems of the annotation and perform a qualitative error analysis on the results. the classifier model is freely available. 2 while our work focuses on german, research on gender-fair language has been conducted for other gendered languages as well (see robiche 2018 for french and verelst 2022 for dutch). thus, a classifier that is able to detect personal nouns could also be fruitful for research in other languages. ## previous work on personal nouns quantitative work on personal nouns in german so far has either looked at pre-chosen lexemes where it is possible to extract all forms of the whole paradigm and thus know the basic population (e.g. @xcite , or has resorted to manually identify personal nouns in a corpus (e.g. @xcite @xcite . @xcite defines personal nouns as "nominal expressions that are used [...] to refer to human beings". while this definition might seem straightforward, it is often difficult to determine if a noun is indeed used to refer to human beings. some problems identified in @xcite include ambiguous nouns and collective nouns that can be used either in a personalized way (1-a) or instead referring to an institution or organization (1-b) @xcite . these issues, especially ambiguity, lead to the problem that even to query specific lexemes in a corpus will yield false positives, for example for nouns such as berliner that can be used both as an adjec-tive and as a noun, and which has a personal and non-personal meaning as a noun on top of that, see the examples in (2). while pos annotation can help to distinguish the adjectival from the nominal use, it does not help to distinguish between the latter two nominal usages. in the context of digital humanities, @xcite trained a classifier to extract and assign gender roles in german literary texts, targeting personal nouns as well as proper names of literary characters. while the task of automatically detecting personal nouns is similar to named-entity recognition as it is a token classification task, it differs insofar as the tokens to be detected are crucially not named entities-proper names are not part of the semantic class of personal nouns. ## the personal noun detection task objective. the objective of the detection task is a binary classification of all tokens in a corpus as either personal noun (pers n) or other (o). conceptually, a personal noun is a token t in a text that meets the following criteria: (i) the lexicalsemantic class of t is [-proper, +human]; (ii) t is used in a context that refers to a person or a group of humans; (iii) the part of speech of t is common noun. following @xcite approach to personal nouns, the detection task targets all noun tokens denoting humans regardless of their referential context, including generic, non-generic, and predicative contexts. metonymic uses of nounssuch as referring to an institution or an organization instead of referring to a person-are labeled "other" (o) according to criterion (ii). for example, gewinnerin ('winner, female') in example (3), which refers to the green party, is labeled o. furthermore, the task excludes personal noun instances that occur as the first part of a compound noun such as bauern in example (4). 3 because the token bauern-proteste refers to the event of 'protest', and only the subtoken bauern fulfills the definition of a personal noun, it is disregarded for the annotation. proper names are, by definition, not personal nouns and are labeled "other" (o). we would like to point out that the task operates on the token level, instead of the phrasal level, because our research interest are forms of gender-fair language in german. this is essentially expressed on the lexical level even if it requires contextual and referential disambiguation. the personal noun detection task is therefore different from, e.g., the task of (phrasal) markable detection in coreference resolution. ## results and discussion the results of the fine-tuned model's performance on the test data are shown in overall, there were 22 cases of false positives (see (7) for an example) and 27 cases of false negatives (see (8) for an example) in the test data. ex-ample (7) showcases an interesting example of a false positive that could be considered a peripheral, non-prototypical personal noun, as a generation is made up of people. the model's classification of this token showcases that some categorization decisions are not as clear-cut as they may seem on the surface. the personal noun heuchler in example (8) was not detected by the model as such. this could be due to its relative infrequency. 10 it was also not part of the training data for the fine-tuning. on closer inspection, though, the false negatives and positives in some cases revealed not a mistake of the model, but an error in the manual annotation. these included errors from the automatic annotation that were not caught and corrected during the manual correction, such as deutschen being categorized as a personal noun in example (9). these oversights stress the importance of using more than one annotator when manually labeling data, so that errors like this can be avoided. apart from looking at the model's performance on the test data, we also tested instances of challenging phenomena as identified by @xcite that make distinguishing between personal nouns and other words difficult. first of all, ambiguity can pose a problem. we tested the two word forms berliner and hamburger that can both be used as an adjective and as a noun, as well as having both a personal noun usage and a 'food' meaning. both word forms were correctly not classified as a personal noun in their adjectival usage, but berliner as a noun was labeled a personal noun in both the 'food' usage and the 'person from berlin' usage. for hamburger, on the other hand, the model correctly only labeled the usage as a personal noun as such. secondly, personalized and institutional usages of collective nouns were tested with the word forms polizei 'police' and menge 'amount', 'crowd'. for both word forms, the model managed to correctly label the personalized usage as a personal noun in example (10-a), and not label the impersonal usage in example (10-b). "the police is part of the executive." proper names could also pose a problem for the classification, as a lot of last names are derived from personal nouns but should not be detected by the model. in fact, the model was able to differentiate correctly between personal noun, in example (11-a), and proper name usage, in example (11-b), for schneider ('tailor'), but it did not detect müller ('miller') as a personal noun in example (11-c), which is the most common family name in germany, 11 but the occupation has become rare, so that müller only appears as a last name in the training data and not in its personal noun usage. "the word man is a noun." (13) "frauen" ist der plural von "frau". "''women' is the plural of 'woman'." interestingly, when adding quotation marks to the sentence in (12-a) as in example (13), the model only labeled frau as a personal noun, but not frauen. for the sentence in example (12-b), though, it did not make a difference whether mann was set in quotation marks or not. another challenge for the study of gender-fair language is that new forms keep evolving. testing the new colon form (e.g. schüler:innen 'students') that became popular only after the training corpus was compiled in 2019, the model still labeled the token demonstrant:innen in example ( 14 ) as a personal noun. this shows that it could be useful for identifying new strategies of gender-fair language emerging in the future as well. die polizei schoss auf demonstrant:innen. "the police shot at protestors." ## conclusion personal nouns, the semantic class of common nouns denoting humans, are of great importance in the context of current discussions and developments in research on gender-fair language and language use in linguistics and digital humanities, as well as gender-fair nlp. in order to facilitate quantitative research, we defined the task of personal noun detection and fine-tuned a pre-trained language model for the detection of personal nouns in german. the fine-tuning yielded surprisingly good results (f1-score: 0.94), considering the small amount of training data and the fact that the actual tokens of interest were not very prevalent. further training on more diverse data including other text types, for example literary texts, which probably contain a range of different personal nouns not covered in news writing or personal blog posts, could improve the results even more. new training data could also include specifically selected sentences containing some of the more difficult to distinguish words as discussed in section 4, like ambiguous words, proper names, and metalinguistic usages. so far, the classifier only detects personal nouns but does not give any additional information on them. ideally, a future version of the model would further enrich this classification. an initial expansion could be to detect grammatical gender. much less trivial, but desirable, would be to implement a further classification of the type of reference, as qualitative research has shown that gender-fair forms tend to be used more frequently in cases of non-generic reference @xcite . incorporating a distinction between generic (15) and non-generic (16) use (see @xcite into the classifier would make it possible to test whether this holds true on a larger scale. similarly, whether a personal noun refers genderspecifically (e.g. masculine lehrer referring to only male teachers) or gender-independently (e.g. masculine lehrer referring to a mixed-gender group of teachers) is necessary information in order to quantify the amount of masculine personal nouns used to refer to gender-diverse groups. training the language model to classify personal nouns in these three categories would thus be a next step.