Gender-tailored Semantic Role Profiling for G erman (2023)

Abstract: In this short paper, we combine the semantic perspective of particular verbs as casting a positive or negative relationship between their role fillers with a pragmatic examination of how the distribution of particular vulnerable role filler subtypes (children, migrants, etc.) looks like. We focus on the gender subtype and strive to extract gender-specific semantic role profiles: who are the predominant sources and targets of which polar events - men or women. Such profiles might reveal gender stereotypes or biases (of the media), but as well could be indicative of our social reality.

## introduction some verbs express a positive or negative relationship (a polar relation) between the fillers of their semantic roles. for example, we can infer from the sentence "he offended her," an even, reciprocally holding, negative relation (e.g. against(he,her). moreover, such semantic roles might bear a polar (i.e. positive or negative ) load, e.g. the agent of cheating might be regarded as a negative actor, a villain. from a pragmatic point of view, it might be interesting to take a closer look at the distribution of particular role fillers or role filler groups of such verbs indicating a polar relation, namely vulnerable groups such as children (pedophilia), migrants (xenophobia), people of color (racist bias), and certain gender identities (gender bias). this could reveal interesting facts about the conceptualization and contextualization of these filler groups in the real world. such an approach could be useful for various kinds of monitoring processes (e.g. discrimination motoring). in this short paper, we focus on gender. our goal is to enable gender-tailored semantic profiling. on a micro level, semantic profiling strives to identify the roles that gender denoting nouns occupy, e.g. that female nouns occur quite often as patients (targets) of physical violence, while male denoting nouns often are filler of the patient role of torture or accusation. on the macro level, a general, crossverb inventory of semantic roles like villain, victim, benefactor, beneficiary could be used to aggregate gender-specific conceptualization. here, we focus on the micro level. we introduce a classifier that determines the grammatical gender of human-denoting german nouns. we combine this with our rule-based sentiment inference system foot_1 @xcite which assigns two types of relations between entities: in favor of, against. each verb of our verb lexicon expresses such a polar relation and has a source (the agent) and a target (the patient, recipient or theme) role. we filtered the output of our system for cases in which the gender classifier labeled at least one of the verb roles as male-or female-denoting foot_2 . with such data, we were able to filter for polar events in which men are sources and women targets (and vice versa). on the basis of statistical tests, cases are found in which female or male denoting nouns are significantly over-or underrepresented. ## related work currently, gender classification is primarily restricted to predicting the gender of text authors of blogs, see @xcite , or to find out whether a headline is about a man or a woman, see @xcite . sun and peng (2021) observe a gender-specific tendency to combine personal and professional events in the wikipedia pages of celebrities, an asymmetric association where e.g. women's personal events appear more often in the career section than for men. they also establish higher efficiency when extracting events (verb denotations) over analyzing raw text for detecting this gender bias. bias detection and debiasing, in general, are important research topics (see @xcite for a survey). researchers use metrics such as pointwise mutual information (pmi) to measure the association of words with gender @xcite . we look into cases in which both grammatical genders co-occur with a verb, i.e. when pmi cannot be used. ## grammatical gender classification the basis for our gender classifier is the freely available resource @xcite of 13,000 german nouns which were manually classified as denoting either animate or inanimate entities foot_3 . in order to create a gold standard for grammatical gender classification, we took a subset containing animate singular nouns and manually foot_4 selected those that can be used to refer to women or men (altogether 4,320). examples of femaledenoting nouns include schwester, gastgeberin, schauspielerin (eng. sister, hostess, actress, respectively). we then saw that the data was imbalanced, namely that there were more male-denoting nouns (2,830) than female-denoting ones (1,490). as such a dataset would have produced a biased classifier with better classification for male-denoting nouns, we searched for more female-denoting nouns, ultimately expanding this set to 3,700. in german, this can generally be carried out by adding the suffix in to the end of male-denoting nouns, e.g. @xmath0. helper). if such a variant is found in a corpus, it is added to the female list. since we found that female nouns in news texts are underrepresented, we decided to keep the whole list of 3,700 female nouns for learning. in @xcite we tested various word embeddings (glove, bert,fasttext) for the training of the animacy classifier (mlp, svm, lr) and found fasttext with logistic regression (lr) to perform best. therefore, we used only fast-text embeddings to train a lr model for genderaware animacy classification. there was no need to carry out extensive experiments, since our initial non-actors female male precision 0.967 0.983 0.973 recall 0.984 0.993 0.927 f1 0.975 0.988 0.949 note: not all german female-denoting nouns possess the "in" ending. in fact, in our list of female-denoting nouns, 50 have endings other than "in" (e.g. frisöse, eng. hairdresser). a rather simple (rule-based) method was to classify a word with an "in" ending as a female-denoting noun. but that would produce quite some error. in a corpus of 25 million nouns, we found 67,823 words (tokens) ending with "in". for 36,247 cases of these "in"-words our classifier predicted female. the remaining 31,576 "in"-nouns correspond to 4,035 types. we manually classified 1,000 and found only 5 female-denoting words. thus, the classifier does not base its decision on the suffix, though this would be a legitimate approach since fasttext uses sub-word splitting. the performance of our classifier with respect to the non-"in" female-denoting nouns cannot reliably be evaluated at the moment. we leave it to future work to train models able to deal with these rarer cases. ## statistical setting our question of interest was that of identifying an imbalance, if any, between men and women, or some gender-specific behavioral semantic profile, as portrayed in newspaper texts. we focused on men and women's roles as positive or negative actors (sources) or as being positively or negatively affected patients (targets). in particular, we looked at all polar verb instantiations, with maleand female-denoting nouns occupying the source and target roles. then, we gathered statistics on how often a positive or negative relation between two gender-denoting nouns (e.g. a female-and a male-denoting noun) was found. we performed this for all gender permutations at the level of a single verb, but we also accumulated this over all verbs. to evaluate whether a verb is more biased towards male or female roles, the (prior) gender distribution in the whole data must be taken into account. in our text corpus, we found a ratio of male-(1,290,415) to female-(283,952) denoting nouns (according to the gender classifier) of about 4:1. that is, the maximum likelihood estimated probability of male denoting nouns is 0.815, that of female 0.185. the data is binomially distributed for each role of a verb frame. for instance, if a transitive (active voice) verb has @xmath1, of which 20 are female, then we determine the cumulative probability of up to 20 cases given 200 trials with @xmath2.185 as 20 @xmath3, 200, 0.185). if this value is below α = 0.05, then we reject h 0 and adopt h 1 , i.e. we can conclude that the verb (usage) is biased, and similarly for the 1 -0.95% interval. spelled out, h 0 claims that female (male) denoting nouns occupy source (target) verb roles according to their prior probability. if this is for some verbs rather unlikely, than h 1 is adopted saying there is a verbrole specific bias, for instance that female denoting nouns are significantly more often targets of (verbs of) physical violence than male denoting nouns. we only looked into verbs for which a normal distribution could be approximately assumed, which is given if n * @xmath4, where n is the number of cases. in our setting, this amounts to a frequency threshold of @xmath5.185 = 27. for each verb above this frequency threshold, we tested the null hypothesis h 0 that male-and female-denoting nouns occupy the role of a verb according to their respective distributions in the whole corpus. ## empirical results we use data from 3 swiss newspapers published between 2004 and 2014. despite the medium corpus size, the cases in which a verb has 2 animate role fillers (singular male or female 6 ) at the source and target positions of that verb are relatively infrequent. this low frequency can be attributed to (1) the abundance of cases written in passive voice (for which there is quite often no source indicating pp) and (2) cases in which the source or holder is a personal pronoun (which, in german, leaves the animacy status of the referent open). in german, 6 we did not take plural nouns into account since german plural male nouns for a long time have been regarded as being generic, denoting all genders. the gender reference of such a noun, thus, cannot be reliably fixed. ## discussion of limitations our method detects gender imbalance by using an existing rule-based system and a new grammatical gender classifier. neither performs perfectly, and we cannot claim that our sampling methods produce representative data drawn from the whole population. rather, we work with a subset that can be identified by our tools. generalizing from the subset to the population is not our intention; rather, our approach is a first step toward gender-tailored sentiment analysis. finally, we do not claim to find biases in the data, but instead speak of imbalance and propose that a qualitative analysis of the results is needed.