Men Are Elected, Women Are Married: Events Gender Bias on W ikipedia (2021)

Abstract: Human activities can be seen as sequences of events, which are crucial to understanding societies. Disproportional event distribution for different demographic groups can manifest and amplify social stereotypes, and potentially jeopardize the ability of members in some groups to pursue certain goals. In this paper, we present the first event-centric study of gender biases in a Wikipedia corpus. To facilitate the study, we curate a corpus of career and personal life descriptions with demographic information consisting of 7,854 fragments from 10,412 celebrities. Then we detect events with a state-of-the-art event detection model, calibrate the results using strategically generated templates, and extract events that have asymmetric associations with genders. Our study discovers that the Wikipedia pages tend to intermingle personal life events with professional events for females but not for males, which calls for the awareness of the Wikipedia community to formalize guidelines and train the editors to mind the implicit biases that contributors carry. Our work also lays the foundation for future works on quantifying and discovering event biases at the corpus level.

## introduction researchers have been using nlp tools to analyze corpora for various tasks on online platforms. for example, pei and jurgens (2020) found that female-female interactions are more intimate than male-male interactions on twitter and reddit. different from social media, open collaboration communities such as wikipedia have slowly won the trust of public (young et al., 2016). wikipedia has been trusted by many, including professionals in work tasks such as scientific journals (kousha and thelwall, 2017) and public officials in powerful positions of authority such as court briefs (gerken, 2010). implicit biases in such knowledge sources ## experimental setup in this section, we will introduce our collected corpus and the event extraction model in our study. ## detecting gender biases in events odds ratio. after applying the event detection model, we get two dictionaries e m and e f that have events as keys and their corresponding occurrence frequencies as values. among all events, we focus on those with distinct occurrences in males and females descriptions (e.g., work often occurs at a similar frequency for both females and males in career sections, and we thus neglect it from our analysis). we use the odds ratio (or) (szumilas, 2010) to find the events with large frequency differences for females and males, which indicates that they might potentially manifest gender biases. for an event e n , we calculate its odds ratio as the odds of having it in the male event list divided by the odds of having it in the female event list: the larger the or is, the more likely an event will occur in male than female sections by equation 1. after obtaining a list of events and their corresponding or, we sort the events by or in descending order. the top k events are more likely to appear for males and the last k events for females. calibration. the difference of event frequencies might come from the model bias, as shown in other tasks (e.g., gender bias in coreference resolution model (zhao et al., 2018)). to offset the potential confounding that could be brought by the event extraction model and estimate the actual event frequency, we propose a calibration strategy by 1) generating data that contains target events; 2) testing the model performance for females and males separately in the generated data, 3) and using the model performance to estimate real event occurrence frequencies. we aim to calibrate the top 50 most skewed events in females' and males' career and personal life descriptions after using the or separately. first, we follow two steps to generate a synthetic dataset: after getting the synthetic data, we run the event extraction model again. we use the detection recall among the generated instances to calibrate the frequency |e| for each target event and estimate the actual frequency |e| * , following: then, we replace |e| with |e| * in equation 1, and get k female and k male events by sorting or as before. note that we observe the model performances are mostly unbiased, and we have only calibrated events that have different performances for females and males over a threshold (i.e., 0.05). foot_5 we predict event types by applying eventplus (ma et al., 2021) on sentences that contain target events and take the majority vote of the predicted types. the event types are from the ace dataset. @xcite we calculate weat scores with all tokens excluding stop words (weat * column) and only detected events (weat column) for career sections. weat score. we further check if the extracted events are associated with gender attributes (e.g., she and her for females, and he and him for males) in popular neural word embeddings like glove (pennington et al., 2014). we quantify this with the word embedding association test (weat) (caliskan et al., 2017), a popular method for measuring biases in text. intuitively, weat takes a list of tokens that represent a concept (in our case, extracted events) and verifies whether these tokens have a shorter distance towards female attributes or male attributes. a positive value of weat score indicates that female events are closer to female attributes, and male events are closer to male attributes in the word embedding, while a negative value indicates that female events are closer to male attributes and vice versa. 7 to show the effectiveness of using events as a lens for gender bias analysis, we compute weat scores on the raw texts and detected events separately. for the former, we take all tokens excluding stop words. @xcite together with gender attributes from caliskan et al. (2017), we calculate and show the weat scores under two settings as "weat * " for the raw texts and "weat" for the detected events. ## results the effectiveness of our analysis framework. ## conclusion we conduct the first event-centric gender bias analysis at the corpus level and compose a corpus by scraping wikipedia to facilitate the study. our analysis discovers that the collected corpus has event gender biases. for example, personal life related events (e.g., marriage) are more likely to appear for females than males even in career sections. we hope our work brings awareness of potential gender biases in knowledge sources such as wikipedia, and urges wikipedia editors and contributors to be cautious when contributing to the pages.