Dissecting Biases in Relation Extraction: A Cross-Dataset Analysis on People’s Gender and Origin (2024)

Abstract: Relation Extraction (RE) is at the core of many Natural Language Understanding tasks, including knowledge-base population and Question Answering. However, any Natural Language Processing system is exposed to biases, and the analysis of these has not received much attention in RE. We propose a new method for inspecting bias in the RE pipeline, which is completely transparent in terms of interpretability. Specifically, in this work we analyze biases related to gender and place of birth. Our methodology includes (i) obtaining semantic triplets (subject, object, semantic relation) involving ‘person’ entities from RE resources, (ii) collecting meta-information (‘gender’ and ‘place of birth’) using Entity Linking technologies, and then (iii) analyze the distribution of triplets across different groups (e.g., men versus women). We investigate bias at two levels: In the training data of three commonly used RE datasets (SREDFM, CrossRE, NYT), and in the predictions of a state-of-the-art RE approach (ReLiK). To enable cross-dataset analysis, we introduce a taxonomy of relation types mapping the label sets of different RE datasets to a unified label space. Our findings reveal that bias is a compounded issue affecting underrepresented groups within data and predictions for RE.

## introduction language technologies are widely spreading throughout our everyday life. however, it has been demonstrated that these technologies are often affected by the presence of gender and racial biases @xcite . "bias" is a cover term for a number of issues, which according to @xcite may emerge at any stage of the natural language processing (nlp) pipeline. they could come from * equal contribution the data curation process @xcite , be intrinsic into the trained model @xcite , or they could derive from the cultural background of nlp practitioners @xcite ). an orthogonal taxonomy of biases distinguishes between allocative and representational ones @xcite . allocative biases regard the unequal distribution of opportunities across different groups, such as disparity in granting loans @xcite or the systematic exclusion of certain minorities from public archives @xcite . representational biases focus on stereotypical associations between groups and certain features @xcite (e.g., women and lexicon about marriage and parenthood). @xcite show that existing works in nlp mainly focus on representational biases while the allocative ones are often overlooked. in this context, relation extraction (re) techniques represent a powerful tool to jointly explore the two types of bias described above. re methods extract fine-grained triples from texts (subject, object, and the semantic relation connecting them), allowing for the discovery of gaps in digital archives. previous work performed event extraction on wikipedia biographies to study the presence of systematic gender biases in this archive @xcite . @xcite collected a distantly supervised dataset from wikipedia for exploring gender bias in re, but they only include four relation types @xcite . despite this preliminary work, standards for the adoption and evaluation of re techniques for bias detection are still missing and are limited to the analysis of gender. furthermore, before using re for bias detection there is the pressing need to explore whether these systems portray any themselves. in this paper, we explore the presence of biases in re, both at the level of data (by analyzing the training data) and model (by analyzing the model predictions). we illustrate our procedure in figure 1. as a first step, in order to enable crossdataset analysis, we introduce a taxonomy of relation types mapping the label sets from different re datasets into a unified label space. then, as a second and third steps we collect information about people mentioned in a text. this includes semantic relations involving people (from re), and meta-information related to them (i.e., 'gender' and 'place of birth'; using entity linking). as a last step, we explore the allocative and representational biases by inspecting potential imbalances into the distribution of the extracted triples across different groups (e.g., men versus women). concretely, we investigate if any relation type (e.g., member, contributes) is more likely associated with one social group (more details in section 5). we repeat our procedure both on the training sets on three widely adopted re datasets: sred fm (huguet @xcite , crossre @xcite , nyt @xcite ; and on the predictions of a state-of-the-art re approach, re-lik @xcite . not only do our findings corroborate existing research regarding the prevalence of gender biases in re but they also broaden the discourse by uncovering biases along additional dimensions, such as origin. to our knowledge, this is the first investigation that examines bias through the lens of transfer learning and reveals the nuanced effects of simplistic interventions like data balancing. while such strategies may reduce biases for certain target groups, they can inadvertently introduce new biases, underscoring the necessity for a more sophisticated, multi-axial approach for bias mitigation. the contributions of this paper are: sun et al. (2019) and blodgett et al. (2020) emphasize current issues in the research about bias detection and mitigation. the first presents a survey aimed at identifying research directions for gender bias detection, while the second criticizes how research in bias detection and mitigation is usually conducted. in order to make explicit potential biases in nlp, bender and friedman (2018) and mitchell et al. (2019) propose to better document datasets and language models (lms) respectively. some works released ad-hoc datasets to explore bias detection. zhao et al. (2018) presented wino-bias, a dataset for coreference resolution aimed at testing stereotypical associations between women and certain types of profession. nadeem et al. (2021) introduced stereoset, for testing the presence of stereotypical knowledge in lms while gehman et al. (2020) released realtoxicityprompt, a list of annotated prompts that is intended to measure the toxicity of text generated by lms. kiritchenko and mohammad (2018) presented the equity evaluation corpus, designed to measure gender and racial biases in models trained for sentiment analysis. several work on bias analysis focuses on inspecting the internal representation of nlp models. caliskan et al. (2017) proposed two metrics for bias detection from word embeddings; may et al. (2019) from sentence encoders; and kurita et al. (2019) from contextualized word embeddings. more recent approaches in this direction use probing strategies @xcite . however, the outcome of these methods is often hard to interpret because of the black box nature of neural models. in order to prioritize interpretability of the results and obtain a more transparent bias analysis, we propose a new procedure for bias detection in re technologies, which is applicable both at the level of data and model. ## methodology we introduce a four-step procedure for detecting biases related to 'gender' and 'place of birth' in the relation extraction pipeline (see figure 1 ). the method can be easily extended to explore other socio-demographic biases. ## experimental setup we follow the four-step procedure described in section 3 to investigate biases in three commonly adopted re datasets, and the predictions of a popular re model. below, we describe our experimental setup in terms of datasets (section 4.1) and modeling (section 4.2). details about their licenses can be found in appendix b. ## social bias analysis in this section we report our bias analysis conducted on the training sets of the datasets described in section 4.1 and on the predictions obtained with our trained models. in line with previous work on 'gender' bias analysis, we consider men versus women @xcite . for biases related to the 'place of birth', instead, we follow previous work and consider global north versus global south @xcite . such a distinction has been introduced by the brandt commission @xcite in the context of an effort of reducing economic issues affecting third world's countries. therefore, we design an operational definition of country belonging to the global south as being a former colony and having a human development index lower than 0.8. we discuss more in details these division in the limitation section. we maintain the distinction between allocative and representational biases and explore both bias types at the level of training sets (sections 5.1 and 5.2) and in the predictions (sections 5.3 and 5.4). ## bias mitigation in this section we look at a common approach to tackle skewed distributions of data by balancing the pre-training data (sred fm ) in order to obtain fairer representations of underrepresented groups. this mitigating strategy was the only one shown to be effective in @xcite . since in as it can be observed in the bottom line of ## conclusion in this paper we address the critical matter of biases within re data and systems, and propose a four-step procedure to analyze them. our approach showcases the widespread nature of biases in the life-cycle of re systems, encompassing datasets, transfer learning and model predictions. our findings reveal a concerning underrepresentation of women and individuals from the global south as well as undesired biases for specific relation types. we demonstrate that tackling bias is a complex and compounded issue which requires careful thought. simple techniques, such as balancing the data for an underrepresented group, may introduce other unwanted biases. we also provide a carefully designed taxonomy of relation types that enables com-parison and effective transfer across re datasets. in conclusion our work serves a dual purpose: on one side, it sheds light on the pervasive biases related to gender and origin within re datasets and systems, on the other it offers a critical perspective on the use of information extraction (ie) techniques for bias exploration. this study emphasizes the need for nuanced, multi-faceted approaches to detect and mitigate biases, urging the community to proceed with caution and depth in developing and applying re technologies.