A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models (2024)

Abstract: Chemical named entity recognition (NER) models are used in many downstream tasks, from adverse drug reaction identification to pharmacoepidemiology. However, it is unknown whether these models work the same for everyone. Performance disparities can potentially cause harm rather than the intended good. This paper assesses gender-related performance disparities in chemical NER systems. We develop a framework for measuring gender bias in chemical NER models using synthetic data and a newly annotated corpus of over 92,405 words with self-identified gender information from Reddit. Our evaluation of multiple biomedical NER models reveals evident biases. For instance, synthetic data suggests that female names are frequently misclassified as chemicals, especially when it comes to brand name mentions. Additionally, we observe performance disparities between female- and male-associated data in both datasets. Many systems fail to detect contraceptives such as birth control. Our findings emphasize the biases in chemical NER models, urging practitioners to account for these biases in downstream applications.

## introduction chemical named entity recognition (ner) is the extraction of chemical mentions (e.g., drug names) from the text. chemical ner is essential in many downstream tasks, from pharmacovigilance @xcite to facilitating drug discovery by mining biomedical research articles @xcite . for instance, chemical ner systems are the first step in pipelines developed to mine adverse drug reactions (adrs) @xcite . however, it is unknown whether these systems perform the same for everyone. who benefits from these systems, and who can be harmed? in this paper, we present a comprehensive analysis of gender-related performance disparities of chemical ner systems. performance disparities have recently received substantial attention in the field of nlp. for example, there are differences in text classification models across sub-populations such as gender, race, and minority dialects @xcite @xcite @xcite . performance disparities can manifest in multiple parts of nlp systems, including the pre-trained models (e.g., word embeddings) and their downstream applications @xcite @xcite . while previous research has explored these disparities for ner systems, the focus has been largely on synthetic data and non-biomedical ner applications @xcite . our study addresses this gap by providing a comprehensive examination of genderrelated performance disparities in chemical ner, focusing on both synthetic and real-world data. this paper is most similar to @xcite with two primary distinctions. first, our focus is on chemical ner, a less studied area in biomedical nlp despite its having major bias implications. second, while @xcite uses synthetic data and templates (e.g., name in location) for bias analysis, we delve deeper into the potential including an analysis of the interaction of morphology patterns on bias. for instance, @xcite highlighted a preference for linguistically feminine brand names in the market, leading drug companies to adopt such naming conventions. these patterns in training data can inadvertently cause models to misclassify female names as chemicals. we also examine real-world data looking at the performance of chemical ner systems on groups that identify as male or female. for instance, @xcite shows that women are more frequently prescribed antidepressants than men. other studies, like @xcite , reveal gender differences in pain sensitivity and opioid prescriptions, with women receiving opioids twice as often. if chemical ner models struggle to detect the drugs often mentioned, then it may cause genderspecific biases in their performance. our analysis identifies some of these patterns in real data. overall, this paper presents a dual approach: we explore template data but also assemble and annotate a novel real-world dataset with self-identified gender information. 1 synthetic data allows us to target specific biases in the models (e.g., morphological issues). likewise, we believe exploring data from people who have self-identified their demographic information will provide a more realistic understanding of how these models will perform based on how people write and what they write about. our main contributions are two-fold: ## related work prior work extensively curated labeled data for chemical ner and developed domain-specific models. for example, the chemdner corpus @xcite was created for the 2014 biocreative shared task on chemical extraction from text. researchers recognize the importance of these systems and are working to make them as fair and accurate as possible. likewise, the cdr @xcite dataset was developed to detect chemical-disease relations for the 2015 shared task. similar to traditional ner tasks @xcite , a broad range of approaches have model that improved on many prior state-of-the-art results. hunflair @xcite introduced a method that matches the word, contextual, and character embeddings into a unified framework to achieve state-of-the-art performance. in this paper, we evaluate several state-of-the-art systems. particularly, we focus on systems that use word embeddings, sub-word embeddings, and character embeddings, which allows us to understand the impact of morphological features of the chemical names on gender bias. several previous works have measured and highlighted bias in different nlp tasks. for instance, @xcite measures the bias of offensive language detection models on african american english. likewise, park et al. (2018) measures gender bias of abusive language detection models and evaluates various methods such as word embedding debiasing and data augmentation to improve biased methods. @xcite shows racial and ethnic bias when identifying hate speech online and that tweets in the black-aligned corpus are more likely to be assigned hate speech. @xcite creates the wikigenderbias dataset to evaluate the gender bias in the relation extraction (re) model, confirming that the re system behaves differently when the target entities are of different genders. @xcite demonstrate that biases in biomedical applications can stem from various sources, such as skewed diagnoses resulting from clinical depression scales that measure symptoms more prevalent in women, potentially leading to a higher reported incidence of depression among this group @xcite . other sources include the underrepresentation of minority populations such as pregnant women (organization and for women's health in society, 2009), non-representative samples in ai training data, and inherent algorithmic discrimination, all potentially contributing to inaccurate and unfair results. recent research has shown that although large language models (llms) are now increasingly being used for tasks such as named entity recognition @xcite and relation classification @xcite , they also have the potential to reinforce or exacerbate gender biases, which emphasizes the importance of careful deployment to prevent the reinforcement of stereotypes @xcite . overall, several metrics have been proposed to measure gender bias. one of the most commonly used metrics involves measuring bias by examining model performance disparities on male and female data points @xcite . performance disparities have been observed across a wide array of nlp tasks such as detecting virus-related text @xcite , language generation @xcite , coreference resolution @xcite , named entity recognition @xcite , and machine translation @xcite . most related to this study, researchers have shown that traditional ner systems (i.e., to detect people, locations, and organizations) are biased concerning gender @xcite . specifically, @xcite demonstrates that female names are more likely to be misidentified as a location than male names. this stream of research underscores the importance of our investigation into performance disparities in nlp. finally, while not directly studied in prior ner experiments, it is important to discuss some background about morphological elements of chemical names. morphological elements often representing masculinity or femininity are frequently used in chemical naming conventions. according to @xcite , consumers perceive linguistically feminine brand names as warmer and more likable. for instance, adding a diminutive suffix to the masculine form of the name usually feminizes it. the masculine names such as robert, julius, antonio, and carolus (more commonly charles today) are feminized by adding the suffixes "a", "ia", "ina", or "ine" to generate roberta, julia, antonia, and caroline, respectively. the suffixes "ia" and "a" is commonly used for inorganic oxides such as magnesia, zirconia, silica, and titania (hepler-smith, 2015). likewise, "ine" is used as the suffix in many organic bases and base substances such as quinine, morphine, guanidine, xanthine, pyrimidine, and pyridine. hence, while these practices were not originally "biased" in their original usage, they can potentially impact model performance (e.g., feminine names can be detected as chemicals). therefore, the patterns can cause biased models. as part of our approach to investigate this potential source of bias, we propose using synthetic data to quantify this phenomenon. ## datasets we use five main datasets used in our experiments: three are publicly-released datasets based on pubmed (cdr @xcite , chemd-ner @xcite , and chebi @xcite ) and two are newly curated datasets, one using social media data and another based on templates. ## methods the goal of ner is to classify words into a sequence of labels. formally, given an input sequence @xmath0, x 2 , . . . , x n ] with n tokens, the goal of ner is to output the corresponding label sequence @xmath1, y 2 , . . . , y n ] with the same length, thus modeling the probabilities over a sequence p(y|x ). for this task, we conducted an experiment evaluating out-of-domain models on the askdoc corpus. specifically, models were trained and optimized on the chemdner and cdr datasets and then applied to the askdoc dataset. all models are evaluated using precision, recall, and f1. to measure bias, we use precision, recall, and f1 differences @xcite . specifically, let m be males' performance metric (e.g., f1), and f represent the female metric. the bias is measured using the difference fm. ## results cdr, chemdner, and chebi results. this finding that female names are classified as chemicals is consistent with prior research on naming conventions for brands being gendered @xcite . to further investigate this, we randomly sampled 100 chemicals from all three datasets and measured the number of brand name mentions. overall, we found one brand name in the chemdner dataset, 19 in the cdr dataset, and 32 in the askdoc dataset, which generally matches the bias performance differences in askdoc results. the askdoc results, as shown in for the fine-grained results, we note that bias and performance can vary depending on unique combinations of the dataset and embedding types. however, for the aggregate results, we have two major findings. first, we find that general domain embeddings are more biased when applied to the chemical ner task (e.g., .0056 vs. .0330 precision). this further emphasizes the results from the synthetic data study. second, we find that word embeddings are generally less fair than flair bert/biobert embeddings for precision (.0071 vs. .0156 and .0352) and f1 (.0158 vs. .0242 and .0245). what does this mean in real-world terms? considering a sample of 1,000,000 chemical mentions across male and female posts (a relatively small number in social media), a 4% recall difference results in an additional 40,000 false negatives for the female group. for example, there are wellknown health disparities between men and women for depression, with absolute differences of less than 3% @xcite . hence, a 4% recall difference can substantially impact findings if applied researchers or practitioners use out-of-domain models to understand medications for this disease. such a considerable gap can markedly affect the utility and trustworthiness of these predictive outcomes in practical scenarios. ## limitation there were several limitations to our study. first, the adjudication of disagreeing items was dependent on the judgment of a single graduate student, potentially introducing human error and bias compared to a multi-adjudicator approach. second, the vast volume of data from the active r/askdoc subreddit community makes the feasibility of one person's comprehensive review debatable. although our annotation method is in line with standard practices, a more multi-faceted approach involving numerous annotators and adjudicators might offer improved accuracy and consistency in future datasets. third, our study focuses on binary representations of gender (ignoring non-binary people). moreover, the social security's most popular baby names (ssn) names may not adequately mention immigrant-related names. hence, the results may be european-specific. ## ethical considerations in this study, we consider binary gender biases. while binary gender is a common area of study in nlp literature @xcite , and we follow best practices of using self-identified gender @xcite , it leaves a large portion of individuals out of the study (i.e., not counted). moreover, we also follow prior @xcite ) by relating names to gender. nevertheless, names are not directly related to gender identity. hence, in future work, we intend to explore data collection methods beyond binary gender. specifically, we plan to collect data from other groups for detailed studies of model performance. additionally, using data from platforms like reddit's r/askdocs, where individuals share personal health experiences, raises ethical concerns about the potential exposure of personally identifiable information (pii) and sensitive personal health information (phi). while our research aims to assess gender bias without examining personal details, the potential for identifiable information necessitates careful handling to protect privacy and confidentiality, following established ethical guidelines for internet research @xcite . ## conclusion in this paper, we evaluate the gender bias of chemical ner systems. moreover, we compare bias measurements from synthetic data with real-world self-identified data. we make two major findings. first, chemical ner systems are biased with regard to gender for synthetic data. specifically, our study found that female name-like patterns feature prominently in chemical naming conventions. this characteristic leads to a notable bias in ner systems, where female names are disproportionately identified as chemicals, inadvertently escalating the gender bias in these systems. second, we explored the performance of these models in real-world scenarios and found that most models perform better on male-related data than female-related data. a striking revelation was the system's poor performance when identifying chemicals frequently found in female-related data, such as mentions of contraceptives. in conclusion, the results of our study emphasize the urgent need for deliberate bias mitigation strategies in chemical ner systems. our findings spotlight the necessity for incorporating both synthetic and real-world data considerations to develop models that are both fair and reliable. there are two major paths for future research. first, while large language models are still behind in terms of performance for ner systems @xcite , they are becoming more common. future work should explore biases in prompting-based ner solutions. second, we plan to explore how the chemical ner biases impact downstream tasks such as relationship classification and question answering.