Investigating Gender Bias in STEM Job Advertisements (2024)

Abstract: Gender inequality has been historically prevalent in academia, especially within the fields of Science, Technology, Engineering, and Mathematics (STEM). In this study, we propose to examine gender bias in academic job descriptions in the STEM fields. We go a step further than previous studies that merely identify individual words as masculine-coded and feminine-coded and delve into the contextual language used in academic job advertisements. We design a novel approach to detect gender biases in job descriptions using Natural Language Processing techniques. Going beyond binary masculine-feminine stereotypes, we propose three big group types to understand gender bias in the language of job descriptions, namely agentic, balanced, and communal. We cluster similar information in job descriptions into these three groups using contrastive learning and various clustering techniques. This research contributes to the field of gender bias detection by providing a novel approach and methodology for categorizing gender bias in job descriptions, which can aid more effective and targeted job advertisements that will be equally appealing across all genders.

## introduction academic institutions in recent decades have strived to launch several initiatives addressing diversity, equity, and inclusion in the hopes of making academic gender representation more equal. however, the problem of gender bias still persists in academia and widens particularly among stem fields. @xcite report that at the top 50 research universities in the u.s., women hold only 31% of the tenured or tenure-track faculty positions. @xcite present that gender disparity in stem persists because of two reasons: women leave stem careers because they feel that their family plans will be hindered because of their professional lives, and due to low self-assessment of their skills in stem's intellectual tasks. they introduce the concept of 'professional role confidence', and argue that women's lack of this confidence, compared to their male counterparts, reduces their likelihood of pursuing careers in engineering. men and women have been found to identify with different goals as core human motivations. men often relate more with agentic goals, while women relate with communal goals @xcite . agentic goals display an affinity to one's status, achievement, and independence, along with speaking assertively, influencing others, and initiating tasks whereas communal goals showcase a drive to contribute to the community, connect, and share with others. @xcite suggest that women may experience intimidation and barriers with job descriptions that are formulated using agentic language. they present a list of masculine-coded and feminine-coded words that represent agentic and communal traits, respectively, and show that women judge jobs with a lot of agentic language as less appealing than jobs containing communal language. building on @xcite 's work, @xcite created gender decoder, a freely available tool that quantifies gender bias in job descriptions by counting the number of masculine and feminine-coded words in them. this simple dictionary look-up approach relies only on the frequency of gender-coded words in the job advertisement and does not consider the contextual meaning of the word or how it is used in the sentence. current studies that deal with gender bias in job descriptions also rely on this method of labeling job descriptions as gender-biased or genderneutral. often, singular words are simply labeled as masculine-coded or feminine-coded. this practice can reinforce traditional gender stereotypes. instead, employing terms like agentic, balanced, and communal offers a more nuanced and inclusive approach to understanding language biases. by categorizing job descriptions based on these dimensions, we move away from reinforcing gender norms and acknowledge the diverse ways in which individuals can express themselves and their abilities. the research question that we address in this work is centered around how we can employ more comprehensive criteria to determine gender bias in job advertisements, moving beyond simply identifying specific masculine-coded words to label an advertisement as biased towards masculinity? this paper makes the following contributions to understanding gender bias in job descriptions: ## related work several research studies have addressed the gender disparity prevalent in academic faculty positions. according to a study by @xcite , citing data from the society for neuroscience, there has been a notable increase in the proportion of women applicants to phd programs in recent years. specifically, the percentage of female applicants increased from 38% in 2000-2001 to 57% in 2016-2017, with a corresponding matriculation rate of 48% for women in the latter year. however, despite these gains in representation among applicants and matriculants, women only accounted for 30% of all faculty positions in phd programs, indicating a significant disparity in gender representation at the faculty level. in stem fields, although there has been a steady rise in the number of female candidates obtaining postgraduate degrees in recent years, the representation of women in faculty positions has remained largely unchanged @xcite . current studies related to gender in nlp have looked at gender bias in the context of large language models @xcite and presented that llms are biased unfavorable for females. in this work, we use nlp to assess gender bias at the beginning of the hiring cycle -in job descriptions. a significant contributing factor to the gender disparity in academia is the significant lack of gender diversity within applicant pools. the initial point of contact between academic employers and job seekers typically occurs through job postings. research indicates that the content and language used in job postings play a crucial role in influencing an applicant's decision to apply for a particular position @xcite . @xcite found that job descriptions in male-dominated fields tend to contain words associated with masculine stereotypes more frequently than those in female-dominated fields. they demonstrated that job advertisements featuring more agentic language were perceived to be more suitable for men, making these positions less appealing to women candidates. @xcite draw inspiration from social science findings and propose language agency as a metric for gender bias evaluation in llm-generated professional documents. they present,"bias in language agency states that women are more likely to be described using communal adjectives in professional documents, such as delightful and compassionate, while men are more likely to be described using "agentic" adjectives, such as leader or exceptional". through their findings, they reveal that chatgpt generates reference letters with biased levels of language agency for male and female candidates. when describing female candidates, chatgpt uses communal phrases such as "great to work with", "communicates well", and "kind". on the other hand, the model describes male candidates as being more agentic, using phrases such as "a standout in the industry" and "a true original". through their study, @xcite demonstrate that there is a distinct difference in the way males and females are described in terms of language agency. different studies have referred to the concept of language agency to evaluate job descriptions as masculine or feminine coded. oldford and fiset (2021) have followed @xcite 's method of annotating job descriptions using a dictionary look-up approach. they focused on classifying finance internship job postings based on masculine and feminine words, as well as evaluating the text based on the percentage of adjectives and verbs that are either agentic (e.g., overcomes, confident, etc.) or communal (e.g., aided, loyal, etc.). their finding revealed that women exhibit greater goal congruity, leading to enhanced motivation and a greater sense of fit when job postings are high in communal language and low in agentic language. tang et al. (2017) adopt the approaches used by textio 1 and unitive 2 , two recruitment assistant services dedicated to promoting inclusivity in job advertisements, to detect gender bias in job descriptions. they observe and adapt the techniques of both these services and introduce two metrics: gender tone and gender target to assess gender bias in the advertisement. gender target follows textio's method and calculates the occurrences of gendered words in the ad, with masculine and feminine terms offsetting each other. meanwhile, gender tone assigns a weight to gendered words based on their specificity, with the cumulative weights reflecting the overall gender tone of the ad. most of the works that evaluate gender bias in job descriptions @xcite @xcite @xcite @xcite , rely on the frequency of individual gender-coded words to assess gender bias in job advertisements, neglecting to explore the contextual positioning of these words within the advertisements. this limitation is noteworthy as it overlooks the nuanced interplay between language and context in conveying bias or its absence. for instance, consider the following two sentences, which illustrate the importance of analyzing the contextual meaning of gender-coded words to detect gender bias: "we want a competitive member to join our team" and "we offer competitive remuneration." in both sentences, the word "competitive" is used. however, the first sentence implies 1 https://textio.com/products/recruiting 2 https://unitive.org/ a competitive environment or culture, potentially favoring traits typically associated with agentic language. in contrast, the second sentence simply indicates that the compensation provided is competitive in the market, without implying any specific genderrelated traits or preferences. these examples show that examining the context in which these words are employed allows for a more accurate assessment of whether gender bias is present and facilitates a clearer understanding of the intended message. ## methodology this section provides an overview of the methodology employed to address the research questions outlined in section 1. figure 1 presents a summary of these steps. ## evaluation, results, and analysis in this section, we present an analysis and visualization of the clustering model, evaluate its perfor-mance, and examine sentence-level label distributions within job advertisements.