Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender? (2024)

Abstract: We examine whether large language models (LLMs) exhibit race- and gender-based name discrimination in hiring decisions, similar to classic findings in the social sciences (Bertrand and Mullainathan, 2004). We design a series of templatic prompts to LLMs to write an email to a named job applicant informing them of a hiring decision. By manipulating the applicant’s first name, we measure the effect of perceived race, ethnicity, and gender on the probability that the LLM generates an acceptance or rejection email. We find that the hiring decisions of LLMs in many settings are more likely to favor White applicants over Hispanic applicants. In aggregate, the groups with the highest and lowest acceptance rates respectively are masculine White names and masculine Hispanic names. However, the comparative acceptance rates by group vary under different templatic settings, suggesting that LLMs’ race- and gender-sensitivity may be idiosyncratic and prompt-sensitive.

## introduction field experiments in prior social science research @xcite @xcite have demonstrated that black-or white-sounding names play a nontrivial role in influencing the hiring decision of candidates with similar qualifications. their results suggest that applicants with names perceived as african american encounter significantly fewer opportunities in comparison to their counterparts with names perceived as european american. following the rapid advancement of large language models (llms; @xcite , a number of studies have examined the ways in which llms exhibit human-like behaviors and cognitive biases @xcite @xcite . in this work, we pose the following question: when prompted to make hiring decisions, do llms exhibit discriminatory behaviors based on the race, ethnicity, and gender associated with a job applicant's name? there are several reasons to study this question: (1) to contribute to scientific understanding of the internal, representational biases of llms, (2) to demonstrate the potential harms of using llms in real-world hiring decisions, and (3) as further validation of llms as a tool for social scientists to cheaply test hypotheses prior to conducting costly, real-world studies. the research question has implications for understanding both representational and allocational harms of llms @xcite @xcite . we design a series of prompts that ask an llm to write an email (e.g., on behalf of a hiring manager) to inform a job applicant about the outcome of their hiring decision. in all settings, the prompt contains the instructions to the llm and the first name of the applicant. we experiment with three additional variables: the job title (position sought), the candidate's level of qualification, and template (para)phrasing. crucially, all prompts do not spec-figure 2 : prompt construction. the cartesian product of the three sets of elements in this figure gives rise to all our 820 templates used in the study. both "[role]" and "[name]" are placeholder tokens that are instantiated with some occupation and some first name, respectively, during the construction of a prompt. if a prompt contains the description of the candidate's qualification, the sentence indicating the qualification is prepended to the base template. *when the role is not specified, the phrase "of [role]" in gray is omitted. with large-scale analysis of these generations (over 2 million emails), we find that llms tend to favor white applicants in making hiring decisions. in contrast, models tend to disadvantage names associated with underrepresented groups. in particular, hispanic names receive the least favorable treatment. while it is (hopefully) unlikely that employers would use llms in precisely this fashion, we believe that, by isolating the influence of names on hiring decisions, these experiments may serve as a "canary in the coalmine," indicating the risk of possible fairness issues with the use of llms at other stages of the hiring pipeline, or in professional workplace settings more generally. ## experiment setup to study the influence of race and gender on llms' hiring decisions, we develop a set of prompt templates instructing models to write an email to a job applicant informing them of a hiring decision. each template contains a "[name]" placeholder, which we substitute with first names statistically associated with a particular race or ethnicity, and gender in the united states. we then measure the average rate of acceptance within each demographic group and compare it to the average acceptance rate over all groups. this methodology of first-name substitution is well established in the social sciences and in nlp research for measuring biased or discriminatory behavior in humans or models @xcite @xcite . ## results and discussion we examine the generated emails from a variety of llms and elaborate how they relate to known labor market discrimination. we present the acceptance rates for every intersectional group in different templatic settings and models in to measure the statistical significance in the difference between the email outcome distributions, we conduct a permutation test between each group's acceptance rate and the population acceptance rate. details about the permutation test are elaborated in appendix c.3. differences are small but statistically significant. we aggregate the acceptance over (1) a subset of 7 occupational roles foot_1 and (2) all 41 occupational roles respectively for different models in we observe that the absolute differences between the highest and lowest acceptance rate for different groups are generally small (between 1.60% and 3.78% across models). despite the small magnitude, our permutation test testifies the statistical significance. a model that discriminates in a small but statistically significant manner can still be problematic. an absolute disadvantage of 3.78% based purely on the racial, ethnic, or gender associations of one's name should be concerning, particularly as such differences, if systematic, can accumulate throughout a pipeline where a series of slightly discriminatory decisions are made @xcite . acceptance rates are uniformly lowest for hispanic male names. hispanic male applicants consistently receive the least favorable treatment in many settings across mistral-7b (tables 1 , 2 , 3), llama2-{7b, 13b, 70b} (tables 1 , 2 , 3), and gpt-3.5 ( some groups exhibit higher acceptance rates. llama2-70b shows least variation across demographic groups. llama2-70b appears to exhibit the least variation in acceptance rates across groups ( ## related work first names, demographic identities, and economic opportunities researchers have been using first names that have strong correlation with some demographic attributes, such as gender, race/ethnicity, and age, to examine the problem of social bias in both social science studies and nlp systems @xcite @xcite . partially due to their association with demographic identities, first names often lead to inequitable distribution of economic opportunities as people build stereotypes in favor of or against names that reveal a person's demographic identity @xcite @xcite . ## conclusion through the use of 820 templates and 300 names, we generate as many as 756, 000 job application outcome notification emails per model that we use to measure llms' discriminatory behavior in labor market decisions. our analyses demonstrate the presence of such discrimination in some llms against some traditionally underrepresented groups, such as hispanic, as their acceptance rates are systematically lower than the average in multiple cases. white applicants, however, are often portrayed in a more positive light with a higher chance of getting accepted. our findings alert the community to be concerned about the implicit biases within the model as they could cause both representational and allocational harms to various demographic groups in downstream tasks.