Sexism in the Judiciary: The Importance of Bias Definition in NLP and In Our Courts (2021)

Abstract: We analyze 6.7 million case law documents to determine the presence of gender bias within our judicial system. We find that current bias detection methods in NLP are insufficient to determine gender bias in our case law database and propose an alternative approach. We show that existing algorithms’ inconsistent results are consequences of prior research’s inconsistent definitions of biases themselves. Bias detection algorithms rely on groups of words to represent bias (e.g., ‘salary,’ ‘job,’ and ‘boss’ to represent employment as a potentially biased theme against women in text). However, the methods to build these groups of words have several weaknesses, primarily that the word lists are based on the researchers’ own intuitions. We suggest two new methods of automating the creation of word lists to represent biases. We find that our methods outperform current NLP bias detection methods. Our research improves the capabilities of NLP technology to detect bias and highlights gender biases present in influential case law. In order to test our NLP bias detection method’s performance, we regress our results of bias in case law against U.S census data of women’s participation in the workforce in the last 100 years.

## introduction are gender biases present in our judicial system, and can machine learning detect them? drawing on the idea that text can provide insight into human psychology (jakiela and ozier, 2019), we look at gender-stereotyped language in case law as a proxy for bias in our judicial system. unfortunately, previous nlp work in bias detection is insufficient to robustly determine bias in our database @xcite . we show that previous bias detection methods all share a common flaw: these algorithms rely on groups of words to represent a potential bias (e.g., 'salary,' 'job,' and 'boss' to represent employment as a potential bias against women) that are not standardized. this lack of standardization is flawed in three main ways. first, these word lists are built by the researchers with little explanation and are susceptible to researchers' own implicit biases. consequently, the words within the word list might not truly describe the bias as it exists in the text. second, the same bias theme (e.g., 'employment') often has different word lists in different papers. inconsistent word lists lead to varied results. as we show, using two different researcher's word lists to represent a bias on a single database can produce almost opposite results. third, there is little discussion about the method of choosing words to represent specific biases. it is therefore difficult to reproduce or extend existing research on bias detection. in order to search meaningfully for gender bias within our judicial system, we propose two methods for automatically creating word lists to represent biases in text. we find that our methods outperform existing bias detection methods and we employ our new methods to identify gender bias in case law. we find that this bias exists. finally, we map gender bias's progress over time and find that bias against women in case law decreases at about the same rate, at the same time, that women enter the workforce in the last 100 years. ## bias statement in this paper, we study gender bias in case law using two new nlp methods. we define gender bias in text as a measurable asymmetry in language when discussing men versus women (excluding group-specific words such as gender pronouns). bias is especially harmful in the context of case law decisions. if case law systematically associates men more positively and powerfully than women, the law creates representational harm by perpetuating unfair and inaccurate stereotypes. further, bias in law could lead to failure to account for gender-related harms that could disproportionately affect women. for example, because of the imposition of restrictions on recovery, there is no reliable means of tort compensation for victims of domestic violence, rape, and sexual assault @xcite . this is just one example where failure to equally consider both genders in law leads to real harm. the proposed bias detection algorithm only detects bias after the case has been written. however, case law is unique in that it sets precedent for other, later cases: judges often cite previous cases as a basis for a new judgment. therefore, we suggest that this bias detection method be used as a way for judges to more deeply understand biases present in the cases they cite. perhaps a deeper understanding of biases in historical cases could prevent biases from reappearing in new judgments. ## related works a variety of bias detection methods have been proposed in gender-related literature. prominent among these methods is the implicit associations test (iat) @xcite @xcite . iat measures the strength of associations between groups (e.g., men, women) and evaluations (e.g., good, bad) or stereotypes (e.g., strong, weak) to which those groups are assigned. the main idea is that classifying a group is easier, and therefore happens more quickly, when the subject agrees with the evaluation. for example, a subject has an implicit bias towards men relative to women if they are faster to classify men as strong / women as weak, than women as strong / men as weak. in nlp literature, the most prominent bias detection method is the word embedding association test (weat). weat measures the association of word lists representing a potentially biased theme (e.g., 'salary', 'job,' and 'boss' to represent employment) to a set of pronoun or otherwise gendered pairs such as (she, he) or (man, woman) @xcite @xcite . the association is measured by first training a word embedding model on text. the researchers then compute the distance of vectors relating to gendered word pairs (e.g., she / he) to words in word lists representing potential bias categories. the average distance of the words in a themed word list is the magnitude of bias. the vector direction (i.e., the positive or negative distance) represents towards which group the bias is directed. weat uses the vector direction and proximity as a proxy for semantic association. weat has been used to uncover biases in many databases. for example, garg et al (2017) used weat to detect bias in google news. other research has used weat to identify bias in twitter posts in 99 countries @xcite . one particularly relevant study to our research uses the same database of case law to study gender bias using weat, finding that cases written by female and younger judges tend to have less bias than their older, male counterparts (ash, chen, and ornaghi, 2020). another particularly relevant work uses weat to study the presence of gender bias in four different databases @xcite . the same work also suggests a preliminary method of automatically detecting word lists to represent gender bias but falls short of suggesting a way to determine the relevance of each bias category. the efficacy of weat in bias detection is inconsistent. weat also fails robustness tests: for example, the average bias magnitude of words in an employment word list might be skewed towards men, but there could be words within the word list whose bias magnitude skews towards women @xcite . even different capitalizations of the same word might have different bias magnitudes ## comparison over time we have shown that automating the formation of word lists to represent biases in weat leads to consistent and robust bias detection in text. using two separate approaches, we created bias word lists to detect gender bias in case law and found that it exists. however, given the time span of our database, the presence of language difference between genders is not surprising. in order to detect meaningful gender bias, i.e., bias that is stronger in text than real-world historical gender gaps, we track the change in bias magnitude over time. we regress the change in bias magnitude against women's participation in the workforce and find they progress at about the same rate. ## conclusion in our research, we analyze 6.7 million documents of case law for gender bias. we find that existing bias detection methods in nlp are not sufficiently consistent to test for gender bias in cap. we attribute this inconsistency to the lack of methodical building of word lists to represent bias. we therefore suggest two new approaches to building word lists for bias representation. we test our two approaches on cap and find both methods to be robust and consistent, and to identify the presence of gender bias. we also show that, when the change in bias magnitude over time is regressed against workforce participation rate in the last 100 years, and find they are heavily correlated. it is worth noting that, although this research focuses specifically on gender bias, the same methodology might be applied to other groups-provided that those groups are identifiable in text. as a future development in this research, we want to explore the results of our data that show that men are overwhelmingly associated with legal language, and women are not, even though women are not less likely to be defendants in certain types of law-such as torts law @xcite . (in fact, chamallas makes the point that torts regulation can sometimes discriminate against women in other ways, and that torts law should in fact have more female defendants than male.) could it be that the law implicitly does not recognize women as independent legal entities in the same way it does men? we also would like to study possible intersections of identity in our judicial system. for example, we show that there is employment labor figure 3: labor slant and wlor thematic clustering regression data.