Are Female Carpenters like Blue Bananas? A Corpus Investigation of Occupation Gender Typicality (2024)

Abstract: People tend to use language to mention surprising properties of events: for example, when a banana is blue, we are more likely to mention color than when it is yellow. This fact is taken to suggest that yellowness is somehow a typical feature of bananas, and blueness is exceptional. Similar to how a yellow color is typical of bananas, there may also be genders that are typical of occupations. In this work, we explore this question using information theoretic techniques coupled with corpus statistic analysis. In two distinct large corpora, we do not find strong evidence that occupations and gender display the same patterns of mentioning as do bananas and color. Instead, we find that gender mentioning is correlated with femaleness of occupation in particular, suggesting perhaps that woman-dominated occupations are seen as somehow “more gendered” than maledominated ones, and thereby they encourage more gender mentioning overall.

heading: 1 Introduction
text: While people of any gender can do any job, most occupations in the current world are not fully genderbalanced, meaning a statistic association exists between particular genders and particular occupations. We can see evidence of such associations in how humans talk about occupations. When talking about a carpenter who is a woman—a maledominated profession in the US—people may refer to her as “female carpenter”. This usage suggests that a woman being a carpenter is rare (and potentially surprising), and thus some find it worth remarking upon explicitly. Men on the other hand, are more likely to be called “carpenter” instead of “male carpenter”, presumably because the latter feels redundant or overinformative somehow.
Typicality effects such as these shape the way we use language in general. For example, Degen et al. (2020) adduced evidence for typicality effects
in the color domain, finding that speakers never referred to an image of a lone yellow banana as “the yellow banana”, but almost always referred to a lone blue banana as “the blue banana”. Because we think of the (stereo-)typical banana as yellow, the blueness of a blue banana is more worth remarking upon than the yellowness of a yellow one. In this work, we explore a parallel phenomenon in the occupation gender space: Are there gender typicality effects for occupation nouns? If so, we expect more mentions of “male” when an occupation is woman-dominated, and more mentions of “female” when an occupation is man-dominated.
Alternatively, gender mentioning may occur more often when the gender associated with an occupation is salient. Past work in social psychology and gender studies has suggested that gender is generally more salient for women than for men. In particular, men are seen as more typical of the concept of “person” than women are, and that women are
4254
more associated with gender than men are (Gilman, 1911; Hamilton, 1991; Bem, 1993; Merritt and Kok, 1995; Bailey et al., 2019, 2020, 2022, 2024). Put simply, according to these theories, the concept of ‘woman’ differs from the concept of ‘man’ in that it has an additional semantic attribute that encodes cultural and/or biological facts or connotations (such as information about clothing, behaviors, etc.) that are associated with women.
In light of this past work, it is also plausible that associating women with “having gender” affects how we conceptualize woman-dominated occupations, such as “nurse” or “hairdresser”. If woman-dominated occupations are indeed conceptualized as “more gendered”, then encountering a woman-dominated occupation might make gender itself more salient, and lead to an increase gender mentioning in corpora. Were this to be the case, we would see higher overall mentioning of gender (with both “male” and “female” adjectives) for woman-dominated occupations.
In this study, we test these two hypotheses about occupation gender typicality using information theoretic approaches to analyze large text corpora. First, we set aside the null hypothesis that gender mentioning (modifying an noun with either “male” or “female” adjective) is completely unrelated to occupation gender (i.e., the empirical gender breakdown in the real population), see Section 5.1.
Next, we test the hypothesis that gender is more salient for women-dominated occupations. This hypothesis is borne out: We find that gender (both male and female) is mentioned more when the occupation is woman-dominated in our largest tested corpus, see Section 5.2. We report medium-to-large correlations between femaleness of occupation (the degree to which it is woman-dominated, according to U.S. labor statistics) and overall gender mentioning (r = 0.49), and for femaleness of mentioning (r = 0.50), maleness of mentioning (r = 0.42), respectively.
Finally, we test whether gender mentioning correlates with surprisingness, or non-typicality, of gender given an occupation. To test this, we define an information theoretic quantity based on the conditional entropy of female/male gendering given a particular job, whereby low occupation genderedness would correspond to a 50-50 gender breakdown. This can be seen as testing the “blue banana” hypothesis for occupation gender typicality, see Section 5.3. We do not find strong evidence for this hypothesis, with very weak correlations emerg-
ing for two different source datasets, Pushshift.io Reddit (r = 0.13) and Wikipedia (r = −0.12).
In sum, our results support the idea that occupation gender typicality is related to gender mentioning, but not in the way predicted by the “blue banana” hypothesis. Instead, gender mentioning appears to be more related to gender salience for woman-dominated occupations, at least for the the corpora we investigated.
To further analyze our results, we also explore three secondary questions. First, we ask: does corpus choice matter (see Section 5.4), and find that our effects are replicated in both Wikipedia and Pushshift.io Reddit, but that they are stronger for the Pushshift.io Reddit dataset. Second, we ask: do the empirical estimates of gender breakdown actually reflect the way people perceive the genderedness of occupations (see Section 5.5). Utilizing a standard word-embedding based method to determine perceived gender, we find that, while similar, perception and empirical observations do meaningfully differ. Third, we ask whether there are trends in the meaning of sentences which mention gender and occupations, such as negative sentiment (see Section 5.6).

heading: 2 Terminology and Notation

heading: 2.1 Terminology
text: Gender is a complex social phenomenon that manifests in how we use language and operate in our societies. Here, we draw upon a distinction described in Ackerman (2019) between social gender, which is how people see themselves in the world, and conceptual gender, which refers to how lexical items are associated with masculine or feminine properties, in the absence of explicit, formal grammatical features such as grammatical gender affixes, gender concord etc. (c.f. Hoyle et al. 2019; Williams et al. 2019, 2021). When we say “gender” in this work, we will be referring to conceptual gender.
Since occupation words can’t encode the social genders of individuals in English (because English doesn’t use grammatical gender marking systematically on all nouns), we decided to use the terms “female” and “male” here.1 We will use the terms “woman” and “man” when referring to the social genders. For the most part, “man” and “woman” will be used for illustrative examples, or when we
1We use these terms instead of “feminine” and “masculine”, because we want to avoid confusing occupation gender with the morphological categories of grammatical gender.
discuss the survey data from the U.S. Bureau of Labor Statistics, which is based on self-report.
In what follows, we have carved out a particular notion of gender which enables us to test existing hypotheses about the gender typicality of occupations. To do this, we have had to code gender as a binary variable. Clearly, this is not empirically sufficient to attain full coverage of all humans, as there are many genders in the English-speaking world, but we are restricted to this coding based on the availability of existing resources, see Section 3.1. Given our precise research questions, the restrictions we have inherited from existing resources do not overly limit or negatively impact our conclusions, as currently, we are aware of no occupations that appear to be “typical” of non-binary individuals or those with other genders. The lack of occupations typical of non-binary people is probably due to the small number of people who currently identify as non-binary, which are estimated to range from approximately 0.68% (Wilson and Meyer, 2021) to 1%2 of the US population. This may change in the future, and we look forward to future research exploring the nature of this change.

heading: 2.2 Notation
text: We describe our notational conventions with respect to random variables, entropy, and mutual information in the Appendix A.
We specifically define three r.v.s in the context of this study; the occupation noun associated with a person J , the gender associated with the occupation noun G, and if that gender was mentioned as a descriptor M . M and G are assumed to have a binary alphabet in the context of this investigation: M = {‘mention’, ‘no mention’ }, and G = {‘female identifier’, ‘male identifier’ }, if appropriate we may also use {0, 1} as shorthand notation for both. The alphabet for J is a list of 37 occupation nouns that we have specifically listed in Appendix B.1.
For example “female physician" is associated with (j =‘physician’, m =‘mention’, g =‘female identifier’). Note that when m =‘no mention’, we can not determine g. Hence, we do not have access to the full joint distribution PJMG. In Section 4, we will show how we can make statements about gender disparities, despite not being able to observe outcomes for g in many cases.
2Pew Research Center, 2022

heading: 3 Data Sources
text: We use four different data sources for this work, which we discuss in turn below.
To address our main research questions, we first find an empirical estimate of occupation gender (from the US Bureau of Labor Statistics). Then, we select two source corpora on which to run our analysis: We use Wikipedia and Pushshift.io Reddit, because they are extensive, openly available resources, which are commonly used in the NLP field, and frequently as training data for Language Models Language Models (LMs). Both should ensure a reasonable coverage of occupations with gender mentions for our analysis.
Finally, we prompt the opensource Llama 2 language model (Touvron et al., 2023) to regenerate text in the style of Wikipedia. We call this new dataset Llama 2 Wikipedia and use it to investigate whether occupation gender typicality measurements change as a result of a corpus being rewritten by an existing language model.

heading: 3.1 US Labor Statistics
text: Following Caliskan et al. (2017); Rudinger et al. (2018); Zhao et al. (2018); Bartl et al. (2020); González et al. (2020) i.a., our empirical estimates of occupation gender breakdowns have been obtained from the U.S. Bureau of Labor Statistics3. These estimates have been collected monthly by the Bureau of Census on behalf of the Bureau of Labor Statistics based on household survey. This survey, the Current Population Survey provides statistics about worker demographics, among other things, where individuals self-report their (binary) genders and occupations. We utilize labor statistics for 2023 (accessed in late fall).
We note that some occupations, such as ‘attendant’ are aggregated to include all roles with ‘attendant’ in their titles, including flight attendant, parking attendant, transportation service attendant, dining room and cafeteria attendants and bartender helpers. In some cases, occupations are missing from the statistics, so we substitute them with synonyms such as “Chief Executives" for “CEO” or “Police Officers" for “sheriff”. The comprehensive aggregated statistics are available in Appendix H.

heading: 3.2 Pushshift.io Reddit
text: Building upon the work of Humeau et al. (2020), we employ a pre-established dataset sourced from
3https://www.bls.gov/cps/cpsaat11.htm
the online social forum, Reddit. This dataset, acquired and made available by an independent third party, can be accessed publicly on pushshift.io (Baumgartner et al., 2020). It encompasses data collected from PushShift4 up to July 2019. We follow a similar data cleaning approach to Roller et al. (2020) to enhance the clarity of the signal. A comment, along with all its subsequent child comments, is eliminated if it fulfills any of the subsequent criteria: 1) The author’s identifier includes the term ‘bot’; 2) The origin of the comment is a known non-English Subreddit; 3) The comment is flagged as removed, deleted, or is devoid of content; 4) The comment comprises less than 70% alphabetic characters; 5) The comment contains a URL. To avoid comment repetition, we flatten each thread tree by concatenating comments in a pre-order traversal sequence. The processed data contains 301M threads.

heading: 3.3 Wikipedia
text: Wikipedia is a fairly large datasource containing encyclopedic information in many languages, which is moderated by volunteers. In conducting our research, we utilize an English Wikipedia dump dated April 2023. The dataset is prepared in accordance with the procedures outlined by Attardi (2015). This extracts articles and associated metadata from the raw XML files. Despite Wikipedia’s general suitability to our research questions, it is worth noting that Wikipedia data tends to be stereotypically skewed towards men, as highlighted by Wagner et al. (2015); Schmahl et al. (2020); Falenska and Çetinoğlu (2021).

heading: 3.4 Linguistic Analysis
text: Upon obtaining the pre-processed datasets, we adhere to the pipeline delineated by Williams et al. (2021). For this endeavor, we utilize the Stanza tool (Qi et al., 2020). The pipeline comprises the following steps: 1) Tokenization & Sentence Segmentation; 2) Part-of-Speech (POS) Tagging: Each token is assigned grammatical information (e.g., noun, verb, adjective); 3) Lemmatization: Each token is reduced to its base form, facilitating the normalization and simplification of the text; 4) Dependency Parsing: This final phase involves analyzing the grammatical structure of each sentence, thereby establishing relationships between words, as described in Chen and Manning (2014). The
4https://files.pushshift.io/reddit/
size of the datasets, following this procedure, is detailed in Table 1.

heading: 3.5 Extraction of gender - occupation pairs
text: Initially, our task involves extracting all occupationrelated words from our text resource and isolating the subset that includes gender mentions. To accomplish this, we commence with a basic list of occupation nouns, sourced from the U.S. Bureau of Labor Statistics (Appendix B.1). Additionally, we utilize a concise list of gender adjectives, namely “male”, “female”, “masculine”, “feminine”, “man”, “woman”, "non-binary” and “nonbinary”.5
Our approach involves identifying instances where an occupation noun appears in the text, followed by determining when a gender adjective modifies this noun. We then extract all amod dependencies, where an occupation noun serves as the head and a gender adjective as the dependent. In this way, we isolate all gender mentions and further categorize them by gender. The statistics pertaining to occupation mentions and gender-specific mentions derived from these pairs can be found in Tables E.1–E.3.

heading: 3.6 Llama 2 Wikipedia
text: Recent work has found that LMs can exacerbate gender imbalances present in their training data (Kotek et al., 2023). To investigate the extent to which LMs generate text that statistically matches what we find for occupation gender typicality, we investigate whether results would change if we examined a different version of Wikipedia, specifically one regenerated by an LM. If the results on LM-restructured Wikipedia are substantially different, it would suggest that the LM either introduces new gender bias, or removes existing gender bias. We utilize Llama 2 70B to generate Wikipedia content and subsequently analyze the distributional differences in occupation gender mentions. We confine our experiments in LM-generated datasets
5We find very few occupation nouns modified with “nonbinary” in Wikipedia (40) and Pushshift.io Reddit (101), therefore exclude them from the analysis for lack of signal.
to Wikipedia, as Pushshift.io Reddit is too large to be tractable.
To generate new text, we provide the first 256 words from the original Wikipedia article, along with the article’s title, from the original content describe above in Section 3.3. We instruct the model to produce an article of a minimum length of 700 words, aligning with approximately the average English Wikipedia article length of 658 words. The prompt used is in Appendix C to enable replicability of our results. We apply the same data processing and extraction procedure as for the original Wikipedia dataset. From Table 1, we can observe that the two are comparable in size.
We evaluate the BLEU score, a text similarity metric, between the original Wikipedia articles and those generated by Llama 2. Our findings reveal low BLEU scores (as detailed in Appendix D), indicating substantial textual differences between the original Wikipedia content and the Llama 2 regenerated versions. This suggests that the Llama 2 Wikipedia dataset warrants statistical analysis.

heading: 4 Methods
text: In this section, we will first describe how observed occurrence counts translate to probability mass functions (pmfs) relating to occupation, gender mentioning and gender. Henceforth, we describe how to compute quantitative markers for gender mentioning and gendered occupations, which we finally use to test our hypotheses from in Section 1.

heading: 4.1 Estimation of probability mass functions
text: The US Labor Statistics is our source to compute the the relationship of gender and occupation independent of gender mentioning in language. Specifically, we compute PG,J by dividing the number of men / women employed in a specific occupation by the total number of employees in the survey. This distribution will be used as the foundation to determine how statistically biased an occupation is with respect to gender.
For each text source, we first determine the likelihood of gender mentioning independent of the specific gender PM . It is the number of gendered occupation mentions divided by the total number of occupation mentions. We further compute, the conditional joint distribution PG,J |M=0. Similarly to the US Labor Statistics, we divide the number of male / female indicated occurrences of a specific occupation by the total number of gendered obser-
vations. Note that it would be more informative if we could compute PG,J,M directly, however since we can not determine gender if it is not mentioned, we need to approximate the joint distribution by combining information from the text source and US Labor Statistics.

heading: 4.1.1 Computing the joint
text: To compute PG,J,M , we need the previously computed pmfs; PG,J , PM and PG,J |M=0, where the latter two are computed from the text corpus, and the the first one is computed via the US Labor Statistics. By applying Bayes’ theorem and the chain rule of probability, we can compute the joint likelihood as follows;
PG,J,M=0 = PG,J |M=0 · PM=0 (1) PG,J,M=1 = PG,J − PG,J,M=0
PM=1 (2)
PG,J,M = PG,J |M · PM (3)
Note, that we assume that there exists a joint likelihood for which the US Labor Statistics is the correct marginal PG,J = ∑ m PG,J,M=m, and our text corpus gives the correct conditional PG,J |M=0. This assumption can be a problem, when there is a statistical bias in what occupations are more prominent in a text sources as opposed to the labor statistics. This can be problematic for multiple reasons: (1) some occupations such as “author" will be overrepresented in a text source such as Wikipedia, and (2) the gender and job distribution of a text source and labor statics needs to match, which can easily not be true because either comes from a different cultural or temporal context. However, in our analysis especially for Pushshift.io Reddit, we believe there to be a good match, as Reddit was created in 2005 and sampled in 2019, and the labor statistics have in all likelihood not changed dramatically since then.

heading: 4.2 Gender bias in occupations
text: We generally cannot access how people perceive the gender typicality of an occupation directly from text, but we can hope their perceived typicality matches the real world gender breakdown6. Consequently, in this study, we propose to measure occupation genderedness, which we define as
1−H(G|J = j). (4) 6For words such as “manager" this might not always be
correct.
Note that, according to our definitions in Appendix A, the entropy term will be in the range between 0 and 1. Consequently, occupation genderedness is low for unbiased occupations and high for gender biased occupations (i.e. man-dominated or womendominated occupations). We further compute the femaleness of occupation;
PG=0|J=j . (5)
Both are computed from the US Labor Statistics joint distribution PG,J .
We introduce both of these measures to test different hypotheses. The former measure is a symmetric estimator. If there is no gender specific surprise, we should find occupation genderedness to correlate well with gender mentioning, In other words we expect “male nurse” and “female CEO” to both be likely expressions. The second measure allows for detecting gender specific mentioning.

heading: 4.3 Gender mentioning in text
text: We measure two ways gender is mentioned in text: by computing (1) PM=0|J=j directly for the likelihood of any gender being mentioned, PM=0|J=j,G=0 for femaleness being mentioned, PM=0|J=j,G=1 for maleness being mentioned, and (2) the mutual information between gender and its mentioning MI(G;M |J = j).
We can estimate the correlation strength of gender bias markers and gender mentioning markers by computing the Pearson correlation coefficient of a linear regression. Further, as discussed in (Caliskan et al., 2017), we can make group comparisons of mutual information between female and male occupations based on distance between text embeddings. The technical details can be found in Appendix G.

heading: 5 Results

heading: 5.1 Rejecting the Null Hypothesis
text: First, we test whether we can reject the null hypothesis that gender mentioning is unrelated with occupation genderedness. For this, we calculate the conditional mutual information, as described in Section 2.2. If the null hypothesis is true, if there is no correlation, and we expect MI(M ;G|J = j) = 0. We present our results in Figure 2, with individual occupation noun results in Appendix I. These findings are compatible with the existence of relationship between gender mentioning and occupation genderness, and the next two sections will attempt to determine its nature.

heading: 5.2 Mentioning gender when it’s female
text: Next, we explore the hypothesis that gender mentioning is correlated with femaleness of occupation PG=f |J=j , under the assumption described in the introduction whereby woman-dominated occupations make gender more salient. We find that this is borne out for Pushshift.io Reddit, see Figure 8. We find significant correlations of moderate size between the femaleness of an occupation and mentioning of either gender (r = 0.51), mentioning of femaleness (r = 0.50), and mentioning of maleness (r = 0.45). Since this finding is robust across all three kinds of mentioning, we take this to be evidence in favor of the hypothesis that the female genderedness of woman-dominated occupations makes gender more salient in general, leading to more mentions of gender.
For Wikipedia, we do not find a strong effect of gender mentioning when female, see Figure 6. This effect is compatible with Wagner et al. (2015) and Schmahl et al. (2020), which find evidence, respectively, for linguistic gender bias in Wikipedia based on adjective sentiment and topic modeling or word embedding approaches. Our findings pertain to gender mention and occupation gender typicality, not to negative sentiment about women or whether stereotypes about family and science exist in the data source—both can be true simultaneously.

heading: 5.3 Ablation: Female Carpenters are not like Blue Bananas
text: We also explored the “blue banana” hypothesis: the gender which is not typical of the occupation will be mentioned. We defined occupation genderedness above as 1 −H(G = g|J = j), where low occupation genderedness meant 50-50 gender breakdown, or gender balance, and high occupation genderedness meant a strongly woman- or a man-dominated occupation. Our results found practically no correlation for either Pushshift.io Reddit or Wikipedia between occupation genderedness and gender mentioning, see Figure 4. We cannot verify a gender-occupation version of the “blue banana” hypothesis, and gender mentions appear to be affected by occupation genderness instead of surprise.

heading: 5.4 Strongest correlations from Pushshift.io Reddit
text: Our findings that gender mentioning is correlated with femaleness of occupation are not equally
present for all tested corpora: See Figure 5 and compare Figures 8-6 in the Appendix. We find that the correlations are medium sized for Pushshift.io Reddit, but very small for Wikipedia and even weaker for Llama 2 Wikipedia. We suspect the difference in correlation strength may be due to dataset size or to domain differences. On size, the Pushshift.io Reddit dataset is several times larger than the Wikipedia datasets, meaning there may be more signal to detect in Pushshift.io Reddit. On text domain, Pushshift.io Reddit has little centralized moderation and is very informal, while Wikipedia has clear style guidelines and an encyclopedic style. For example, Wikipedia policy describes “neutral point of view”, or a prohibition on subjectivity, which may generally decrease the prevalence of adjectives thought to be evaluative or unnecessary.

heading: 5.5 Female-coded vs female-dominated
text: Above, we compared ground truth occupation gender breakdown (from the US labor statistics) against gender mentioning. This presumes that
the ground truth statistics accurately represent the way humans think about occupations and gender, although this may not always be the case. For example, Kotek et al. (2023) reported that people prefer stereotypical gender association with occupations over genders that actually match the ground truth. As an expression of this mismatch, people may presume, for example, that the typical ‘CEO’ is a man in our society, even when the occupation is not man-dominated according to our empirical statistics. To determine whether peoples’ impression of an occupation’s gender (and genderedness) differs from the empirical gender (and genderedness), we utilize cosine similarity as a measure of similarity. Following Bolukbasi et al. (2016) and Caliskan et al. (2017), we deem occupations female- or malecoded by computing cosine distance of word embeddings of an occupation and (fe)male attribute words (see more details in Appendix G). Depending on the distance between an occupation and an attribute word, we can resplit our occupation list into two groups (male- and female- coded),
and compute the average mutual information for both. For both Pushshift.io Reddit and Wikipedia, we find a significant group difference between the mutual information of female- and male- coded occupations. The mutual information for the female coded occupations is 0.0187±0.0024 bits and 0.0161± 0.0012 bits, respectively. In contrast, the mutual information for the male coded occupations is 0.0026±0.00003 bits and 0.0037±0.00013 bits. In both cases, the female- vs male- coded groups are about an order of magnitude apart, indicating further evidence that mentioning is more correlated with perceived female occupations.

heading: 5.6 What is discussed when people mention gender?
text: One of our main findings is that occupations associated with women are more likely to be modified with adjectives signaling gender. However, these findings only point to the existence of an effect, they do not tell us, qualitatively, whether the gender
mentions occur because of semantic trends present in the comments.
During qualitative exploration of the Pushshift.io Reddit dataset, we noticed that the vast majority of samples containing woman-dominated occupations (for “female/male nurse”, for example) included derogatory, and/or sexually explicit content. Others examples discussed gender balance (or lack thereof) in particular occupations, or the experience of people holding non-gender typical roles. Another trend for sentences containing a womandominated occupation modified with “male” is that these were often negative sentiment. In a smaller number of examples, we also noticed discussions where the gender of a caregiver might be relevant (people expressing that they would prefer a gender matched nurse to care for them during a hospital stay).
To adduce a bit more quantitative description of these anecdotal observations, we sampled 80 comment threads that mentioned occupations and con-
tained an amod dependency with a gender adjective (20 of each types) to better understand what was being discussed when gender was mentioned. We performed three quantitative measurements: first, we measured comment sentiment, using the Stanza sentiment analysis tool (Qi et al., 2020). Next, we handcoded the samples for whether they contain offensive or inappropriate content, and/or whether the topic of discussion is gender balance in occupations. As anticipated, we find that sentiment is higher (0.6) when the occupation is man-dominated and modified with “male” than for the other three types (0.4 for “female carpenter” and “male nuse”, and 0.3 for “female nurse”). See Table 8 for more summary statistics for percentage offensive and percentage discussing gender, respectively.
We also ran a logistic regression on offensiveness and occupation gender discussions, to visualize how strongly they might affect gender mentioning. In Figure 8, we report that more offensive language was present in discussions pertaining to woman-coded occupations like “nurse” than in man-coded ones like “carpenter”. Gender-related conversations also highlight non-typical gender adjectives modifying occupations (e.g. “female carpenter” and “male nurse”). In sum, these exploratory results indicate that our main findings likely derive from offensive or negative sentiment conversations about women-coded occupations, or general conversations about gender-balance in occupations.

heading: 6 Related Work
text: Information Theory for Corpus Analysis. Increasingly, corpus analysis works have relied on information theoretic tools, particularly those pertaining to the lexical semantics of grammatical gender and related morphological specifications (Liu et al., 2019; McCarthy et al., 2020; Williams et al., 2019, 2020; Rathi et al., 2021; Williams et al., 2021; Chen et al., 2022; Stańczak et al., 2023).
Gender in Occupations. Occupations are interesting for studying gender in NLP. Early work on gender bias in word embeddings (Bolukbasi et al., 2016; Caliskan et al., 2017) spawned a wealth of work on social bias and occupations in sentiment analysis (Bhaskaran and Bhallamudi, 2019), coreference resolution (Zhao et al., 2018; Rudinger et al., 2018), probing for gender bias (Touileb et al., 2022), and multilingual applications (Stanovsky et al., 2019; Prates et al., 2020; Troles and Schmid,
2021; Corral and Saralegi, 2022).

heading: 7 Conclusion
text: We perform a corpus statistical analysis of Wikipedia and Pushshift.io Reddit, and find that there is a relationship between gender mentioning and occupation genderedness using information theoretic techniques. Unlike in other contexts, we find no evidence that gender mentioning is correlated with gender surprise. Instead, we find evidence that gender is more likely to be mentioned for woman-dominated occupations than man-dominated ones.

heading: 8 Limitations
text: While we have relied pretty heavily on Wikipedia, Pushshift.io Reddit, and the U.S. Labor Statistics as the basis of many of our quantities, other options may be possible. Some issues include: Wikipedia and the Labor statistics do not come from the same joint distribution, but are rather proxies with different bias problems. For instance, English Wikipedia has a fame bias (which could mean our estimates for less lucrative or impressive occupations are noisier), and the labor statistics has a location bias (being US specific, but not English specific). Moreover, there is potentially some temporal discrepancy between these two data sources, in that the Labor Statistics numbers are specific to a recent year (2023), but Wikipedia and Pushshift.io Reddit have been collectively edited and added to over decades.
As with all corpus analyses, our conclusions are limited by the available resources. The occupations we consider are restricted to conventionalized job titles, and therefore we may not be able observe the full range of gender mentioning phenomena. However, what corpus analyses lack in flexibility, they make up for in scale: this work is able to draw conclusions from a large corpus of existing text produced by English speakers, which can give us insights into occupation gender typicality.

heading: 9 Acknowledgments
text: We thank Mark Tygert for helpful discussions about our statistical methodology. We also thank our anonymous reviewers for suggesting a few interesting experiments, which we have incorporated into the paper in Section 5.6 and the Appendix.

heading: A Notation
text: Throughout this work, we follow Polyanskiy and Wu (2014) for our notation. We denote a probability space as (Ω,F ,P). We define random variables (r.v.) as X : Ω → X . We denote an r.v. as a capital letter, X , while their realizations are denoted by a lower case letter, x. The distribution of X is represented as PX , which is a probability measure on the alphabet X . We use pX for probability mass function of PX , we may often drop the subscript if the context is unambiguous. We use PX|Y for a conditional distribution, which can be thought of as a collection of probability measures on X , one PX|Y=y for each value of y. Expectations are denoted either as E[X], or Ex∼PX [x], and similarly, E[f(X)] can also be denoted by Ex∼PX [f(x)]. A concept of particular importance in this study is the entropy of a random variable, which is defined as the expected information content of a random variable X; H(X) = Ex∼PX [− log2 p(x)]. We further will rely on mutual information, denoted as I(X;Y ). It quantifies the mutual dependence between two random variables X and Y . It is defined as I(X;Y ) = E(x,y)∼PX,Y [log2 pX,Y (x,y) pX(x)pY (y)
] and represents the reduction in uncertainty about X given the knowledge of Y , and vice versa.

heading: B List of nouns
text: B.1 Occupation
accountant analyst assistant attendant auditor baker carpenter cashier ceo chief cleaner clerk constructor cook counselor designer developer driver editor farmer guard hairdresser housekeeper janitor laborer lawyer librarian manager mover nurse physician receptionist salesperson secretary sheriff supervisor teacher writer

heading: C Prompt for Llama 2 Wikipedia
text: Generate a Wikipedia article on the topic of {title}. Use the following first paragraph from the original Wikipedia article as a starting point: {first 256 words} Now, expand upon the provided paragraph by providing additional details, historical context, notable events, key figures, and any relevant subtopics. Aim for a well-structured and informative Wikipedia style article with a minimum length of 700 words. Ensure that the content is factually accurate, well-written, and on Wikipedia writing style.

heading: D BLEU score between Wikipedia and

heading: E Occupation mentions
text: E.1 Pushshift.io Reddit
E.2 Wikipedia
E.3 Llama 2 Wikipedia
F Correlation between femaleness of an occupation and mentioning in Wikipedia is weak
We found only weak correlations (r ≤ 0.20) between the femaleness of an occupation (according to US labor statistics) and gender, femaleness, maleness mentioning in Wikipedia. The strongest correlation (r = 0.20) was for overall gender mentions.

heading: G Female-coded and women-dominated
text: The y-axis, representing woman-dominatedness, is defined by the percentage of female labor representation in each occupation according to the US Bureau of Statistics. The x-axis, representing female-codedness, is the perceived femaleness of an occupation, calculated by average cosine similarity of word embedding between the occupation and two groups of gendered adjectives 7 as a proxy. Defined as similarity_female if similarity_female > similarity_male else -similarity_male Occupations with a y-axis value exceeding 0.5 are considered women-dominated, while those with a positive x-axis value are deemed female-coded.
7[“male”, “masculine”, “man”] for the male group and [“female”, “feminine”, “woman”] for the female group.

heading: H Comparison between Pushshift.io Reddit, Wikipedia and Labor Statistics

heading: I Mutual information for Pushshift.io Reddit and Wikipedia

heading: J Annotation of samples from Pushshift.io Reddit

heading: Future Work
text: This may change in the future, and we look forward to future research exploring the nature of this change