Disagreeable, Slovenly, Honest and Un-named Women? Investigating Gender Bias in English Educational Resources by Extending Existing Gender Bias Taxonomies (2024)

Abstract: Gender bias has been extensively studied in both the educational field and the Natural Language Processing (NLP) field, the former using human coding to identify patterns associated with and causes of gender bias in text and the latter to detect, measure and mitigate gender bias in NLP output and models. This work aims to use NLP to facilitate automatic, quantitative analysis of educational text within the framework of a gender bias taxonomy. Analyses of both educational texts and a lexical resource (WordNet) reveal patterns of bias that can inform and aid educators in updating textbooks and lexical resources and in designing assessment items.

heading: 1 Introduction
text: Educational materials for children such as reading comprehension articles or test assessments often protagonize real or fictional characters with gender information, rendering the materials more engaging (Brugeilles et al., 2009). They, however, could carry implicit gender bias and thus potentially reinforce gender stereotypes via children’s learning process (Waxman, 2013; Doughman et al., 2021).
One example of such gender bias in educational materials lies in the asymmetrical distribution of males and females in human-generated text such as textbooks, where male and female characters tend to take on different social roles (Brugeilles et al., 2009). Additionally, such gender bias surfaces in the lexical entries and definitions in dictionaries. An open letter (Flood, 2023) calls on Oxford University Press to change its "sexist" definitions of the word "woman."
Most research on gender bias in the educational field relies on qualitative methodologies suitable for small-scale analyses (e.g., Namatende-Sakwa (2018); Phan and Pham (2021)). In contrast, gender bias studies in the field of NLP mostly attempt to identify, quantify and mitigate gender bias in NLP
applications (Savoldi et al., 2021; Zhao et al., 2019; Bordia and Bowman, 2019), with few looking at educational texts (Li et al., 2020).
Towards the aim to identify and analyze gender bias in educational data using NLP methods, in this paper, we first review recently developed gender bias taxonomies (§3) with an extension to incorporate new types of bias in text. Using NLP techniques, we extract gendered mentions1 from educational materials (e.g. textbooks, reading materials, etc.) and a lexical resource (WordNet2 (Miller, 1992)). We quantify different types of gender bias therein to reveal the linguistic patterns most closely associated with such bias. Our contributions include: (1) adopted and extended existing gender bias taxonomies and developed a pipeline for the extraction of person mentions and linguistic features (§4); (2) designed an analysis method for identifying various types of gender bias in text in different dimensions (§5); and (3) applied the analysis method to educational datasets to demonstrate the presence of different types of gender bias.

heading: 2 Bias Statement
text: In this work, we attempt to examine gender bias in human-generated text and specialize it to educational resources such as textbooks, test assessment items and lexicons. We adopt the definition of gender bias as given in Doughman et al. (2021): "an exclusionary, implicitly prejudicial, or generalized representation of a specific gender as a function of various societal stereotypes." Here we employ and extend existing gender bias taxonomies (Hitti et al., 2019) and examine different types of gender bias in educational resources.
People implicitly associate certain behaviors or
1We recognize and acknowledge that gender is a spectrum rather than binary; however, in this work, we focus solely on investigating gender bias concerning male and female genders, as explicit non-binary entries in available data are scarce.
2 https://wordnet.princeton.edu/
219
traits to a specific gender, creating gender stereotypes. Such bias in educational resources can be learned by children through the early process of learning (Waxman, 2013; Doughman et al., 2021) and further perpetuates gender stereotypes. For example, it has been shown that women are generally less represented in textbooks and often associated with family-related roles and traits, whereas men are over-represented and often associated with work-related roles. Such differentiated representation of male and female genders in textbooks, which often serve an instructional purpose, creates a false imagery for children with respect to what roles men and women are expected to undertake, producing unnecessary and harmful gender stereotypes. Furthermore, lexical resources such as WordNet are often used to train NLP systems or as external knowledge bases. The implicit bias within these resources can be passed on to produce biased system outputs that can potentially cause representational harms (Blodgett et al., 2020).
Here, we investigate gender bias in educational resources only for male and female genders for the following reasons: (1) the datasets used for analysis are not recent and up-to-date (all educational datasets are published before 2018). Therefore, the number of people mentioned in those datasets whose gender is non-binary gender is limited; (2) the NLP systems such as coreference resolution in the current pipeline to extract person mentions cannot reliably detect and extract people of non-binary gender. In future work, once trustworthy NLP systems that can reliably detect and extract people of non-binary gender become accessible, the analyses can be extended to incorporate the comparison between binary and non-binary genders by using the same overall pipeline and analysis methods (e.g. odds ratio analysis).

heading: 3 Related Work
text: In this study, we focus on gender bias in educational data. We first discuss a taxonomy of gender bias in human-generated text and then review previous research on gender bias in the educational field and in NLP research.

heading: 3.1 Taxonomy of Gender Bias
text: To meaningfully categorize various kinds of gender bias, Hitti et al. (2019) propose two types of gender bias in text: structural and contextual bias. Structural bias "occurs when bias can be traced down from a specific grammatical construc-
tion," including gender generalization (e.g., generic he) and explicit marking of sex (e.g., "chairman" vs. "chairwoman"). Contextual bias "requires the learning of the association between gender marked keywords and contextual knowledge," which includes societal bias, where traditional gender roles reflect social norms, and behavioral bias, which is a generalization of attributes and traits onto a gendered person. Examples are given in Table 1 (B3 (1) and (2)).
Based on Hitti et al. (2019), Doughman et al. (2021) and Doughman and Khreich (2022) provide a more fine-grained taxonomy with five types of gender bias, linking each type to possible realworld implications. Our work builds on and expands the taxonomies, as further described in §4.2.

heading: 3.2 Gender Bias Studies in Educational Field
text: There exists substantial research on gender bias in educational settings for various languages and regions, including: English textbooks in Uganda (Namatende-Sakwa, 2018) and Vietnam (Phan and Pham, 2021), in Vietnamese story textbooks (Vu, 2008) and Arabic textbooks (Izzuddin et al., 2021).
Research on gender bias in educational corpora mostly resorts to traditional approaches such as content analysis (Stemler, 2001) and critical discourse analysis (CDA) (Locke, 2004). Despite their obvious strengths in providing in-depth understanding of gender bias, manual coding is required, which is impractical for widespread use.
In this work, we study gender bias in an educational setting by building on linguistic constructs associated with qualitative categories of bias, but enable scalable quantitative analysis by applying NLP methods.

heading: 3.3 Measuring Gender Bias in Text
text: Cryan et al. (2020) explore automating bias analysis in text by developing lexicon-based and machine learning algorithms for gender stereotype detection from a corpus manually coded for gender stereotypes. This approach is limited to the particular gender stereotypes used in annotation.
An alternative approach is to compute some statistic associated with gendered mentions in different linguistic contexts, leveraging NLP analysis tools to automatically annotate linguistic contexts. For example, Zhao et al. (2017) investigate and define gender bias based on the ratio of the joint probability of an activity (e.g., a verb) and a gender group (e.g., female). Bordia and Bowman (2019)
use a point-wise mutual information (PMI) based statistic. The odds ratio (OR) is often adopted statistic for measuring gender bias in text (Valentini et al., 2023), and will be adopted in our work. An advantage of this approach of using statistics on a range of linguistic contexts is that it can reveal biases not anticipated in manual coding.
Studies that have taken this approach with texts for children include Li et al. (2020), which explores gender and cultural bias in U.S. history textbooks used in Texas and Toro Isaza et al. (2023), which investigates gender bias in fairy tales for children. Our work is informed by these studies, but it is grounded in a bias taxonomy, and we also investigate a lexical resource.

heading: 3.4 Gender Bias Studies in NLP research
text: For NLP models, researchers look at the existence of gender bias in word embeddings (Bolukbasi et al., 2016; Caliskan et al., 2017; May et al., 2019), large language models (LLMs) (Bordia and Bowman, 2019; Fatemi et al., 2023), and in tasks such as coreference resolution (Zhao et al., 2018), machine translation (Savoldi et al., 2021), among others. Another important aspect of gender bias studies in NLP concerns bias mitigation in NLP applications (Savoldi et al., 2021; Bolukbasi et al., 2016; Park et al., 2018). These efforts are ultimately concerned with downstream application impact. In our work, the use of NLP is as a linguistic annotation tool, and bias detection is aimed to support human authors of educational texts.

heading: 4 Methodology
text: In this work, we adopt and expand the existing taxonomies for gender bias in human-generated text and attempt to identify different types of gender
bias in our datasets. We look at two types of data3: educational corpora (denoted corpora henceforth) and lexical resource (WordNet).

heading: 4.1 Datasets
text: There are two major types in the educational corpora: Content and Exam (listed in Table 2). Content datasets mainly include open source textbooks (Michigan, 2014; Siyavula, 2014; CK12, 2007) and reading articles for K-12 education (e.g., CCS_doc
4, wee_bit (Vajjala and Meurers, 2012), and OneStop (Vajjala and Lučić, 2018)); Exam datasets contain test items administered either in the U.S. or internationally, including pisa (Pisa, 2015), naep_science and naep_math.5 These educational corpora cover a wide range of subjects such as math, science, history etc., and diverse linguistic phenomena, offering a rich source for the investigation of gender bias.
For lexical resources, we opt for WordNet 6 for a few reasons. It is widely used in the NLP field and may thereby perpetuating potential biases in downstream tasks. Also, it serves as a rich lexical resource with definitions and semantic relationships among words, which benefits our analysis. Lastly, it offers users convenient and free access to word entries and related information.
3Both types of educational materials examined in this paper are in English.
4 https://corestandards.org/assets/Appendix_B.
pdf 5 https://nces.ed.gov/nationsreportcard/
6The latest version 3.1 contains only database files but no code is available, therefore we use Version 3.0. https: //wordnet.princeton.edu/

heading: 4.2 Different Types of Gender Bias to Identify
text: As noted earlier, important related work on detecting gender bias in text (e.g., Li et al. (2020); Toro Isaza et al. (2023)) does not incorporate recent taxonomies of gender bias. To systematically understand what kinds of gender bias exist in educational materials, we adopt and extend the gender bias taxonomy from Hitti et al. (2019) and Doughman et al. (2021). In our study, we first consider structural bias and contextual bias (as defined in §3.1). We also add three new types of bias: distributional bias, definitional bias and namedness. Table 1 lists all bias types and the datasets used to conduct the analyses, along with examples.

heading: 4.2.1 Structural Bias
text: Explicit Marking of Sex (B1): At the morphological level, explicit marking of sex7 manifests when gender-neutral entities are denoted using gender marker such as "-man" and "-woman." Here, the term "gender marker" refers not to markers of grammatical gender but to free morphemes (e.g., "-woman" in "needlewoman") or head nouns in compound phrases (e.g., "woman" in "slovenly woman"). B1 in Table 1 presents an example where "policeman" contains the marker "-man" but the definition denotes a gender-neutral meaning. Generic he (B2): We also examine the generic usage of gendered pronoun "he" where the pronoun is co-indexed with a neutral common noun. As shown in the example from B2 of Table 1, the word scientist is gender neutral but is co-indexed with a male reflexive pronoun "himself ".

heading: 4.2.2 Contextual Bias
text: In Hitti et al. (2019), contextual bias has two subtypes: societal bias, where a gender is stereotypically assigned a social role, and behavioral bias, where certain attributes or traits associated with a gender can lead to generalized gender stereotypes.
7The word "sex" in this terminology is used by the original author. We keep this terminology in this work for the sake of consistency but do not use sex and gender interchangeably.
In our work, we use the same word contextual bias (B3) to refer to societal and behavioral bias due to the nuanced distinction between societal and behavioral bias. For example, the sentence from Doughman et al. (2021) illustrates societal bias: "The event was kid-friendly for all the mothers working in the company," where "mothers" are stereotypically assigned the role of caretakers, representing societal bias. However, "mothers" are also stereotypically associated with the trait of "caring for kids", which falls under behavioral bias. In our study, stereotypical bias emerges when a specific gender is stereotypically ascribed a social norm or attributed certain traits.

heading: 4.2.3 Additional Bias
text: We add three gender bias types to the taxonomy: Distributional Bias (B4): This type of bias refers to the uneven distribution of different genders. For example, in our textbook dataset, male mentions appear more frequently than female ones. Namedness (B5): People in text can be mentions with a real or fictional name or referred to with a common noun such as "scientist." Through preliminary examination of the educational corpora, we found that female characters show up as anonymous more frequently than their male counterparts (e.g. "mother" vs. "John"). Thus, we choose to explore this bias type where males are often given names while females are not. For example, in a corpus, the percentage of male proper nouns is higher than that of females (see statistics B5 in Table 1). This issue is denoted as namedness bias in our taxonomy. Definitional Bias (B6): The nuanced definitions given to male and female words implicate the differentiated representation of men and women in lexical resources, which we denote definitional bias. As shown in B6 in Table 1, the definition given to "horseman" only refers to men and is detailed, whereas "horsewoman" is defined solely based on the male version: "horseman".

heading: 4.3 Analysis Methods
text: We detect different bias types in our datasets by employing a generic pipeline comprising four steps: (1) preprocessing, (2) person mention extraction, (3) gender labeling, (4) bias analysis.

heading: 4.3.1 Preprocessing
text: Corpora: In preprocessing, we use the Stanford CoreNLP package8 (Manning et al., 2014) with steps of sentence segmentation, tokenization, truecasing, POS tagging, named entity recognition, dependency parsing and coreference resolution. WordNet: In WordNet, an entry can either be a single word (e.g., "horsewoman") or a compound phrase (e.g., "honest woman"). If a word or phrase has multiple senses, each sense is treated as a distinct entry. Each entry includes a definition and additional details such as syntactic category (e.g., "NOUN") and lexicographer (e.g., "noun.person"). We extract entries and their definitions from WordNet using the NLTK package9 (Bird et al., 2009) and analyze the dependency structure of the definitions using CoreNLP.

heading: 4.3.2 Person Mention Extraction
text: Corpora: We first extract all proper nouns, common nouns and pronouns as mention candidates. We use named entity information and the WordNet sense (i.e., "noun.person") information to determine if each candidate is a person. Lastly, in coreference chains, if at least one mention in a chain is considered a person from the previous step, then the rest of the chain is also considered a person. Implementation detail is given in Appendix A. WordNet: For WordNet, we extract all entries in the "noun.person" lexicographer file. We consider these entries as the ones denoting people.

heading: 4.3.3 Gender Labeling
text: Gender labeling procedure outputs three labels: M for male, F for female and N for neutral 10. Corpora: We label the gender of mentions in corpora based on a two-step heuristic. First, we determine the gender of individual mentions using a list of seed words for pronouns (e.g., "she", "he") and common nouns (e.g., "woman", "man") and
8Version 4.5.3, release date: 3/15/2023, https:// stanfordnlp.github.io/CoreNLP/index.html
9Version 3.8.1, https://www.nltk.org/index.html 10The label N for neutral gender can refer to person mentions of either gender (e.g., "someone") and groups of people of mixed genders (e.g., "they").
the Gender Guesser API11 for the first names of proper nouns. Then, using coreference chains, we resolve the gender for mentions whose gender is not determined from the previous step. For example, for common nouns such as "scientist," the gender cannot be determined in the first step because it is a profession that can be undertaken by any gender. Through coreference chain where it is co-referred by a gendered pronoun, its gender then can be resolved. Implementation detail is given in Appendix B. WordNet: The extracted entries are grouped into the three gender categories based on gender indications in their definitions. We create three seed word lists containing terms with obvious gender information (e.g., colored words in the first three examples in Table 3). If the root of the dependency structure of the entry definition or the modifier of the root matches predefined terms, we assign the corresponding gender label to the entry.
Then, unlabeled entries are categorized using those labeled entries. If the root of a definition matches a labeled entry, the unlabeled entry is assigned the corresponding gender label. As the last example in Table 3 shows, the gender of "roughrider" is assigned based on the gender of "horseman." This iterative process repeats until no further male or female labeling occurs, leaving the remaining unlabeled entries as neutral.

heading: 4.3.4 Pipeline Validation
text: To validate the accuracy of the person mention extraction and gender labeling components in our NLP pipeline, we manually labeled 100 examples from the pisa, naep_math and naep_science datasets. All gendered person mentions (pronouns, proper nouns and common nouns) are annotated with respect to gender. The annotated validation set contains 365 mentions in total (176 male mentions and 189 female mentions). The system identified 368 mentions and the number of correctly extracted mentions is 345.
11 https://pypi.org/project/gender-guesser/
The pipeline can achieve high precision, recall and F-1 scores in extracting the person mentions (see Table 4). The extraction module can produce some false positive extractions such as animal names (e.g., "Dolly" (the famous clone sheep)) and planet names (e.g., "Venus"). The named entity recognition package can miss some human names (e.g., "Stacie", "Sue").
For the gender labeling component, the labeling accuracy is 100% for the 100 validation instances where the gold standard mentions match the extracted mentions, because all person mentions in the validation set are in coreference chains and they are co-referred with a gendered pronoun. For larger datasets, the accuracy is not perfect because of several limitations. First, the Gender Guesser API is based on a list of proper first names. If a name is not in the list, then the gender cannot be correctly resolved. Second, for non-English names such as Chinese first names, most of the time the gender cannot be determined without further coreference information.

heading: 4.3.5 Bias Analysis
text: Corpora: For distributional bias (B4), we count the frequencies of males and females. Linguistic features are extracted to assess their association with gender to examine generic he (B2), contextual bias (B3) and namedness (B5).
First, we correlate the POS tags of gendered mentions with gender to investigate generic he (B2) and namedness (B5). By categorizing the verbs that serve as the root of gendered mentions using the agency connotation framework (Sap et al., 2017), we examine what types of verbs are more likely to be associated with a specific gender (B3). Agency is attributes of the agent of the verbs, denoting whether the action implies power and decisiveness. For example, "he obeys" implies the person "he" has low agency, while "he chooses" implies "he" has high agency. We also extract gendered possessive pronouns and the possessed common nouns. Via a list of kinship terms (e.g., "mother", "father") (full list in Appendix D), the association between gender of possessive pronouns and kinship terms is measured (B3).
WordNet: Initially, we extract proper nouns (usually names of famous persons or fictional figures) from person entries using heuristics, and look into distributional bias (B4) based on the frequency of their gender labels. Next, we investigate the use of gender pronouns such as "he" (B2) in defining gender-neutral entries. Additionally, we employ rule-based techniques to extract person entries ending with gender markers of "-man," "-woman," and "-person"12 and assess the tendency for genderspecific markers to encompass gender-neutral connotations, indicative of explicit marking of sex (B1). Lastly, we scrutinize potential stereotypical bias (B3) in entries associated with gender-specific markers and definitional bias (B6) by examining how roles marked by "-man" and "-woman" are depicted.

heading: 4.3.6 Gender Bias Statistic
text: In the analysis of feature bias, we conduct significance testing on the association between gender and a binary feature of interest using Fisher’s exact test13 to obtain p-values14 at α = 0.05 level. In addition, we use odds ratio (OR) to determine the direction and magnitude of association. The odds ratio of a binary related feature x ∈ X that measures gender bias in favor of males is given by:
ORx = Mx/Mnot x Fx/Fnot x (1)
where Mx is the count of male mentions with feature x and Mnot x without x. Fx and Fnot x are defined similarly. If the p-value ≤ 0.05, the association is deemed significant. If OR > 1, then we observe gender bias toward men, and toward women for OR < 1. We choose odds ratio as the statistic to measure association between a specific gender and a feature because it is interpretable and commonly used to measure association between binary categorical variables and it is independent of the marginal distributions, which is desirable for our case since the distributions of male and female mentions are highly asymmetrical.

heading: 5 Experiments and Results
text: In this section, we present our experimental design and results for the corpora and WordNet.
12We plan to analyze more gender markers such as "-or" in "actor" and "-ess" in "actress" in future works.
13We opt for Fisher’s exact test instead of Chi-square test because the number of co-occurrences of gender and certain features is too small.
14Adjusted via False Discovery Rate for multiplicity.

heading: 5.1 Educational Corpora
text: By extracting gendered mentions with their linguistic features, we investigate four types of gender bias in corpora.

heading: 5.1.1 Distributional Bias (B4)
text: Distributional bias in corpora is examined through comparing the number of extracted male and female mentions. We have observed the evidence for distributional bias in favor of male mentions for all content corpora (Table 5), which adheres to our hypothesis that male mentions are over-represented in text while females are under-represented with respect to mention frequency.

heading: 5.1.2 Generic He in Corpora (B2)
text: To inspect the usage of generic he in corpora, we look at extracted mentions that are only common nouns with no gender information per se in comparison to those that are inherently gendered common nouns. Generic common nouns such as "researcher" denote nouns that can address any person in general, while gendered common nouns such as "mother" refer to a specific gender in particular. Our finding (Table 6) shows that for all datasets examined, male common noun mentions are typically generic rather than gendered, while female mentions are more likely to be gendered.

heading: 5.1.3 Possessive Pronoun and Kinship (B3)
text: To approach contextual bias where a specific gender is associated with certain societal roles, we create a list of kinship terms such as "mother" and "father" to categorize the common nouns possessed by a gendered possessive pronoun. Possessive pronouns (e.g., "his", "her") that occur frequently in the datasets carry important gender information. We examine which gender is more likely to be associated with kinship terms, indicating a stereotypical
association of a specific gender with family-related roles. Significant association with kinship terms is observed for the OneStop and CCS_doc datasets with OR < 1: female possessive pronouns (e.g., "her") are more likely to co-occur with kinship nouns, while male ones do not.

heading: 5.1.4 Agency of Gendered Mentions (B3)
text: In addition to the previous finding on contextual bias, to examine what kinds of behavior are stereotypically associated with a specific gender, we categorize the verbal roots that head the person mentions in the nominal subject position in the sentences according to the connotation framework in Sap et al. (2017). Significant association (Table 7) between female mentions and low agency verbs in the textbook dataset is detected with an OR < 1, indicating females mentions in textbook are more often associated with low-agency verbs than males do, consistent with the findings in Sap et al. (2017). For the other datasets except naep_math, while insignificant, the OR < 1, displaying a similar trend to textbook.

heading: 5.1.5 Namedness of Gendered Mentions (B5)
text: We investigate namedness using the POS tags of gendered mentions. There are three types of male and female person mentions: pronoun (PRP ),
common noun (NN ) and proper noun (NNP ). By comparing the distribution of NN and NNP , we discover that males are more likely to be tagged as proper nouns, while females tend to be common nouns. Proper nouns have explicit name information, whereas common nouns can refer to any person in general. The significant correlation (Table 8) between males and whether or not they are proper nouns implies that males tend to receive names, but females typically remain more generic and anonymous. This observation represents previously unreported structural bias where females appear less identifiable through proper names.

heading: 5.2 WordNet
text: We conduct experiments on the person entries and definitions extracted from WordNet to elucidate instances of five bias types.

heading: 5.2.1 Distributional Bias (B4)
text: Table 9 shows the number of entries we extract from WordNet. Among all entries in WordNet, 21,463 are person entries.
Among person entries, we define 8,652 proper nouns (e.g., names of famous persons or fictional figures). Labeling the gender of proper names by their definitions is challenging (e.g., the definition of "Sand" is "French writer known for ...," exhibiting no gender cue). Therefore, we randomly pick 100 proper nouns and determine their gender based on the information on their Wikipedia pages: 85 of them are males, 14 are females, and 1 entry ("salian") refers to a group of people. Among the 99 entries that are individuals, 91 are real persons, 8 are fictional. This adheres to the distributional bias that males are represented more in this lexical resource, possibly due to historical reasons.
The rest of person entries are grouped into M, F, and N based on their definitions (see Section 4.3.3).

heading: 5.2.2 Generic He (B2)
text: Among the neutral person entries (column N in Table 9), we find there are 100 entries wherein the roots in the dependency structures of the definitions are either co-referred or co-indexed with gendered pronouns such as "himself " (see example in B2 of Table 1). We count the frequency of gendered pronouns and gender-inclusive pronouns (e.g., "he or she" or "they"). We find that usage of generic he widely occurs in WordNet definitions. Among the 100 definitions, the male generic pronoun is employed in 67 definitions to denote gender-neutral roots, whereas only 33 instances feature genderinclusive language.

heading: 5.2.3 Explicit Marking of Sex (B1)
text: For person entries that are not proper nouns, we collect those ending with the gender markers ("- man," "-woman," and "-person"). Table 10 displays the breakdown of their gender labels determined by the definitions.
There are notably 303 entries ending with "- man" featuring gender-neutral definitions. Also, while the neutral label of the 16 entries with "- woman" may seem perplexing, they are deemed neutral due to the absence of gender-specific words in their definitions (see example of "needlewoman" in Table 3). We consider gender markers ("-man" vs. "-woman") and the gender labels of the definitions (M and F vs. N) and observe that the marker "-man" is inclined towards denoting gender-neutral entries,15 providing evidence for explicit marking of sex.
15Fisher’s exact test: OR = 14.623, p ≪ 0.05.

heading: 5.2.4 Contextual Bias (B3)
text: In Table 10, some entries have variants representing the same role. For instance, "chairman," "chairwoman," and "chairperson" share the same root morpheme but differ in markers. We classify person entries containing gender markers based on the number of associated variants in Table 11 (Full word lists in Appendix F and example definitions in Appendix G).
In Table 11, row (1a) shows that out of the 310 entries marked only with "-man", 50 are defined as male, lacking corresponding "-person" or "-woman" variants. These entries typically pertain to occupational roles (e.g., "seaman", "mailman"). Row (1b) identifies 11 entries solely marked with "- woman", some of which carry sexist connotations like "loose woman", "kept woman", and "honest woman", where asymmetric social expectations are imposed on women in contrast to men.
Row (2) shows entries with only two markers. Specifically, Row (2b) features 3 entries without the "-man" variant, all of which ("disagreeable woman", "slovenly woman", and "unpleasant woman") convey negative connotations. Row (2c) highlights 10 entries lacking the "-woman" version. Notably, the two male entries with "-man" ("rich man" and "wealthy man") lack female counterparts.
In this table, 52 male entries lack "-woman" variants16 and 14 female entries lack "-man" variants.17 We perform Sentiment Analysis on the definitions of these two entry groups using the vaderSentiment (Hutto and Gilbert, 2014) API.
1652 is the sum of 50 from (1a) and 2 from (2c) in Table 11 1714 is the sum of 11 from (1b) and 3 from (2b)
Results reveal a significant difference,18 with female entries having a lower average sentiment score (-0.141) compared to male ones (0.056).19
The presence of entries like "disagreeable woman" and "rich man" raises initial concerns, since the modifiers directly convey their meaning, rendering their inclusion in lexical resources less necessary. Moreover, these entries may reinforce gender stereotypes. These observations indicate societal bias, reflecting not only the allocation of certain social roles exclusively to males but also the differentiated sentiment associated with gender.

heading: 5.2.5 Definitional Bias (B6)
text: Furthermore, we examine the definitions of the 62 entries that have both "-man" and "-woman" variants.20 We find 10 entries whose definitions for "-man" variant are detailed, whereas the corresponding "-woman" entries receive simpler definitions derived from their "-man" or "-person" counterparts (see example of "horseman" and "horsewoman" in row B6 in Table 1). This approach renders the understanding of "horsewoman" reliant on the definition of "horseman." For the purpose of ensuring semantic comprehensiveness, meticulous definitions for all variants should be provided, incorporating senses conveyed by all morphemes within the entries to facilitate reader comprehension and mitigate potential bias.

heading: 6 Discussion
text: Our investigation has revealed the pervasive existence of various types of gender bias within both educational corpora and WordNet. Specifically, we have noted the prevalence of distributional bias evidenced by the uneven distributions of males and females across both datasets, alongside explicit marking of sex and the generic use of male pronouns within WordNet. Additionally, a diverse array of syntactic patterns within the corpora has been identified as displaying gender bias.
In this work, we only explore gender bias in English educational materials. The extraction pipeline and gender labeling procedure proposed contain language-dependent components that are unique to English (e.g. using a coreference resolution system to determine gender of a common noun based on gendered pronouns). For languages such as Man-
18Unpaired two-sample t-test: t = −2.15, p = 0.035. 19The sentiment score ranges from -1 to 1, where [-1, 0)
indicates negative sentiment, and (0, 1] indicates positive. 2062 is the sum of (2a) and (3a) totals in Table 11
darin Chinese where the gender of the pronouns is indistinguishable without orthographic information, the pipeline may integrate language-specific NLP systems to resolve the gender of person mentions. Moreover, the way that gender bias manifests in text can differ from language to language (and culture to culture). Thus, the bias patterns used to detect gender bias in this work will be different.
The presence of gender bias in educational resources carries significant implications. Exposure to those materials can potentially shape children’s perceptions through implicit gender bias, fostering the development of gender stereotypes. This perpetuation of biased narratives has far-reaching consequences for societal attitudes and inequality. Moreover, NLP models reliant on lexical resources such as WordNet, wherein gender bias is discernible in multiple forms, may inadvertently perpetuate said biases in downstream tasks.
However, our work offers actionable insights for educational resource developers, offering guidance on elements to consider during the creation process to mitigate bias. Moreover, our study on WordNet pinpoints the bias issues that warrant monitoring and maintenance by developers.

heading: 7 Conclusion
text: In this study, based on the existing taxonomy of gender bias in text, we have examined 7 types of gender bias in educational corpora and WordNet. The analysis has shown that many types of gender bias exist in both types of data, emphasizing the necessity for meticulous examination of such biases in associated resources. Our future work aims to identify additional linguistic features correlated with gender. Furthermore, deeper exploration is warranted into corpora from other domains and lexical resources beyond WordNet.

heading: 8 Limitations
text: There are several limitations to our study: (1) we only consider binary gender in this paper; (2) the small data size of some of the assessment items limits the use of statistical analyses; (3) WordNet as a proxy for a dictionary does not suffice due to its lack of comprehensive entries and definitions and it is not regularly maintained; (4) in this study, we employ odds ratio as the statistic for gender bias, which only considers correlation instead of causation; (5) in this work, we only work with the English language, while gender bias can appear in educational materials in other languages as well.

heading: 9 Ethical Considerations
text: We identify several ethical considerations that are related to our work. (1) First, the educational assessment items typically are not made publicly available, which presents a challenge for multiple researchers to compare methods on the same data and to reproduce our analysis results. However, this type of educational data assumes vital importance to look at, so mechanisms are needed to enable these types of studies. (2) This work is not subjected to privacy concerns since the datasets do not contain identifiable information about individuals. However, famous people (dead or alive) appear in our datasets, and they are potentially used for analysis. (3) Our gender labeling procedure only labels male, female and neutral gender, without consideration of non-binary genders. Such limited consideration and inclusion of binary gender constrains the scope of our study within the binary gender framework, particularly in neglect of stereotypes and bias directed towards non-binary gender community.

heading: A The Pipeline for Extracting Person Mentions from Educational Corpora
text: This appendix describes in detail the implementation of the person mention extraction procedure for educational corpora. The corpora first are preprocessed by using the Stanford CoreNLP package. After preprocessing the educational corpora, we extract individual person mentions. Person mentions include three kinds: pronouns, proper nouns
and common nouns. We first recognize the three types of mentions from text as individual mention candidates using their POS tag information. Using named entity recognition (NER) information and the supersense obtained from WordNet, we determine if each candidate mention is a person if and only if the NER assigns a "PERSON tag or its supersense is "noun.person". By leveraging coreference resolution, we then form coreference chains. In each coreference chain, if at least one mention in the chain is determined as a person in the previous step, the rest of the chain is deemed as person mentions. The last step is to ensure that common nouns that are missed from the second step are correctly extracted.

heading: B Gender Labeling for Corpora
text: In this appendix, we describe the gender labeling procedure for the educational corpora.
After extracting person mentions from the corpora, we resolve the gender of the mentions based on a two-step heuristic:
The first step in gender labeling is to check whether or not a mention is in fixed lists of pronouns and common nouns that have salient gender information: for example, "he", "she", "woman", "man" (full lists in Appendix C). If a mention is in the list, then the gender labeling function will output a label from the set {M,F,N}, where N stands for neutral gender. If a mention is not in the list, we then send the first token of the mention (assuming that the remaining mention is a proper noun) to the Gender Guesser API21. This API has a list of first names from various countries that have corresponding gender information. If the mention is in the name list, then it will output one label from {male, female, mostly_male,mostly_female,andy, unknown}, where andy stands for androgynous, meaning a name that is equally probable for male and female. If a mention is not in the name list, then the API will return unknown. We group male and mostly_male to be M and female and mostly_female to be F.
Note that there are some issues with this Gender Guesser API: it does not predict gender of mentions with only last names. Within the datasets used in this project, there are many last names of famous people of whom the gender is clearly retrievable. Also, the word lists for pronouns and common nouns in Appendix C are not comprehen-
21 https://pypi.org/project/gender-guesser/
sive. To resolve these two concerns, we choose to leverage the coreference cluster information, where we obtain the gender of a mention by the genders of its cluster, if any. The next issue with this API is that it is largely US-centric (although it has an option for country) and does not consider variations across different cultures. We do not attempt to solve this issue in this work.
The gender labeling function using cluster information works as follows:
1. Remove all unknown genders from the cluster if there are other genders in the cluster, e.g.{M,F, unknown} becomes {M,F}
2. If there is a three-way tie between M , F and andy, return andy.
3. If there is a two-way tie between M and F , return andy.
4. If there is a two-way tie between either M or F and andy, return M or F . For example, for{M,M, andy, andy}, return M .
5. If there is no tie, return the most frequent gender.

heading: C Word Lists for Person Pronouns and Person Common Nouns
text: This appendix contains the word lists for male, female and neutral gendered and neutral person pronouns (excluding "it") and for male, female and neutral gendered person common nouns. The list for common nouns are not exhaustive. Neutral Pronouns: I, me, we, our, us, myself, ourself, ourselves, let’s my, mine, they, them, their, you, your, themself, themselves, yourself, yourselves. Male Pronouns: he, him, his, himself. Female Pronouns: she, her, hers, herself. Female common nouns: girl, woman, mrs, ms, mother, mom, aunt, niece, sister, wife, daughter, grandmother, grandma, grandmom, granddaughter, bride, girlfriend, gal, madam, lady, female, waitress, actress, governess, spinster, empress, heroine, hostess, landlady, stewardess, princess. Male common nouns: boy, man, mr, father, dad, uncle, nephew, brother, husband, son, grandfather, grandpa, granddad, grandson, groom, boyfriend, guy, gentleman, bachelor, male, actor, emperor, prince. Neutral Person Common Nouns: people, adult, adults, person, people, child, children.

heading: D Kinship Terms for Detecting Societal Bias (B3)
text: This appendix provides the list for kinship terms for the analysis of stereotypical bias (B3) for educational corpora. family, son, daughter, brother, child, sister, father, mother, dad, daddy, mum, mom, mummy, niece, nephew, parent, sibling, stepdaughter, wife, husband, spouse, stepfather, stepdad, stepmother, stepmom, grandchild, grandfather, grandmother, grandma, grandmom, grandpa, granddad, grandson, granddaughter, baby22.

heading: E Example of Instances from the Educational Corpora
text: This appendix provides instance examples for all educational corpora used in this study.
E.1 CCS_doc
A medieval fisherman is said to have hauled up a three-foot-long cod, which was common enough at the time. And the fact that the cod could talk was not especially surprising. But what was astonishing was that it spoke an unknown language. It spoke Basque. This Basque folktale shows not only the Basque attachment to their orphan language, indecipherable to the rest of the world, but also their tie to the Atlantic cod, Gadus morhua, a fish that has never been found in Basque or even Spanish waters. The Basques are enigmatic. They have lived in what is now the northwest corner of Spain and a nick of the French southwest for longer than history records, and not only is the origin of their language unknown, but also the origin of the people themselves remains a mystery also. According to one theory, these rosy-cheeked, dark-haired, longnosed people where the original Iberians, driven by invaders to this mountainous corner between the Pyrenees, the Cantabrian Sierra, and the Bay of Biscay. Or they may be indigenous to this area. They graze sheep on impossibly steep, green slopes of mountains that are thrilling in their rare, rugged beauty. They sing their own songs and write their own literature in their own language, Euskera. Possibly Europe’s oldest living language, Euskera is one of only four European languages–along with
22The term "baby" is tricky because it can be used for intimate, non-family members, but when its possessive pronouns are gendered such as "his", "her", it is more likely that "baby" refers to a child.
Estonian, Finnish, and Hungarian–not in the IndoEuropean family. They also have their own sports, most notably jai alai, and even their own hat, the Basque beret, which is bigger than any other beret.
E.2 naep_math A bag contains two red candies and one yellow candy. Kim takes out one candy and eats it, and then Jeff takes out one candy. For each sentence below, fill in the oval to indicate whether it is possible or not possible.
E.3 naep_science Bacteria and laboratory animals are sometimes used by scientists as model organisms when researching cures for human diseases such as cancer. Describe one possible advantage and one possible disadvantage of using bacteria as models to help find cures for human diseases. Advantage: Disadvantage: Describe one possible advantage and one possible disadvantage of using laboratory animals such as mice, guinea pigs, and monkeys as models to help find cures for human diseases.
E.4 OneStop The Duke and Duchess of Cambridge have won the first part of their fight for privacy. A French magazine was told to stop selling or reusing photos of the royal couple. The pictures show the duchess sunbathing topless while on holiday in the south of France. It is possible that the magazine editor and the photographer or photographers will also have to go to a criminal court. The French magazine Closer was told to give digital files of the pictures to the couple within 24 hours. Closers publisher, Mondadori Magazines France, was also told to pay 2,000 in legal costs. The magazine will have to pay 10,000 for every day it does not give the couple the files. The court decided that every time Mondadori the publishing company owned by the ex Italian Prime Minister Silvio Berlusconi publishes a photograph in the future in France, they will get 10,000 fine. The couple welcome the judges decision. They always believed the law was broken and that they had a right to their privacy. The royal couple are pleased with the decision, but they want to have a much more public criminal trial against the magazine and photographer or photographers. Under French law, if you do not respect someones privacy, you may have to spend a maximum of one year in prison and pay a fine of 45,000. This punishment would send a message to the world and,
the couple hope, stop paparazzi taking photos like this in the future. On Saturday the Irish Daily Star also published the photos. And the Italian celebrity magazine Chi published a special edition of 26 pages with the photos of the future queen.
E.5 pisa Mimi and Dean wondered which sunscreen product provides the best protection for their skin. Sunscreen products have a Sun Protection Factor (SPF) that shows how well each product absorbs the ultraviolet radiation component of sunlight. A high SPF sunscreen protects skin for longer than a low SPF sunscreen. Mimi thought of a way to compare some different sunscreen products. She and Dean collected the following: ... Mimi and Dean included mineral oil because it lets most of the sunlight through, and zinc oxide because it almost completely blocks sunlight. Dean placed a drop of each substance inside a circle marked on one sheet of plastic, and then put the second plastic sheet over the top. He placed a large book on top of both sheets and pressed down. Mimi then put the plastic sheets on top of the sheet of light-sensitive paper. Light-sensitive paper changes from dark gray to white (or very light gray), depending on how long it is exposed to sunlight. Finally, Dean placed the sheets in a sunny place.
E.6 textbook Conclusions The scientist must next form a conclusion. The scientist must study all of the data. What statement best explains the data? Did the experiment prove the hypothesis? Sometimes an experiment shows that a hypothesis is correct. Other times the data disproves the hypothesis. Sometimes it’s not possible to tell. If there is no conclusion, the scientist may test the hypothesis again. This time he will use some different experiments. No matter what the experiment shows the scientist has learned something. Even a disproved hypothesis can lead to new questions. The farmer grows crops on the two fields for a season. She finds that 2 times as much soil was lost on the plowed field as compared to the unplowed field. She concludes that her hypothesis was correct. The farmer also notices some other differences in the two plots. The plants in the no-till plots are taller. The soil moisture seems higher. She decides to repeat the experiment. This time she will measure soil moisture, plant growth, and the total amount of water the plants consume. From now on she will use no-till methods of farm-
ing. She will also research other factors that may reduce soil erosion.
E.7 wee_bit
Nicole Thompson and her third-grade social studies students at Greenbriar Academy in North Carolina wanted to learn about world geography. So late last year, they sent an e-mail message to 100 people. Readers were asked to send the e-mail message to people in other places. Readers were also asked to write something about themselves as well. About six weeks later, Thompson and her students received more than 60,000 e-mail replies! Messages came from every state in the United States and from 120 countries. According to Thompson, the students’ favorite response was written by a carpenter at McMurdo Station in Antarctica. "It was a huge deal. We didn’t think we would hear from Antarctica!" Thompson said.

heading: F Full Word List for Table 11
text: This appendix provides the comprehensive word list corresponding to each row of Table 11.
F.1 Row 1a (310 entries that only have -man marker)
freshman, ablebodied seaman, able seaman, abominable snowman, adman, aircraftman, aircraftsman, aircrewman, alderman, apeman, artilleryman, assistant foreman, backup man, backwoodsman, baggageman, bagman, bandsman, bargeman, barman, barrowman, batman, batsman, beadsman, bedesman, beef man, bellman, best man, big businessman, boatman, bookman, border patrolman, bowman, brahman, brakeman, broth of a man, bushman, busman, cabman, cameraman, career man, cattleman, cavalryman, cave man, caveman, chapman, chargeman, chinaman, churchman, city man, clergyman, coachman, coalman, coastguardsman, college man, company man, con man, confidence man, conjure man, corner man, cousingerman, cow man, cowman, cracksman, craftsman, cragsman, crewman, "customers man", dairyman, dalesman, deliveryman, deskman, dirty old man, divorced man, doorman, dragoman, draughtsman, dustman, earthman, elder statesman, elevator man, end man, ent man, everyman, exserviceman, exciseman, family man, feral man, ferryman, fieldsman, fingerprint man, fireman, first baseman, fisherman, foeman, footman, fourminute man, frogman, front man, fugleman, gman, gagman, garbage man, garbageman, gasman, "gentlemans gentleman", government man, groomsman, groundsman, guardsman, gunman, handyman, hangman, hardwareman, hatchet man, heman, head linesman, headman, headsman, heidelberg man, helmsman, henchman, herdsman, highwayman, hired man, hit man, hitman, hodman, holdup man, hotelman, houseman, huntsman, husbandman, iceman, infantryman, ingerman, iron man, ironman, jazzman, journeyman, klansman, "ladies man", landman, landsman, lawman, leading man, ledgeman, lensman, letterman, liegeman, liftman, lighterman, lineman, linesman, linkman, linksman, liveryman, lobsterman, lockman, longbowman, longshoreman, lookout man, lowerclassman, lumberman, machoman, mailman, maintenance man, maltman, marksman, matman, meatman, medical man, medicine man, medieval schoolman, merman, middleaged man, middleman, midshipman, military man, military policeman, militiaman, milkman, minuteman, miracle man, moneyman, motorcycle policeman, motorman, mountain man, muffin man, muscleman, navy man, night watchman, nurseryman, oddjob man, oilman, ombudsman, organization man, outdoor man, packman, pantryman, party man, patrolman, penman, pigman, piltdown man, pitchman, pitman, pivot man, placeman, plainclothesman, plainsman, plantsman, ploughman, plowman, pointsman, posseman, postman, potman, poultryman, pr man, preacher man, pressman, privateersman, property man, propman, publicity man, quarryman, raftman, raftsman, railroad man, railway man, railwayman, red man, remittance man, renaissance man, repairman, rewrite man, rhodesian man, rifleman, righthand man, roadman, roundsman, sandwichman, schoolman, seaman, second baseman, section man, seedman, seedsman, service man, serviceman, sheepman, showman, sidesman, signalman, skilled workman, soundman, spaceman, sporting man, squaw man, stableman, steelman, steersman, stickup man, stockman, straw man, strawman, strongman, superman, swagman, switchman, swordsman, tman, tallyman, taximan, taxman, third baseman, timberman, tollman, townsman, tradesman, trainbandsman, trainman, traveling salesman, travelling salesman, trencherman, tribesman, triggerman, tv newsman, underclassman, utility man, vice chairman, vigilance man, visiting fireman, warehouseman, watchman, waterman, weatherman, widowman, wild man, wingman, wireman, wise man, wolfman, woodman, woods-
man, workingman, workman, yardman, yeoman, yesman
F.2 Row 1b (12 entries that only have -woman marker)
charwoman, cleaning woman, comfort woman, foolish woman, honest woman, kept woman, lollipop woman, loose woman, needlewoman, washwoman, widow woman, wonder woman
F.3 Row 1c (85 entries that only have -person marker)
abandoned person, aliterate person, bad person, bereaved person, bisexual person, blind person, british people, clumsy person, color-blind person, colored person, crabby person, creative person, dead person, deaf-and-dumb person, deaf person, deceased person, diseased person, displaced person, disreputable person, dutch people, eccentric person, emotional person, english people, english person, epicene person, famous person, fat person, forgetful person, french people, french person, good person, handicapped person, heterosexual person, homeless person, hunted person, illiterate person, important person, incompetent person, inexperienced person, influential person, insured person, irish people, irish person, juvenile person, large person, learned person, literate person, nonperson, nonreligious person, nude person, oriental person, poor person, primitive person, professional person, psychotic person, religious person, retired person, scholarly person, self-employed person, selfish person, shy person, sick person, silent person, slavic people, sleepless person, small person, spanish people, stateless person, street person, stupid person, swiss people, thin person, uneducated person, unemotional person, unemployed person, unfortunate person, ungrateful person, unkind person, unperson, unskilled person, unsuccessful person, unusual person, unwelcome person, very important person, visually impaired person
F.4 Row 2a (47 entries that have -man and -woman markers)
-man airman, assemblyman, beggarman, bionic man, bondsman, bondsman, bondsman, bondman, bond-
man, clansman, committeeman, congressman, cornishman, councilman, countryman, countryman, englishman, fancy man, fancy man, freedman, freeman, frenchman, frontiersman, gay man, gentleman, horseman, irishman, juryman, laundryman, madman, newspaperman, nobleman, oarsman, outdoorsman, point man, policeman, scotchman, scotsman, selectman, sportsman, statesman, stunt man, unmarried man, vestryman, washerman, yachtsman, yellow man -woman airwoman, assemblywoman, beggarwoman, bionic woman, bondswoman, bondswoman, bondswoman, bondwoman, bondwoman, clanswoman, committeewoman, congresswoman, cornishwoman, councilwoman, countrywoman, countrywoman, englishwoman, fancy woman, fancy woman, freedwoman, freewoman, frenchwoman, frontierswoman, gay woman, gentlewoman, horsewoman, irishwoman, jurywoman, laundrywoman, madwoman, newspaperwoman, noblewoman, oarswoman, outdoorswoman, point woman, policewoman, scotchwoman, scotswoman, selectwoman, sportswoman, stateswoman, stunt woman, unmarried woman, vestrywoman, washerwoman, yachtswoman, yellow woman
F.5 Row 2b (3 entries that have -woman and -person markers)
-woman disagreeable woman, slovenly woman, unpleasant woman -person disagreeable person, slovenly person, unpleasant person
F.6 Row 2c (10 entries that have -man and -person markers)
-man anchorman, common man, draftsman, holy man, layman, public relations man, rich man, straight man, wealthy man, working man -person anchorperson, common person, draftsperson, holy person, layperson, public relations person, rich person, straight person, wealthy person, working person
F.7 Row 3a (15 entries that have -man, -woman and -person markers)
-man black man, businessman, chairman, counterman, enlisted man, foreman, foreman, kinsman, married man, newsman, old man, salesman, spokesman, white man, young man -woman black woman, businesswoman, chairwoman, counterwoman, enlisted woman, forewoman, forewoman, kinswoman, married woman, newswoman, old woman, saleswoman, spokeswoman, white woman, young woman -person black person, businessperson, chairperson, counterperson, enlisted person, foreperson, foreperson, kinsperson, married person, newsperson, old person, salesperson, spokesperson, white person, young person
G Example Definitions of Entries in Table 11
This appendix provides the example definitions of entries from Table 11.
G.1 Examples from the 50 entries in row (1a)
able-bodied seaman: a seaman in the merchant marine; trained in special skills able seaman: a seaman in the merchant marine; trained in special skills backwoodsman: a man who lives on the frontier bagman: a salesman who travels to call on customers beef man: a man who raises (or tends) cattle best man: the principal groomsman at a wedding career man: a man who is a careerist cattleman: a man who raises (or tends) cattle coachman: a man who drives a coach (or carriage) cow man: a man who raises (or tends) cattle dirty old man: a middle-aged man with lecherous inclinations divorced man: a man who is divorced from (or separated from) his wife elevator man: a man employed to operate an elevator family man: a man whose family is of major importance in his life ferryman: a man who operates a ferry
G.2 Examples from the 11 entries in row (1b)
charwoman: a human female employed to do housework cleaning woman: a human female employed to do housework comfort woman: a woman forced into prostitution for Japanese servicemen during World War II foolish woman: a female fool honest woman: a wife who has married a man with whom she has been living for some time (especially if she is pregnant at the time) kept woman: an adulterous woman; a woman who has an ongoing extramarital sexual relationship with a man lollipop woman: a woman hired to help children cross a road safely near a school loose woman: a woman adulterer washwoman: a working woman who takes in washing widow woman: a woman whose husband is dead especially one who has not remarried wonder woman: a woman who can be a successful wife and have a professional career at the same time
G.3 Examples from the 47 entries in row (2a)
airman: someone who operates an aircraft airwoman: a woman aviator
assemblyman: someone who is a member of a legislative assembly assemblywoman: a woman assemblyman
oarsman: someone who rows a boat oarswoman: a woman oarsman
policeman: a member of a police force policewoman: a woman policeman
statesman: a man who is a respected leader in national or international affairs stateswoman: a woman statesman
G.4 Examples from the 3 entries in row (2b)
disagreeable woman: a woman who is an unpleasant person disagreeable person: a person who is not pleasant or agreeable
slovenly woman: a dirty untidy woman
slovenly person: a coarse obnoxious person
unpleasant woman: a woman who is an unpleasant person unpleasant person: a person who is not pleasant or agreeable
G.5 Examples from the 2 entries in row (2c) rich man: a man who is wealthy rich person: a person who possesses great material wealth
wealthy man: a man who is wealthy wealthy person: a person who possesses great material wealthy
G.6 Examples from the 15 entries in row (3a) businessman: a person engaged in commercial or industrial business (especially an owner or executive) businesswoman: a female businessperson businessperson: a capitalist who engages in industrial commercial enterprise
newsman: a person who investigates and reports or edits news stories newswoman: a female newsperson newsperson: a person who investigates and reports or edits news stories

heading: Future Work
text: The court decided that every time Mondadori the publishing company owned by the ex Italian Prime Minister Silvio Berlusconi publishes a photograph in the future in France, they will get 10,000 fine. The couple welcome the judges decision. They always believed the law was broken and that they had a right to their privacy. The royal couple are pleased with the decision, but they want to have a much more public criminal trial against the magazine and photographer or photographers. Under French law, if you do not respect someones privacy, you may have to spend a maximum of one year in prison and pay a fine of 45,000. This punishment would send a message to the world and,
the couple hope, stop paparazzi taking photos like this in the future. On Saturday the Irish Daily Star also published the photos. And the Italian celebrity magazine Chi published a special edition of 26 pages with the photos of the future queen. E. 5 pisa Mimi and Dean wondered which sunscreen product provides the best protection for their skin. Sunscreen products have a Sun Protection Factor (SPF) that shows how well each product absorbs the ultraviolet radiation component of sunlight. A high SPF sunscreen protects skin for longer than a low SPF sunscreen. Mimi thought of a way to compare some different sunscreen products. She and Dean collected the following:. Mimi and Dean included mineral oil because it lets most of the sunlight through, and zinc oxide because it almost completely blocks sunlight. Dean placed a drop of each substance inside a circle marked on one sheet of plastic, and then put the second plastic sheet over the top. He placed a large book on top of both sheets and pressed down. Mimi then put the plastic sheets on top of the sheet of light-sensitive paper. Light-sensitive paper changes from dark gray to white (or very light gray), depending on how long it is exposed to sunlight. Finally, Dean placed the sheets in a sunny place. E