Tales and Tropes: Gender Roles from Word Embeddings in a Century of Children’s Books (2022)

Abstract: The manner in which gender is portrayed in materials used to teach children conveys messages about people’s roles in society. In this paper, we measure the gendered depiction of central domains of social life in 100 years of highly influential children’s books. We make two main contributions: (1) we find that the portrayal of gender in these books reproduces traditional gender norms in society, and (2) we publish StoryWords 1.0, the first word embeddings trained on such a large body of children’s literature. We find that, relative to males, females are more likely to be represented in relation to their appearance than in relation to their competence; second, they are more likely to be represented in relation to their role in the family than their role in business. Finally, we find that non-binary or gender-fluid individuals are rarely mentioned. Our analysis advances understanding of the different messages contained in content commonly used to teach children, with immediate applications for practice, policy, and research.

## introduction educators and parents use books to teach children messages about society, conduct, and the world. these messages may be encoded in how different identities are, and are not, represented. if there are systematically different associations between specific identities and depictions, such messages can shape how children view the roles that they themselves, as well as others, can occupy in society. in this paper, we apply natural language processing (nlp) tools to analyze the gendered association of different attributes (e.g., traits, occupations, physical characteristics) to measure how females and males are portrayed in children's books. 1 to measure portrayal, we use word embeddings, a prediction-based method for analyzing the semantic meaning of words in context using highdimensional vectors. we supplement our analysis by training a model to detect individual sentences containing stereotypes in order to gain a deeper understanding of the implicit and explicit messages conveyed to children by the books they read. this awareness can, in turn, also help inform contentselection decisions of educators and caregivers. messages about gender-specific abilities and roles may influence children's beliefs and career paths @xcite @xcite . gender representation in children's content has traditionally been measured by manual content analysis, in which one or multiple annotators slowly read through the text of written content to classify the messages within one or multiple dimensions @xcite . the key advantage of this approach is that it is able to measure deep meaning in books; the main disadvantage is that it is highly labor-intensive, making it prohibitively costly to comprehensively characterize representation in large bodies of content, and requires a high degree of fidelity in the management and training of the coders @xcite . it is difficult to avoid human biases in this type of traditional content analysis, though these biases are of course also baked into any content analysis, including computerized approaches @xcite . advances in computer-driven content analysis began to address these concerns through automation. early efforts focused on a numerical accounting of words which represented different genders -such as counts of pronouns and the genders of named entities -and these counts were then compared across bodies of text @xcite of a subset of books which center lgbtqia+ experiences, we only found two characters with non-binary identities (0.37% of total characters). because there would not be a sufficient sample size to estimate embeddings for this group, we limit our main analysis to females and males. @xcite . simple token counts, however, primarily capture superficial representation. if a female or male is frequently present but portrayed in a stereotypical or narrow manner, then the mere existence of representation will not only be insufficient but also possibly counterproductive. in this paper, we address this gap by using word vectors to measure how different genders are depicted, vis-a-vis societal roles, in english-language, award-winning children's books commonly found in schools and homes over the past century, complementing existing measurement of whether they appear @xcite . this involves converting high-dimensional measures of the semantic meaning of words in text into one-dimensional measures of gender representation in children's books. our study makes two primary contributions: ## related work external stimuli may have important influences in shaping beliefs, actions, and outcomes @xcite @xcite . for example, his-torical analysis of changes in textbooks using a quasi-experimental framework has shown that such changes shape both people's preferences and their view of history @xcite . less is known about the representation of identities in the content in these books and how these identities are depicted. recent work has attempted to address this question by estimating the frequency of female and male presence in stories. research enumerating gender counts in children's books shows inequality in how frequently females are present in the text relative to males over time regardless of the measure, for example, in gendered pronouns and in the gender of named characters @xcite . while these findings are illustrative, they show only superficial representations and neglect to demonstrate whether the trend towards numeric equality is inclusive or rather one of an increased incidence of imbalanced representations. if the frequency of inclusion of underrepresented identities increases without a change in the underlying equity in the manner of representation, simple frequency-based measures might overstate the equity of representation in books that children are given. recent work has addressed how characters of different genders are portrayed. @xcite analyzes female characters' emotional dependency on male characters in a collection of books, movie synopses and movie scripts. that study defines narratives in which a man serves as a woman's path to a happy, fulfilling life as characterized by the 'cinderella complex'. using pretrained word2vec models, they constructed a vector representing the dimension of happy vs. unhappy that was used for calculating the 'happiness scores' of words surrounding specific female and male characters. they first selected the movie synopsis of cinderella; calculating happiness scores for it, the study shows that the happiness of cinderella depends on the prince, but not vice versa. further testing on different movie genres showed that the happiness score of the female characters portrayed in the same context as male characters was higher than when the females were portrayed alone. they also find that male characters are more likely to be described using verbs, while female characters are more likely to be described using adjectives. ## results and discussion word embeddings. for conducting the experiments, we relied on the word embedding fairness evaluation (wefe) framework @xcite , an open source software that encapsulates, evaluates and compares different fairness metrics proposed in the literature. here, we present the results obtained by using the weat metric from @xcite , the most commonly used association test for word embeddings. 16 weat assesses the extent to which a model associates two sets of target words (i.e., females and males) with sets of attribute words (i.e., appearance and competence, family and business, female and male professions). with values that can range between -2.0 and 2.0, a positive score means that females are more associated with words related to appearance, family or female professions, and a negative score means that males are more associated with the aforementioned attributes. the representation and visibility of women has increased substantially over the last century, including in occupations that have been traditionally dominated by men @xcite . despite this considerable progress, differential treatment of women in many dimensions of the economy persists into the 21st century @xcite . our analysis reflects these patterns, highlighting the incidence and persistence of professional role stereotyping in these corpora (cf. figure 2 (a) ). in addition, our results also highlight that females are more likely to be associated with words related to family than words related to business, relative to these likelihoods for males (cf. figure 2 (b) ). while the association between females and family appears to decrease slightly in the histwords data over time, we see no evidence of a similar decline in our children's book collections (c.f. figure 2 (e) ). finally, we quantify the degree to which the language used to describe females and males is different. figure 2 (c ) shows that females are much more likely to be associated with words related to their looks (as opposed to males, who tend to be associated with words related to their competence). this association between females and appearance is decreasing over time in the histwords and mainstream collections, but increases in the most recent decades within the children's book collections. these results show that all three collections (i.e., diversity, mainstream, and histwords) contain biased representations. we found no evidence that the diversity collection, meant to highlight typically excluded or marginalized identities, portrays females more equitably than the 'general' efforts of the mainstream collection. stereotype detection. we next apply the method for detecting particularly salient incidences of gender stereotypes in text, as described in section 4.3. first, we access reviews from the common sense media platform to identify books that are likely to possess highly gender stereotypical language. we then apply stereotype detection to identify specific sentences associated with the themes that were highlighted in the review and that present egregious cases of this type of messaging. here we present two sentences from a book in our corpora that this method identifies as containing such stereotypes. this book lends a commentary on rape culture.: future work should more deeply interrogate the stereotypes being transmitted in these children's books, as highlighted by these few examples. such work should also include an exploration of the context of such phrases. ## conclusion in this paper we demonstrate how nlp tools can be used to investigate the incidence of systematically different associations between females and males and their societal roles, as transmitted through children's stories. these findings underscore the importance of tracking not only whether different identities are included in stories, but also how they are portrayed. we make two primary contributions. first, we analyzed how gender roles are portrayed in children's literature. second, we created the first word embeddings trained on a century of awardwinning children's literature, storywords 1.0. consistent with previous research, our results show that females are more likely than males to be represented in relation to their appearance and their roles in the family. while we cannot speak to what 'optimal' representation would look like, our tools make it possible for practitioners, policymakers, and parents with a given goal to measure representation in a given set of books in order to help them make their choices. important directions for future work include using more precise tools, such as coreference resolution, to better understand and disentangle the indirect and direct messages contained in these texts. in addition, researchers or practitioners using these tools could expand their analysis to other targets: different groups (to understand how other identities may be differentially represented), as well as additional attributes that convey different societal meanings. furthermore, researchers must expand definitions of gender to account for non-binary and gender-fluid identities. in the future, we also plan to account for polysemous words by using contextualized word vectors.