Cross-Cultural Analysis of Human Values, Morals, and Biases in Folk Tales
Authors: Winston Wu
Lu Wang
Rada Mihalcea
Venue: EMNLP 2023 Main (2023)
Keywords: values
morality
bias
folk tales

Abstract: Folk tales are strong cultural and social influences in children's lives, and they are known to teach morals and values. However, existing studies on folk tales are largely limited to European tales. In our study, we compile a large corpus of over 1,900 tales originating from 27 diverse cultures across six continents. Using a range of lexicons and correlation analyses, we examine how human values, morals, and gender biases are expressed in folk tales across cultures. We discover differences between cultures in prevalent values and morals, as well as cross-cultural trends in problematic gender biases. Furthermore, we find trends of reduced value expression when examining public-domain fiction stories, extrinsically validate our analyses against the multicultural Schwartz Survey of Cultural Values and the Global Gender Gap Report, and find traditional gender biases associated with values, morals, and agency. This large-scale cross-cultural study of folk tales paves the way towards future studies on how literature influences and reflects cultural norms.

Cross-Cultural Analysis of Human Values, Morals, and Biases in Folk Tales
Winston Wu Lu Wang' Rada Mihalcea²
'Department of Computer Science, University of Hawai'i at Hilo CCmmputer Science and Engineering, University of Michigan - Ann Arbor wswu@hawaii edu, (wangluxy, mi ciahalcea)@@mich edu
Abstract
aspects are expressed across cultures. To do so, we first compile a large corpus of over 1,900 folk tales originating from 27 cultures from multiple online sources. We then employ lexicon-baac based cor- relation analyses to investigate human values in folk tales across cultures. To extrinsically validate our analyses, we compare values expressed in folk tales with values expressed in long-form novels, as well cultural values from the Schwartz Cul- tural Values Orientation Survey (Schwartz, 2006). In similar manner, we analyze morals in folk tales, using the framework of Moral Foundation Theory (Graham et al., 2013, 2018). Further, we investigate the representation of male and female characters in folk tales, as well as gender associ- ations with values, morals, and verbs of agency and power. Our analyses of values, morals, and gender bias in folk tales across cultures reveal sta- tistically significant differences between cultures, as well as cross-cuss cross-cultural trends, which we detail in the following sections.
Folk tales strong cultural and social influ- ences in children's lives, and they are known teach morals and values. However, exist- ing studies folk tales are largely limited to European tales. In our study, we compile a large corpus of over 1,900 tales originat- ing from 27 diverse cultures across six conti- nents. Using a range of lexicons and corre- lation analyses,
examine how human val- ues, morals, and gender biases are expressed in folk tales across cultures. We discover differ- ences between cultures in prevalent values and morals, as well as cross-cultuall trends in proh- lematic gender biases. Furthermore, we find trends of reduced value expression when ex- amining public-domain fiction stories, extrin- sically validate our analyses against the mul- ticultural Schwartz Survey of Cultural Values, find traditional gender biases associated with values, morals, and agency. This large- scale cross-cultural study of folk tales paves the way for future studies on how literature in- fluences and reflects cultural norms.
This paper presents several novel contributions: (1) the collection of large dataset of folk and fairy tales from across 27 cultures, (2) the application of automatic lexicon-basee measurements of val- ues and morals folk tales across cultures, which have traditionally been manual efforts, (3) cross- cultural analyses of these folk tales with respect to values and morals, revealing values corresponding to human connection, and (4) cross-cultural analy- ses of gender biases with respect to values, morals, agency, and power, revealing associations with tra- ditional gender roles. To our knowledge, this is the largest cross-cultural computational study of folk tales, covering cultures from six continents. Code and data for reproducing our experiments can be found at github. com//wswu/folktales
1 Introduction
Folk tales are stories told in the oral tradition. All cultures tell stories, though the manner in which the stories are told can differ (McCabe, 1997). Ac- cording to folklorist William Bascom (1954), folk- tales have four purposes: (1) to let people escape from society, (2) to validate and justify culture, (3) to reinforce morals and values, and (4) to ap- ply social pressure. Most of the previous work on folk and fairy tales has focused on a limited set of Anglo-American or European cultures, ignoring large swaths of the world's 's diverse cultures. In this paper, we present cross-cultural analysis of folk tales covering wide range of cultures around the world, focusing on Balcom's latter two purposes. Specifically, we examine the values, morals, and gender biases expressed in folk tales, quan- tifying similarities and differences in how these
2 Data
Folk tales have largely been passed down through oral tradition but have recently been digitized. We

---

compile large dataset of folk tales across mul- tiple cultures from three online sources: the Al- ishman Folk Texts (AFT; Hagedorn and Darányi, 2022), the Multilingual Folk Tale Database' (MFTD), and Project Gutenberg.2 AFT is a dig- itized compilation of tales collected and annotated by the late Professor D. L. Alishman. The Mul- tilingual Folk Tale Database is collection of tales shared by various users in multiple languages. While these are the largest available collections of folk tales online, they are largely limited to Euro-centric cultures. To enable a wider analy- sis across cultures and cover all six continents of the world (except Antarctica), we augment this set with tales from public domain tale compilations from Project Gutenberg, representing the follow- ing cultures: Africa, Australia, Brazil, Canada, China, Japan, and the Himalayas. The list of sources for these tales can be found in Table 1. We manually cleaned each text by removing Project Gutenberg boilerplate text, image captions, and ex- cess whitespace.
300
METD
230
Guardbrgg
200
130
100
50
Ausralio
Ageiia
Bazil
Cunoda
Dommakk
Seglind
France
Gresz
Himalay ayaa
Icelind
Icda
Indand
bally
Noolay
Pakcom
Haligiine
Russi
Scallani
Soald
Shidkn den
Switzesion
USA Walds
Ri
Figure 1: Number of fairy tales per culture.
tales come from MFTD, and 257 tales come from Project Gutenberg, with an average of 1,036 words per tale. Figure presents the total number of tales from each culture.
Human Values
One of the functions of folklore is to teach and reinforce values (Bascom, 1954). Values are in- extricably connected with culture, and naturally, people in different cultures hold different values, A long line of research by social psychologist Shalom Schwartz has proposed the Theory of Ba- sic Human Values (Schwartz and Bilsky, 1987; Schwartz, 2012), which are universal guiding prin- ciples for a person or group. Schwartz originally identified 10 human values (Schwartz and Bilsky, 1987), later 19 (Schwartz et al., 2012), which were validated be comprehensive across cul- tures through surveys in over 65 countries. Values and morals fairy tales have also been manually analyzed and annotated, especially in the educa- tion literature (e.g. Moss, 1988; Guroian, 1996). However, these studies are usually limited to a small set of tales of interest. In addition, existing work measuring cross-cultural values (Rokeach, 1973; Inglehart et al., 2000) often rely surveys, which are time-consuming and require human la- bor.
We combine tales from these three data sources with the following modifications: (1) we consider tales whose provenance in AFT is "Aesop" to be from Greece;3 (2) we remove duplicate tales (the culture of origin is identical and the Jaccard sim- ilarity between the lowercased words in the title is greater than 0.8); (3) remove tales shorter than 50 or longer than 4,000 tokens (for speed; 13 tales were removed); (4) for adequate representa- tion, we only keep tales when there are at least 10 tales from that culture; (5) to facilitate our analy- sis, we remove tales not available English. We emphasize that while the text is in English, the con- tent of the tales is representative of the tales' cul- ture. For tales not originally written in English, some tales teractions with native speakers, while others were translated into English from existing sources. To our knowledge, all folk tale compilers and trans- lators were professional folklorists who have dedi- cated their lives to studying a specific culture, and thus have an intimate understanding of the culture of the tale, well native-level proficiency in English and the original language.
compiled in English based in-
To enable the analysis of folk tales at a large scale, we propose an automated lexicon-based method to measure human values. We use the Val- ues Lexicon, a crowd-sourced lexicon of 49 hu- man values (Wilson et al., 2018) containing 1,267 English words and phrases. The Values Lexicon was constructed based on crowdsourcing by users from the United States, India, Kenya, the Philip- pines, and Trinidad and Tobago. has been val- idated against human-labeled word intrusion and category matching tasks with high accuracy. The
In total, our datasct comprises 1,925 tales across 27 cultures. 788 tales come from AFT, 880
'www.mf'te org www.gutenberg.com .rg Folk tales do not include mythology.

---

Origin
Title
Author/Editor
Japan
Japanese Fairy Tales
Yei Theodora Ozaki
Himalayas
Simla Village Tales, or Folk Tales from the Himalayas
Alice Elizabeth Dracott
China
A Chinese Wonder Book
Norman Hinsdale Pitman
China
The Chinese Fairy Book
Richard Wilhelm
Africa
West African Folk-Tales
W. H. Barker and Cecilia Sinclair
Australia
Australian Fairy Tales
Atha Westbury
Canada
Canadian Fairy Tales
Cyrus Macmillan
Brazil
Fairy Tales from Brazil: How and Why Tales from Brazilian Folk-Lore
Elsic Spicer Fells
Table 1: Folk tales compiled and cleaned from Project Gutenberg.
Acceping-otterss
Achievement
Advice
the Social value overlap with Society and Rela- tionships words. These values reveal that across cultures. tales emphasize values related to hu- man connection. Several other prevalent values, including Relationships. Helping-others, Family, Significant-other, and Children, are directly re- lated to interpersonal relationships. This is signifi- cant because humans are wired to form bonds with one another (Fishbane, 2007). Many folk tales, which were originally orally passed down, may have been created to reinforce the importance, even usefulness, of human connections in past so- cieties.
Animals
Art
Autonomy
Career
Children
Cognition
Creativity
Dedication
Emotion
Family
Feeling-good
Forgiving
Friends
Future
Gratitude
Hard work
Health
Helping-ofe
Honesty
Inner-peace
Justice
Leaming
Life
Marriage
Moral
Nature
Optimism
Order
Purents
Perseverance
Purpose
Relationships
Religion
Respect
Responsible
Security
Self-confidence
Siblings
Significant-other
Social
Society
Spirituality
Thinking Work-ethic
Truth
Wealth
Table 2: The 49 values from the Values Lexicon (Wil- son et al., 2018).
What are the most common values by culture? Next, we examine whether there are differences in the values expressed in folk tales from differ- ent cultures. Figure 2 plots the top 10 values expressed in each culture's tales, where the dark color indicates that the value more prevalent (the full figure with all values is presented in Figure 6). A visual inspection reveals that there are differ- ences within cultures, and this is natural because each culture has unique socictal norms (Schwartz, 1999). For example, research has shown that there are differences in basic societal values between Eu- rope and Asia (Blondel and Inoguchi, 2006; Hof- stede, 2007). Indeed, our analysis shows that the cultures of the Himalayas, Hungary, and the Philip- pines show the strongest expression of Social val- ues, compared to Denmark, France, and Greece, which show the weakest expression.
list of values in the Values Lexicon is shown Ta- ble We use this lexicon to analyze the human values expressed in folk tales.
We analyze folk tales for human values using the following process. First, tokenize and low- ercase each tale with spaCy.* We then compute a distribution of human values for each tale by counting tokens associated with a human value and normalizing by the total number of tokens in the text. We compare and analyze these distribu- tions across cultures to answer several research questions below.
What values are expressed in folk tales? To identify the most common values expressed in folk tales, we examine the top three most prevalent values within each tale. Across all cultures, the most common values are Social values, followed by Society and Relationships, ure computed by examining the top three most prevalent values within each tale. The Social value contains words like mother, father, son, together, house, home, family, and live. Many words in
To test whether these differences are statistically significant, we perform multiple two-sided T-tests, with Bonferroni correction, because we are mak- ing thousands of comparisons. We find total of 229 statistically significant differences (p < 0.05) when comparing any two culture-value pairs. For example, Hungary's expression of Social values is statistically significantly different from Italy, Nor- way, Denmark, France, and Greece. While these
shown in Fig-
'spacy.io 1o

---

Chilres
Culture
Tales
Gutenberg
Africa
5.06
Parecta
Rekteniza
Australia
4.52
4.11
Austria
5.09
Secate
scead
Brazil
3.60
Chis
sueek Soree tar
Canada
4.86
China
5.22
Denmark
4.35
England
4.32
3.96
Figure 2: The top 10 human values in fairy tales across cultures. The darker the color, the more prevalent that value is expressed in that culture.
France
5,05
3.84
Germany
4.84
3.49
Greece
4.39
Himalayas
5.28
Hungary
5.52
Iceland
5.84
results are significant, the interpretation of these results requires in-depth knowledge of the respec- tive cultures. For example, the Philippines is said to have "no culture" but had a strong social hier- archy before being colonized by Spain (Rosaldo, 1988; Junker, 1990), which may explain the high expression of Social words in its tales. We empha- size that these kinds of large cros-cultural analy- ses of values are possible without the of veys, is the norm in the existing literature on cross-cultural values, but will require collabora- tion with cultural anthropologists and other so- cial scientists to fully understand the findings from such analyses.
India
5.23
Ireland
4.66
4.05
Italy
4.45
3.69
Japan
4.46
Norway
3.77
Pakistan
3.62
Philippines
5.05
Romania
5.32
Russia
4.91
5.91
Scotland
4.38
Serbia
3.42
Spain
4.75
4.92
Sweden
4.63
Switzerland
4.64
Turkey
4.86
USA
4.47
4.03
Wales
4.09
Mean
4.64
3.99
How do folk tales compare to fiction books? One outstanding question is whether the values expressed in folk tales are also reflected in the broader literature. Tales are often written to teach lesson explain how the world works, but they usually depict fictional events. Fiction novels also fictional, but the purpose of fiction is often to entertain (Follett, 1923). Perhaps the authors who grew up in a culture perpetuate values their own works that they learned from that culture's folk tales. To investigate this question, we com- pile a set of public-domain fiction books written in English from Project Gutenberg whose authors were born in country covered by folk tale dataset. This set of fiction comprises 260 books across 9 cultures: Australia, England, France, Ger- many, Ireland, Italy, Russia, Spain, and USA. We then perform the same analysis of values on this dataset of novels.
Table 3: Percentage of words in folk tales and Guten- berg fiction novels that express values. Overall, novels contain a smaller percentage value words compared to folk tales.
from Project Gutenberg. On average. 4.64% of words in folk tales are value-cxpressing, compared to 3.99% in novels. Thus, across cultures, novels express values lower rate than folk tales. This supports Bascom's (1954) notion that one purpose of folk tales is reinforce values, in comparison to novels, which are primarily written to entertain broader target audience.
Next, examine if there change in val- ues expressed berween folk tales and longer nov- els. To do this analysis, we only consider value- expressing words, comparing the average expres- sion of values between folk tales and longer novels. We find that even after controlling for the length of the novels, the expression of most values de- creases. Figure presents the top five expressed values. A marked decrease in the percentage of value-expressing words across different values re- inforces that novels express values at a lower rate compared to folk tales.
As these books are substantially longer than folk tales (on average, 92,845 tokens per novel 1,036 tokens for folk tales), we first examine whether there difference in the percentage of words expressing values in folk tales and longer- form novels. Table presents the percentage of value-expressing words folk tales and novels

---

Are values in folk tales reflective of broader cul- ture? We also seek to extrinsically validate the values in folk tales. addition to human values, Schwartz also studied cultural values (Schwartz, 2006). This cross-cultural study identified seven cultural values- - Hierarchy, Embeddedness, Mas- tery. Affective Autonomy, Intellectual Autonomy, and Egalitarianism by surveying K-12 teachers and college students in 80 countries, who provided scores for these cultural values ranging from 1 and 5. For our analysis, we identify human values that most closely align with these Cultural Values Ori- entation Scores from Schwartz's s cultural value sur- vey. We examine 19 countries which are covered by Schwartz's study and also represented in our folk tales. For each pair of cultural value and hu- man value, rank the countries by their cultural value score as well as by our average human value measurement in their folk tales. Next, we compute the Spearman rank correlation between these two rankings. The most positively and negatively cor- related cultural and human values are presented in Table 4.
Cultural Value
Human Value
AffAuton
Wealth
0.65
Hierarchy
Marriage
0.52
Mastery
Nature
0.50
Mastery
Self-confidence
0.43
Harmony
Family
0.41
AffAuton
Children
-0.56
Embedded
Career
-0.60
Egalitarianism
Work-ethic
-0.62
Embedded
Siblings
-0.64
Hierarchy
Respect
-0.77
Table 4: The most positively and negatively correlated cultural values and human values computed by ranking 19 countries. AffAuton refers Affective Autonomy. is Spearman's 's rho.
Moral Foundations
Folk tales also teach morals (Bascom, 1954), and culture is an important variable that shapes one's morals (Prinz, 2014). To examine morals in folk tales, we employ Moral Foundations Theory (MFT; Graham et al., 2013, 2018), which was cre- ated by social psychologists to explain the basis of moral actions. MFT defines five moral founda- tions, each with a positive and negative aspect: Care/Harm, Fairness/Cheating. Loyalty/Betrayall Authority/Subversion.. and Sanctity/Degrraattin MFT has recently been used analyze a variety of text, most notably in social media and politics (e.g. Fulgoni et al., 2016; Roy and Goldwasser, 2021; Asprino et al., 2022). Analogous to our analysis of human values, we perform a eexicon-base analy- sis of moral foundations on folk tales by employ- ing the Moral Foundations Dictionary 2.0 (Frimer et al., 2019), which associates 2,103 words and phrases with the 10 moral foundations in MFT. Figure displays the distribution of cultures ex- pressing moral foundations through folk tales.
We briefly interpret some of these findings. Af- fective Autonomy refers to the pursuit of "affec- tive positive experiences", such as enjoyment of life (Schwartz, 2006). China, India, and Turkey are the countries with the highest Affective Au- tonomy score. The Wealth value is strongly cor- related with Affective Autonomy, because wealth gives one the means to enjoy life. In contrast, the Children value negatively correlated; perhaps having children takes away time for these positive affective pursuits. The Hierarchy and Marriage values also have a natural explanation. In hicrar- chical societies, individuals are socialized to con- form the roles of the hierarchy (Schwartz, 2006). The act of marriage comes with roles and responsi- bilities that must be upheld. Unexpectedly, the Re- spect value strongly negatively correlated with Hicrarchy. Since the Respect value associated with words related to respecting (respect, honor- ing) and the older generation (elders, seniority), we would expect that respecting authority is an in- tegral part of hierarchical society. This may indi- cate that human values and cultural values not always aligned.
Are there cross-cultural trends in moral foun- dations expressed in folk tales? We first ex- amine cross-cultural trends. Overall, we find a stronger expression of positive dimensions (Care, Loyalty, Authority, and Sanctity) compared to the negative dimensions. This may be because pos- itive aspects of morality ('thou shalt") promote human welfare in comparison to negative aspects
Auustria, China, Denmark, France, Greece, Hungary, In- dia, Ireland, Italy, Japan, Norway, Pakistan, Philippinees Ro- mania, Russia, Serbia, Spain, Sweden, and Turkey.
"A later version of this theory defines a sixth dimension. I siherty/Oppression which we do not consider because avail- able lexicons do not cover this dimension.

---

China
Spain
Word
# Tales (%)
Word
# Tales (%)
Faimes
Cheriig
Loyalty
order
41 (0.08)
ordered
9 (0.14)
Authorty
father
34 (0.07)
father
7 (0.11)
Subvesioe
Degratation
Sacciit
master
29 (0.06)
order
6 (0.09)
Afre
Austalli
Acer
uradi
Chins
Denmate
Eugaas
Trum
Cree
July
paitte
shillgge Rodbam
Switin
Will
emperor
23 (0.05)
master
5 (0.08)
ordered
23 (0.05)
commanded
4 (0.06)
bowed
23 (0.05)
servants
4 (0.06)
servants
(0.04)
servant
4 (0.06)
Figure Moral foundations expressed in fairy tales
honor
14 (0.03)
elder
3 (0.05)
across cultures.
queen
13 (0.03)
respected
2 (0.03)
willing
(0.03)
protect
2 (0.03)
Table 5: The most frequent words associated with the Authority moral dimension from Chinese and Spanish folk tales, and the number of tales including each word. Though historically China and Spain were both monar- chies, the emperor is mentioned in larger proportion of Chinese folk tales.
("thou shalt not") (Staub, 2013). As tools for teaching, folk tales need convey their sage, and positive interactions are a more effec- tive way to do so. In addition, we find that the Fairess/Cheettnng dimension more weakly ex- pressed compared to the other dimensions.
Gender Bias
Looking across cultures, we identify statisti- cally significant differences in the expressions of morality by performing multiple T-tests with Bon- ferroni correction. Our analysis identified 54 sta- tistically significant differences. The full listing of these culture pairs is presented in Figure 8. As a case study, we consider folk tales from China, which shows significant differences from other cul- tures in its expression of Authority. known that Chinese culture values deference au- thority (Hwang, 2012; Zhai, 2017). How is this value reflected in folk tales? We examine the most frequent words associated with the Authority di- mension in Chinese and Spanish folk tales, shown in Table 5, because tales from both of these cul- tures express strong Authority, We find that order, both
Finally, we turn our focus to gender bias, which is pervasive in children's literature (Louie, 2001). Studies have shown that gender stereotypes are picked up by the children reading the tales. For example, in study of primary-school children who read Cinderella, girls did not identify with Cinderella due to her lack of independence, but boys identified with Prince Charming due his independence (Westland, 1993). An investigation of gender stereotypes in 247 children's 's books in- dicated that gender bias is indicative of the tar- get audience (Lewis et al., 2022). Other stud- ies show that in children's 's books, mothers play dominant role in parenting, while fathers un- derrepresented (Anderson al., 2021). Some of these gender discrepancies disappear if the author is female (Nagaraj and Kejriwal, 2022). There has been growing interest investigating gender bias in children's literature, specifically children's books, but there is little work on large collections of folk tales across multiple cultures. Our work seeks to rectify this shortcoming in the literature.
is well
and verb, father, and master the top Authority terms in both Chinese and Span- ish folk tales. However, the next most frequent term, emperor, occurs over 5% of Chinese folk tales, but does not appear in the top terms for Span- ish tales. Historically, both China and Spain were monarchies. However, in Chinese tradition, the emperor is iraditionally believed to be of divine descent (Werner, 2020). Because of this, the em- peror features prominently in tales, similar other deities in tales from other cultures. Spanish tales, on the other hand, often feature king. Thus, an analysis of morals from folk tales can also reveal historical phenomena in culture.
To support the analyses in this paper, we first identify the gender of characters in each tale. We follow the process in Lucy and Bam- (2021) of performing dependency parsing on each tale using spaCy with the en_core_wed web_sm model and coreference resolution using fast-coref (Toshniwal et al., 2021). Within each coref- erence cluster, we assign the character's gen-

---

tales were written teach boys and girls differ- values and attitudes appropriate for their gen- der. To determine which differences, we examine expresexpressing words that are associated with mentions of gendered characters. Specifically, we compute the co-occurrence of value-expressing words within 5-word window of each mention of gendered characters. Table 6 presents the per- centage of male and female characters associated with each value. We find that on average, female characters are associated with values that are tra- ditionally considered feminine, such as Children, Family, and Marriage (Fox and Murry, 2000). contrast, male characters have significant differ- ences in traditionally masculine values such Ca- reer, Wealth, Friends, and Helping-others, and cog- nitive values such Cognition, and Thinking. Table contains a listing of values associated with female and male characters in folk tales.
Aftica
Austrlla
Austia
Brazil
Canada
China
Denmark
England
France
Germany
Greece
Himalayas
Hungary
Ireland
Italy
Japan
Norway
Pakistan
Phil Ppiines
Romania
Russa
Scodland
Serbia
Spain
Sweden
Swirzerland
Turkey
USA
Wales
0.0
0.2 All Female
0.4
0.6
0.8
1.0
All Male
What is the relationship between gender and moral foundations? Using the same procedure for analyzing gender and human values, we com- pute the association between gender and moral foundations (Table 7). We find that in folk tales, female characters are strongly associated with the Care dimension, while male characters are more associated with Authority. This seems cor- roborate existing findings that women are more concerned about preventing harm (Efferson et al., 2017) and that women are depicted as caring and nurturing, while men are depicted as authority fig- ures (Tronto, 1995). We also find that male char- acters more associated with the dimensions of Cheating, Betrayal, and Degradation; the differ- ences are smaller but still statistically significant. These may correspond with the "bad guy" trope that occurs frequently in tales (Davies, 1995).
Figure 4: Average gender ratios characters in folk tales across cultures. average ratio of 0.72 across all cultures indicates that tale has average 2.57 male characters for every female character. Error bars indi- cate standard error.
der based on which gendered pronoun is more frequent: male (he/him/his/himself) or female (she/her/her//herrslf),, We only consider charac- ters if they have at least three mentions in a story. We now investigate the representation of male and female characters in the story.
How are genders represented in folk tales? Across cultures, stories simply have more male characters than female characters." In total, our dataset contains 1,680 female characters and 3,853 characters (a 2.29x difference). When computing gender proportions on a per-tale basis, we find that a tale has on average 2.57 male characters for every female character. Figure shows the stark differences in the ratio of male characters to female characters in folk tales. Similar gender disparities have been found in fairy tales (Zhou et al., 2022), children's picture books (Gooden and Gooden, 2001), and movies (Smith et al., 2015).
What is the relationship between gender, agency, and power? Characters can be per- ceived to have agency and power based on ac- tions they take. As an example from Sap et al. (2017), consider the verbs "accept" and "assess". A character who accepts things is perceived to be more passive (lower agency) than someone who assesses. Sap al. (2017) found that male charac- ters portrayed with more agency than female characters in movie scripts. Can we also see these differences in folk tales? Following Sap et al. (2017), we perform dependency parsing on our folk tales using spaCy to identify verbs, the sub-
What the relationship between gender and values? While some folk tales were written for educational value, Zipes (2012) argues that many
Many tales have animals as characters, but these animals are anthropomorphized and given a gender.

---

Foundation
Female
Male
Care Harm
26.91
14.61
10.45
11.34
Value
Female
Male
Fairness
2.30
2.30
Accepting-othen
0.49
0.46
Cheating
2.51
3.82
Achievement
0.11
Advice
0.62
0.76
Loyalty
15.15
14.55
Betrayal
0.35
0.90
Animals
0.99
1,86
0.65
0.79
Authority
15.57
21.73
Autonomy
0.07
0.08
Suhversion 1.00
0.99
Career
0.95
1.48
Sanctity
14.04
14.91
Children
13.41
9.84
Degradation 3.36
4.60
Cognition
5.31
Creativity
0.06
0.08
Table 7: Percentage of female and male characters as- sociated with moral foundations. number is bolded it is greater than the opposite gender, and this differ- ence is statistically significant.
Dedication
0.06
0.08
Emotion
0.01
Family
24.01
14.81
Feeling-good
2.58
2.73
Forgiving
0.20
0.22
Friends
0.86
Future
0.81
0.89
ject of the verb (dependency relation nsubj), and object of the verb (dependency relation nsubjpass and dobj). For this analysis, 2,155 verbs annotated for agency (positive, nega- tive, equal) and power (agent, theme, equal) (Sap et al., 2017).
Gratitude
1.16
1.34
Hard-work
0.35
Hcalth
0.10
0.11
lexicon of
Helping-others
9.02
10.96
Honesty
0.07
Inner-peace
0.80
0.75
Justice
0.94
0.67
Learning
0.66
1.14
We first examine agency. After accounting for the difference in number of male/female charac- ters, find that male characters are more likely than female characters be agents of verb Fig- ure 5. The verbs themselves also warrant investi- gation; for some verbs, the agent plays a more ac- tive role, whereas for other verbs, the agent plays more passive role. To investigate the interplay between gender and agency, we count the gender of verbs' agents. The results are presented in Ta- ble 8. We find that overall, positive agency verbs prevalent than negative agency words. When separated by the gender of the agent, male characters are more likely to be agents of a posi- tive agency verb, while female characters are more likely to be agents of a negative agency verb. Our findings across cultures are similar to Sap et al. (2017)'s findings movie scripts.
Life
3.74
Marriage
12.62
6.34
Moral
0,06
0.14
Nature
0.08
Optimism
0,20
0.24
Order
1.41
1.47
Parents
10.36
5.90
Perseverance
0.21
Purpose
0.42
0.69
Relationships
37.19
25.05
Religion
2.39
3.57
Respect
6.64
8.62
Responsible
0.10
0.13
Security
0.33
0.52
Self-confidence
0.11
0.11
Siblings
4.07
2.57
Significant-other
14.05
7.53
Social
47.86
36.38
Society
11.93
15.06
Spirituality
1.39
1.47
Thinking
4.77
5,58
Truth
1,03
1.68
Next, we examine power. Sap et al. (2017) an- notates verbs for power in one three ways: ei- ther the agent has stronger power than the theme, the theme (object) has stronger power than the agent, or the power between the agent and theme equal. We examine the association between male and female characters agents and themes of powerfll vs. theme-powerful verbs in Ta- ble For Agent-powenn verbs, we find that male characters are 2.67x more likely to the agent than the theme (compared to 10x for female char-
Wealth
1.85
3.39
Work-ethic
0.75
0.92
Table 6: Percentage of female and male characters as- sociated with human values. A number is bolded if it is greater than the opposite gender, and this difference statistically significant. Female characters average are more associated with values that are traditionally feminine.

---

A>T
Equal
Aftcca
Australia
P/Agent
9985 (0.21)
3504 (0.21)
9750 (0.24)
Ausra
F/Theme
4760 (0.10)
1504 (0.09)
2849 (0.07)
Brazil
M/Agent
24075 (0.50)
8158 (0.49)
22613 (0.56)
Canada
China
M/Theme
9016 (0.19)
3317 (0.20)
4963 (0.12)
Denmark
England
France
Table 9; Association between power and gender. A>T means the verb is where the agent has power than the theme in the event, while T>A means the theme has more power than the agent. Male charac- ters more likely than female characters to be agents of Agent-powerful verbs.
Germany
Greece
Himalayas
keland
India
Italy
Female
Japan
Norway
Pakistan
Phil hilippines
Romania
Russia
Basic Human Values and Moral Foundations, we analyzed word usage in folk tales to identify cross- cultural trends of values involving human connec- tion, as well as culture-specffe preferences in the values and morals expressed in folk tales. Our analysis of gender bias in folk tales reveals that across cultures, male and female characters in the tales are associated with values and morals that are traditionally thought of as masculine and feminine, respectively. addition, we found male charac- ters have more agency in events than female char- acters.
Scodland
Spain
Sweden
Switzerlant
Turkey
USA
Wales
0.4
0.5
0.6
0.9
Non-Agent
Agent
Figure 5: Proportion of male/female characters as agents of events. Across cultures, male characters more likely to be the agent.
Agency
Female
Male
This large-scale cross-cultural study of folk tales paves the way forward for future studies on how literature influences and reflects cultural norms. While folk tales are reflection of their re- spective cultures, they can also have profound in- fluences hundreds of years down the road on chil- dren reading these tales. Understanding the true effect of reading folk tales may require unrealistic longitudinal studies. However, machines mak- ing great strides in language understanding. One interesting avenue of future research is NLP models to create new folk tales. Indeed, there have been some (manual) efforts to purposefully create or rewrite stories that subvert established gender biases (Crew, 2002). With recent advances in language modeling, we would like to automat- ically write stories using controllable generation and style transfer techniques. The automatic pro- duction of such debiased literature will hopefully bring awareness to long-standing biases that per- vade our society.
Positive
16,687 (0.65)
39,851 (0.67)
Negative
6,108 (0.24)
13,151 (0.22)
Equal
3,037 (0.12)
6,846 (0.11)
Table 8: Association between agency and gender. Posi- tive means that the agent has an active role in the event, while Negative means the agent plays more passive role. Male characters are more likely to be agents of positive agency verb, while female characters are more likely to be agents of negative agency verb.
acters. For Theme-powerful verbs, male charac- ters are 2.46x more likely to be the agent than the theme of an Agent-powerful verb (compared to 2.32x for female characters). This decrease in the ratio between male and female characters in- dicates gender imbalance: male characters are more likely to be agents an agent-powerful verb, than female characters. Again, this corroborates (Sap et al., 2017)'s findings.
6 Conclusion
Acknowledgments
This paper presented large-scale cross-cultural analysis of values, morals, and gender bias in folk tales. We first collected a large dataset of over 1,900 folk tales from 27 diverse cultures around the world. Building upon well-known theories of
We thank the anonymous reviewers for their constructive feedback, and the members of the Language and Information Technologies (LIT) and LAnguage Understanding and generatioN re-

---

searCH (LAUNCH) labs the University of Michigan for the insightful discussions during the early stage of the project. This project was partially funded by an award from the Temple- ton Foundation (#62256), a grant from the Air Force Office of Scientific Research (#FA9550-22- 1-0099), and grant from the Department of State (#STC10023GR00144) Any opinions, findings, and conclusions recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Templeton Foundation or the Air Force Office of Scientific Research or the Department of State.
values in our culture, and develop more socially just and equitable language technologies.
References
David A Anderson, Mykol C Hamilton, Gabrielle M Gilkison, and Skyler Palmer. 2021. Gender social- ization, family leadership. and the invisible father in childrens literature. Journal of Family Issues, page 0192513X21 1058820.
Luigi Asprino, I Luana Bulla, Stefano De Giorgis, Aldo Gangemi. Ludovica Marinucei, and Misael Mon- giovi. 2022. Uncovering values: Detecting latent moral content from natural language with explain- able and non non-trained methods. In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop Knowledge Extraction and Integra- tion for Deep Learning Architectures, pages 33-41, Dublin, Ireland and Online. Association for Compu- tational Linguistics.
Limitations
A selection of folk tales cannot fully represent the morals, values, and biases of culture. Unfortu- nately, the analyses in this paper also limited by the availability of folk tales online, which do not span every culture in the world. We also recog- nize that the coreference- based heuristic for gen- der inference has limitations (Cao and Daumé III, 2020), though it an established approach for in- ferring characters' genders. In addition, this study analyzes folk tales written in English, due to the lack of existing tools in other languages for ana- lyzing values and morals. As mentioned in Sec- tion 2, the texts of these tales, though written in English, are indeed representative of their origi- nal culture. For tales not originally written in En- glish, some tales were compiled English based on interactions with native speakers, and others were translated into English from existing sources. To our knowledge, all compilers and translators of the tales in our dataset were professional folk- lorists who have dedicated their lives to studying specific culture, and thus have intimate un- derstanding of the culture of the tale, as well as native-level proficiency in English and the original language. Future work can investigate translating existing into English, or developing new lexicons or more sophisticated tools for analyzing tales in other languages.
William R Bascom. 1954, Four functions of folklore. The Journal of American Folklore, 67(266):333 349.
Jean Blondel and Takashi Inoguchi. 2006. Political cul- tures in Asia and Europe: citizens, states and soci- etal values. Routledge.
Yang Trista Cao and Hal Daumé III. 2020. Toward coreference resolution. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4568-4595, Online. Association for Computational Linguistics.
Hilary S Crew. 2002. Spinning new tales from tra- ditional texts: Donna jo napoli and the rewriting of fairy tale. Children's 's Literature Education, 33(2):77-95.
Jude Davies. 1995. i'm the bad guy?falling down and white masculinity in 1990s hollywood. Journal of Gender Studies, 4(2):145-152.
Leah Efferson, Andrea Glenn, Rheanna Remmel, and Ravi Iyer. 2017. The influence of gender on the rela- tionship between psychopathy and five moral foun- dations. Personaliry and mental health, 11(4):335- 343.
Mona Dekoven Fishbane. 2007. Wired connect: Neuroscience, relationships, and therapy. Family process, 46(3):395-412.
Ethics Statement
Folk tales have an important connection with cul- ture. This work is making important progress in understanding how folk tales reflect culture well as how folk tales can influence writers. Under- standing the interplay between tales and culture will allow us to better understand the biases and
Wilson Follett. 1923. The Modern Novel: A Study of the Purpose and the Meaning of Fiction. AA Knopf.
Greer Litton Fox and Velma McBride Murry. 2000. Gender and families: Feminist perspectives and family research. Journal of Marriage and Family, 62(4): 1100-1772

---

Jeremy A Frimer, Reihane Boghrati, Jonathan Haidt, Jesse Graham, and Morteza Dehgani. 2019. Moral Toundations dictionary for linguistic analyses 2.0.
Li Lucy and David Bamman. 2021. Gender and repre- sentation bias in GPT-3 generated stories. In Pro- ceedings of the Third Workshop on Narrative Un- derstanding, pages 48-55, Virtual. Association for Computational Linguistics.
Unpublished manuscript.
Dean Fulgoni, Jordan Carpenter, Lyle Ungar, and Daniel Preofiuc-Piettr. 2016. An empirical ex- ploration of moral foundations theory in partisan news sources. In Proceedings of the Tenth Inter- Language Resources and
Allyssa McCabe. 1997. Cultural background and sto- rytelling: A review and implications for schooling. The Elementary School Journal, 97(5):453-47) 473..
national Conference Evaluation (L.RE'166) 16), pages 3730-3736, Portorož, Slovenia. European Language Resources Associa- tion (ELRA).
Anita Moss. 1988. Mothers, monsters, and morals in victorian fairy tales. The Lion and the Unicorn, 12(2):47-60.
Akarsh Nagaraj and Mayank Kejriwal, 2022. Robust quantification of gender disparity in pre-modemn en- glish literature using natural language processing. arXiv preprint arXiv:2204.05M:
Angela M Gooden and Mark A Gooden. 2001. Gender representation in notable children's picture books: 1995-1999. Sex roles, 45(1):89-101.
Jesse Graham, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P Wojcik, and Peter H Ditto. 2013. Moral foundations theory: The pragmatic va- lidity of moral pluralism. In Advances experimen- tal social psychology, volume 47, pages 55-130 EI-
Jesse Prinz. 2014. Where do morals come from?- a plea for a cultural approach. In Empirically in- formed ethics: Morality between facts and norms, pages 99-116. Springer.
sevier.
Milton Rokeach. 1973. The nature of human values. Free press.
Jesse Graham, Jonathan Haidt, Matt Motyl, Peter Meindl, Carol Iskiwitch, and Marlon Mooijman. 2018. Moral foundations theory. Atlas of moral psy- chology. 211.
Renato Rosaldo. 1988. Ideology, place, and people without culture. Cultural Anthropology. 3(1):77-87.
Shamik Roy and Dan Goldwasser. 2021. Analysis of nuanced stances and sentiment towards entities of US politicians through the lens of moral founda- tion theory. In Proceedings of the Ninth Interna- tional Workshop on Natural Language Processing for Social Media, pages 1-13, Online. Association for Computational Linguistics.
Vigen Guroian. 1996. Awakening the moral imagina- tion: Teaching virtues through fairy tales. The Inter- collegiate Review, 32(1):3.
Joshua Hagedorn and Sándor Darányi. 2022. Bearing a bag-of-tales: An open corpus of annotated folktales for reproducible research. Journal of Open Humani- ties Data, 8(16).
Maarten Sap, Marcella Cindy Prasettio, Ari Holtzman, Hannah Rashkin. and Yejin Choi. 2017. Connota- tion frames of power and agency in modern films. In Proceedings of the 2017 Conference on Empiri- cal Methods in Natural Language Processing, pages 2329 2334, Copenhagen, Denmark. Association for Computational Linguistics.
Geert Hofstede. 2007. A european in asia. Asian Jour- nal of Social Psychology, 10(1):16-21.
Kwang-Kuo Hwang. 2012. Foundations of Chinese psychology: Confucian social relations, volume I. Springer.
Shalom Schwartz. 2006. A theory of cultural value ori- entations: Explication and applications. Compara- live sociology, 5(2-3):137-182
Ronald Inglehart, Miguel Basanez, Jaime Diez- Medrano, Lock Halman, and Ruud Luijkx. 2000. World values surveys and european values surveys, 1981-1984, 1990-1993, and 1995-1999 1997. Ann Arbor- Michigan, Institute for Social Research, ICPSR ver- sion.
Shalom H Schwartz. 1999, A theory of cultural values and some implications for work. Applied psychol- ogy, 48(1):23-47.
Shalom H Schwartz. 2012.
overview of the schwartz theory of basic values. Online readings in Psychology and Culture, 2( 2(1):2307-0919
Laura Lee Junker. 1990. The organization of intra- regional and long-distance trade in prehispanic philippine complex societies. Asian Perspectives. 1902):167-209.
Shalom H Schwartz and Wolfgang Bilsky. 1987. To- ward universal psychological structure of human values. Journal of personality and social psychol- ogy, 53(3)5550.
Molly Lewis, Matt Cooper Borkenhagen, Ellen Con- verse, Gary Lupyan, and Mark S Seidenberg. 2022. What might books be teaching young children about gender? Psychological science, 33(1):33-47.
Shalom H Schwartz, Jan Cieciuch, Michele Vecchione, Eldad Davidov, Ronald Fischer, Constanze Beier- lein, Alice Ramos, Markku Verkasalo, Jan-Erik Lön- nqvist, Kursad Demirutku, et al. 2012. Refining the
BY Louie. 2001. Why gender stereotypes still persist in contemporary childrens literature.

---

theory of basic individual values. Journal of person ality and social psychology, 103(():663.
Stacy Smith, Mare Choueiti, Katherine Pieper, Traci Gillig, Carmen Lee, and Dylan DeLuca. 2015. In- equality in 700 popular films: Examining portrayals of gender, race, & lgbt status from 2007 to 2014.
Adiroanes:
Adhis
Askeramy
Carer
Cauaasity
Dedeaioon
Ervin Staub. 2013. Development and maintenance of prosocial behavior: International perspectives positive morality, volume 31. Springer Science Business Media.
Feraning
Geattzde
Hart-work
timll
Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021. On gener- alization in coreference resolution. In Proceedings of the Fourth Workshop Computational Models of Reference, Anaphora and Coreference, pages 111- 120, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Hannmy
Iraan-puaee
-ucce
Life
Mtarase
Crder
Parmas
hunase
Iodateedipp
soligaus
Joan C Tronto. 1995. Women and caring: What can feminists learn about morality from caring? 11999. Justice and care, pages 101-116. Routledge.
Kesporable
Searty
Selfoomisleeee
Sodal
Sedee
Edward Theodore Chalmers Werner. 2020. Chinese Mythology. e-artnow.
Takknng
trah
Mb
heerss
Booll
Cave
Chts
Gomay
Heggie motes
Intast
Iily
Hilpetee
Rigad
selo
Spot
Ella Westland. 1993, Cinderella in the classroom. chil- dren's 's responses gender roles in fairy-tales, Gen- Education, 5(3):237-249.
Figure 6: Complete listing of human values in fairy tales across cultures. The darker the color, the more prevalent that value is expressed that culture.
Steven R Wilson, Yiting Shen, and Rada Mihalcea. 2018. Building and validating hicrarchical lexicons with a case study on personal values. Interna- tional conference on social informatics, pages 455- 470. Springer.
Yida Zhai. 2017. Values of deference to authority in japan and china. International Journal of Comparu- live Sociology, 1802):120-139
Social
Sociity
Relationsips
Zhixuan Zhou, Jiao Sun, Jiaxin Pei, Nanyun Peng, and Jinjun Xiong. 2022. A moral-and event-centric in- spection of gender bias in fairy tales at a large scale. arXiv preprint arXiv:2211. 143558.
words
Helping- other
Family
30
20
Jack Zipes. 2012. Fairy tales and the art of subversion. Routledge.
10
10
Folk Tiles
Novels Pulk Tales
Novels
USA
England
A Appendix
Figure 7: Percentage of value words representing the top values. A value of c.g. 40 indicates that 40% of value-expressing words express that particular value. Note that these values may add to more than 100% because word can associated with more than sin- gle value. Error bars denote standard error.
The appendix contains figures that were not able to fit in the main content. Figure 6 shows heatmap of values expressed across all cultures. Figure presents the top five expressed values in novels and folk tales. Figure shows statistically sig- nificant differences in Moral word expression be- tween cultures.

---

Moral
Culturel
Culture2
Ustatisti
Harm
China
Hungary
4.23
0.026
China
Canada
Horm
Denmark
Germany
4,59
<0.001
Harm
Denmark
Greece
4.78
<0.001
Harm
Denmark
Himalayas
4.72
<0.001
Harm
Denmark
Hungary
-6.21
<0.001
Ireland
Harm
Denmark
Italy
5.29
<0.001
Denmark
Japan
<0.001
Harm
Denmark
Philiipiiees
4.59
<0.001
Harm
Denmark
Romania
5.05
<0.001
Harm
Denmark
Australia
5.66
<0.001
Harm
Denmark
Canada
-7.88
<0.001
Germany
Norway
Harm
Greece
Norway
4.48
<0.001
Harm
Himalayas
Norway
<0.001
Harm
Hungary
Norway
<0.001
Harm
India
Norway
4,01
0.042
Harm
Ireland
Norway
4.19
0.022
Horm
Italy
Norway
5.14
<0.001
Japan
Norway
Harm
Norway
Philippines
-4.65
<0.001
Harm
Norway
Romania
-483
<0.001
Harm
Narway
Australia
55.81
<0.001
Harm
Norway
Canada
-7.71
<0.001
Harm
Russin
Canada
4.10
0.037
Harm
Scoeland
Canada
-.27
0.025
Canada
Chesting
China
Spain
-448
<0.001
Cheating
Denmark
Spain
4.63
<0.001
Cheating
England
Spain
-5.58
<0.001
Cheating
England
Turkey
-4.62
<0.001
Cheating
England
Canada
Cheating
Russia
Spain
-456
<0.001
Betrayal
England
Iceland
0.011
Authority
China
Denmark
8.00
<0.001
Authority
China
England
5.57
<0.001
Authority
China
France
0.012
Authority
China
Germany
4.77
<0.001
Auilority
Authority
China
Ireland
4.91
<0.001
Authority
China
<0.001
Authority
China
Norway
7.13
<0.001
Auilority
China
Russia
4.83
<0.001
Authority
Authority
China
Wales
4.11
0.039
Authority
China
Canada
0.024
Authority
China
Brazil
4,26
0.022
Authority
Denmark
Spain
-.16
0.024
Sabversion
Germany
Africa
4.78
<0.001
Sanctity
China
England
4.71
<0.001
Sanctity
Sanctiyy
China
Ialy
4.79
<0.001
Sanctity
China
Norway
<0.001
Figure 8: Statistically significant differences (p 0.05
after Bonferroni correction) cultures by expression of Moral Foundations in folk tales.


Reviews:
[{"review_id": "8MBKUINB6s", "reviewer": "Reviewer_bzwS", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive review of our work. We are glad you found our findings interesting.\n\nTo address your comments:\n\nWe study human values, morals, and biases because they are closely intertwined, and they influence one another. In addition, in our work, we found that these three aspects can be analyzed using similar methods.\n\nThank you for the Zhou et al. (2022) reference. Indeed, they found similar results, although on a significantly smaller set of seven cultures (as compared to the 27 diverse cultures included in our study). We also note that this paper has not been peer-reviewed, but we will include it in the camera-ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The computational approach to understanding folk tales is novel, compared to the large body of qualitative and literature research.\n2. The dataset is carefully collected and valuable for future research.\n3. The findings are interesting.", "reasons_to_reject": "1. There's a lack of disucssion regarding why Human Values, Morals, and Biases are discussed at the same time.\n2. There's a lack of engagement with prior literature in computationally understanding bias or values in literature. For example, [1] is a paper on fairy tales and has some similar findings about gender bias and moral foundations: \"female characters turn out more associated with care-, loyalty- and sanctity- related moral words, while male characters are more associated with fairness- and authority- related moral words.\" Engaging with such literature will further extend the paper's contribution to computationally understanding values and biases in literature.\n\n[1] Zhou, Z., Sun, J., Pei, J., Peng, N., & Xiong, J. (2022). A Moral-and Event-Centric Inspection of Gender Bias in Fairy Tales at A Large Scale. arXiv preprint arXiv:2211.14358.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper compiles a dataset of folk tales and conducts a cross-cultural analysis of human values, morals, and biases. The computationally-aided linguistic analysis of folk tales is novel. The dataset can facilitate more reseach into this important research topic.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jlafz8464O", "reviewer": "Reviewer_irgT", "summary": "", "questions": "Did you conduct qualitative analyses to assess whether the scores assigned using the lexicon-based approaches are in line with your own estimations?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive comments on our work! We are glad you found our analyses extensive and rigorous, and the results well presented.\n\nTo address your comments: \n\nMethods: The novelty of our paper lies in the large cross-cultural dataset and the insights drawn from applications of the methods across cultures, rather than in the methods themselves.\n\nHuman annotations: The lexicons we employ have been validated against human annotations in prior work. For example, the values lexicon was validated against human-labeled word intrusion and category matching tasks, with up to 0.72 accuracy [1], and Moral Foundations Dictionary has been used in a classifier and tested against hand-annotated Twitter posts with 0.66 F1 score [2]. We will include a discussion of these points in the camera ready.\n\n[1] Wilson et al. (2018). Building and Validating Hierarchical Lexicons with a Case Study on Personal Values\n[2] Hoover et al. (2020). Moral foundations twitter corpus: A collection of 35k tweets annotated for moral sentiment\n\nPresentation: Thank you for the presentation suggestions. With the extra page in the camera ready, we can move Table 10 into the main content.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* This paper provides an extensive and rigorous analysis of a novel corpus of 1900 folk tales across 27 cultures. \n* The introduced corpus, as well as the reported results, address a gap in the literature and are as such likely interesting to researchers working in this area. \n* The paper is well-written, and the results are overall well-presented. Statistical testing is used to provide assessments of statistical significance across comparisons. \n", "reasons_to_reject": "* The methods used in this paper have little to no technical novelty, and all experiments are based on lexicon-based approaches to measure textual characteristics. While this shows to provide interesting results, these methods have their limitations, and contextualized representations of text might be useful to better estimate concepts such as human values, morals, and gender biases. \n* Related to the point above, it would have been interesting to see how human annotations correlate with the lexicon-based approach. The concepts measured in this work are complex, and a small-scale experiment to assess how well the lexicon-based approach measures them (based on how humans would evaluate these) would have strengthened the paper.\n", "questions_to_authors": "Did you conduct qualitative analyses to assess whether the scores assigned using the lexicon-based approaches are in line with your own estimations?", "paper_topic_and_main_contributions": "This paper presents a dataset of 1925 folk tales representing 27 cultures and provides an analysis of human values, morals, and gender biases in individual tales across cultures. The authors argue that no such corpus currently exists, since existing works mainly focus on European tales.  \nFor human values, the analysis consists of a lexicon-based approach using the Values Lexicon, which contains keywords for 49 human values. Experimental results show that the most common values expressed across cultures are social values, but that different cultures have tendencies to different human values. Moreover, the authors compare the occurrence of such value-representing words between novels (sourced from Project Gutenberg) and the folk tales, finding that folk tales have a stronger representation of such words. \nFor morals, the authors also employ a lexicon-based approach, using the Moral Foundations Dictionary 2.0 which contains keywords for 10 moral foundations. Here, it can be seen that positive aspects of morality are more strongly represented than negative ones and that morality is expressed differently across cultures. \nFor gender biases, the authors first identify the genders of characters in the tales, showing that male characters are overrepresented across cultures (2.29 times as many male characters as female ones) and that characters for both genders are associated with traditional values (e.g., children and family for female, career, wealth for male). \n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Justification_for_Ethical_Concerns": "N/A"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* Line 285: one “are” too much in “…are they are…”\n* For Figures 2 and 4, it would be helpful to see (normalized) numbers for each box, instead of only relying on the color intensity.\n* I recommend not to refer to Tables in the Appendix from the main manuscript (e.g., Table 10) if they are crucial to understanding the results described in the text. The Appendix should serve as a space for supplementary material that is not essential to understand the main findings presented in the manuscript. \n"}}, {"review_id": "5k9kqOmq7Y", "reviewer": "Reviewer_Vntz", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive review of our work. We are glad you found our analyses to be extensive and the findings insightful.\n\nTo address your comments:\n\nCultures: We agree that Africa and the Middle East (and South America, Southeast Asia, and many more) are underrepresented in our dataset. Unfortunately, we are limited by our source: Project Gutenberg has very few books of tales from these regions. We will consider adding another book, South African Folk Tales, to the dataset.\n\nAccording to the introduction of West African Folk Tales, the stories are folklore of the “Gold Coast peoples”, which is modern-day Ghana (when this book was written, this area consisted of several territories rather than a single country). \n\nIncluding cultures with <10 tales: We actually considered this when performing our analyses but decided that the analyses would not be representative with such a small sample size. One possibility could be to include two versions of the dataset, one with cultures with >=10 tales, and another with all tales, and we will consider this for the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well written and the analyses performed by the authors are extensive and insightful. The work is the first of its kind to perform such a cross-cultural study across a wide range of countries, and the dataset created will be a valuable resource for future studies. The results also revealed multiple interesting insights, in particular related to what moral foundations are most reflected within the tales of each culture, while the gender bias analyses were less surprising.", "reasons_to_reject": "The following are few concerns that the authors may be able to address:\n\n\nAfrica is considered as one culture, whereas the rest of the cultures considered are based on specific countries. The resource used for those tales states “West African Folk-Tales”, but it is not clear what countries are included in it, which could be problematic since, for example, Morocco and Ghana are completely different cultures. \n\n\nDespite the paper mentioning that most previous work is focused on Anglo-American and European cultures, European countries are over-represented in the dataset (19 countries), compared to only 1 country (Brazil) from Latin America, 1 combined African culture, and no representation of Middle Eastern countries. I believe more representation of cultures from those areas would have allowed for a stronger cross-cultural study, although it is understandable that data may not be available. However, a suggestion would be the inclusion of tales from such cultures even if the number of tales available is less than 10.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a cross-cultural study of value, morals, and gender biases in folk stories across a big number of countries. The authors perform analyses using English lexicon datasets for value/moral analyses and tools for dependency parsing and coreference resolution for gender bias analyses. To carry out the study, a dataset of English-translated folk tales from multiple sources was collected. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper compiles a large corpus of over 1,900 tales from 27 diverse cultures across six continents, and conducts language and statistical analyses to study how human values, morals, and gender biases are expressed in folk tales across cultures. Three reviewers reviewed the submission, highlighted the strengths of the paper in terms of novelty (R1, R3), data contributions (R1, R2), presentation and readability (R2, R3), and findings (R1, R2, R3). During the rebuttal phase, the authors also responded that they would address the reviewers’ concerns through engagement with prior literature and additional analyses.
Decision: Accept-Main