Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models
Authors: Lina Conti
Guillaume Wisniewski
Venue: EMNLP 2023 Main (2023)
Keywords: Language Model
Transformer
LM Analysis
Gender Bias
TL;DR: by learning language models on artificial corpora, highlight the conditions under which linguistic properties such as gender can emerge; our experiments show in particular that gender bias can appear even in perfectly gender-balanced corpora

Abstract: Numerous studies have demonstrated the ability of neural language models to learn various linguistic properties without direct supervision. This work takes an initial step towards exploring the less researched topic of how neural models discover linguistic properties of words, such as gender, as well as the rules governing their usage. We propose to use an artificial corpus generated by a PCFG based on French to precisely control the gender distribution in the training data and determine under which conditions a model correctly captures gender information or, on the contrary, appears gender-biased.

Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models
Lina Conti se 4,6 and Guillaume Wisniewski
Sp Fondazione Bruno Kessler, Italy
University of Trento, Italy
Université Paris Cité, LLF, CNRS, 75313 Paris, France Ivarel @fbk. eu, gui guillaume. wisni wwskiwwskiee wwsiiiwwwkkiaaaaaee -arriss..
Abstract
NLM's ability to capture gender information. By answering these two questions, we hope to enhance our comprehension of how an NLM determines a word's gender (or other linguistic properties) and gain insights into the conditions that give rise to a biased representation of gender in these models.
Numerous studies have demonstrated the ability neural language models to learn various linguistic properties without direct supervision. This work takes an initial step towards exploring the less researched topic of how neural models discover linguistic properties of words, such as gender, as well as the rules governing their usage. We propose by PCFG based on French to precisely control the gender distribution determine under which conditions model correctly captures gender information or, on the contrary, appears biased.
To achieve this, we propose to use artificial corpora generated by PCFGs ($2). This approach allows us to have complete control over the gender of nouns and the contexts in which they are presented. Following the methodology from Kim and Linzen (2020), we evaluate the NLM's capacities on a test set specifically designed to contain examples that differ from those of the training set based on well-defined criteria. By precisely manipulating the contexts in which words appear when training and testing, we can establish causal relations between changes in these contexts and the NLM's ability to assign gender.
artificial corpus generated
training data and
1 Introduction
Numerous studies have demonstrated NLMs' ability to capture linguistic properties, such as grammatical categories and syntactic rules like number and gender agreement, without any direct supervision (Belinkov et al., 2020; Manning al., 2020). However, they leave unanswered the fundamental question of how NLMs uncover these properties from raw textual data alone. This study aims to address this question by focusing on the specific case of gender in French. Our motivation for working gender is twofold. Firstly, the presence of gender bias in NLMs raises significant societal concerns (Gaucher al., 201 I; Cao and Daumé III, 2020) and understanding how these models capture and use gender information is a crucial step towards addressing such biases. Secondly, the grammatical expression of gender in languages such as French makes it possible to construct testbeds to explore how this linguistic infor- mation can emerge in an unsupervised way in LMs.
While the scope of this study is limited to examin- ing the impact of certain training data characteristics NLMs' gender learning, our aspiration is that it serves important step in larger process towards understanding and rectifying biased behaviour in NLMs. The subsequent phase entails evaluating how alterations in the training corpus and therefore on how gender is learnt manifest in the text generated by these models. Once a causal relation- ship is established between potential factors influ- encing gender learning and the behaviour of NLMs, the objective is to use this knowledge to mitigate bias in "real" language models trained on natural data by specifically targeting the factors identified as playing an important role in our controlled experiments.
Contrary to prior research, our results ($3) show that transformer language models do not necessarily amplify gender imbalances present in the data. More- over, observe the presence of gender bias even in cases where the training data is perfectly balanced.
This work addresses two questions: i) whether the gender that should be assigned to word is dynamically
ingrained in its representation computed based on the linguistic context and ii) which characteristics of the training set, such as word frequency or gender imbalance, affect an
'Code available at: lina-come contoconii//ee conti//rtificiall data gender.
https: htt.g/gtttuut..eee/e//eeee..e com/

---

2 Observing the Emergence of Gender Information in NLMs
involves three levels of rules. The first level determines whether the noun occurs in context where gender information is revealed by determiners and adjectives or in a gender-ambiguee context:
To investigate how NLMs discover gender infor- mation, we propose to train models on controlled sets where gender expression is precisely regulated. We then use a linguistic probe (Belinkov, 2022), to assess the model's ability to learn this property in various scenarios. Following current line of re- search (Kim and Linzen, 2020; White and Cotterell, 2021), we use artificial language corpora generated by PCFGs to have full control over the expression of gender. This control encompasses, for example, the frequency of each gender and the informativeness of word occurrences' context with regard to gender.
NP NP gendered
NP ambiguuus
(1)
The second level of rules, following a similar structure, describes the constituents of each of these contexts. Mcanwhile, the third level establishes the lexical rules that associate terminal symbols (words) to respective word categories. For example:
NP Pmmbiguous + DETepic (ADlepie |t) NOUN DETopic + chaque leur ...
(2)
Through the manipulation of rules and their prob- abilities, can determine the gender of nouns and exert precise control over the gender-related infor- mation accessible the model during training (e.g., by ensuring that some nouns only appear in gender- neutral contexts). We can then use the following experimental pipeline to assess the LM's capacity to assign gender to noums in different scenarios: 1. we generate two train sets (1m_ train and probe_train) train) and test set (probe_test) using slightly different PCFGs;
Our PCFGs aim mimic subset of the French language focusing on the expression of gender within noun phrases (NPs). While some prior studies in the literature (Kim and Linzen, 2020) aim to maintain semantic coherence in their artificial data, others (Gulordava et al., 2018) leverage the absence of such coherence to disentangle syntax from semantics. In this work, we chose to focus solely on the grammatical aspect of gender to eliminate potential confounding factors introduced by semantics, such as stereotypes propagated by the distributional environment of words. This deliberate decision narrows down the potential sources of bias under investigation, at least for the present. When constructing the syntax of our language, we also prioritise simplicity over realism, particularly concerning elements unrelated to the specific aspects we investigate in relation to gender learning.
2. we train a LM on 1m_tmmaaaa ttaar;i;
using probe probe_train, we train a linguistic probe (Belinkov, 2022) to predict the gender (masculine or feminine) to be assigned to noun occurrences based on their contextualised representation at the final layer of the LM; 4. we evaluate the probe's performance on probe_test.
Our main motivation for selecting French as the basis for our PCFGs stems from the opportunity it provides to construct contexts that either mark gender or not. French has two grammatical genders: feminine and masculine. Nouns, determiners, adjectives and pronouns can have fixed genders or be epicene, meaning they can denote individuals of any gender. In this case, the gender must be inferred from the context. For instance, when the epicene noun "artiste" is accompanied by the feminine determiner "une", feminine gender can be inferred. However, when used with an epicene determiner as in "chaque artiste," the gender becomes ambiguous since the context provides no information about it. The noun phrase generation in the PCFGs
Hewitt and Liang (2019) highlight that by relying solely on probe accuracy we cannot tell whether a high score means the information is encoded in the representations or whether the probe has just learnt the task. In order to avoid this problem and ensure that a high probe accuracy indicates the encoding of gender information within the language model's 's representations, nouns present in probe_trail ttaain are excluded from probe_tesss _etst.
In all our experiments, we used the transformer LM from White and Cotterell (2021) (see ŞB). We generated 10, 000 sentences for lm_train 1,000 for probe_ probe_train train and 1,000 x n for probe_ test where represents the number of distinct groups considered in our analyses.
PPropsslls exist for gender-neutral morphosyntactic alter- natives in French, such as the neo- pronoun iel. However, since as of now their adoption remains limited, we align our PCFGs with standard French, which recognises only two genders.
All aspects of French syntax relevant to estab- lishing PCFGs, and those regarding gender Forr clarity, we omit rule probabilities.

---

in particular, are very similar to other romance languages, such as Spanish, Portuguese and Italian. Changing the lexical rules would be enough to make the PCFGs generate sentences emulating another one of those languages. Since such a PCFG would be equivalent from the point of view of the NLM, the results of our experiments can be generalised to these other languages. However, different PCFGs would be necessary to study how NLMs learn gender in lan- guages with vastly different gender systems, such as languages with more than two genders or languages that have no gender system (Corbett, 2013). The ex- ploration of this topic is deferred to future research.
that exclusively appear in ambiguous contexts, the model's performance is closer to chance level.
Our observations reveal potential cause of bias emergence: when a word is consistently associated with specific gender in the training set, that gender becomes ingrained in its representation. It may then be challenging to revise this association when con- textual cues contradict the learnt gender. This leads to language errors when a model relies on stereo- types rather than context, for example, assigning "secretary" a female gender even when the syntax of the sentence indicates that the word refers to mas- culine entity (Zhao et al., 2018). The presence of this form of gender bias in natural language models that deployed in industry leads to representational harms (Crawford, 2017), by perpetuating stereo- types associated to each gender and contributing to confine individuals in restrictive gender roles.
3 Experimental Results
Decoupling contextual gender information from static gender associations Our first experiment aims to assess whether NLMs can use contextual information during inference to find the gender of words that occur only in ambiguous contexts during training, thus assessing whether the model's repre- sentation of gender relies on information memorised from training data is dynamically constructed during inference based on linguistic cues.
Assignment of gender to epicene nouns To gain deeper insights into the relationship between gender encountered during training and gender indicated in the context, we incorporated epicene nouns into our PCFG. This allowed us to investigate the impact of confficting genders between the model's 's training data and the context during inference.
Training context
Inference context
More specifically, we constructed a PCFG (Ap- pendix A.2) that includes five noun categories: fem- inine nouns, masculine nouns and three categories of epicene nouns. Each of the five noun categories appears in both gendered and ambiguous contexts in 1m train. When they appear in gendered contexts, one epicene category has an equal likelihood of be- ing associated with either gender; another has a 25% chance of appearing in feminine context and a 75% chance of appearing in a masculine one; the third category has the opposite probabilities. We eval- uated the NLM's ability to identify the gender of the referent of nouns in these five categories using the same methodology in the previous experiment.
Ambiguous
Gendered 95.84 +0.20
Ambiguous
50.37 40.49
Gendered
99.26
99.99
Table I: Accuracy (in %) when predicting the gender of words according to whether they occurred in gendered ambiguous contexts in 1m_ train and grobe_testt,
More precisely, we used a PCFG (Appendix A.1) where all nouns are either masculine or feminine and consistently appear in either gendered or non-gendered contexts within a given dataset. Table 1 reveals that gender identification is nearly flawless for words encountered in gendered contexts in 1m train. Even when these words appear in ambiguous contexts in probe_tess the model's identification of gender remains highly accurate. This suggests that gender information is inherently encoded within the model's representations, reflecting the associations it has learned during the training process.
As reported in Figure 1, the model demonstrates remarkable accuracy determining the gender of the noun's referent based on contextual cues when it appears in gendered context. It consistently assigns a feminine gender when the noun appears in a feminine context, achieving an accuracy rate close 100%, and hardly ever assigns femininc gender when the noun is in a masculine context, maintaining an assignment rate close to 0%. This holds true for all categories, even when the noun consistently appeared in the opposite gender context during training. However, in ambiguous
The model exhibits reasonable ability to infer gender solely from context for words whose gender was not revealed in 1m train. This highlights the model's 's capacity to extract gender information from the sentence structure. As expected, for words

---

Percentage of feminine probe preditions
100
Noun frequency
Inference context
Ambiguous
Gendered
very
#0.48
98,40 +: 0.14 98.08 = 0.14
75
#0.45 74.80 #0.45
frequent
76.25 #0.42
98.13
context feminine ambiguous
very frequent
77.30
98.19 0.11
Table Probe accuracy (%) by noun's frequency 1m train and context in probe_ test.
25
frequency, indicating that assigning gender to words that have been encountered more frequently in the training data is relatively easier.
100 Percentage of feminine occurrences in the training data
25
50
75
The difference in accuracy between very infre- quent and very frequent nouns may not appear substantial, with a mean accuracy of 73.9% for very rare nouns compared to 77.3% for very frequent nouns. However, the absence of overlapping con- fidence intervals among the quartiles indicates that the influence of noun frequency on the occurrence of gender assignment errors statistically significant."
Figure 1: Percentage of feminine probe predictions for epicene nouns by percentage of feminine occurrences of those nouns in 1m_ train and context in which they appear in probe_tess test.
contexts, the model tends to assign the noun its most frequently associated gender in 1m train. Unlike the observations by Stanczak and Augen- stein (2021) for models trained natural language, model does not appear to amplify imbalances present in the training data. For instance, when epicene words were encountered in feminine context 75% the time during training, the model signs a feminine gender to referent only 60.93% #0.48 of the time in an ambiguous context, rather than maintaining 75% or higher assignment rate.
Accuracies for gendered contexts are more uniform across all quartiles, suggesting that word frequency plays comparatively minor role. This observation suggests that when gender information is readily available in the context during inference, the impact of frequencies in the training corpus becomes less critical.
Influence of the gender distribution on default gender guessing now now turn to the impact of the imbalance in gender distribution in the training data, which is often cited as the reason for the overuse of masculine gender by NLMs. The disproportionally low representation the feminine gender constitutes form of bias referred to as reder-repressntatioon, which results the marginalisation and invisibility of women (Crawford, 2017). To assess the extent to which this over-assignment of masculine gender can be attributed to gender frequency imbalances in the training corpus, we conducted an experiment using five different PCFGs. These PCFGs assigned vary- ing probabilities to feminine and masculine noun phrases, allowing to generate training sets that ranged from perfectly gender-balanced to exhibiting strong bias towards one gender. We then assessed the frequency with which models trained on data gen- crated by each PCFG assigned each gender to nouns
This observation indicates that the inability of LMs assign alternative genders epicene words, such as occupational nouns, which have been extensively studied to uncover gender bias NLMs, cannot be solely attributed to a gender frequency imbalance in their occurrences in the training data.
Impact of word frequency We now explore the influence of word frequency on the ability of NLMs to capture gender information and examine whether models more prone to making errors words that occur less frequently in the training data. To investigate this, we categorised the nouns in prope_rteete test based on their probability of being generated 1m 1m_trrre_e train: the 25% of nouns with the lowest probability of being generated were classified as "very rare", the subsequent 25% of nouns as "rare", and so on.
Table 2 presents the average probe accuracy for nouns in each quartile based on whether they appeared in gendered or ambiguous context the test set. The performance the probe in ambiguous contexts exhibits a positive correlation with word
"Given that our work is based on artificial data, we cannot make conclusive assertions regarding the magnitude of this effect for models trained on natural data. Nevertheless, employing artificial data enables us to identify the factors that can play substantial role in gender assignment.

---

for which they had explicit gender information." In an unbiased model, we would expect the probe to predict approximately 50% of nouns to be of each gender. However, if models exhibited a bias towards the majority class the probe would more frequently predict the gender that was more prevalent in the train set. We quantify the degree of bias as follows:
initially thought. It raises the question of whether training models on balanced data, which has been explored as a strategy to mitigate gender bias, is sufficient, as there are likely other factors at play.
4 Conclusions
In this study, we artificial language corpora to gain deeper insights into how NLMs acquire linguistic knowledge from unannotated texts. Our experiments shed light on the complex factors influencing the emergence of gender (as an illustra- tive linguistic property) in NLMs. It is crucial to underscore that, being derived from artificial data, our observations offer insights into the potential mechanisms involved in learning gender, but they do not provide definitive proof that these mechanisms are the core factors driving NLMs ability acquire and use linguistic knowledge when working with nat- ural language data. However, our findings on NLMs' learning process can suggest future directions for bias mitigation. For instance, our results challenge the assumption that training data imbalance is the sole cause of gender bias in NLMs and thus shed doubt on the idea that training or fine-tuning gender-balanced corpora would be enough to avoid gender bias. They also show that an imbalanced gender distribution in the training data does not nec- essarily lead a transformer based LM to wer-assign the majority class gender. This is cause for optimism as it suggests that transformers' learning process is not inherently biased or irremediably predisposed to reproducing the most frequent patterns. Our future work will concentrate generalising our observa- tions to models trained on natural language data.
x number of majority class predictions
total number of predictions
A degree of bias of 0 indicates that the probe predicted 50% of nouns for cach gender, while a value of (resp. -1) suggests a strong bias toward the majority (resp. minority) class.
1.00
0.75
Degine
0.00
-0.25
-0.50
-1.00
50%
40%
30%
20%
10%
Proportion of feminine nouns in the training data
Figure 2: Degree bias exhibited by models trained on сотрoга with different gender distributions,
Figure 2 displays scatter plot illustrating the degree of bias observed in models trained on different corpora. Each corpus contained varying proportions of feminine noun phrases, ranging from 50% down 10%, with the remaining noun phrases being masculine. For each gender distribution, twenty models were trained.
A Acknowledgments
We sincerely thank the reviewers for their careful reviews and insightful comments, which were of great help in improving the manuscript. We would also like gratefully acknowledge support from the Labex EFL (ANR-10-LABX-008))
The degree bias observed in models can vary significantly even within single gender distribution. Models may exhibit bias towards the majority class, the minority class, or any point in between. Surpris- ingly, the gender distribution in the models' training data does not seem to have substantial impact on the degree of bias. One possibility explore in future research whether the examples seen at the beginning training have a disproportionate influ- ence on the likelihood that the model attributes to each gender. These findings suggest that the origin of bias in language models is more complex than
Limitations
The first limitation of our work concerns the use of probes. While we have carefully distinguished between information learnt from the probe's training process and information inherent to the model's representations as advocated by Hewitt and Liang (2019), it is worth noting that using probes introduces an additional system to examine gender, rather than directly studying the behaviour of LMs.
"These nouns exclusively appeared in gender-ambiguous contexts in hoth 1m, trairin and probe_ probe_test test and were not included in prope_traaaeee train.

---

In future research, it would be valuable explore more intricate PCFGs that could support the corefer- ence resolution challenges usually used to highlight gender bias in NLMs, allowing us to compare gender assignment measurements obtained through this method with the results obtained using probes.
Kate Crawford. 2017. The trouble with bias. Conference on Neural Information Processing Systems, invited speaker.
Danielle Gaucher, Justin Friesen, and Aaron C Kay. 201 I. Evidence that gendered wording in job advertisements exists and sustains gender inequality. Journal of personality and social psychology, 101(1):109.
In addition our previous considerations, the extent to which our findings can be extrapolated to "real" models trained on natural language also war- rants further investigation. Current state-of-the-art LMs are significantly larger than the models used in our study and exhibit emergent capacities resulting from their increased size and training data (Manning et al., 2020; Wei et al., 2022). Therefore, it would be relevant to examine how model and dataset sizes impact our experiments. Additionally, comparing the variability of gender expression and the gender distributions in our data those observed in natural language would offer valuable insights. By further exploring these points, we could gain a better understanding of the robustness and applicability of our findings to real-world language models.
Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. Colorless green recurrent networks dream bierarchically. In Proceed- ings of the 2018 Conference of the North American the Chapter the Association for Computational Linguis- ties: Human Language Technologies, Volume (Long Pupers), pages 1195-1205, New Orleans, Louisiana. Association for Computational Linguistics.
John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJCLP)), pages 2733-2743, Hong Kong, China. Association for Computational Linguistics.
Najoung Kim and Tal Linzen. 2020. COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 9087-9105, Online. Association for Computational Linguistics.
It should also be kept in mind that while all the results described in this work are based on the analysis of an artificial corpus of data, the design of our PCFGs draws inspiration from the expression of gender in French. The expression of gender varies significantly across languages and exploring the abil- ity of systems to learn more complex gender systems would be an interesting avenue for future research.
Christopher D Manning, Kevin Clark, John Hewitt, Ur- vashi Khandelwal, and Omer Levy. 2020. Emergent linguistic structure in artificial neural networks trained by self-supervision. Proceedings of the National Academy of Sciences, 117748):00446-00044
A final limitation of this work is the consideration of binary gender system, which will be revisited in Section 6.
Karolina Stanczak and Isabelle Augenstein. 2021. A survey on gender bias in natural language processing. arXiv preprint arXiv:21
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Iliia Polosukhin. 2017. Attention is all you need, Advances in neural information processing systems, 30.
References
Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 2801)2007-2..
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph. Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models. Transactions on Machine Learning Research. Survey Certification.
Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick, 2020. Interpretability and analysis in neural NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tatorial Abstracts, pages 1-5, Online. Association for Computational Linguistics.
Yang Trista Cao and Hal Daumé III. 2020. Toward gender-inclusive coreference resolution. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages Association for Computational Linguistics.
Jennifer C. White and Ryan Cotterell. 2021. Examining the inductive bias of neural language models with artificial languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1thh International Joint Confer- ence on Natural Language Processing (Volume I: Long Papers), pages 454-463, Online, Association for Computational Linguistics.
Online.
Greville Corbett. 2013, The expression of gender. De Gruyter.

---

Tatu Ylonen. 2022.
Wiktextract: Wiktionary as machine reatable structured data. Proceedings of the Thirteenth I Language Resources and Evaluation Conference, pages 1317-1325, Marseille, France. European Language Resources Association.
are tokenised into words, we ensure that the lexical rules generate 1m lm_dev, probe_ probe_train train and probe_ test only contain words that were present in 1m_ train. This approach ensures that we avoid incorporating tittof-vocabblayy words in the sets used for inference.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias coreference resolution: Evaluation and debiasing methods. In Proceedings the 2018 Conference of the North Chapter the A Association for Compu- tational Linguistics: Human Language Technologies, Volume 2 (Shorl Papers), pages 15-20, New Orleans, Louisiana. Association for Computational Linguistics.
A.1 PCFG for the first experiment
In this section, report the PCFGs used in the experiment "Decoupling contextual gender information from static gender associations." We categorise feminine and masculine nouns into four groups based on their appearance in gendered (G) or gender-ambiguous (A) contexts within 1m train and whether they are present in gendered context (G) in probe_ ttaaeetrannnn remain unscen (U) in this set. The results presented in Table concern exclusively nouns in the latter category. Thus, any gender information detected by the linguistic probe at inference time encoded the LM's representation and not merely memorised by the probe.
6 Ethical Issues
One ethical issue central to our work concerns the treatment gender as a binary variable. While this aligns with the grammatical conventions of standard French, it falls short from a social perspective as it disregards individuals who do not identify within this binary framework. By imposing binary choice, assumptions about gender need be made, which can potentially result in inaccuracies, the strategy masculine default adopted, further marginal- ising genders other than the masculine. Proposed solutions exist to make French more gender- inclusive, and it is crucial to contemplate their integration into NLP systems. Using artificial data, have done, provides opportunity to explore the integration of gender-ing sender-inclusive inclusive language in NLP, even in the absence of natural data, which is currently an obstacle. Therefore, artificial languages can serve as an initial step in determining the requirements for gender-inclusive language models, conducting experiments, for instance, to ascertain the number of gender-neutral language instances necessary for the model to comprehend non-binary gender.
The following PCFG is used to generate 1m train.
NP VP " [1.0]
PP -> PREP NP [1.0]
VP -> VERB [0.5] VERB NP [0.5]
NP -> NPGend [0.4] NPAmb [0.4]
NP -> NP PP [0.2]
NPGend -> DETFem npGendFem [0.5]
NPGend -> DETMasC npGendMasc [0.5]
npGendFem -> NOUNFemGend [0.4]
npGendFem -> ADJFem NOUNFemGend [0.3]
npGendFem -> NOUNFemGend ADJFem [0.3]
A PCFGs to model gender in French
npGendMasc -> NOUNMascGend [0.4]
Below, we present the complete set of non- non-terminal rules used in our experiments. To construct the terminal symbols, we draw from a vocabulary extracted from the Universal Dependencies project and from Wiktionary using the wiktextract library (Ylonen, 2022). For generating lexical rules, we randomly select 400 nouns, 300 adjectives, 20 verbs, prepositions and 15 determiners from our vocabulary. These lexical items are then assigned to respective non-terminal symbols with probabilities following a Zipfian distribution. Since our sentences
npGendMasc -> ADJMasc NOUNMascGend [0.3]
npGendMasc NOUNMascGend ADJMasc [0.3]
NPAmb DETEpic npAmbFem [0.5]
NPAmb -> DETEpic npAmbMasc [0.5]
npAmbFem em -> NOUNFemAmb [0.4]
npAmbFem -> ADJEpic NOUNFemAmt [0.3]
npAmbFem -> NOUNFemAmb ADJEpic [0.3]
npAmbMasc NOUNMasCAmb [0.4]
npAmbMasc ADJEpic NOUNMascAmb [0.3]
npAmbMasc NOUNMasCAmb ADJEpic [0.3]
"We sample the words in our PCFGs from the dataset found at httpps//gel //g1 thub. com/ /ni versal SeeaaDeeeenn Dependencies French-GSD.
NOUNFemAmb -> NOUNFemAG [0.5]
NOUNF Ambb -> NOUNFemAU [0.5]

---

NOUNMascAmb NOUNMasCAG [0.5]
npGendFem ADJFem NOUNFem [0.3]
NOUNMascAmb NOUNMaSCAU [0.5]
upGendFem NOUNFem ADJFem [0.3]
NOUNFemGend NOUNFemGG [0.5]
upGendMasc NOUNMasC [0.4]
NOUNFemGend -> NOUNFemGU [0.5]
upGendMasc -> ADJMasc NOUNMasC [0.3]
NOUNMascCend -> NOUNMascCG [0.5]
npCendMasc -> NOUMMasC ADJMasc [0.3]
NOUNMascGend -> NOUNMascGU [0.5]
If the nouns are to appear in gender-ambiguous contexts, the other hand, the rules that follow applied.
The PCFG presented below is employed for generating probe_tore train.
NP -> NPAmb [0.8] NP PP [0.20]
S -> NP VP " [[1.]]
NPAmb -> DETEpic npAmbFem [0.5]
PP PREP NP [1.0]
NPAmb -> DETEpic npAmbMasc [0.5]
VP VERB [0.5] VERB NP [0.5]
npAmbFem -> NOUNFem [0.4]
npAmbFem ADJEpic NOUNFem [0.3]
NP -> NPGend [0.8] NP PP [0.20]
npAmbFem NOUNFem ADJEpic [0.3]
npAmbMasc NOUNMasC [0.4]
NPGend -> DETFem npGendFem [0.5]
npAmbMasc ADJEpic NOUNMasc [0.3]
NPGend -> DETMaSC upGendMasc [0.5]
npAmbMasc NOUNMasC ADJEpic [0.3]
npGendFem -> NOUNFemGend [0.4]
upGendFem -> ADJFem NOUNFemGend [0.3]
A.2 PCFG the second experiment
npGendFem NOUNFemGend ADJFem [0.3]
upGendMasc NOUNMascGend [0.4]
In this section, we report the PCIGG used in the ex- periment "Assignment of gender to epicene nouns."
upGendMasc ADJMasc NOUNMascGend [o.3]
upGendMasc MOUNMascGend ADJMasc [0.3]
The following PCFG is used to generate 1m 1m_ train.
NOUNFemGend -> NOUNFemGG [0.5]
-> NP VP [1.0]
NOUNFemGend -> NOUNFemAG [0.5]
NOUNMascGend -> NOUNMasCGG [0.5]
PP -> PREP NP [1.0]
NOUNMascGend NOUNMasCAG [0.5]
VP -> VERB [0.5] 5] VERB NP [0.5]
To create probe_ trobe_test, we use variations of the following rules. Placeholder X denotes the context in which noun appears in 1m, train and can assume the values G or A. On the other hand, Y represents the context in probe_ train and can be either G or U.
NP NPGend [0.4] NPAmb [0.4]
NP PP [0]
NPAmb DETEpic NOUN [0.4]
NPAmb -> DETEpic ADJEpic NOUN [0.3]
NPAmb -> DETEpic NOUN ADJEpic [0.3]
S -> NP VP [[1.0]
NOUN -> NOUNMasC [0.35] | NOUNFem [0.35]
NOUN -> NOUN25 [o.1] NOUN50 [o. 1]
PP -> PREP NP [1.0]
NOUN NOUN75 [0.1]
VP -> VERB [0.5] I VERB NP [0.5]
NPGend -> NPFem [0.35] NPMasc [0.35]
NOUNFem NOUNFemXY [1.0]
NPGend -> NP25 [] NP50 [0.1]
NOUNMasc NOUNMascXY [1.0]
NPGend -> NP75 [0.1]
If the selected noun category is to be tested in gendered contexts during inference, the following rules are also added to the PCFG.
NPFem -> DETFen em NOUNFem [0.4]
NPFem -> DETFem ADJFem NOUNFem [0.3]
NP NPGend [0.8] NP [0.20]
NPFem DETFem NOUNFem ADJFem [0.3]
NPMasc DETMasc NOUNMasC [0.4]
NPGend DETFem npGendFem [0.5]
NPMasc DETMasc ADJMasc NOUNMasC [0.3]
NPGend -> DETMasC npGendMasc [0.5]
NPMasc -> DETMasc NOUNMasc ADJMasc [0.3]
npGendFem -> NOUNFem [0.4]

---

NP25 -> DETFem np25Fem [0.25]
mp25Fem NOUN25 ADJFem [0.4]
NP25 -> DETMaSC np25Masc [0.75] 75]
mp25Masc NOUN25 [o. 4]
mp25Fem NOUN25 [0.4]
np25Masc ADJMasc NOUN25 [0.3]
up25Fem ADJFem NOUN25 [0.3]
np25Masc -> NOUN25 ADJMasc [0.3]
up25Fem NOUN25 ADJFem [0.3]
np25Masc -> NOUN25 [0.4]
NP50 -> DETFem mp50Fem [0.50]
np25Masc -> ADJMasc NOUN25 [0.3]
NP50 -> DETMasc mp50Masc [0.50]
np25Masc -> NOUN25 ADJMasc [0.3]
mp50Fem -> NOUN50 [0.4]
mp50Fem ADJFem NOUN50 [0.3]
NP50 DETFem np50Fem [0.50]
mp50Fem NOUN50 ADJFem [0.3]
NP50 DETMaSс mp50Masc [0.50]
mp50Masc NOUN50 [0.4]
mm50Fem NOUN50 [0.4]
np50Masc ADJMasc NOUN50 [0.3]
mp50Fem ADJFem NOUN50 [0.3]
mp50Masc -> NOUN50 ADJMasc [0.3]
mp50Fem -> NOUN50 ADJFem [0.3]
mp50Masc -> NOUN50 [0.4]
NP75 -> DETFem np75Fem [0.75]
np50Masc -> ADJMasc NOUN50 [0.3]
NP75 -> DETMasc mp75Masc [0. [5]
np50Masc NOUN50 ADJMasc [0.3]
mp75Fem -> NOUN75 [0.4]
mp75Fem ADJFem NOUN75 [0.3]
NP75 DETFem np75Fem [0.75]
mp75Fem NOUN75 ADJFem [0.3]
NP75 -> DETMasC np75Masc [0.25]
np75Masc -> NOUN75 [0.4]
up75Fem -> NOUN75 [0.4]
mp75Masc ADJMasc NOUN75 [0.3]
mp75Fem -> ADJFem NOUN75 [0.3]
np75Masc -> NOUN75 ADJMasc [0.3]
mp75Fem -> NOUN75 ADJFem [0.3]
mp75Masc -> NOUN75 [0.4]
np75Masc -> ADJMasc NOUN75 [0.3]
The subsequent PCFG used for generating probe_tests The placeholders Y and can be substituted with specific values to represent the noun category being tested and the context in which these nouns should appear, respectively. For Y, the options include Fem, Masc, 25, 50, 75. As for X. it can be replaced with Fem, Masc, Amb.
np75Masc NOUN75 ADJMasc [0.3]
The PCFG described is used in the generation Trope_ttreee train.
S -> NP VP " [[1.0]
PP PREP MP [1.0]
VP VERB [0.5] | VERB NP [0.5]
-> NP VP [[.0]
NP -> NPGend [0.8] NP PP [0.20]
PP -> PREP NP [1.0]
VP -> VERB [0.5] VERB NP [0.5]
NPGend -> NPFem [0.35] NPMasc [0.35]
NPGend NP25 [0.1] NP50 [o.1] 1]
NP -> NPX [0.80] NP PP [0.20]
NPGend NP75 [0.1]
NPAmb DETEpic NOUN [0.4]
NPFem -> DETFem NOUNFem [0.4]
NPAmb DETEpic ADJEpic NOUN [0.3]
NPFem em
-> DETFem ADJFem NOUNFem [0.3]
NPAmb -> DETEpic NOUN ADJEpic [0.3]
NPFem Femm -> DETFem NOUNFem ADJFem [0.3]
NPFem -> DETFem NOUN [0.4]
NPMasc -> DETMaSC NOUNMasc [0.4]
NPFem -> DETFen em ADJFem NOUN [0.3]
NPKasc -> DETMaSc ADJMasс NOUNMasC [0.3]
NPFem -> DETFem NOUN [0.4]
NPMasc -> DETMaSc NOUNMasc ADJMasc [0.3]
NPMasc -> DETMaSс NOUN [0.4]
NPMasc DETMasC ADJMasc NOUN [0.3]
NP25 -> DETFem np25Fem [0.25]
NPMasc DETMasC NOUN ADJMasc [0.3]
NP25 DETMaSC np25Masc [5.
np25Fem NOUN25 [0.4]
NOUN -> NOUNY [1.0]
np25Fem ADJFem NOUN25 [0.3]

---

B Transformer Language Model
We used the same model size as in (White and Cotterell, 2021), with an embedding size and an output size of 256, 3 hidden layers with 4 attention heads each and feed-forward networks with hidden size of 1024. We also experimented with larger models, such the original transformer size in (Vaswani al., 2017), but we found that did not significantly improve perplexity or the probe's accuracy. Therefore, we opted for smaller models to save computational resources and training time. Our models were trained for 100 epochs, with one warmup epoch. We used a learning rate of 0.0005, dropout rate of 0.3 and batch size of 64. The weights from the epoch that performed best on the development were selected as the final model. To ensure the statistical significance of our find- ings and mitigate the influence of model variance, we conducted each experiment by training 20 distinct language models and probes. The resulting probe accuracies were averaged, allowing us to compute 95% confidence intervals for our measurements.


Reviews:
[{"review_id": "Z0iqHtAgv1", "reviewer": "Reviewer_bPtj", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer bPtj,\n\nThank you for your valuable time, feedback and style and grammar corrections.\n\n> “this has only studied using French and using artificial data (thus, results cannot be extrapolated to other languages and/or data)”\n\nThe goal of this short paper is to investigate whether some precise factors can influence a transformer-based LM’s ability to assign gender by working in a controlled setting, and **not** to make claims about why real-world language models trained on natural data are biased, which would be impossible using artificial data. We believe that this first objective is a necessary step towards understanding the biases in the models used today and that by making it possible to carry out experiments under perfectly controlled conditions, the methodology we are introducing allows us to draw general conclusions about the capacities of the transformer architecture. For example, our results show that having an unbalanced gender distribution in the training data does not necessarily lead a transformer-based LM to over-assign the majority class gender. This suggests that transformers’ learning process is not inherently biased or irremediably predisposed to reproducing the most frequent patterns.  \nWe agree that further work is necessary to assess if our conclusions still hold for models trained on natural language or for languages with different gender systems and these are points we discussed in the “Limitations” section. However, we think the results we present here are already important and can help better direct these future works, by suggesting factors whose influence could be tested and a methodology for doing so.\n\nConcerning reproducibility, our code and dataset will be made available upon publication, which should make it easy to reproduce our results. We will also add the average runtime and number of parameters in the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Interesting study that could benefit future studies on the field.", "reasons_to_reject": "* Results cannot be directly extrapolated.", "questions_to_authors": "", "paper_topic_and_main_contributions": "By using an artificial, controlled dataset, this work studies how well Transformer language models are able to capture gender information under different circumstances (which are controlled creating artificial data using PCFG). Despite that this has only studied using French and using artificial data (thus, results cannot be extrapolated to other languages and/or data), the most interesting conclusion is that \"the origin\nof bias in language models is more complex than initially thought\".", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "l. 019: (Manning et al., 2020; Belinkov et al., 2020) -> (Belinkov et al., 2020; Manning et al., 2020)\nl. 042: a NLM -> an NLM\nl. 050: methodology of -> methodology from/by\nl. 070: (White and Cotterell, 2021; Kim and Linzen, 2020) -> (Kim and Linzen, 2020; White and Cotterell, 2021)\nl. 126: LM of -> LM from/by\nl. 197: of -> by\n"}}, {"review_id": "8EiJXJgMHP", "reviewer": "Reviewer_WdXg", "summary": "", "questions": "- Why was it not possible to extract the masculine/feminine noun constructions from natural data? I expect that the simplicity of these constructions would make this possible, and it would remove the caveat of the constructions being semantically implausible.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer WdXg,\n\nThank you for your detailed review and for the reference pointers you provided. We address each of your concerns separately below.\n\n> “The paper does not engage with the literature on probing, which has shown that solely using accuracy as a metric is insufficient and not necessarily reflective of actual model behaviour.”\n\nWe are well aware of the limitations of using probe accuracy highlighted in the papers you put forward and have discussed the problems raised by the use of probes in our “Limitations” section. Hewitt et al. (2019) call attention to the fact that by relying solely on probe accuracy we cannot tell whether a high score means the information is encoded in the representations or whether the probe has just learnt the task. The strategy we have used to avoid this problem is explained in footnote 4: “to ensure that a high probe accuracy indicates the encoding of gender information within the language model’s representations rather than its memorisation by the probe, nouns present in probe_train are excluded from probe_test.”\n\nElazar et al. (2020) highlight that behavioural conclusions should not be drawn from probing results. We agree that our results do not make it possible to draw conclusions about the behaviour of language models with regard to gender. However, the aim of our work is to shed light on how transformer LMs are able to learn gender and, in the process, identify the possible causes of bias in this family of models, rather than to determine exactly why a particular model behaves in a biased way. We believe probes to be a fitting tool to study this question.\n\n>“The context-free assumption of the PCFGs loses the semantic plausibility of the constructions, which might make it challenging to relate these findings to natural settings. A formal language setting that takes this better into account would strengthen the results.”\n\n> “Why was it not possible to extract the masculine/feminine noun constructions from natural data? I expect that the simplicity of these constructions would make this possible, and it would remove the caveat of the constructions being semantically implausible.”\n\nWe explain our choice to use PCFGs in Appendix A (l. 470):\n“In our case, we chose to disregard the semantics of our artificial languages, focusing solely on the grammatical aspects of gender. This deliberate choice serves to narrow down the range of possibilities for how bias can manifest and emerge, at least for the present.”\nThanks to the additional page in the final version, we will be able to move this discussion to the “main” paper as, as you pointed out, it is an important question. Gulordava et al. (2018) show that syntax can be learnt independently from semantics, so we chose to focus only on the grammatical aspect of gender since semantics could introduce potential confounding factors (such as stereotypes introduced by the distributional environment of words) and we wanted to isolate the factors we were testing, such as the grammatical informativity of the context at training and inference time and the overall gender distribution in the training data. This is also why we did not extract our constructions from natural data, but generated them using PCFGs instead.\n\n> “The paper barely engages with the current literature on (gender) bias.”\n\nWe are familiar with the references you provided but we did not elaborate on these points due to lack of space. We will use the extra page in the final version to address these issues more thoroughly.   \nWe have, for example, taken into account Blodgett et al. (2020)’s suggestion that papers on bias explicitly state what is meant by the word bias and their normative assumptions about what model behaviours are good and which ones are bad. In the section “Influence of the gender distribution on default gender guessing” (l. 246), we define precisely what we mean by “degree of bias” and what would be the expected results for a biased and an unbiased model. We do, however, agree that it would be beneficial to further expand on these topics making our assumptions about why these model behaviours are bad and who exactly would be the wronged party explicit instead of taking them to be self-evident. \n\n> “The paper presents little to no pointers how its findings can be of use to analysis of bias in LLMs in general.”\n\nPlease see the response to Reviewer bPtj for this point.\n\nLastly, thank you for the suggestion to turn Table 2 into a line plot, it does indeed make it a lot clearer and easier to interpret. We will use this representation in our final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A focused and systematic approach to investigate model bias in isolation.\n- Surprising finding that biases do not seem to be amplified as has been reported in earlier work on bias.\n- The usage of artificial languages for model interpretability is a fruitful line of work, and I am excited to seeing it be applied in a gender bias context.", "reasons_to_reject": "- The paper barely engages with the current literature on (gender) bias. \n- The paper presents little to no pointers how its findings can be of use to analysis of bias in LLMs in general.\n- The paper does not engage with the literature on probing, which has shown that solely using accuracy as a metric is insufficient and not necessarily reflective of actual model behaviour.\n- The context-free assumption of the PCFGs loses the semantic plausibility of the constructions, which might make it challenging to relate these findings to natural settings. A formal language setting that takes this better into account would strengthen the results.", "questions_to_authors": "- Why was it not possible to extract the masculine/feminine noun constructions from natural data? I expect that the simplicity of these constructions would make this possible, and it would remove the caveat of the constructions being semantically implausible.", "paper_topic_and_main_contributions": "The paper presents an investigation into gender bias using artificial languages. It is often said that one of the major causes for gender bias in LMs stems from unbalance in the training data, and the paper proposes to investigate this in a controlled setup using PCFGs. \n\nIt is shown that contrary to previous work, gender unbalances are not amplified. Furthermore, it is shown that gender bias can persist even in the case where the training data is completely balanced.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Table 2 could perhaps be better represented as a line plot, right now the trend is much harder to interpret."}, "extra_text": {"Missing_References": "The paper would benefit from being better rooted within bias research, for example:\n\n- *How Does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?* - Gonen et al. (2019)\n- *Language (Technology) is Power: A Critical Survey of “Bias” in NLP* - Blodgett et al. (2020)\n- *The Birth of Bias: A case study on the evolution of gender bias in an English language model* - Van der Wal et al. (2022)\n\nFor probing there is a extensive line of work that has raised doubts about interpreting probing accuracy as a direct measure for model performance. Useful references here could be:\n\n- *Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals* - Elazar et al. (2020)\n- *Designing and Interpreting Probes with Control Tasks* - Hewitt et al. (2019)"}}, {"review_id": "LLnpavqTx1", "reviewer": "Reviewer_B4CF", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer B4CF,\n\nThank you for your positive feedback! We are pleased that you found our work novel, rigorous and convincing.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The problem studied in this work is critical and the analysis is in good scientific rigor with convincing experimental results and insightful analysis. The paper is written in a clear and concise style.\n\n2. This work provides a novel perspective on how gender bias emerges in language models. The work may help determine under which conditions a model correctly captures gender information or, on the contrary, appears gender-biased.", "reasons_to_reject": "This is overall a solid short paper. I do not find any major concerns.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work focuses on the problem of how neural language models (NLMs) discover information. In particular, this work takes an initial step toward exploring the less researched topic of how neural models discover linguistic properties of words, such as gender, as well as the rules governing their usage. By evaluating the NLM's capacities on a test set specifically designed to contain examples that differ from those of the training set based on well-defined criteria, this work finds that NLMs can memorize gender information from the training data, they also learn to use sentence structure to infer the gender of a word occurrence.\n\nOverall, this work studies an interesting problem concerning the emergence of gender bias in Transformer language models. The findings challenge the assumption that training data imbalance is the sole cause of gender bias in NLMs. This work may shed light on the salient factors influencing the emergence of gender bias in NLMs. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces an artificially developed French corpus to better understand the circumstances under which transformer models capture gender information. Its findings are especially impactful, suggesting that gender biases can persist even when training corpora appear fully balanced. This contradicts previous research which often attributes gender biases to imbalanced training data. While the paper was acknowledged as sound (two 4s, one 3), there are potential issues in regards to the extensibility of this work to other languages and the lack of engagement with existing research on gender bias. The authors addressed these concerns in their rebuttals, however, and acknowledged that missing discussions would be included in the final version of the paper.

In light of this, only minor revisions, addressing the aforementioned concerns, need to be made to ensure this paper is camera ready.

*As an additional note for the authors, I highly recommend adding to the title and at minimum, the abstract that the corpus was made for French. I would also like to reiterate Reviewer WdXg's recommendation to explicitly state how you believe "experiments on artificial data will ultimately lead to 'understanding the biases in the models used today'."
Decision: Accept-Main