For notational simplicity, we define INLINEFORM12 to be the set of all length- INLINEFORM13 sequences of morpho-syntactic tags. We define the probability of INLINEFORM0 given INLINEFORM1 and INLINEFORM2 as DISPLAYFORM0   where the binary factor INLINEFORM0 scores how well the morpho-syntactic tags INLINEFORM1 and INLINEFORM2 agree given the POS tags INLINEFORM3 and INLINEFORM4 and the label INLINEFORM5 . For example, consider the INLINEFORM6 (adjectival modifier) edge from experto to ingeniero in fig:tree. The factor INLINEFORM7 returns a high score if the corresponding morpho-syntactic tags agree in gender and number (e.g., INLINEFORM8 and INLINEFORM9 ) and a low score if they do not (e.g., INLINEFORM10 and INLINEFORM11 ). The unary factor INLINEFORM12 scores a morpho-syntactic tag INLINEFORM13 outside the context of the dependency tree. As we explain in sec:constraint, we use these unary factors to force or disallow particular tags when performing an intervention; we do not learn them.

Extrinsic Evaluation	We extrinsically evaluate our approach by assessing the extent to which it reduces gender stereotyping. Following DBLP:journals/corr/abs-1807-11714, focus on neural language models. We choose language models over word embeddings because standard measures of gender stereotyping for word embeddings cannot be applied to morphologically rich languages. As our measure of gender stereotyping, we compare the log ratio of the prefix probabilities under a language model INLINEFORM0 for gendered, animate nouns, such as ingeniero, combined with four adjectives: good, bad, smart, and beautiful. The translations we use for these adjectives are given in sec:translation. We chose the first two adjectives because they should be used equally to describe men and women, and the latter two because we expect that they will reveal gender stereotypes. For example, consider DISPLAYFORM0  If this log ratio is close to 0, then the language model is as likely to generate sentences that start with el ingeniero bueno (the good male engineer) as it is to generate sentences that start with la ingeniera bueno (the good female engineer).

Conclusion	We presented a new approach for converting between masculine-inflected and feminine-inflected noun phrases in morphologically rich languages. To do this, we introduced a Markov random field with an optional neural parameterization that infers the manner in which a sentence must change to preserve morpho-syntactic agreement when altering the grammatical gender of particular nouns. To the best of our knowledge, this task has not been studied previously. As a result, there is no existing annotated corpus of paired sentences that can be used as “ground truth.” Despite this limitation, we evaluated our approach both intrinsically and extrinsically, achieving promising results. For example, we demonstrated that our approach reduces gender stereotyping in neural language models. Finally, we also identified avenues for future work, such as the inclusion of co-reference information.

Intervention	As explained in sec:gender, our goal is to transform sentences like sent:msc to sent:fem by intervening on a gendered word and then using our model to infer the manner in which the remaining tags must be updated to preserve morpho-syntactic agreement. For example, if we change the morpho-syntactic tag for ingeniero from [msc;sg] to [fem;sg], then we must also update the tags for el and experto, but do not need to update the tag for es, which should remain unchanged as [in; pr; sg]. If we intervene on the INLINEFORM0 word in a sentence, changing its tag from INLINEFORM1 to INLINEFORM2 , then using our model to infer the manner in which the remaining tags must be updated means using INLINEFORM3 to identify high-probability tags for the remaining words. Crucially, we wish to change as little as possible when intervening on a gendered word. The unary factors INLINEFORM0 enable us to do exactly this.

That said, with the exception of Hebrew, our approach either sacrifices less grammaticality than naïve swapping of gendered words and sometimes increases grammaticality over the original corpus. Given that we know the model did not perform as accurately for Hebrew (see tab:intrinsic), this finding is not surprising.

This is because two nouns do not normally need to have the same gender when they are conjoined. Moreover, our MRF does not include co-reference information, so it cannot tell that, in this case, both nouns refer to the same person. Note that including co-reference information in our MRF would create cycles and inference would no longer be exact. Additionally, the lack of co-reference information means that, for Spanish, our approach fails to convert nouns that are noun-modifiers or indirect objects of verbs. Somewhat surprisingly, the neural parameterization does not outperform the linear parameterization. We proposed the neural parameterization to allow parameter sharing among edges with different parts of speech and labels; however, this parameter sharing does not seem to make a difference in practice, so the linear parameterization is sufficient.

In these languages, the words in a sentence are marked with morphological endings that reflect the grammatical gender of the surrounding nouns. This means that if the gender of one word changes, the others have to be updated to match. As a result, simple heuristics, such as augmenting a corpus with additional sentences in which he and she have been swapped  , will yield ungrammatical sentences. Consider the Spanish phrase el ingeniero experto (the skilled engineer). Replacing ingeniero with ingeniera is insufficient—el must also be replaced with la and experto with experta. In this paper, we present a new approach to counterfactual data augmentation  for mitigating gender stereotypes associated with animate nouns (i.e., nouns that represent people) for morphologically rich languages. We introduce a Markov random field with an optional neural parameterization that infers the manner in which a sentence must change when altering the grammatical gender of particular nouns. We use this model as part of a four-step process, depicted in fig:pipeline, to reinflect entire sentences following an intervention on the grammatical gender of one word.

fig:bias demonstrates depicts gender stereotyping and grammaticality for each language using the original corpus, the corpus following CDA using naïve swapping of gendered words, and the corpus following CDA using our approach. It is immediately apparent that our approch reduces gender stereotyping. On average, our approach reduces gender stereotyping by a factor of 2.5 (the lowest and highest factors are 1.2 (Ita) and 5.0 (Esp), respectively). We expected that naïve swapping of gendered words would also reduce gender stereotyping. Indeed, we see that this simple heuristic reduces gender stereotyping for some but not all of the languages. For Spanish, we also examine specific words that are stereotyped toward men or women. We define a word to be stereotyped toward one gender if 75% of its occurrences are of that gender. fig:espbias suggests a clear reduction in gender stereotyping for specific words that are stereotyped toward men or women. The grammaticality of the corpora following CDA differs between languages.

If the log ratio is negative, then the language model is more likely to generate the feminine form than the masculine form, while the opposite is true if the log ratio is positive. In practice, given the current gender disparity in engineering, we would expect the log ratio to be positive. If, however, the language model were trained on a corpus to which our CDA approach had been applied, we would then expect the log ratio to be much closer to zero. Because our approach is specifically intended to yield sentences that are grammatical, we additionally consider the following log ratio (i.e., the grammatical phrase over the ungrammatical phrase): DISPLAYFORM0  We trained the linear parameterization using UD treebanks for Spanish, Hebrew, French, and Italian (see tab:data). For each of the four languages, we parsed one million sentences from Wikipedia (May 2018 dump) using  's parser and extracted taggings and lemmata using the method of  . We automatically extracted an animacy gazetteer from WordNet  and then manually filtered the output for correctness.

We intrinsically evaluate our approach using Spanish and Hebrew, achieving tag-level INLINEFORM0 scores of 83% and 72% and form-level accuracies of 90% and 87%, respectively. We also conduct an extrinsic evaluation using four languages. Following DBLP:journals/corr/abs-1807-11714, we show that, on average, our approach reduces gender stereotyping in neural language models by a factor of 2.5 without sacrificing grammaticality.

Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology	Gender stereotypes are manifest in most of the world's languages and are consequently propagated or amplified by NLP systems. Although research has focused on mitigating gender stereotypes in English, the approaches that are commonly employed produce ungrammatical sentences in morphologically rich languages. We present a novel approach for converting between masculine-inflected and feminine-inflected sentences in such languages. For Spanish and Hebrew, our approach achieves F1 scores of 82% and 73% at the level of tags and accuracies of 90% and 87% at the level of forms. By evaluating our approach using four different languages, we show that, on average, it reduces gender stereotyping by a factor of 2.5 without any sacrifice to grammaticality.

Related Work	In contrast to previous work, we focus on mitigating gender stereotypes in languages with rich morphology—specifically languages that exhibit gender agreement. To date, the NLP community has focused on approaches for detecting and mitigating gender stereotypes in English. For example,  proposed a way of mitigating gender stereotypes in word embeddings while preserving meanings;  studied gender stereotypes in language models; and  introduced a novel Winograd schema for evaluating gender stereotypes in co-reference resolution. The most closely related work is that of  , who used CDA to reduce gender stereotypes in co-reference resolution; however, their approach yields ungrammatical sentences in morphologically rich languages. Our approach is specifically intended to yield grammatical sentences when applied to such languages.  also focused on morphologically rich languages, specifically Arabic, but in the context of gender identification in machine translation.

Introduction	One of the biggest challenges faced by modern natural language processing (NLP) systems is the inadvertent replication or amplification of societal biases. This is because NLP systems depend on language corpora, which are inherently “not objective; they are creations of human design”  . One type of societal bias that has received considerable attention from the NLP community is gender stereotyping  ,  ,  . Gender stereotypes can manifest in language in overt ways. For example, the sentence he is an engineer is more likely to appear in a corpus than she is an engineer due to the current gender disparity in engineering. Consequently, any NLP system that is trained such a corpus will likely learn to associate engineer with men, but not with women  . To date, the NLP community has focused primarily on approaches for detecting and mitigating gender stereotypes in English  ,  ,  . Yet, gender stereotypes also exist in other languages because they are a function of society, not of grammar. Moreover, because English does not mark grammatical gender, approaches developed for English are not transferable to morphologically rich languages that exhibit gender agreement  .

We provide the size of the languages' animacy gazetteers and the percentage of automatically parsed sentences that contain an animate noun in tab:anim. For each sentence containing a noun in our animacy gazetteer, we created a copy of the sentence, intervened on the noun, and then used our approach to transform the sentence. For sentences containing more than one animate noun, we generated a separate sentence for each possible combination of genders. Choosing which sentences to duplicate is a difficult task. For example, alemán in Spanish can refer to either a German man or the German language; however, we have no way of distinguishing between these two meanings without additional annotations. Multilingual animacy detection  might help with this challenge; co-reference information might additionally help. For each language, we trained the BPE-RNNLM baseline open-vocabulary language model of  using the original corpus, the corpus following CDA using naïve swapping of gendered words, and the corpus following CDA using our approach. We then computed gender stereotyping and grammaticality as described above. We provide example phrases in tab:lm; we provide a more extensive list of phrases in app:queries.

Extrinsic Evaluation Example Phrases	For each noun in our animacy gazetteer, we generated sixteen phrases. Consider the noun engineer as an example. We created four phrases—one for each translation of The good engineer, The bad engineer, The smart engineer, and The beautiful engineer. These phrases, as well as their prefix log-likelihoods are provided below in tab:query.

A Markov Random Field for Morpho-Syntactic Agreement	In this section, we present a Markov random field  for morpho-syntactic agreement. This model defines a joint distribution over sequences of morpho-syntactic tags, conditioned on a labeled dependency tree with associated part-of-speech tags. Given an intervention on a gendered word, we can use this model to infer the manner in which the remaining tags must be updated to preserve morpho-syntactic agreement. A dependency tree for a sentence (see fig:tree for an example) is a set of ordered triples INLINEFORM0 , where INLINEFORM1 and INLINEFORM2 are positions in the sentence (or a distinguished root symbol) and INLINEFORM3 is the label of the edge INLINEFORM4 in the tree; each position occurs exactly once as the first element in a triple. Each dependency tree INLINEFORM5 is associated with a sequence of morpho-syntactic tags INLINEFORM6 and a sequence of part-of-speech (POS) tags INLINEFORM7 . For example, the tags INLINEFORM8 and INLINEFORM9 for ingeniero are INLINEFORM10 and INLINEFORM11 , respectively, because ingeniero is a masculine, singular noun.

Parameterization	We consider a linear parameterization and a neural parameterization of the binary factor INLINEFORM0 . We define a matrix INLINEFORM0 for each triple INLINEFORM1 , where INLINEFORM2 is the number of morpho-syntactic subtags. For example, INLINEFORM3 has two subtags INLINEFORM4 and INLINEFORM5 . We then define INLINEFORM6 as follows: INLINEFORM7   where INLINEFORM0 is a multi-hot encoding of INLINEFORM1 . As an alternative, we also define a neural parameterization of INLINEFORM0 to allow parameter sharing among edges with different parts of speech and labels: INLINEFORM1   where INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 and INLINEFORM3 define the structure of the neural parameterization and each INLINEFORM4 is an embedding function. We use the unary factors only to force or disallow particular tags when performing an intervention. Specifically, we define DISPLAYFORM0  where INLINEFORM0 is a strength parameter that determines the extent to which INLINEFORM1 should remain unchanged following an intervention. In the limit as INLINEFORM2 , all tags will remain unchanged except for the tag directly involved in the intervention.

eq:dist is normalized by the following partition function: INLINEFORM14   INLINEFORM0 can be calculated using belief propagation; we provide the update equations that we use in sec:bp. Our model is depicted in fig:fg. It is noteworthy that this model is delexicalized—i.e., it considers only the labeled dependency tree and the POS tags, not the actual words themselves.

Experiments	We used the Adam optimizer  to train both parameterizations of our model until the change in dev-loss was less than INLINEFORM0 bits. We set INLINEFORM1 without tuning, and chose a learning rate of INLINEFORM2 and weight decay factor of INLINEFORM3 after tuning. We tuned INLINEFORM4 in the set INLINEFORM5 and chose INLINEFORM6 . For the neural parameterization, we set INLINEFORM7 and INLINEFORM8 without any tuning. Finally, we trained the inflection model using only gendered words. We evaluate our approach both intrinsically and extrinsically. For the intrinsic evaluation, we focus on whether our approach yields the correct morpho-syntactic tags and the correct reinflections. For the extrinsic evaluation, we assess the extent to which using the resulting transformed sentences reduces gender stereotyping in neural language models.

Inference	Because our MRF is acyclic and tree-shaped, we can use belief propagation  to perform exact inference. The algorithm is a generalization of the forward-backward algorithm for hidden Markov models  . Specifically, we pass messages from the leaves to the root and vice versa. The marginal distribution of a node is the point-wise product of all its incoming messages; the partition function INLINEFORM0 is the sum of any node's marginal distribution. Computing INLINEFORM1 takes polynomial time  —specifically, INLINEFORM2 where INLINEFORM3 is the number of morpho-syntactic tags. Finally, inferring the highest-probability morpho-syntactic tag sequence INLINEFORM4 given INLINEFORM5 and INLINEFORM6 can be performed using the max-product modification to belief propagation.

Parameter Estimation	We use gradient-based optimization. We treat the negative log-likelihood INLINEFORM0 as the loss function for tree INLINEFORM1 and compute its gradient using automatic differentiation  . We learn the parameters of sec:param by optimizing the negative log-likelihood using gradient descent.

As described in the previous section, the strength parameter INLINEFORM1 determines the extent to which INLINEFORM2 should remain unchanged following an intervention—the larger the value, the less likely it is that INLINEFORM3 will be changed. Once the new tags have been inferred, the final step is to reinflect the lemmata to their new forms. This task has received considerable attention from the NLP community  ,  . We use the inflection model of D18-1473. This model conditions on the lemma INLINEFORM0 and morpho-syntactic tag INLINEFORM1 to form a distribution over possible inflections. For example, given experto and INLINEFORM2 , the trained inflection model will assign a high probability to expertas. We provide accuracies for the trained inflection model in tab:reinflect.

Adjective Translations	tab:fem and tab:masc contain the feminine and masculine translations of the four adjectives that we used.

Intrinsic Evaluation	To the best of our knowledge, this task has not been studied previously. As a result, there is no existing annotated corpus of paired sentences that can be used as “ground truth.” We therefore annotated Spanish and Hebrew sentences ourselves, with annotations made by native speakers of each language. Specifically, for each language, we extracted sentences containing animate nouns from that language's UD treebank. The average length of these extracted sentences was 37 words. We then manually inspected each sentence, intervening on the gender of the animate noun and reinflecting the sentence accordingly. We chose Spanish and Hebrew because gender agreement operates differently in each language. We provide corpus statistics for both languages in the top two rows of tab:data. We created a hard-coded INLINEFORM0 to serve as a baseline for each language. For Spanish, we only activated, i.e. set to a number greater than zero, values that relate adjectives and determiners to nouns; for Hebrew, we only activated values that relate adjectives and verbs to nouns. We created two separate baselines because gender agreement operates differently in each language.

To evaluate our approach, we held all morpho-syntactic subtags fixed except for gender. For each annotated sentence, we intervened on the gender of the animate noun. We then used our model to infer which of the remaining tags should be updated (updating a tag means swapping the gender subtag because all morpho-syntactic subtags were held fixed except for gender) and reinflected the lemmata. Finally, we used the annotations to compute the tag-level INLINEFORM0 score and the form-level accuracy, excluding the animate nouns on which we intervened. We present the results in tab:intrinsic. Recall is consistently significantly lower than precision. As expected, the baselines have the highest precision (though not by much). This is because they reflect well-known rules for each language. That said, they have lower recall than our approach because they fail to capture more subtle relationships. For both languages, our approach struggles with conjunctions. For example, consider the phrase él es un ingeniero y escritor (he is an engineer and a writer). Replacing ingeniero with ingeniera does not necessarily result in escritor being changed to escritora.

Acknowledgments	The last author acknowledges a Facebook Fellowship.

Belief Propagation Update Equations	Our belief propagation update equations are DISPLAYFORM0 DISPLAYFORM1   where INLINEFORM0 returns the set of neighbouring nodes of node INLINEFORM1 . The belief at any node is given by DISPLAYFORM0

Gender Stereotypes in Text	Men and women are mentioned at different rates in text  . This problem is exacerbated in certain contexts. For example, the sentence he is an engineer is more likely to appear in a corpus than she is an engineer due to the current gender disparity in engineering. This imbalance in representation can have a dramatic downstream effect on NLP systems trained on such a corpus, such as giving preference to male engineers over female engineers in an automated resumé filtering system. Gender stereotypes of this sort have been observed in word embeddings  ,  , contextual word embeddings  , and co-reference resolution systems  ,  inter alia.

Question: How do they measure grammaticality?
Answer: by calculating log ratio of grammatical phrase over ungrammatical phrase