A Rewriting Approach for Gender Inclusivity in Portuguese
Authors: Leonor Veloso
Luisa Coheur
Rui Ribeiro
Venue: EMNLP 2023 Findings (2023)
Keywords: nlp
portuguese
gender neutrality
gender inclusivity
machine translation

Abstract: In recent years, there has been a notable rise in research interest regarding the integration of gender-inclusive and gender-neutral language in natural language processing models. A specific area of focus that has gained practical and academic significant interest is gender-neutral rewriting, which involves converting binary-gendered text to its gender-neutral counterpart. However, current approaches to gender-neutral rewriting for gendered languages tend to rely on large datasets, which may not be an option for languages with fewer resources, such as Portuguese. In this paper, we present a rule-based and a neural-based tool for gender-neutral rewriting for Portuguese, a heavily gendered Romance language whose morphology creates different challenges from the ones tackled by other gender-neutral rewriters. Our neural approach relies on fine-tuning large multilingual machine translation models on examples generated by the rule-based model. We evaluate both models on texts from different sources and contexts. We provide the first Portuguese dataset explicitly containing gender-neutral language and neopronouns, as well as a manually annotated golden collection of 500 sentences that allows for evaluation of future work.

A Rewriting Approach for Gender Inclusivity in Portuguese
Leonor Veloso and Luísa Coheur and Rui Ribeiro
INESC-ID, Lisbon, Portugal Instituto Superior Técnico, University of Lisbon, Portugal
leonor wel.oo@tecncooo.ooooo www.ooa....aaa..aa. luisa www.ereeeeeenniccooooooo rui m. www.eeiirreeeniiiee ulisboa..p.. .t
wttp:/tteennnn...oooooo.oeeee ulisboa. pt
Abstract
with the advent of social media, there has been an increase in the visibility and usage of new sets of gender-neutral pronouns (Pinheiro, 2020) (some- times referred to as neo-pronomes in Portuguese). These neopronouns are preferred by many individu- als to refer to themselves (Miranda, 2020), but can also be used to refer to mixed-gender group of people avoiding the default masculine plural for example, um grupo de alunes instead of um grupo de alunos (EN: group of students).
In recent years, there has been notable rise in research interest regarding the integration of gender-inclusive and gender-neutral language in natural language processing models. A spe- cific area of focus that has gained practical and academic significant interest gender-neutral rewriting, which involves converting binary- gendered text to its gender-neutral counterpart. However, current approaches to gender-neutral rewriting for gendered languages tend to rely on large datasets, which may not be an op- tion for languages with fewer resources, such as Portuguese. In this paper, we present rule-based and neural-based based tool for gender- neutral rewriting for Portuguese, heavily gen- dered Romance language whose morphology creates different challenges from the ones tack- led by other gender-neutral rewriters. Our neu- ral approach relies on fine-tuning large multilin- gual machine translation models examples generated by the rule-bane mule-baeed model. We evalu- ate both models on texts from different sources and contexts. We provide the first Portuguese dataset explicitly containing gender-neutral lan- guage and neopronouns, as well as manually annotated golden collection of 500 sentences that allows for evaluation of future work.
Although the lack of gender inclusion in NLP datasets is a multilingual problem (Zhou et al.. 2019), most efforts being made towards the pro- cessing of gender-neutral language are focused on Germanic languages (Vanmassenhove et al., 2021; Sun et al., 2021; Amrhein et al., 2023). As such, we set out to create NLP models that allow for the processing of Portuguese gender-neutral language in the same way that is currently being achieved for other languages. The three main contributions of our work consist of:
As far as we concern, the first Portuguese parallel datasets explicitly containing gender- neutral language and neopronouns, made pub- liely available as well as manually curated set of 500 sentences. These datasets are comprised sentences belonging to five dif- ferent text categories: literary texts, journal- istic texts, dialogues, social media posts and comments, and simpler sentences. This al- lows for the evaluation of Portuguese gender- neutral rewriters in different contexts. We hope this contribution will increase visibility of gender-neutral language and neopronouns in the landscape of Portuguese NLP datasets, as well as allow for future research regarding these topics;
1 Introduction
The relationship between language and gender, as well its effects on socictal gender dynamics, has been examined and documented as early as the 70s (Gal, 1989, 1978). In more recent years, there has been a push towards the usage of gender-neutral (or gender-fair) language. How this gender-fairness is achieved, is, however, highly dependent on the specifics of each language. During the course of this work, we will concern ourselves with the case of Portuguese.
Like most Romance languages, Portuguese is characterized by binary grammatical gender sys- tem in which nouns belong to one of two classes: masculine or feminine. During the last decade,
A rule-based gender-neutee neutral rewriter based on

---

handcrafted rules, for which we provide open access?;
rely on fine-tuning large pre-trained model, described in section 5.2.
A neural gender-neutral rewriter developed via fine-tuning a large multilingual machine translation model. This method requires rela- tively smaller sized datasets (when compared to training a model from scratch), and thus al- lows for the development of gender-rewriters for lower-resource languages.
2.2 Gender-fair rewriting for Other Languages
Alhafni et al. (2022) developed gender rewriter for Arabic, combining both rule-based and neu- ral models. The authors used the Arabic Parallel Gender Corpus (APGC), which contains gender labels for each word specifying whether the word is gendered or not, female or male, and referring to the first-person or the second-person. The target gender label is appended to the input words as special token and the assumption is that the neural model pays attention the label to output the cor- reet gender alternatives. Section 2.3 details other sender-identifying systems that rely upon gender- tagged data.
2 Related Work
As previously mentioned, our system falls into the category of what we will call gender-neutral rewriter systems. These rewriters take as input some form of gendered text and output a gender- neutral version of that same text. What is consid- ered be gender-neutral greatly varies between proposed systems and languages.
Amrhein et al. (2023) propose novel approach to the gender-rewriting task. While the previous models rely forward augmentation, this ap- proach relies on backward augmentation and round- trip translation to create parallel dataset. This approach used to create an English rewriter that matches or outperforms the results of Vanmassen- hove et al. (2021). The backward augmentation approach consists of retrieving gender-fair data from large monolingual corpora and creating rule- based pipeline to derive artificially biased text. The round-trip translation approach relies on the fact that most current machine translation models are socially biased. This can be exploited by using a biased model to translate from gender-fair text to a pivot language. This output is then translated back to the original language, creating a biased version of the original gender-fair text. The authors use the round-trip translation method create German rewriter, using English pivot language.
2.1 Gender-fair rewriting for English
Gender-ncuträl rewriting currently modeled as a text generation problem, solved with sequence-to- sequence rule-based or neural models trained with parallel data.
Vannassenhove et al. (2021) and Sun et al. (2021) have developed similar rewriters in terms of approach and model architecture. They pro- pose two models: a rule-based rewriter, and neural model. The neural model trained on a parallel dataset composed the original, binary- gendered version of the dataset, and the respec- tive output of the rule-based rewriter. The neural model developed by Vanmassenhove et al. (2021) outperforms their rulc-based rewriter, while the rule-based rewriter developed by Sun al. (2021) outperforms their neural model. Both models from all authors achieve a WER (Woodard and Nelson, 1982; Morris et al., 2004a) of less than 1% in the respective test sets.
We identify two issues with the round-trip trans- lation approach when developing a rewriter for the Portuguese language:
In these approaches, a gender-fair version of a bindre-gendered dataset is synthetically gener- ated through a rule-based pipeline. This method, which we employ in work, can be referred to forward augmentation (Amrhein et al., 2023). Similarly Vanmassenhove et (2021) and Sun et al. (2021), we train our neural model on data gen- erated by the rule-based model. However, while the cited authors train a neural model from scratch,
The lack of very large Portuguese monolin- gual datasets containing gender- language;
The lack of consistency of the existing gender- fair data regarding the usage and choice of neopronouns: due the diversity of Por- tuguese gender-neutral language proposals and neopronouns (detailed section 3), it is often the case that real examples of gender- neutral language are not consistent in terms
https:///tthubb.oommmmooorrggege neutralize http:://hhugginggacccco///o/oo// neutralizer

---

of gender agreement. Examples of this phe- nomenon are depicted in Table 6, in Appendix A.I.
Portuguese are characterized by an -e an termi- nation in gendered terms. These proposals tend arise in the form of practical guides, both by Brazil- ian and European Portuguese authors, and tend to be based on informal studies and observations of the neutral language used by queer communities. Since the usage of gender-neutral language in Por- tuguese is an ongoing debate, in this section we present some guides reporting on the community usage of gender-neutral language.
Amrhein et al. (2023) resort to LLMs in order generate additional gender-fair examples. We address this approach as future work in section 7.
2.3 Other Gender-Identiffying Systems
At the time of writing, no gender-neutral rewrit- ers have been developed for Romance languages. However, Jain et al. (2021) have developed neu- ral machine translation system, for the Spanish language, trained to translate from one (binary) gender alternative to another, as a method for gen- erating augmented data. This task is very similar gender-neutral rewriting, since it requires identi- fying and rewriting gendered terms. The parallel dataset is enriched with re-genderabliltty tags, i.e., if the source and the target parts differ, then the sentence is considered re- re-genderrble. Using these tags results in a reduced error rate when compared to standard neural system (without any tags).
3.1 Proposals for a Neutral Grammatical Gender Within Portuguese
The earliest source we were able to find is the "Man- ifesto ILE" (P. Berlucci, 2015), which suggests the gender-neutral ile/dile pronouns. However, the au- thors of this guide do not provide any rules or rec- ommendations the usage of this pronoun and agreement with nouns and adjectives.
Caê (2020) presents the Elu, Ile, Ilu, and El neo- pronoun systems. While these systems differ in terms of the pronoun in use, the agreement with nouns and adjectives is similar, requiring an ter- mination. For example, filho (EN: son) becomes filhe. The author also provides some general tips for rephrasing sentences to omit gender. For exam- ple, ela caiu (EN: she fell) becomes aquela pessoa caiu (EN: that person pelso.felll.
Bellandi and Siccardi (2022) have developed system that identifies gender-diseriminatory lan- guage for Italian. The authors describe two possi- ble ways in which language can be gender discrim- inatory: when sentences contain "only the male form of a noun having a different female form", and "sentences containing nouns having the same male and female form, without any other grammatical element to stress reference to both genders". The latter case is also common in Portuguese (e.g. using os docentes with the male article os, when docentes is both the male and female term for "teacher"). The authors trained a neural model to recognize these situations, assigning a label to each.
Marques and Santos (2021) recommend the us- age of the neopronoun éle/déle and the termina- tion system -e. An example sentence formed with the rules from this guide is: O professor deu as boas-vindas a todes es alunes. (EN: The teacher welcomed all the students.). The authors also rec- ommend the usage of already existent neutral forms c.g., monarca (EN: monarch) instead of reilrainha (EN: king/queen).
As far as we are aware, datasets that contain gender tags are not available for Portuguese, and therefore an approach that relies on gender tags (such as the approaches of Bellandi and Siccardi (2022), Jain al. (2021), and Alhafni et al. (2022)) is currently not viable.
The only other European Portuguese guide we found sets rules for system using the pronoun elu/delu (Valente, 2020). The authors provide gen- eral grammatical rules and example sentences, such as Aquelu menine minhe filhe (EN: That kid is my child).
3 Grammatical Gender and Gender-neutral Language in Portuguese
Most of the systems we observed converge on how the agreement of the neutral pronoun with nouns and adjectives should be done. For exam- ple, the neutral termination tends to be -e (e.g. "filho" "filhe"), since -e is vowel that provides contrast in speech with -o and -a. An exception is often made for terms whose masculine form ends with -e: in those cases, the corresponding neutral
Like most Romance languages, Portuguese has two genders: masculine (characterized by an termina- tion in gendered terms) and feminine (characterized by an -a termination in gendered terms). Most of the current proposals for a third neural gender for

---

form ends with -u in order to be distinguished from the masculine form (e.g. "aquele" "aquelu"). An example sentence using different neopronouns can be observed Table 8.
ing cleaning of the dataset, we removed speaker tags.
Social Media Random sample of tweets from the Portuguese Tweets for Sentiment Analysis dataset, which contains examples retrieved mainly from 01/08/2018 to 20/10/2018. We only used tweets from the "no theme" partition of the dataset. We removed links, mentions, hashtags, and emojis.
3.2 Rejection of -x and @@ Terminations
The characters -X and @@ have also been adopted gender-neutrality markers (eg: todxs or tod @ s), but there is concern with the legibility and usability of said terminations. Due to people and reading softwares not being able to pronounce words that terminate in -X or -@,, their usage has been cate- gorized as ableist (P. Berlucci, 2015; Marques and Santos, 2021; Valente, 2020). For these reasons, these terminations tend to be rejected by the com- munity. Bearing this in mind, in this work we focus on the implementation of rules based on -e and -u terminations.
Simple Sentences Samples from the Portuguese dataset of the Tatoeba (Tiedemann, 2020) corpus, consisting of relatively simpler (both in terms of vo- cabulary and syntactic construction) and less noisy sentences.
We chose to divide the data into five categories analyze the performance of our models in different types of text, and how they account (or not) for variability of sentence structure and vocabulary.
4 Datasets
4.1 Automatically Curated Set
We curated sets of 5,000 sentences from each text category, amounting to total of 25,000 examples. The gender-neutral alternatives of each example are generated by the rule-based model. This parallel dataset, containing the original (binary-gendered) sentences and the respective gender-neutral ver- sion, was used for training our neural model. Ta- ble (from Appendix A.4) depicts one example for each dataset category. This automatically cu- rated set is composed of approximately 60% gen- dered (14874 sentences) and 40% non-gendered sentences (10126 sentences). Due to the novelty of this line of research, the impact of the percent- age of gendered sentences on the performance of gender-rewriter models is still understudied. We leave that particular topic for future work.
Our dataset is split into five text categories:
Literary Texts Selected works found in the DIP collection" from Linguateca' resource center. DIP is a shared task whose goal is to identify of char- acters and respective attributes in literary works (Santos al., 2022). In order to avoid examples with an orthography that might be too different from modern Portuguese, we have only selected works released after 1910. Cleaning and processing the data consisted of the removal of tags (such as chapter indications, language tags, etc.) and author notes.
Journalistic texts Random sample of sentences found in the NaturaPublico94 dataset, from Pro- jecto Natura The original corpus contains the first 2 paragraphs of each article in the Portuguese news- paper Público, retrieved during the period of 1991 to 1994. NaturaPublico94 was retrieved during 1994.
4.2 Manually Curated Test Set
For curating our test set, we selected an additional 100 sentences from each of the five categories de- scribed in section 4. All 500 examples gen- dered, containing either named entities, human ref- erents, or personal pronouns.
Dialogues Random sample of extracted sen- tences from the SubTle corpus (Ameixa and Co- heur, 2013; Ameixa et al., 2014). SubTle aggre- gates dialogues from movie subtitles, extracted from 7MDB/ and pertaining to one of four movie genres: Horror, Scifi, Western, and Romance. Dur-
The 500 sentences in the collection were man- ually annotated by an NLP researcher. A sample of 100 examples belonging to the collection was annotated by other 5 fellow researchers in order to calculate the annotator agreement. The annotators followed an annotation guide for the elu system,
"htps://www.liibee httts::/www..inuuteec.pppm "htts://haturaaa di uminho, pt/ jj/pln/corpora/ httpss//www.immmccom
http://wwww..oaaaaaaaaaannpeeeee twwett-fforseemiimnt-ammta

---

whose rules are consistent with the ones employed in our rule-based model and are inspired by the proposals presented in section 3.1, particularly the ones authored by Caè (2020) and Marques and San- tos (2021). A sample of the said guide can be found in Appendix C.
preprocessing pipeline currently makes use of the entity recognition model for Span- ish, chosen due to the similarity between the two languages.
Stanza named
5.1.2 Human Referents Extractor
We calculate the agreement using the metrics WER, CER, and exact match. We achieve WER of 2.15%, CER (Morris et al., 2004b) (character error rate) of 0.54%, and an exact match score of 82%. We assume that these results reflect the quality of the full collection.
The extractor module sweeps the input text to find all proper nouns, nouns, and pronouns that refer people. Their positions in the text, as well as the positions of their heads the dependency parsing graph are then stored and used in the rewriter mod- ule. This task requires access to a Portuguese word- net (Miller, 1995). We have chosen OpenWWr PT (de Paiva et al., 2012), due to its integration with the NLTK library (Loper and Bird, 2002). NLTK allows access to the lexicographer file of each word sense. Lexicographer files are split into 45 categories. Category 18 contains "nouns denot- ing people". Our extractor module checks if the lexicographer file name of the current synset corre- sponds to this category. If so, that noun is referent to human, and, such, terms related to it should have the correct gender form.
Annotation disagreements often arise either from the existence of several possible gender-neutral al- temaaties, from uncertainty over if certain term should be neutralized. Examples of these types of disagreements are found in Table 1. In the first sentence, the devil (diabo) may be considered genderless entity, and therefore terms related to should not be rewritten. However, the female form of diabo, diaba, can be used, which may be used an argument in favor of diabo being gendered term. In sentence 2, the genderless term doentes is a synonym to the gender-neutral term enfermes, derived from the term enfermnos/enfermas)
5.1.3 Rewriter
The final module the RBM rewrites gendered terms related to human referents, whose positions in the sentence are stored by the extractor module. In Portuguese, nouns, pronouns, clitic pronouns, determiners, adjectives, and verbs can be gendered. each term
Annotator X
Annotator Y
Está o diabo à solta.
Está è diabe à solta.
ESs enfermes eram, muitas vezes, ex- muitas vezes, ex- postes na rua.
Es doentes eram,
The rules used for "gender-nnuuralizii depend on their word class, as described in further detail below.
postes na rua.
Table Annotator disagreements are marked in bold.
Nouns & Pronouns Since the neopronoun "elu" is the most used in the third neutral gender pro- posal we have presented in section 3, the system we employ in our model currently only uses that specific neopronoun. Certain gendered nouns have an existent gender-neutral synonym, which can be used instead of rewriting the gendered noun us- ing rules. The rewriter module performs a lookup to a table (depicted in Appendix A.2) containing some of these gender-neutral terms and uses them accordingly.
5 Models
5.1
Rule-based Model
As depicted in Figure 3 (Appendix B.1), the rule- based model (RBM) composed of three main modules, described in the following sections.
5.1.1 Preprocessing Pipeline
The preprocessing pipeline consists of tokeniza- tion, POS- roS-tagging, dependency parsing, and named entity recognition models made available by the Stanza (Qi et al., 2020) toolkit. At the time of writ- ing, Stanza does not make available any Portuguese named entity recognition model. Therefore, the
Clitic Pronouns a clitic pronoun is gendered, then it is rewritten (e.g. "Eu vou vê-la."" wll 4 "Eu vou vê-le.".
Definite articles that precede a proper noun are omitted (e.g. "O João fe-
Determiners
"EN: "The devil on the loose."
10 EN: "the sick were often exposed in the street."
"EN: "I'm going to see her."

---

liz "João é feliz."). Other types of determin- ers are neutralized their head in the dependency graph is referent to a human (e.g "João um ra- paz." n13 - "João é ume jovem.").
"João foi levado jantar." "João foi levade a jantar.") The RBM rewrites verb forms that require an auxiliary verb different from the verb "ter" (to have), since that particular verb is used in several non-gendered tenses. We make an exception for gerund, which can require an auxiliary verb dif- ferent from "ter", but is not gendered form (e.g "João foi andando." "15 + "João foi andando.").
Adjectives The task of checking whether an ad- jective refers to certain referent is complex, since an adjective can be in multiple positions in sen- tence. The rule-based rewriter module assumes that an adjective should be rewritten if either the adjec- tive itself or its head the dependency graph has been marked by the extractor module as either refer- ent to a human or head of term marked as refer- ent to a human. This rule tends to correctly rewrite adjectives in sentences with relatively simple con- struction. However, fails in sentences where adjective and human-referee noun share root term in the dependency graph. This is illustrated in Figure and Figure 2.
5.2 Neural Model
We fine-tuned the M2M100 (Fan et al., 2021) mul- tilingual encoder-decoder, setting both the source and target languages as Portuguese. For training, we used the original sentences of our automatically curated set source, and the respective versions rewritten by the RBM as target. We followed 80-10-18 split for training, validation, and test sets. For more details about the chosen hyperparameters and model please refer to Appendix B.
Experiments
6.1 Metries
NSUBJ XCOMP
Due to the novelty of the task of gender-neutral rewriting, we find that there is lack of general consensus useful metrics for evaluating these types of rewriter systems.
Fred
bonito
Fred é
bonite
3
For comparison with the works described in section 2, we employ WER and BLEU-4 (Pa- pineni et al., 2002) (usually the default n-gram value for BLEU scores) as metrics. Since gender- neutral language in Romance languages often con- sists in single character change (e.g. "João é bonito," 116 + "João é bonite."), we also employ CER (character error rate) metric. Additionally, we calculate BLEU-1 and ROUGE-1 (Lin, 2004).
Figure 1: Since both terms Fred and bonito share a head node, and Fred is human referent, the adjective is rewritten. This sentence can be translated as "Fred is pretty."
ROOT
APPOS
AMOD
PUNCT
6.2 Results
Bom
dia
senhora
+
Boe dia
senhore
We tested the RBM and the NM on our manually curated test set. We compare the results against a baseline metric (Base), which computes the met- rics between the original examples and the gender- neutral versions.
2
4
Figure 2: The same rule that worked for the on Figure fails with this sentence structure. Since senhora is a human referent, neutralized, even though it refers the term dia. This sentence can be translated as "Good morning, madam!"
adjective hom is incorrectly
The results are detailed Table 2. The superior performance of the neural model on the dataset with simpler sentences suggests that the neural ap- proach may be better suited for rewriting source sentences with simpler syntax. While the aver- age performance of the rule-based model tends to
Verbs Most verb tenses not gendered, with the exception of some main verbs that require an auxiliary verb, such as past participle forms (e.g
"EN: "João taken dinner."
12 PEN: "João is happy." 13 EN: "João is boy."
1 EN: "João walked."
EEN: "João is pretty."

---

be slightly better compared to the performance of the neural model, the higher ROUGE score of the neural model suggests that this approach tends to rewrite fewer terms, since a higher ROUGE score corresponds to a higher rate of true positives.
capturing if the term is related (or not) to a human referent.
Similarly to Vanmassenhove et al. (2021), we have found that neural model is able to general- ize over the training data. For instance, the fifth sen- tence of Table 4 contains the expression soprador de apito (EN: whistle blower). Although the word soprador does not exist in the model training data, the model is able to rewrite the term as the respec- tive gender-neutral form sopradore. However, this is not always the case: in the first sentence of Table 3, the neural model is not able to correctly rewrite the term ricas (EN: rich). We hypothesize that this may be either due the higher complexity of the respective gender neutral form (ricas -> riques) or due the lack of representation of female gender forms in data (both in our own datasets and the training data of our base neural model).
6.3 Discussion
Rule-based Approach If the RBM produces wrong output, the error can arise from one and/or two circumstances:
One of the preprocessing pipelines has pro- duced an error: for instance, the named en- tity recognition model may fail and wrongly tag noun human referent, as in the case of the first example found in Table 4. Since we are using a Spanish NER model, this is common mistake with names that may be un- usual in Romance languages (such as "Bill", found in the fourth example found in Table 4.) It might be worth to note that this problem is not specifically due to the NER model being focused on the Spanish language, but due it being focused on Romance language. These types of errors are not directly caused by the rules we have defined, and as such can only be mitigated by using different preprocessing tools.
Conclusion
In this paper, we present the first Portuguese dataset explicitly containing gender-neutral lan- guage, along with rule-based and neural gender- neutral rewriters. Additionally, provide manu- ally annotated collection of 500 original sentences and a respective gender-neutral version. One entry of our automatically curated dataset (used for train- ing the NM) consists of a binary-gendered sentence with the respective gender-neutral version provided by the RBM. We provide the first benchmarks of the gender-neutre rewriting task for the Portuguese language.
The rewriter rules have failed: The rules we have found to be most susceptible to errors are the ones regarding adjectives, as we have described in section 5.1.3. Since Portuguese adjectives diverse in terms of gendered terminations, creating rules that encompass all adjectives in the language may be complex and consuminn.
A Although the neural model can generalize over the seen data, we hypothesize that fails to internal- ize the context of sentences and whether gendered terms refer to humans or objects.
Neural Approach If the NM produces wrong output, the error arise from:
In future work, we hope to bypass this issue ei- ther by training larger model with more quality data, and/or developing a hybrid bale-based/neural model. Another possible approach is data augmen- tation using LLM prompting. replicate this approach, we have prompted Chat- GPT!7 the sibling model to InstructGPT (Ouyang et al., 2022), to generate Portuguese gender-neutral sentences. Results are depicted in Appendix D. Al- though promising approach, we hypothesize that the generated sentences do not have the necessary consistency and variety to create quality dataset. We expect that, in the future, with the advancement
The model has learned a wrongly rewritten form of a certain term: as is the case in the incorrect sentence depicted in Table 5, These types of errors may be mitigated by improving the quality of the rewriting rules.
first attempt to
The model is either ignoring terms that should be rewritten, rewriting terms that should be ignored: jectives are more susceptible to these types of errors, most likely due to the model simply learning how to rewrite a certain term, but not
have noticed that ad-
"chat..penai.com

---

deputades para Par- Social Media
Literary
Journalistic
Dialogue
Simple
Average
Base
15.07
13.43
21.93
14.22
26.45
18.63
WER %
RBM
7.89
4.98
7.95
5.27
7.02
6.62
NM
7.56
8.03
8.19
7.35
6.2
7.47
Base
3,90
2.86
7.20
3.75
7.36
5,01
CER %
RBM
1.63
1.03
2.26
1.23
1.72
1.57
NM
1.55
2.61
2.47
1.72
1.39
1.95
Base
57.90
71.79
42.93
65.97
32.89
54.30
BLEU-4
RBM
76.28
87.36
72.92
80.40
74.37
78.27
NM
79.77
83.91
71.60
78.53
76.72
78.106
Base
81.00
85.13
74.53
83.72
71.23
79.12
BLEU-1
RBM
90.75
94.32
90.75
93.25
93.00
92.414
NM
92.11
92.65
90.70
91.74
93.83
92.21
Base
83.15
90.80
79.15
86.63
74.41
82.93
ROUGE-1
RBM
91.67
95.00
91.04
93.77
93.38
93.17
NM
92.95
93.78
92.35
93.38
94.59
93.41
Table Metrics for the manually curated test for each data category. The best model for each category/metric pair is marked in bold.
RBM
NM
RBM
NM
Vocês são riques.
Vocês são ricas. Elu está parado na parte mais frie do
Irene é de Peru.
Irene é do Peru.
Elu está parade na parte mais fria do complexo.
Elu é a maiora pes- que viveu.
Elu é a maior pes- que já viveu. Eu dormi com a vi- tima número dois. Minhe parente não me deixa sair com Bill.
complexo.
Eu dormi com a ví- tima número dues. Minhe parente não me deixa sair com o Bill.
[...]
foram convi- [...] foram convi-
dadas dues profis-
dadas duas profis-
sionais vindes da
sionais vindas da
RTP [...].
RTP [..]]
Em 1989
Em 1989 con- seguiram eleger
[-..] Tom trabalhava como soprador de apito [...].
[...] Tom trabalhava como sopradore de apito [...].
seguiram eleger
deputades para
Parlamento lamento Europeu.
Boe dia criança !!! Bom dia criança !!!
Europeu.
Oi sou sub dele- gade
Oi sou a sub dele- gada
Example sentences where the neural model performs better than the rule-based model.
Table 4:
Caso Haddad seja
Caso Haddad seja eleito, eu fazer diferente de alguns you pensar positivo [...]
RBM
NM presidente reunirá amanhā com ês empresáries mais impor-
eleito, fazer
Correct
È presidente reunirá amanhā com és empresáries
diferente de alguns you pensar positivo [...]
Table 3:
Example sentences where the rule-based model performs better than the neural model.
mais impor- tantes do país.
tantes do país.
Incorrect
Você é muito gentile.
Você muito gentile.
of these types of models and optimized prompt- ing, we are able to generate quality nender-neutral examples that allow us to create larger inclusive datasets.
Table 5:
Example sentences where both models produce the same output (correctly incorrecty).

---

Limitations
References
Bashar Alhafni, Nizar Habash, and Houda Bouamor. 2022. User-centric gender rewriting. arXiv preprint arXiv:2205.02211
The usage of gender-neutral language in Portuguese-speaking communities is a diverse and ever-changing linguistic phenomenon. While we present some of the third neutral gender pronouns found in literature in section 3, our models only process the neopronoun elu and follow rewriting rules that are not universally agreed upon.
David Ameixa and Luísa Coheur. 2013. From subtitles to human interactions: introducing the subtle corpus. Technical report.
David Ameixa, Luisa Coheur, Pedro Fialho, and Paulo Quaresma. 2014. Luke, am your father: dealing with out-of-domain requests by using movies subti- tles. In Intelligent Virtual Agents: 14th International Conference, IVA 2014, Boston, MA, USA. August 27-29, 2014. Proceedings 14, pages 13-21. Springer.
Another limitation is that our rule-based model suffers from low scalability to long text. Handmade rules often fail to correctly rewrite long sentences due to their more complex and unpredictable struc- ture.
Chantal Amrhein, Florian Schotmmann, Rico Sennrich, and Samuel Läubli, 2023. Exploiting biased models to de-bias text: A gender-fair fair rewriting model. arXiv preprint arXiv:2305. 14000.
While using pre-trained large multilingual trans- lation models may be an option for developing gender-neutral rewriters for lower-resource lan- guages, this method is dependent on the existence of said models for the target language. Languages with very low resources are often not represented in such models.
Valerio Bellandi and Stefano Siccardi. 2022. Gender discriminatory language identification with hy- brid algorithm based on syntactic rules and machine learning.
Ethics Statement
Lukas Biewald. 2020. weights and biases. Software available from wandb.com.
Experiment tracking with
Our publicly available datasets were not filtered for harmful hateful content. While we fine-tuned our neural model on this data, since the output of our neural model consists of the input text with minor changes, we do not believe it susceptible to hallucinations of harmful text.
Gioni Caè. 2020. Manual para uso da linguagem neutra língua portuguesa. https://portall nila. edu, br wtr.ioorieee/aaall-----eeuuuuu nneurra/Manualde htaptc////hanooeeeeemmggggneeerr wnguagemeeutaaaprtttooooom
Accessed: 2023-06-23.
The annotation was voluntary and the partici- pants received no financial compensation for the task.
Valeria de Paiva, Alexandre Rademaker, and Ger- ard de Melo. 2012. Openwordnet-pt: An open Brazilian Wordnet for reasoning. In Proceedings of COLING 2012: Demonstration Papers, pages 353-360, Mumbai, India. The COLING 2012 Or- ganizing Committee. Published also as Techreport hww.d/hhhnneeeeeen4433550..
We do not claim that any Portuguese gender- neutral grammar system, gender-neutral termina- tion, neopronoun is in any way superior or should be used to detriment of others. In future work, we expect develop models that accommo- date the addition new systems.
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centre multi- lingual machine translation. The Journal of Machine Learning Research, 22(1):483944886
Acknowledgements
We would like thank the voluntary annotators for their contributions.
Susan Gal. 1978. Peasant men can't get wives: Lan- guage change and sex roles in bilingual community. Language in society, 7(1):1-16.
This research was supported by the Portuguese Recovery and Resilience Plan through the project C645008882-000000 (Center for Responsible AD), and through Fundação para a Ciência a Tecnologia (FCT), specifically through the INESC-ID multi-annual funding with reference UIDB/50021/2020.
Susan Gal. 1989. Between speech and silence: The problematics of research on language and gender. 1PrA Papers in Pragmatics, 3(1):1-38.
Nishtha Jain, Maja Popovic, Declan Groves, and Eva Vanmassenhove. 2021. Generating gen- der augmented data for nlp. arXiv preprint arXiv:2107.05998.

---

Chin-Yew Lin. 2004. ROUGE: package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguisties.
Challenges of an international shared task. Com- putational Processing of the Portuguese Language: 15th International Conference, PROPOR 2022, For- taleza, Brazil, March 21-23, 2022, Proceedings, pages 413-419. Springer.
Edward Loper and Steven Bird. 2002. Nitk: The natural language toolkit. arXiv preprint cx/0205028.
Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, and Melvin Johnson. 2021. They, them, theirs: Rewriting with gender english. arXiv preprint arXiv:2102.06788
V. Marques and D. Santos. 2021. Guia prático para um portugués inclusivo. http: http://queerist /queerist..eeteeeeeeeee. tt tecnico .ulisboa. httpsssaaptpppjjeeeeee pt/projjetos/linguggee 2022-10-28.
Accessed:
Jörg Tiedemann. 2020. The tatoeba translation Mhallenge-realistie data sets for low resource and multilingual mt. arXiv preprint arXiv:2010.063: 0..6544.
George A Miller. 1995. Wordnet: lexical database for english. Communications of the ACM, 38(11):39-44
Pedro Valente. 2020. Sistema elu, linguagem neutra elu
Maria Juliana Ravália Miranda. 2020. Português para todes?: um diálogo entre análise de discurso crítica sociolinguística sobre linguagem não binária. Ac-
em género. https: /ttp///eeeaeeee/ee///////aaaeeee -inguagem= neutra em genero- 1317469. Ac- cessed: 2022-10-30.
https: //bdm. unb. br htthhh/nlee1144///2// cessed: 01-12-2022.
Eva Vanmassenhove, Chris Emmery, and Dimitar Shterionov. 2021. Neutral rewriter: A rule- based and neural approach to automatic rewriting into gender-neutral alternatives. arXiv preprint arXiv:2109.06105
Andrew Morris, Viktoria Maier, and Phil Green. 2004a. From and ril to mer and wil: improved evaluation measures for connected speech recognition.
Andrew Morris, Viktoria Maier, and Phil Green. 2004b. From wer and ril to mer and wil: improved evaluation measures for connected speech recognition.
J.P. Woodard and J.T. Nelson. 1982. An information theoretic measure of speech recognition performance.
Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang. Muhao Chen, Ryan Cotterell, and Kai- Wei Chang. 2019. Examining gender bias in lan- guages with grammatical gender. arXiv preprint arXiv:1909.02224.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models follow instruc- tions with human feedhack. Advances Nearal Information Processing Systems,
C. Souza P. Berlucci, A. Zanell. 2015. Manifesto ILE. tybbox. com/mani festo-ile--ee ara- uma comuni wawp:/ccccicccccaaraaaaaaaa cacao- wawaaao-raaaaaalleaeeaaaaaaaaaee mente-innll va/. Accessed: 00-2-10-28.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei jing Zhu, 2002. Bleu: a method for automatic evalu- ation of machine translation. pages 31 1-188.
Larissa Roberta Rosa Pinheiro. 2020. Linguagem neutra: reestruturação do gênero no português brasileiro frente às mudanças sociais. https:// bdm. unb. bt 200addll/114332222222
Matt Post. 2018. A call for clarity in reporting BLEU scores. Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Belgium, Brussels. Association for Computa- tional Linguistics.
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. arXiv preprint arXiv:2003.07082
Diana Santos, Roberto Willrich, Marcia I Langfeldt, Ri- cardo Gaiotto de Moraes, Cristina Mota, Emanoel Pires, Rebeca Schumacher. and Paulo Silva Pereira. 2022. Identifying literary characters in portuguese:

---

A Gender-neutral Grammar Details and Examples
A.4 Dataset Examples
Table 9 depicts examples for each dataset category.
A.1 Examples from Users of Portuguese Gender-neutral Language
B Model Details
B.1
Rule-based Model Pipeline
Algum amigue pelo litoral de cabedelo para um rolè na praia?
Figure 3 depicts the three main modules and spective dependencies of our RBM.
Minha amigue tirou runa das bruxas para mim e eu estou impactada
B.2 Hyperparameter Search Results
Todas todes recebendo trafamento
igualitário...
We performed automated hyperparameter search using Weights & Biases (Biewald, 2020). We ran total of ten sweeps, exploring different combina- tions of values for learning rate and weight decay. The results are depicted in Figure 4.
Procuro um namorade que me dè carinho
aateççoo
Table 6: Example excerpts retrieved from Twitter in 20/02/2023. We slightly modified the examples to lower searchability increase the privacy the authors.
B.3 Training Hyperparameters
We fine-tuned the M2M100 1111000_888M0 418M model for 5 epochs, using learning rate of 0.00005569, weight decay of 0.02, batch size of 8, and eval batch size of 8. We maximum target length of 128. The rest of the parameters are left as Hugging- Face TraSeeqraininggArrumenn
A.2
Gender-neutral Expressions
Original Expression
Gender-neutral
Expression
homem/mulher
pessoas
defaults. We used the Seq2Seq Trainer class to complete the fine-tuning procedure, using sacrebleu (Post, 2018) for our the compute compute_metries fuction.
rapaz/rapariga
jovem
menino/menina
criança
pai/măe
parente
aluno/aluna
estudante
C Annotation Guidelines Sample
professor/professoraa
docente
esposo/esposa cônjuge
The annotation guidelines contain rewriting rules for every gendered word class. For the sake of brevity, in this section present only the rules for determiners (determinantes, in Portuguese) and contractions with prepositions (contrações com preposições). Although the original guide was writ- ten in Portuguese, here we also present an English version of the text.
rei/rainha
monarca
Table 7: Binary-gendered expressions and respective
gender-neutral alternative expressions.
A.3
Usage of Different Portuguese Neopronouns
Personal Pronoun
"Ele comeu a pizza dele. 18v
C.1 Determinantes
Se precedidos por um nome próprio, os deter- minantes artigos definidos (o. a, os, as) devem ser omitidos. Se precedidos por um nome comum, não devem ser omitidos. Os determinantes artigos in- definidos (um, uma, uns, umas) nunca são omitidos se referentes uma pessoa, devem ser reescritos respetiva forma neutra.
PT
clu
"Elu comeu pizza delu."
ile
"Ile comeu a pizza dile."
ilu
"Ilu comeu pizza dilu."
éle
"Éle comeu a pizza déle."
"El comeu a pizza del."
httgs://tuuuiinnaeeccccccceo//o/ coffaceeoook/222100000 418M
Table 8: Usage of different Portuguese neopronouns.
coldocs/transformerss/
main httpsseeeetiietttttttaooomeeeees
"https:/huggingglai main, Classeeraannctttttoooooe...eeeo
18 EEN: "He ate his pizza."

---

Category
Miguel é ume Original Sentence
Gender-neutral Sentence
Literary
orgulhoso de opinião, como ele só! (EN: He is proud and opinion- ated, only he can be!)
orgulhose de opinião, como elu só!
Journalistic
Fomos procura deles organizámos um almoço comemorativo. (EN: We went looking for them and organized a celebratory lunch.)
Fomos procura delus organizámos um almoço comemorativo.
Dialogue
Precisa saber só de olhar para mul- her, sem ela dizer. (EN: He needs know just by looking at the woman, without her saying so.)
Precisa saber só de olhar para a pes- soa, sem elu dizer.
Social Media
foi ela quem fez exorcismo. (EN: she was the one who performed the exorcism.)
foi elu quem fez exorcismo.
Simple Sentences
Eu estou viciado em mascar chiclete. (EN: I' I'm addicted to chewing gum.)
Eu estou viciade em mascar chiclete.
Table 9: Dataset categories and respective examples. The original sentences contain idiomatic expressions, which we tried to capture the English translation.
Wordnet
Gender-neutral Grammar
imput
Preprocessing Pipeline
Human Referents Extractor
Rewriter
outpul
tokenizer
mwt
pos
lemma
depparse
ner
Figure 3: RBM Pipeline.
EN If preceded by a proper noun, the definite article determiners (o, a, os, as) must be omitted. preceded by a common noun, they must not be omitted. The indefinite article determiners (um, uma, uns, umas) are never omitted and, if they refer to a person, must be rewritten according to the respective neutral form.
Miguel escritor. escritore.
C.2 Contrações com Preposições
Os casos específicos das contrações pelo/pela ao/à possuem uma particularidade: no caso de serem precedidas optar pela la forma neutra apresentada (por ou a). No caso de serem precedidas por outro tipo de nome, deve-se optar pela 2 forma neutra apresen- tada (pele ou ae).
um nome próprio, deve-se
(s))a(()) é(s)ooo
O Sérgio é amigo da Mariana. 22 Sérgio amigue de Mariana.
EN For the particular cases of the contractions pelo/pela and ao/à: in case they are preceded by a proper noun, one should opt for the 1st presented neutral form (por or a). If they are preceded by
O João é simpático." 23 João simpátique.
um/uma(s) ume(s)
22 EN: "Sérgio is Mariana's 's friend."
23 EN: "João is nicc."
2'EN: "Miguel writer."

---

weight_decay
learning_rate
eval/loss
eval/bleu
0.050
0.000100
0-160
94.2
0.045
0.981090
0.158
94.1
0.156
94.0
0.040
0.009080
0.154
93.
0.035
0.00962g
0.152
93.8
0.150
982
0.000060
0.025
0.148
93.6
0.000050
0.020
0.145
93.5
0,000040
0.144
93.4
0.015
0.142
93.3
0.009030
0.010
0.140
93.2
0.005
0.000000
0.138
93.1
0.000
0.000010
0.136
93.0
Figure 4: Hyperparameter search results, provided by Weights & Biases (Biewald, 2020). The best configuration consists of weight decay of 0.02 and learning rate of 0.00005569.
another type of noun, one must choose the 2nd neutral form presented (pele or ae).
Generated Sentences Wrong gender-neutral terms or terms whose gender agreement is cor- rect are marked in bold.
pelo(s)/pela(s) por/pele(s)
Elu um estudante dedicade. (EN: They are a dedicated student.)
A Mariana fez tudo pela Lúcia. + Mariana fez tudo por Lúcia.
Elu adora explorar novos lugares e culturas. (EN: They love exploring new places and cul- tures.)
A Mariana fez tudo pelas amigas. 26 Mariana fez tudo peles amigues.
Elu trabalha como profissionale da área de tecnologia. (EN: They work as &eechnology professional.)
ao(s)/ā(s) a(S)/ae(s)
A Teresa deu o livro ao Pedro. 27 + Teresa deu livro a Pedro.
Elu uma pessoa criative e talentose. (EN: They are a creative and talented person.)
A Teresa deu livro amigo. 28 + Teresa
Elu está escrevende um livro incrível. (EN:
deu livro amigue.
They are writing an amazing book.)
D LLM Prompting
Prompt "Generate examples of Portuguese gender-neutral sentences, following the "elu" sys- tem. Bear in mind that, in the "clu" system, terms that refer to humans tend to have an -e termination. Make sure to use sentences that include nouns, ad- jectives, and verbs."
25 EEN:
"Mariana did everything for Lúcia."
EN: "Mariana did everything for her friends."
27 EN:
"Teresa gave Pedro book."
28 EN:
"Teresa gave the book to her friend."


Reviews:
[{"review_id": "FSfGGTtsbC", "reviewer": "Reviewer_jbYX", "summary": "", "questions": "* Why stop only on  25,000 augmented examples to train on? \n* Did you try sub-sampling or active learning on the augmented data?\n", "limitations": "", "rating": 3, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. If the paper is accepted, we will take into account all your comments.\n\nRegarding your questions and reasons to reject:\n1. The fact that the neural model was trained on annotations created by the rule-based model is not necessarily the reason why the neural model's performance is lower. This was also the method used by Vanmassenhove et al. (2021), whose neural model achieves a better performance than the rule-based model. Of course, English and Portuguese are languages with very different morphology and in future work we plan on doing a more thorough error analysis.\n2. We have trained a neural model using 50,000 augmented examples, which achieves similar performance to the one trained with 25,000 examples. Since creating these examples using the rule-based model can be time-consuming and does not entail better results, we have not reported this experiment in the paper.\n3. We have not sub-sampling or active learning on the data, but we will certainly consider it in future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Well written paper.\n* Interesting new contribution - most previous work was Germanic languages; this is on Portuguese, which is quite under represented in NLP.\n* New public data set. \n* New benchmarks on this task.", "reasons_to_reject": "* Golden data set is too small (500 examples)\n* The annotations used to train the neural model are based on the rule based model... which explains the neural model lower performance than the rule based model. ", "questions_to_authors": "* Why stop only on  25,000 augmented examples to train on? \n* Did you try sub-sampling or active learning on the augmented data?\n", "paper_topic_and_main_contributions": "This paper presents an important contribution in the space of modeling gender-inclusive language, specifically in Portuguese.  \nThe paper created a gold test set and compared two models for the task of gender-inclusive rewriting.\nWhile it may be argued that the task is \"artificial\" it is still important as current LLM mods will likely suffer from the limited resources in the target form of the language. So, the big picture is timely.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "* Cite Arabic Parallel Gender Corpus (APGC) when your refer to it: https://aclanthology.org/2022.lrec-1.199/\n* Please cite the ACL Anthology instead of Arxiv when possible!"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1 . As far as we concern \n>> As far as we are concerned \nOR To the best of our knowledge.\n\n2 .  Prepositions in headers should be lower cased:\nProposals for a Neutral Grammatical Gender Within Portuguese \n>>\nProposals for a Neutral Grammatical Gender within Portuguese \n\n"}}, {"review_id": "0B3yFVSIFZ", "reviewer": "Reviewer_CAWR", "summary": "", "questions": "See above.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. If the paper is accepted, we will take into account all your comments.\n\nRegarding your questions and reasons to reject:\n\n1.1 The \"development set\" examples are not necessarily gendered. We have trained the neural model with examples that are not necessarily gendered in the hope that it would learn which terms should be rewritten and which terms should not. We have created a gendered manually curated test set due to the core of the work being gender-neutral rewriting. Using a gendered test set helps us to easily understand the performance of the models, since these models would most likely be used with gendered sentences (in a real-world scenario). Vanmassenhove et al. (2021) have also created a gendered test set.\n\n1.2 You are absolutely right. What we call \"development set\" is the dataset used during the training process of the neural model.\n\n2.1 We have not been clear about the NER problems. Using a Portuguese NER model would not solve the data errors that are created due to the named entity recognition module. The actual problem is that we are using a Romance language NER model. For instance, the example we provide in the paper mentions the name \"Bill\", which is not tagged as a human referent due to it not being picked up by the NER model. A Portuguese model would most likely also not recognize \"Bill\", since it is an English name. Since we are creating resources for the Portuguese language, we have decided to use a NER model trained on a language similar to Portuguese, even though this may cause errors when encountering a non-Spanish/Portuguese name. \n\n2.2 We have not been clear about the \"development set\" splits. Our \"development set\" was split into training, validation, and test sets because it is required by the HuggingFace framework we have used during the neural model's training process. Those splits are different from what we call \"Test Set\", described in Section 4.2. Our test set consists of 500 manually curated gendered sentences, and was used for evaluating both the neural model and the rule-based model. There is no intersection between the \"development set\" and the test set.\nNote: the rule-based model was not trained (due to its rule-based nature).\n\n2.3 We have named the \"Do Nothing\" system \"Base\", because it is the term used by Vanmassenhove et al. (2021). We have found that authors of gender-neutral rewriters tend to report the results of  \"doing nothing\" for easier comparison with the results of their models. Amrhein et al. (2023) call this \"Source\" system, for example.\n\n3. You are absolutely right, we are planning a more thorough error analysis as future work. We were not able to report a manual error analysis with a quantification of errors because it can be a very time consuming task, mainly due to the complexity of the morphology of the Portuguese language and the large amount of error types.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The dataset will be made publicly available and would be very useful in building gender-neutral NLP models. This will reduce the biases that are embedded with the models, and which are amplified further in downstream tasks. Moreover, the dataset constitutes the first of its kind on the problem of gender-neutral rewriting in Portuguese, which is a gender-marking language and morphologically richer than English. Further, the authors do a good job of explaining grammatical gender in Portuguese and the various ways to express gender neutrality.   ", "reasons_to_reject": "I found parts of the paper to be confusing and somewhat lacking across various sections:\n\n\n1. Data:\n    * The authors mention that all examples in the test set are gendered, what about the examples in the so-called \"development\" set? Are they all gendered? If so, what is the percentage of the sentences containing at least one masculine vs feminine gendered reference? If not all the sentences are gendered, how many sentences do not contain any gender references? Adding these statistics to the dataset description would make things much clearer to the readers.\n\n    * I also found calling a dataset a \"development\" set is confusing in the context of this paper. Particularly because a development set (or a dev set), has the same meaning as a \"validation\" set. So I would avoid using the term development to describe the dataset that encompasses both training, validation, and testing.\n\n    * Most people reading this paper do not understand Portuguese, so including English translations within the tables and highlighting the gendered words will make things easier to read. \n\n2) Models & Experiments:\n\n    * Rule-based system: The authors mention that Stanza doesn't have a NER system for Portuguese so they used the Spanish NER system. Why didn't the author train a Portuguese-specific NER model instead? That should be trivial given that there are Portuguese NER datasets available. The authors in fact mention that this could lead to data errors (Section 6.3) and therefore the models would not be able to be trained and evaluated properly. This makes the quality and usability of the dataset questionable.\n\n    * Neural Model: The authors mention that split the \"development\" data twice and I found that to be confusing. It's unclear to me if they follow the same splits that were used to train the rule-based and the neural models.\n\n    * The \"base\" system should really be named a \"Do Nothing\" baseline because it basically outputs the input to the output as it is.\n\n3) Error Analysis:\n\n    * The authors discuss the main sources of where the errors could be coming from when evaluating the models. But they do not provide any sort of quantification on each of these sources. Providing examples indicating that the rule-based system is better than the neural model (or vice-versa) is not sufficient to understand the models' behaviors quantitatively. I would advise the authors to either do a manual error analysis on a subset of examples to quantify the types of errors across different models.\n\n", "questions_to_authors": "See above.", "paper_topic_and_main_contributions": "In this paper, the authors present the first rule-based and neural-based gender-neural rewriting models for Portuguese. They built the first Portuguese parallel dataset for gender-neutral rewriting across five domains: literary texts, journalistic texts, dialogues, social media posts and comments, and simpler sentences. The dataset was created automatically by using a rule-based model which the authors built to gender-neutralize gendered sentences. The dataset contains 25000 parallel sentences (5000 sentences per domain). Further, the dataset contains 500 sentences that were manually annotated and used for testing the gender-neutralizing rewriting models' performances. Moreover, the authors demonstrate the usage of the dataset by building different gender-neutral rewriting models and evaluating their performance.\n\n\n\n\n\n\n", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1) I find the interchangeable usage of gender-neutral and gender-fair to be confusing. I think sticking to one would make things easier.\n\n2) Table 9: what do the footnotes stand for? I am assuming English translations but those are not provided.\n"}}, {"review_id": "F70KgRELlo", "reviewer": "Reviewer_KYF6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback. If the paper is accepted, we will take into account all your comments.\n\nYou are right, we do not claim to make technical contributions. We are providing resources for the Portuguese language in the form of datasets containing gender-neutral language and gender-neutral rewriter models.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Valuable resources: The papers provides new resources for gender-neutral rewriting in Portuguese: a small-scale, manually created test set as well as two different rewriting systems (I could not verify the resources because the code and data were not provided to me).\n- Linguistic expertise: The paper presents information on grammatical gender and gender-neutral language in Portuguese, which is helpful for future work.\n- Clear presentation: The paper is well-written and easy to follow.\n", "reasons_to_reject": "- Limited technical novelty: The main contribution of the paper are resources and baseline experiments for gender-neutral rewriting in Portuguese, for which such resources did not previously exist. The paper does not make novel technical (e.g., algorithmic or modeling) contributions (nor does it claim to do so).", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is about gender-neutral rewriting for the Portuguese language. The authors describe the specific difficulties that Portuguese poses for gender-neutral languages. They then apply some established techniques, namely a rule-based rewriter and a paraphrasing model trained on augmented data, to a new, manually created test set across five domains. An automatic and a qualitative evaluation indicate that the neural approach shows some generalization, but is overall less accurate than the rule-based approach.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work presents a rule-based and a neural-based tool for gender-neutral rewriting for Portuguese and provide the first Portuguese dataset explicitly containing gender-neutral language, including a manually annotated gold standard.

The reviews show ambivalent scores with respect to soundness and excitement, which mostly fall on the upper bound, as it is recognized to be an interesting contribution and topic. Yet, the concerns raised are sensible, and have to do with a fundamental part of the work: the strategy of using a rule-base system to create the data to train the neural system, something which may explain why the rule-based system performs better, even though the authors do not sufficiently discuss this issue. The remaining issues should also be clarified in the final version if accepted (the NER problems, data description, Portuguese examples translated or glossed to facilitate comprehension of gendered terms, etc.)

I should say that a gold standard or 500 sentences for a task like this does not seem too small for a low-resource context. Furthermore, the fact that the paper does not make novel technical (e.g. algorithmic or modelling) contributions should not preclude its publication, as it is  not the work's objective.
Decision: Accept-Findings