The Shared Task on Gender Rewriting (2022)

Abstract: In this paper, we present the results and findings of the Shared Task on Gender Rewriting, which was organized as part of the Seventh Arabic Natural Language Processing Workshop. The task of gender rewriting refers to generating alternatives of a given sentence to match different target user gender contexts (e.g., a female speaker with a male listener, a male speaker with a male listener, etc.). This requires changing the grammatical gender (masculine or feminine) of certain words referring to the users. In this task, we focus on Arabic, a gender-marking morphologically rich language. A total of five teams from four countries participated in the shared task.

## introduction the problem of gender bias in natural language processing (nlp) systems has been receiving a lot of attention across a variety of tasks such as machine translation, co-reference resolution, and dialogue systems. research has shown that nlp systems do not only have the ability to embed societal biases, but they also amplify and propagate them in ways that create representational harms and degrade users' experiences @xcite . the main cause of this problem is usually attributed to inherently biased data that is used to build these systems and which mirrors the inequalities of the world we live in. therefore, many approaches were proposed to mitigate this problem by either using counterfactual data augmentation techniques @xcite @xcite or by debiasing pretrained representation that is trained on biased data @xcite @xcite . however, even the most balanced of models can still exhibit and ## task description the task of gender rewriting was introduced by @xcite and it refers to generating alternatives of a given arabic sentence to match different target user gender contexts. we focus on contexts involving two users (i and/or you)first and second grammatical persons with independent grammatical gender preferences. this requires changing the grammatical gender (masculine or feminine) of certain words referring to the users (speaker/first person and listener/second person) in the input sentence. therefore, given an arabic sentence as an input, the goal is to generate four different gender rewritten alternatives to match the different target user gender contexts (i.e., female speaker with a male listener, a male speaker with a male listener, a male speaker with a female listener, and a female speaker with a female listener). notation we use the notation that is defined by @xcite . namely, we use four elementary symbols to facilitate the discussion of this task: 1m, 1f, 2m and 2f. the digit part of the symbol refers to the grammatical person (1 st or 2 nd ) and the letter part refers to the grammatical gender (masculine or feminine). additionally, we use b to refer to invariant/ambiguous gender. ## participants and systems five teams from four countries participated in the shared task. ## outlook and lessons learned we organized this shared task on gender rewriting for arabic to raise awareness in the arabic nlp community of the problem of gender bias in arabic nlp systems, and to encourage the community to come up with new approaches to alleviate this problem. although the shared task received some interest from the community, the participation was limited foot_7 when compared to other shared tasks organized at recent editions of wanlp foot_8 or osact. 10 we believe that this is due to a couple of factors. first is the skewed interest towards sentencelevel classification tasks within the arabic nlp community and the lack of novel open-vocabulary sequence transduction tasks. for instance, most of the shared tasks organized at wanlp over the past few years focused on sentence-level classification to tackle dialect identification: madar and nadi @xcite @xcite ; or arabic sarcasm detection: arsarcasm @xcite . the last shared task that featured a generation problem in arabic was the qalb shared task on grammatical error correction @xcite . we acknowledge the importance of working on sentence-level classification problems, but there are many natural language generation tasks where arabic is still lagging behind compared to other languages. examples of such tasks include dialectal machine translation, grammatical error correction, text simplification, and style transfer, to name a few. we envision that the development of resources and models for such tasks would re-spark the interest of the arabic nlp community in a wide range of exciting, yet unsolved problems in arabic nlp. second is the novelty and difficulty of the gender rewriting problem compared to other conven-tional sequence transduction tasks. approaching the problem correctly requires developing controlled generation models that are able to make subtle, yet complex and grammatically correct, edits at the word level. in retrospect, we recognize that we could have organized this shared task as two subtasks: one on gender identification at the word or sentence levels, and the other on sentencelevel gender rewriting. this could have served as a bridge between classification and generation tasks, too, and allowed more people to participate for part if not the whole of the task. as such, we recommend that organizers of novel and nontraditional tasks to break the problem into subtasks to encourage more participation. lastly, the main goal of participating in a shared task is to learn about a new problem by introducing an interesting solution, which could benefit the community as a whole, as a positive or negative result. being on top of the leaderboard should not be the only motive; we encourage organizers within the community to echo this sentiment when running their shared tasks.