Abstract: Vast availability of text data has enabled widespread training and use of AI systems that not only learn and predict attributes from the text but also generate text automatically. However, these AI models also learn gender, racial and ethnic biases present in the training data. In this paper, we present the first system that discovers the possibility that a given text portrays a gender stereotype associated with an occupation. If the possibility exists, the system offers counter-evidences of opposite gender also being associated with the same occupation in the context of user-provided geography and timespan. The system thus enables text de-biasing by assisting a human-in-the-loop. The system can not only act as a text pre-processor before training any AI model but also help human story writers write stories free of occupation-level gender bias in the geographical and temporal context of their choice.

Introduction: AI systems are increasing and Natural Language Generation is getting ever more automated with emerging creative AI systems. These creative systems rely heavily on past available textual data. But often, as evident from studies done on Hollywood and Bollywood story plots and scripts, these texts are biased in terms of gender, race or ethnicity. Hence there is a need for a de-biasing system for textual stories that are used for training these creative systems. Such de-biasing systems may be of two types 1) an end-to-end system that takes in a biased text and returns an unbiased version of it or 2) a system with a human-in-the-loop that takes a text, analyzes it and returns meaningful clues or pieces of evidence to the human who can appropriately modify the text to create an unbiased version. Since multiple types of biases may exist in the given text, the former de-biasing system requires identifying which biases to focus on and how to paraphrase or modify the sentence to de-bias it. These notions can often be subjective and it might be desirable to have a human-in-the-loop. This is the focus of the latter de-biasing system as well as the approach taken by us in the paper. Gender stereotyping with respect to occupations is one of the most pervasive biases that cuts across countries and age groups BIBREF0 . In this paper, we focus on de-biasing with respect to gender stereotyping in occupations. This bias has also been recently noted in machine translation systems BIBREF1 . In this translation tool, the sentences “He is a nurse. She is a doctor" were translated from English to Turkish and back to English which inappropriately returned “She is a nurse. He is a doctor"! In this paper, our system takes a piece of text and finds mentions of named entities and their corresponding occupations. From the gender of the named entities, the system suggests examples of real people with alternate gender who also had the corresponding occupation. The rest of the paper is organized as follows - Section 2 describes the related work, Section 3 discusses about the design and Section 4 lays out the implementation of our de-biasing system. In Section 5 we describe a walk-through of our system and in Section 6 we conclude our paper.

Question: Which dataset do they use to train their model?
Answer: A dataset they created that contains occupation and names data.