Easy Adaptation to Mitigate Gender Bias in Multilingual Text Classification (2022)

Abstract: Existing approaches to mitigate demographic biases evaluate on monolingual data, however, multilingual data has not been examined. In this work, we treat the gender as domains (e.g., male vs. female) and present a standard domain adaptation model to reduce the gender bias and improve performance of text classifiers under multilingual settings. We evaluate our approach on two text classification tasks, hate speech detection and rating prediction, and demonstrate the effectiveness of our approach with three fair-aware baselines.

## introduction recent research raises concerns that document classification models can be discriminatory and can perpetuate human biases @xcite @xcite @xcite . building fairness-aware classifiers is critical for the text classification task, such as hate speech detection and online reviews due to its rich demographic diversity of users. the fairness-aware classifiers aim to provide fair and non-discriminatory outcomes towards people or groups of people based on their demographic attributes, such as gender, age, or race. fairness has been defined in different ways @xcite across downstream tasks; for mitigating biases in the text classification, existing research @xcite @xcite has focused on group fairness @xcite , under which document classifiers are defined as biased if the classifiers perform better for documents of some groups than for documents of other groups. methods to mitigate demographic biases in text classification task focus on four main directions, data augmentation @xcite @xcite , instance weighting @xcite , debiased pre-trained embeddings @xcite , and adversarial training @xcite @xcite . the existing studies have been evaluated on english datasets containing rich demographic variations, such as wikipedia toxicity comments @xcite , sentiment analysis @xcite , hate speech detection @xcite . however, the methods of reducing biases in text classifiers have not been evaluated under multilingual settings. in this study, we propose a domain adaptation approach using the idea of "easy adaptation" @xcite and evaluate on the text classification task of two multilingual datasets, hate speech detection and rating prediction. we experiment with non-debiased classifiers and three fairaware baselines on the gender attribute, due to its wide applications and easily accessible resources. the evaluation results of both non-debiased and debiased models establish important benchmarks of group fairness on the multilingual settings. to our best knowledge, this is the first study that proposes the adaptation method and evaluates fair-aware text classifiers on the multilingual settings. ## multilingual data we retrieved two public multilingual datasets that have gender annotations for hate speech classification @xcite and rating reviews @xcite . 1 the hate speech (hs) data collects online tweets from twitter and covers four languages, including english (en), italian (it), portuguese (pt), and spanish (es). the rating review (review) data collects user reviews from trustpilot website and covers four languages, including english, french (fr), german (de), and danish (da). the hs data is annotated with binary labels indicating whether the tweet is related to hate speech or not. the review data has five ratings from 1 to 5. to keep consistent, we removed reviews with the rating 3 and encoded the review scores into two discrete categories: @xmath0. all the data has the same categories for the gender/sex, male and female. we anonymized tweets, lowercased all documents, and tokenized each document by nltk @xcite , which supports processing english and the other six languages. we summarize the data statistics in ## easy adaptation framework previous work has shown that applying domain adaptation techniques, specifically the "frustratingly easy domain adaptation" (feda) approach @xcite , can improve document classification when demographic groups are treated as domains @xcite . based on these results, we investigate whether the same technique can also improve the fairness of classifiers, as shown in figure 1 . with this method, the feature set is augmented such that each feature has a domain-specific version for each domain, as well as a domain-independent (general) version. specifically, the features values are set to the original feature values for the domain-independent features and the domain-specific features that apply to the document, while domain-specific features for documents that do not belong to that domain are set to 0. we implement this via a feature mask by the element-wise matrix multiplication. for example, a training document with a female author would be encoded as [f general , f domain,f emale , 0], while a document with a male author would be encoded as [f general , 0, f domain,male ]. at test time we only use the domain-independent features. while the feda applies to non-neural classifiers, we treat neural models as feature extractors and apply the framework on neural classifiers (e.g., rnn). we denote models with the easy adaptation with the suffix -da. ## experiments demographic variations root in documents, especially in social media data @xcite . in this study, we present a standard domain adaptation model on the gender factor, and we treat each demographic group as a domain (e.g., male and female domains). we show the domain adaptation method can effectively reduce the biases of document classifiers on the two multilingual corpora. each corpus is randomly split into training (80%), development (10%), and test (10%) sets. we train the models on the training set and find the optimal hyperparameters on the development set. we randomly shuffle the training data at the beginning of each training epoch. ## conclusion we present an easy adaptation method to reduce gender bias on two downstream tasks (hate speech detection and user rating prediction) under the multilingual setting. the experiments show that by treating demographic groups as domains, we can reduce biases while keeping relatively good performance. our future work will solve the limitations of this study, including non-binary genders, multiple demographic factors, embedding sources, and label imbalance. code and data instructions of our work are available at https://github.com/ xiaoleihuang/domainfairness .