Disentangling Document Topic and Author Gender in Multiple Languages: Lessons for Adversarial Debiasing (2021)

Abstract: Text classification is a central tool in NLP. However, when the target classes are strongly correlated with other textual attributes, text classification models can pick up “wrong” features, leading to bad generalization and biases. In social media analysis, this problem surfaces for demographic user classes such as language, topic, or gender, which influence the generate text to a substantial extent. Adversarial training has been claimed to mitigate this problem, but thorough evaluation is missing. In this paper, we experiment with text classification of the correlated attributes of document topic and author gender, using a novel multilingual parallel corpus of TED talk transcripts. Our findings are: (a) individual classifiers for topic and author gender are indeed biased; (b) debiasing with adversarial training works for topic, but breaks down for author gender; (c) gender debiasing results differ across languages. We interpret the result in terms of feature space overlap, highlighting the role of linguistic surface realization of the target classes.

## introduction natural language processing, and machine learning more generally, has recently received a significant deal of criticism because of the frequent presence of bias in the predictions, where we define bias as a systematic difference in system performance on one set of instances compared to another. such biases have been identified in nlp tasks such as word representation @xcite , textual inference @xcite , coreference resolution @xcite ), text classification @xcite and emotion intensity prediction @xcite . in text classification tasks, a principal source of such biases are demographic attributes of authors, such as gender, age, or race foot_0 . the reason is that these attributes shape speakers' language use substantially @xcite . nlp models are not only able in principle to pick up such cues, as studies on modeling demographic attributes show @xcite , but they actually have a motivation to do so whenever some demographic attribute is strongly correlated with the model's classification target and therefore supports its recognition. as an example, in social psychology, @xcite report that elderly people experience and express their emotions less intensely than younger people. therefore, in a corpus of emotional expressions across age groups, it is reasonable for a model that predicts emotion intensity to look out for linguistic cues regarding author age, even if these cues are not really related to emotion intensity per se, such as typical markers of youth language ("rad", "fam", "ftw", etc.). this focus is arguably problematic, though, since it can give rise to a form of age bias -namely, overestimating emotion intensity for documents exhibiting youth language. more generally, the bias-inducing role of demographic attributes is dangerous for studies that use texts from a multitude of authors -often gathered from social mediato draw inferences about the authors @xcite . in such studies, demographic biases can lead to erroneous causal attributions (as our case will illustrate). to counteract the presence of biases in nlp, researchers have devised debiasing methods. due to its general applicability and high effectiveness, adversarial debiasing has become one of the most widely used methods for bias mitigation @xcite @xcite . unfortunately, these advances are not accompanied by an analysis of the prerequisites that need to be satisfied for adversarial training to perform successfully. it has been established empirically is that adversarial training works well for many cases in nlp; nevertheless, we demonstrate that there are relatively simple setups where it can fail. we analyze what factors contribute to the failure. concretely, we consider the correlated attributes of document topic (scientific / non-scientific) and author gender on a self-collected multilingual corpus of ted talk transcripts in french, german, spanish, and turkish. this setup enables us to observe the interplay between linguistic properties and adversarial debiasing. our investigation proceeds in three steps. first, we train independent classifiers for each attribute and evaluate them with regard to overall performance and with regard to the bias they exhibit. in the second step, we apply adversarial debiasing to the predicting of each attribute with respect to the other, and re-evaluate the debiased models. finally, in the third step we discuss the differences observed in the previous step: (a), both document topic and author gender can be classified reasonably well by independent classifiers, but exhibit considerable bias; (b), author gender bias in topic classifiers can be reduced by adversarial training; however, adversarial debiasing in the opposite direction fails completely; (c) this effect is true for all languages except french. our interpretation is that the failure of adversarial debiasing is due to the fact that feature space for author gender is subsumed the topic feature space for all languages except french, where gender is expressed overtly by morphological cues that can be picked up by the model. ## related work regarding bias analysis at the representation level, the most important source of bias is arguably formed by corpus-derived embeddings which are used by virtually all current nlp systems. there have been several efforts to investigate the amount of bias within monolingual @xcite @xcite and multilingual embeddings. @xcite . bias analysis at the system level has investigated a range of applications such as ner @xcite , machine translation @xcite , natural language inference @xcite , emotion intensity prediction @xcite , coreference resolution @xcite and text classification @xcite . a number of strategies have been explored for bias mitigation. one common strategy for reducing the bias is to target the representational level again, that is, corpora @xcite and word embeddings @xcite . other methods target the model architecture in various ways. for example, @xcite introduce an additional term to be used in loss function of language generation model, seeking to reduce the gender bias exhibited by the model. a more fundamental idea is to adopt adversarial training @xcite to other tasks. for instance, @xcite adapted adversarial training to the task of domain adaptation by introducing gradient reversal layer (grl) which acts as an identity function during forward pass and reverses the gradient by multiplying it by a negative scalar during the backward pass. @xcite apply the idea to the removal of demographic bias; @xcite remove publication source as a bias variable from a satire detection model. @xcite reported that adversarial training with grl layer can remove unintended bias from the representations of pos tagging and sentiment analysis models while maintaining task performance. @xcite show that adversarial training mitigates the bias in word embeddings while maintaining its performance on word analogies task. finally, @xcite demonstrate how adversarial learning can be used for debiasing knowledge graph embeddings. ## dataset in order to conduct a study on the relationship between topic and author gender in multiple languages, we require a multilingual comparable corpus for which topic and gender information are available. the corpus should be as parallel as possible so that any differences in outcome across languages are not simply due to differences in the evaluation data. among the available multilingual parallel data sets, arguably the two most prominent ones are wit 3 and opus. wit 3 @xcite consists of lecture translations automatically crawled from the ted talks in a variety of languages and was used in the evaluation campaigns iwslt 2013 and 2014. opus @xcite is a collection of data from several sources which provides sentence alignments as well as linguistic markup (for some languages). unfortunately, neither corpus provides topic or gender labels. for this reason, we create a new multilingual parallel dataset with these annotations, based on ted talks ( http://ted.com/talks ). a ted talk is a presentation at the ted conference or one of its international partner events. ted talks are limited to a maximum length of 18 minutes and may be on any topic. ted talks are rehearsed talks and at least semi-formal, while still definitely belonging to the category of spoken language. in this regard, they are comparable to the widely used europarl corpus @xcite . the talks are divided according to the languages, topics and posted dates. all original talks are presented in english, but volunteers provide (and double check) translations into other languages. authors are identified by name. checking for which languages the ted webpage provided substantial numbers of transcripts (as of february 2020) led us to select german (de), spanish (es), french (fr) and turkish (tr) as target languages. we crawled all 1518 ted talks for which transcripts in all four target languages were available. we conducted some preprocessing: we cleaned transcripts by removing extra line breaks, extra spaces, and punctuation marks. inspired by the work in open-domain question answering @xcite , we then segmented the transcripts into a sequence of segments. rather than using paragraphs or sentences as segments directly, we split articles into segments with the length of 60 words by sliding window as @xcite demonstrated that splitting articles into non-overlapping fixed-length segments leads to better results in question answering. finally, we annotated the transcripts with topic and author gender information. for topic, we grouped transcripts into two classes according to the community-provided tags. the instances that have either technology or science tag were labeled as scitech while the rest was labeled as other. this grouping strategy led to a balanced dataset (53% science, 47% other). for author gender, we assume a binary gender classification (male/female) to be compatible with existing datasets @xcite . this should not be understood as a rejection of non-binary gender. we manually determine the author's gender infor ## experimental design ## experiment 1: simple classification and bias analysis in our first experiment, we set up neural classification models for the two tasks of topic and gender classification individually and evaluate them for the presence of bias in their predictions. ## discussion the results of our two experiments show an intriguing asymmetry between the two tasks of topic and author gender classification when debiased for the respective other attribute. reducing author gender bias in topic classification with adversarial training proceeds as expected, is relatively robust to the choice of λ in the interval between 0 and 1, and shows a consistent pattern across languages which can be explained by the properties of the languages involved. in contrast, reducing topic bias in author gender classification relies heavily on λ, quickly deteriorating to baseline level for large values of λ, and does not consistently manage to reduce bias in any case. this asymmetry cannot be an artifact of model architecture or data alone, since we use the same model architecture on the same data. instead, we believe that these patterns result from an interaction between the representation learning of the model and the information that the model can draw from the data. they can be understood through the latent feature space of the final shared layer in our architecture below the two heads (cf. figure 2 ), where each class can be characterized by a region of informative features. figure 5 shows venn diagram-style depictions of the three possible cases for a pair of attributes. in the left-hand case, (a), there is no overlap between the latent features of the two attributes. that is, the two attributes are independent of one another, and so is learning. however, this is by definition the case without correlations among attributes that we do not consider. in the center case, (b), there is an intersection between the latent features of the two attributes. the classifiers' use of this overlap potentially creates biases, but adversarial training exactly punishes the use of this region of latent feature space. thus, debiased classifiers can learn either attribute to the extent that the part of the feature space outside the intersection is still sufficiently informative. the right-hand case, (c), is the limit case when one of the two attributes does not have an independent standing, that is, the informative latent features of attribute 1 are completely contained in the informative feature space of attribute 2. this leads to biases in either classifier just as case (b), but also creates an asymmetry in the effect of adversarial debiasing: attribute 2 can be debiased by simply 'cutting out' the informative space of attribute 1, but debiasing attribute 1 in the opposite manner results in an empty feature space for attribute 1, and we would expect the classifier to revert to baseline performance. this set theoretic visualization is a major simplification of the latent feature space in neural models, where the three cases cannot apply categoriallythey rather represent different points on a continuum. nevertheless, the predictions of the subsumption case, (c), match our experimental results well: assuming that author gender features are included in topic features, we would expect to find successful debiasing of the topic classifier, but breakdown of the debiased author gender classifier. this is exactly the pattern of results that we have observed. note that this analysis builds on the behavior of the features of the attributes in the training data, in particular in a representation learning approach like the one we have pursued. in other words, changes of the data -or differences within the data, such as between languages -are expected to influence the outcome. again, this is what we see: french, due to its consistent morphological marking of gender, is closer to case (b), while the other languages are closer to case (c). ## conclusion this paper was concerned with text classification for correlated attributes, which pose an important but often overlooked challenge to model fairness -in particular, as we have argued, in the case of demographic attributes. we specifically analyzed the relationship between document topic and author gender. we established that topic classifiers exhibit gender bias and author gender classifiers show topic bias; that adversarial debiasing corrects gender bias in topic classification but breaks down in the opposite di-rection; and that this effect varies by language. beyond the concrete study, our contribution is to draw attention to the general question of prerequisites for successful adversarial debiasing, which, to our knowledge, has not received much attention. our results indicate that when the target attribute and the bias attribute are too strongly correlatedor, indeed, when the target attribute is subsumed by the bias attribute -adversarial debiasing fails: with a small weight on the bias component, no debiasing takes place; with a large weight, target attribute classification deteriorates to baseline level. furthermore, we find that the linguistic expression of the attributes matters greatly: the only language for which we achieved satisfactory results was french, due to the consistent morphological marking of gender which can be captured independently of topic @xcite . this highlights the importance of understanding the differences between languages regarding how they encode content @xcite , and underscores the importance of cross-lingual methods. in future work, we plan to develop a diagnostic to recognize potentially problematic constellations of correlated attributes and improve debiasing. overall adv baseline de 39.8 67.0 es 22.6 66.2 fr 14.0 67.6 tr 22.6 69.0 as we swapped the main and adversarial tasks, we modify the baseline in the same way too. we start with training the feature extractor and gender classifier head on the topic classification task. then,we freeze the feature extractor, remove the gender classifier and train the topic classifier on top of freezed feature extractor.