Causal ATE Mitigates Unintended Bias in Controlled Text Generation (2024)

Abstract: We study attribute control in language models through the method of Causal Average Treatment Effect (Causal ATE). Existing methodsfor the attribute control task in Language Models(LMs) check for the co-occurrence of words in a sentence with the attribute of interest, and control for them. However, spurious correlation of the words with the attribute in the training dataset, can cause models to hallucinate the presence of the attribute when presented with the spurious correlate during inference. We show that the simple perturbation-based method of Causal ATE removes this unintended effect. Specifically, we ground it in the problem of toxicity mitigation, where a significant challenge lies in the inadvertent bias that often emerges towards protected groups post detoxification. We show that this unintended bias can be solved by the use of the Causal ATE metric. We provide experimental validations for our claims and release our code (anonymously) here: [github.com/causalate-mitigates-bias](https://github.com/causalate-mitigates-bias/causal-ate-mitigates-bias).

## introduction controllable text generation methods are often used to guide the text generated by language models (lms) towards certain desirable attributes @xcite @xcite . the goal herein is to generate sentences whose attributes can be controlled @xcite . language models, which are pre-trained only for next word prediction, cannot directly control for attributes in their outputs. on the other hand, one may wish to alter words in the autoregressively produced sentences, either accentuating or mitigating the desired attributes. attributes such as sentiment, writing style, language precision, tone, and toxicity are key concerns for control in language models, with particular emphasis on toxicity mitigation due to its relevance in sensitive contexts @xcite . regularizers in the reward models are often employed during training to alter the output sentences towards certain desirable attributes @xcite . such regularization penalities (or rewards) often rely on models trained on real-world datasets. such datasets contain spurious correlates -words that correlate with certain attributes without necessarily causing them @xcite . in the context of toxicity mitigation, prior works show that detoxification methods inadvertently impact language model outputs concerning marginal-ized groups @xcite . words such as 'gay' or 'female' are identified as being toxic, as they co-occur with toxic text, and hence the lm stops speaking about them @xcite . this is called the unintended bias problem. in this paper we provide experimental and theoretical justifications for the use of causal ate to mitigate the unintended bias problem in text classification. we prove theoretically that for spurious correlates, the causal ate score is upper-bounded. we also show through extensive experiments on two popular toxicity classification datasets @xcite that our method shows experimental promise (see figure 1 ). we provide a full list of related works in related works section 6. ## notations and methodology consider a sentence s, made up of tokens (words) from some universe of words w . let the list of all sentences s in our dataset be denoted s. let each sentence @xmath0. so the dataset, which we can call d, consists of tuples (s, a(s)) for all @xmath1. let the cardinality of the labelled dataset be |d| = |s| = n. from such a dataset, it is possible to construct an attribute model that gives us an estimate of the probability of attribute a, given a sentence s. i.e. it is possible to construct a model a(•) such that a(s) = p{a | s} for any given sentence s. now such a model may rely on the words in s. let @xmath2, . . . , w n }. we now define an attribute model a(•) given a word as follows: definition 1 (attribute model a(w i ) for any word w @xmath3. note that such a model is purely correlation based, and can be seen as the proportion of sentences containing an attribute amongst those containing a particular word. i.e. it is an estimate of the co-occurrence of attribute with the word. based on attribute model a(•) we can define an attribute model a(•) for any sentence @xmath4, . . . , w k } as follows: definition 2 (attribute model a(s) for a sentence @xmath5. a(@xmath6, . . . , w k }) := max note that such a model is conservative and labels a sentence as having an attribute when any word in the sentence has the attribute. for the purpose of attributes such as toxicity, such an attribute model is quite suitable. ## theory and background now that we have laid the groundwork, we can make proceed to make the central claims of this work. lemma 1. consider sentence @xmath7, . . . , w k }. we will make two simple claims: this lemma is straightforward to prove from definition 7. we will now make a claim regarding the ate score of the given words themselves. recall that c i is the context for the word w i from a sentence s. given c i , w i is replaced by w ′ i by a perturbation model (through masked language modelling). towards our proof, we will make two assumptions: nn2layer105 neural network with 2 hidden layers (10 neurons and 5 neurons, respectively). 7 nn3layer20105 neural network with 3 hidden layers (20, 10, and 5 neurons, respectively). assumption 1. we make a mild assumption on this replacement process: a(w ′ i ) < a(c i ). grounding this in the attribute of toxicity, we can say that the replacement word is less toxic than the context. this is probable if the replacement model has been trained on a large enough corpus. see @xcite for empirical results showing this claim to be true in practice. assumption 2. we make an assumption on the dataset. a spurious correlate has a word with a higher attribute score in the rest of the sentence for sentences labelled as having the attribute. for example, in the case of toxicity, a spurious correlate like muslim, has a more toxic word in the rest of the sentence, when the sentence is labelled as toxic. given these assumptions, we have the following theorem: theorem 1. given assumptions 1 and 2 for a spurious correlate w i , ate(w i ) ≤ 0.25. proof. if we consider three numbers { a(c i ), a(w i ), a(w ′ i )}, there are six possible orderings of this set. we can subsume these orderings into two cases: within these cases, we study the variation of at e(w i ) with a(w i ). we plot these in the figure 3 . using a case-by-case analysis over these possibilities, we prove the statement. the full proof of the theorem is provided in appendix a. based on theorem a and lemma 1, a(s) ≤ 0.25 if each w @xmath8, i.e. non-causal, for attribute a. in the following section we provide experimental justification for our work through experimental results. 4 experiments in this section, we present experimental evidence demonstrating the efficacy of the causal average treatment effect (causal ate) method for mitigating unintended bias in text classification tasks. our experiments focus on toxicity detection, utilizing two widely recognized datasets. the results provide both theoretical and practical support for the utility of causal ate in addressing bias associated with protected groups. ## related works in this section we will look at five related lines of work: (a) controlled generation (b) unintended bias problem (c) toxicity mitigation (d) toxicity detection (e) causal methods for text controlled generation can be broadly categorized into fine-tuning methods @xcite @xcite , decoding-time approaches using attribute classifiers @xcite and causality based approaches @xcite . majority of these techniques were tested on toxicity mitigation and sentiment control. the dependence of attribute regularizers on probabilistic classifiers make them prone to such spurious correlations @xcite . in the unintended bias problem lms which are detoxified inherit a tendency to be biased against protected groups. lm quality is compromised due to a detoxification side-effect @xcite . some works address lm control through improving datasets @xcite . unfortunately, this makes annotation and data curation more expensive. as an alternative, there is growing interest in training accurate models in presence of biased data @xcite . our work fits into this framework. in the context of toxicity mitigation, @xcite highlight that detoxification methods have unintended effects on marginalized groups. they showcased that detoxification makes lms more brittle to distribution shift, affecting its robustness in certain parts of language that contain mentions of minority groups. concretely, words such as "female" are identified as being toxic, as they co-occur with toxic text, and hence the lm stops speaking about them @xcite . this is called the unintended bias problem. this unintended bias problem can manifest as differences in performance of the lm for different demographic groups. toxicity detection toxicity is a well studied problem in context of responsible and safe ai effort. hence, we foucs our experiments on toxicty mitigation in this study. several works have also studied the angle from toxic text detection. numerous studies have explored toxic text detection, including hatebert @xcite , hate-check @xcite , and perspective api @xcite . we employ the hate-bert model for assessing local hatefulness and utilize perspective api for third-party evaluation, where we report the corresponding metrics. causal methods for text spurious correlations between protected groups and toxic text can be identified is by understanding the causal structure. @xcite emphasizes on the connect between causality and nlp. towards mitigation of the bias problem @xcite proposed the use of causal ate as a regularization technique and showed experimentally that it does indeed perform as intended. in this paper, we probe the causal ate metric theoretically, and prove that the causal ate metric is less susceptible to false positives. an attribute control method based on this metric would mitigate unintended bias. we provide a theoretical basis from which to understand the causal ate metric and showcase that this causal technique provides robustness across contexts for attribute control in language models. ## conclusion in conclusion, our work provides a theoretical justification for using the causality-based concepts of counterfactuals, and ate scores for controlled text generation. we provide experimental results that validate these claims. we show that the simple perturbation-based method of causal ate removes the unintended bias effect through reduction of false positives, additionally making systems more robust to biased data.