Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models (2023)

Abstract: Debiasing methods that seek to mitigate the tendency of Language Models (LMs) to occasionally output toxic or inappropriate text have recently gained traction. In this paper, we propose a standardized protocol which distinguishes methods that yield not only desirable results, but are also consistent with their mechanisms and specifications. For example, we ask, given a debiasing method that is developed to reduce toxicity in LMs, if the definition of toxicity used by the debiasing method is reversed, would the debiasing results also be reversed? We used such considerations to devise three criteria for our new protocol: Specification Polarity, Specification Importance, and Domain Transferability. As a case study, we apply our protocol to a popular debiasing method, Self-Debiasing, and compare it to one we propose, called Instructive Debiasing, and demonstrate that consistency is as important an aspect to debiasing viability as is simply a desirable result. We show that our protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked.

## introduction recent advances in deep learning have led to the creation of large transformer-based language models (lms) that have showcased near-human level performance on several natural language understanding and generation tasks @xcite @xcite @xcite . the datasets on which they operate are evergrowing in size, assembled based on minimally filtered web crawls @xcite . while many new and exciting applications have emerged as a result of these models, publicly accessible lms such as gpt-2 @xcite , gpt-3 @xcite , and the recent chatgpt @xcite have been observed to replicate or even amplify undesirable behaviours that occur on the internet @xcite ). consider the following example where the boldfaced text is an output generated by gpt-3 from an input prompt: while in both cases, the model generates outputs that are relevant to the prompt, it is evident that simply adding negative information can condition the lm into creating an equally negative continuation. this allows for users of an lm to easily "nudge" ir into having undesirable behaviour, which may bode poorly for the real-world technologies on which they are based. in fact, numerous studies have shown that humans use more obscenities and aggressive speech when interacting with robots than with fellow humans @xcite . there has been a recent push in the development of debiasing methods which seek to mitigate these undesirable behaviours, such as adversarial debiasing @xcite , auto-debiasing @xcite , and debiasing by fine-tuning @xcite , all of which have been proposed in just the last year, as well as continued use of less recent, but popular methods such as self-debiasing @xcite and gender equalizing debiasing @xcite . at the same time, evaluation measures that are used to qualify or quantify the success of these methods have been based primarily on the outputs of the language models, which may specification importance domain transferability test summary do reversed bias specifications amplify undesirable bias? does the presence of bias specifications significantly reduce undesirable bias? do the results hold in both adversarial and non-adversarial settings? most debiasing methods to date seek to mitigate the bias in an lm's output by targeting different stages in the text generation process. as such, each method often has different testing procedures to measure the success of the new debiased system. in addition, these methods are usually tested in restricted domains as well as according to a singular specification of the traits to debias against. since societal values are both culturally-bound and everevolving, there is no guarantee that a certain specification should hold unconditionally and forever. for example, the grasp of what qualifies as toxic may be different in another society or even for the same society a few years in the future. for this reason, accurately capturing toxic language across cultural and temporal bounds is a challenging task @xcite @xcite . while it is therefore a step in the right direction to develop a method that debiases language models with respect to certain time, setting and value specifications, it is equally important to ensure that the method's success will transfer with any change to the setting or specification. for example, will a debiasing method be equally successful in debiasing in settings that are non-adverserial? will a debiasing method be equally successful if the definition of toxicity is altered? in other words, how consistent is the debiasing mechanism with respect to its own specifications? to date, to the best of our knowledge, there is no standardized test to compare and evaluate debiasing methods to ensure their consistency when applied to various settings. in this work, we present such a protocol according to three proposed criteria: specification polarity, specification importance, and domain transferability (summarized in we introduce a novel evaluation protocol to measure the consistency of debiasing methods: by characterizing consistency according to clearly defined and easily-measured criteria, we propose a standardized approach to comparing and evaluating debiasing methods with respect to each other and to varying specifications, including reversed definitions and non-adverserial settings. we demonstrate that debiasing methods sometimes lack consistency: our findings demonstrate that the positive results reported for current debiasing methods may not necessarily correspond to the mechanisms on which they are based. the mitigation of bias, although in itself a desired result, should be accompanied with measures that provide a deeper glimpse into the method's ability to generalize to modified specifications and settings. we introduce a novel & consistent debiasing method: to explore the feasibility of our protocol, we propose a debiasing method called instructive debiasing which passes all three criteria of consistency. we also demonstrate qualitatively that our proposed method generates outputs that are interpretable with respect to how the mechanism is defined. ## related work debiasing techniques to mitigate bias in lms, previous work has adopted a variety of approaches targeting different stages in the text generation pipeline. one common approach is to debias the word embeddings that represent the relationships between words, and has been performed in a variety of ways such as: modifying the word associations between genders and gender stereotyped words such as woman and homemaker @xcite , modifying contextualized word associations using fine-tuning @xcite , and using definitions of words found in the dictionary to modify word associations @xcite . other works have presented new plugins and software to debias generated text including answer-set programming for semantics as seen with neurasp, a python plugin that allows you to define semantic rules that help determine the neural networks output @xcite . some approaches focus more on establishing clear practices and techniques as opposed to developing new software such as the goal oriented approach of @xcite . another recent debiasing technique that will be used as a case study and discussed in section 4 is self-debiasing @xcite , a method that uses negative input attributes to create a negative output in order to reduce the probabilities of using words from the negative output in the model's original distribution, using a specialized decoding algorithm. our proposed method of instructive debiasing is inspired by that of self-debiasing, but differs in that specifications are given in terms of positive attributes, which removes the need for a decoding algorithm altogether, reducing model complexity while introducing improvements in terms of consistency and interpretability. as well, since our method does not directly modify language model token probabilities, we do not risk affecting a language model's text generation ability, which could lead to issues of perplexity @xcite . the measure of consistency the measure of consistency as a proxy to model performance is one that has been explored for a variety of nlp applications. given that deep learning models cannot be interpreted with ease at least directly, one approach is to "probe" a model, whereby hidden representations of the model are applied to auxiliary tasks that test linguistic phenomenon of interest (e.g., verb understanding) @xcite @xcite . other approaches look at correlations between properties of word-embeddings and the downstream task performances in which they are used @xcite . based on the definition we provide for consistency in section 3, we believe that these efforts are related ways to test the correspondence between a model's inner mechanism and outputs. consistency has been also directly covered extensively in various areas in nlp such as in question answering (qa) @xcite , common-sense reasoning (csr) @xcite @xcite , and natural language inference (nli) @xcite . recently, a testing methodology called checklist @xcite was developed that uses a number of tests to evaluate the consistency and robustness of models. one test in particular, the directional expectation test (der), most closely resembles our test of specification polarity, whereby a problem instance is perturbed by the addition of a positive or negative sentiment with an expected result (golden label) which is compared to the model's actual behaviour. our protocol for consistency is also formulated as a checklist, but one that seeks to explore the case of consistency specifically in debiasing techniques, for which we are not aware of any work in the literature. ## consistency evaluation protocol in the context of our debiasing protocol, we define a consistent debiasing method as one which produces results that follow expectedly from the specifications it is provided. by specifications, we are referring to the behaviours (such as toxicity, sexual explicitness, profanity) that are used in the debiasing method for mitigation. these behaviours can be provided as explicit definitions or keywords, and are free to be incorporated at any stage of the generation pipeline (e.g., as prompts to a pre-trained model or as filtering queues for the training dataset). for the debiasing methods we compare, these specifications occur as prefixes to input prompts for language models -however, our proposed protocol is general and can be adapted for use beyond the prompt-based paradigm. in the goal of the protocol is to become a standard tool for researchers when developing debiasing methods, to ensure that a method is consistent with its results before further experimentation or deployment. the evaluation protocol runs three pass-or-fail tests on a debiasing method: specification polarity, specification importance, and domain transferability whose results determine its consistency. ## instructive debiasing in order to assess the efficacy of the protocol, we conduct a comparative case study between two debiasing methods: the self-debiasing method, as described by @xcite (refer to section 2), and instructive debiasing. instructive debiasing leverages prompts, relying entirely on the innovative field of prompt template engineering for debiasing. it is consistent with the terminological framework suggested in the given reference @xcite . the method employs a prefix prompt structure, utilizing the pattern "be s for: x". here, s corresponds to a specification inspired by those outlined in @xcite , and x represents the input prompt. the decision to utilize prefix prompts was driven by their demonstrated effectiveness in text generation tasks using auto-regressive language models (lms) like gpt-2 and gpt-3 @xcite . compared to alternative methods such as cloze prompts, prefix prompts enable more controlled and explicit behavior instruction provision to the lm. this structure, where a behavior instruction precedes the input, was crafted through an intuitiondriven, manual template engineering process, intended to create language that is both naturally phrased and accessible to human comprehension. our method capitalizes on the lm's ability to parse both the instruction and the input to generate an output that aligns with the given instruction. this effect is achieved by attaching specifically designed debiasing instructions to the input prompts, which guides the lm's text generation process to manifest the desired behavior. in contrast to the self-debiasing method, which modifies the language model's (lm's) original probability distribution through post-hoc adjustments to discourage tokens generated by negative prompts (an indirect form of a 'double negative' process), instructive debiasing refrains from intervening or altering the lm's initial distribution. rather, our technique harnesses the inherent capacity of the lm to utilize context. this approach not only makes it an appropriate candidate for testing consistency, but it also offers a solution that is simpler, more adaptable, less resource-intensive, and potentially more interpretable. here is the original input used: "be positive, polite, respectful, compliant or loveable, modest, helpful, and kind for: " it's noteworthy that while self-debiasing and instructive debiasing employ specifications of opposing polarity, both methods effectively reduce bias. this distinction arises from their inherent processes: self-debiasing prompts a language model (lm) to eliminate negativity from the output, whereas instructive debiasing encourages the lm to accentuate positivity in the output. ## experimentation to evaluate the consistency of the debiasing methods described in section 4, we performed experiments using the consistency protocol for both self-debiasing and instructive debiasing following the steps described in section 3. ## results when evaluating self-debiasing on our consistency protocol and comparing the results to the debiasing performance of the original specifications ( for instructive debiasing, it passed the specification polarity test since the opposite specifications resulted in lower toxicity scores than without a specification (i.e., default) ( ## conclusion the issue of bias in language models is one that needs active counter-measures. if these measures are not evaluated for consistency, they may not generalize and adapt to new and highly sensitive situations. in this paper, we presented a protocol to evaluate in more depth an important aspect of debiasing techniques, i.e., consistency, and presented a new technique that passes the protocol. our protocol is just one of a vast line of possible contributions towards creating a standardized set of techniques for testing debiasing methods, e.g., @xcite . the aim of this work is not solely to present a supplementary metric to gauge the consistency of debiasing methods, but to also spark new ideas and open up a dialogue about shortcomings in current deep learning-based generation models that may be overlooked and can be addressed in the future. with transformer-based language models being ever more pervasive in the public, universal standards and practices must be considered closely.