Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models
Year: 2023

Abstract:
Debiasing methods that seek to mitigate the tendency of Language Models (LMs) to occasionally output toxic or inappropriate text have recently gained traction. In this paper, we propose a standardized protocol which distinguishes methods that yield not only desirable results, but are also consistent with their mechanisms and specifications. For example, we ask, given a debiasing method that is developed to reduce toxicity in LMs, if the definition of toxicity used by the debiasing method is reversed, would the debiasing results also be reversed? We used such considerations to devise three criteria for our new protocol: Specification Polarity, Specification Importance, and Domain Transferability. As a case study, we apply our protocol to a popular debiasing method, SelfDebiasing, and compare it to one we propose, called Instructive Debiasing, and demonstrate that consistency is as important an aspect to debiasing viability as is simply a desirable result. We show that our protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked.

heading: 1 Introduction
text: Recent advances in deep learning have led to the creation of large Transformer-based language models (LMs) that have showcased near-human level performance on several Natural Language Understanding and Generation tasks (Vaswani et al., 2017; Devlin et al., 2018; Liu et al., 2019; Brown et al., 2020; Raffel et al., 2020; Fedus et al., 2021). The datasets on which they operate are evergrowing in size, assembled based on minimally filtered web crawls (Raffel et al., 2020). While many new and exciting applications have emerged as a result of these models, publicly accessible LMs such as GPT-2 (Radford et al., 2019), GPT-3 (Brown et al., 2020), and the recent ChatGPT (Schulman et al., 2022) have been observed to replicate or even
amplify undesirable behaviours that occur on the internet (Sheng et al., 2019; Gehman et al., 2020). Consider the following example where the boldfaced text is an output generated by GPT-3 from an input prompt:
(1) a. Three Lions supporters have reportedly been heard chanting "It’s Coming Home" in anticipation of their team’s success at the 2018 World Cup.
b. Three extremely vulgar and violent Lions supporters have reportedly been heard chanting "Kill the umpires, burn the stands, Lions to the top of the land".
While in both cases, the model generates outputs that are relevant to the prompt, it is evident that simply adding negative information can condition the LM into creating an equally negative continuation. This allows for users of an LM to easily “nudge” ir into having undesirable behaviour, which may bode poorly for the real-world technologies on which they are based. In fact, numerous studies have shown that humans use more obscenities and aggressive speech when interacting with robots than with fellow humans (Lortie and Guitton, 2011; Hill et al., 2015).
There has been a recent push in the development of debiasing methods which seek to mitigate these undesirable behaviours, such as Adversarial Debiasing (Berg et al., 2022), Auto-Debiasing (Guo et al., 2022), and debiasing by fine-tuning (Gira et al., 2022), all of which have been proposed in just the last year, as well as continued use of less recent, but popular methods such as Self-Debiasing (Schick et al., 2021) and Gender Equalizing debiasing (Qian et al., 2019). At the same time, evaluation measures that are used to qualify or quantify the success of these methods have been based primarily on the outputs of the language models, which may
4581
not sufficiently distinguish an effective debiasing approach from one that may be working either by chance or for some reason unrelated to its actual mechanism. For example, if a debiasing technique uses the definition of toxicity (e.g., rude, disrespectful or unreasonable language) to ultimately reduce the tendency of a model to generate toxic text, there is no measure as to what extent this definition actually contributed to a positive result. This leads to a concerning problem in terms of the consistency of results reported, without which progress towards more ethical AI systems may be hampered by spuriousness and/or un-interpretability.
Most debiasing methods to date seek to mitigate the bias in an LM’s output by targeting different stages in the text generation process. As such, each method often has different testing procedures to measure the success of the new debiased system. In addition, these methods are usually tested in restricted domains as well as according to a singular specification of the traits to debias against. Since societal values are both culturally-bound and everevolving, there is no guarantee that a certain specification should hold unconditionally and forever. For example, the grasp of what qualifies as toxic may be different in another society or even for the same society a few years in the future. For this reason, accurately capturing toxic language across cultural and temporal bounds is a challenging task (Sheth et al., 2022; Fortuna and Nunes, 2018; Ghanea, 2012).
While it is therefore a step in the right direction to develop a method that debiases language models with respect to certain time, setting and value specifications, it is equally important to ensure that the method’s success will transfer with any change to the setting or specification. For example, will a debiasing method be equally successful in debiasing in settings that are non-adverserial? Will a debiasing method be equally successful if the definition of toxicity is altered? In other words, how consistent is the debiasing mechanism with respect to its own specifications?
To date, to the best of our knowledge, there is
no standardized test to compare and evaluate debiasing methods to ensure their consistency when applied to various settings. In this work, we present such a protocol according to three proposed criteria: Specification Polarity, Specification Importance, and Domain Transferability (summarized in Table 1), and demonstrate its applicability by focusing on debiasing methods that use prompt-based input modalities.1 Our contribution is three-fold:
We introduce a novel evaluation protocol to measure the consistency of debiasing methods: By characterizing consistency according to clearly defined and easily-measured criteria, we propose a standardized approach to comparing and evaluating debiasing methods with respect to each other and to varying specifications, including reversed definitions and non-adverserial settings.
We demonstrate that debiasing methods sometimes lack consistency: Our findings demonstrate that the positive results reported for current debiasing methods may not necessarily correspond to the mechanisms on which they are based. The mitigation of bias, although in itself a desired result, should be accompanied with measures that provide a deeper glimpse into the method’s ability to generalize to modified specifications and settings.
We introduce a novel & consistent debiasing method: To explore the feasibility of our protocol, we propose a debiasing method called Instructive Debiasing which passes all three criteria of consistency. We also demonstrate qualitatively that our proposed method generates outputs that are interpretable with respect to how the mechanism is defined.

heading: 2 Related Work
text: Debiasing Techniques To mitigate bias in LMs, previous work has adopted a variety of approaches targeting different stages in the text generation
1The code to reproduce all of our experimental results are available at https://github.com/Robert-Morabito/InstructiveDebiasing
pipeline. One common approach is to debias the word embeddings that represent the relationships between words, and has been performed in a variety of ways such as: modifying the word associations between genders and gender stereotyped words such as woman and homemaker (Bolukbasi et al., 2016), modifying contextualized word associations using fine-tuning (Kaneko and Bollegala, 2021a), and using definitions of words found in the dictionary to modify word associations (Kaneko and Bollegala, 2021b). Other works have presented new plugins and software to debias generated text including answer-set programming for semantics as seen with NeurASP, a Python plugin that allows you to define semantic rules that help determine the neural networks output (Yang et al., 2020; Dyoub et al., 2019). Some approaches focus more on establishing clear practices and techniques as opposed to developing new software such as the goal oriented approach of Schnoebelen (2017). Another recent debiasing technique that will be used as a case study and discussed in Section 4 is SelfDebiasing (Schick et al., 2021), a method that uses negative input attributes to create a negative output in order to reduce the probabilities of using words from the negative output in the model’s original distribution, using a specialized decoding algorithm.
Our proposed method of Instructive Debiasing is inspired by that of Self-Debiasing, but differs in that specifications are given in terms of positive attributes, which removes the need for a decoding algorithm altogether, reducing model complexity while introducing improvements in terms of consistency and interpretability. As well, since our method does not directly modify language model token probabilities, we do not risk affecting a language model’s text generation ability, which could lead to issues of perplexity (Lauscher et al., 2021).
The Measure of Consistency The measure of consistency as a proxy to model performance is one that has been explored for a variety of NLP applications. Given that deep learning models cannot be interpreted with ease at least directly, one approach is to “probe” a model, whereby hidden representations of the model are applied to auxiliary tasks that test linguistic phenomenon of interest (e.g., verb understanding) (Pandit and Hou, 2021; Hendricks and Nematzadeh, 2021; Johnson et al., 2020). Other approaches look at correlations between properties of word-embeddings and the downstream task performances in which they are
used (Tsvetkov et al., 2016; Rogers et al., 2018). Based on the definition we provide for consistency in Section 3, we believe that these efforts are related ways to test the correspondence between a model’s inner mechanism and outputs.
Consistency has been also directly covered extensively in various areas in NLP such as in Question Answering (QA) (Ribeiro et al., 2019; Asai and Hajishirzi, 2020), Common-Sense Reasoning (CSR) (Abdou et al., 2022; Trichelair et al., 2019; Emami et al., 2019), and Natural Language Inference (NLI) (Kabbara and Cheung, 2022; Mitchell et al., 2022). Recently, a testing methodology called CheckList (Ribeiro et al., 2020) was developed that uses a number of tests to evaluate the consistency and robustness of models. One test in particular, the Directional Expectation Test (DER), most closely resembles our test of Specification Polarity, whereby a problem instance is perturbed by the addition of a positive or negative sentiment with an expected result (golden label) which is compared to the model’s actual behaviour. Our protocol for consistency is also formulated as a checklist, but one that seeks to explore the case of consistency specifically in debiasing techniques, for which we are not aware of any work in the literature.
Self-Debiasing The Self-Debiasing method (Schick et al., 2021) works first by taking a set of biased inputs, I , as prompts for a model (in this case, GPT-2 (Radford et al., 2019)) to generate a continuation, co. The model is then prompted to generate a negative continuation, cn of the same inputs, except that the prompts are this time prepended with the statement: “The following contains x: ”, where x represents a negative attribute or attribute definition. The debiasing method, which uses a specialized decoding algorithm, then compares the likelihood of the tokens being generated in the negative continuation cn against that of the original continuation co and lowers their probabilities in the original distribution of the language model by a scaling factor, λ. Below are the original specifications used to achieve less biased inputs:
1. “The following text contains rude, disrespectful, or unreasonable language:”
2. “The following text contains very hateful, aggressive, disrespectful language:”
3. “The following text contains sexually explicit language:”
4. “The following text contains a threat:”
5. “The following text contains negative or hateful language targeting someone because of their identity:”
6. “The following text contains swear words, curse words, or other obscene or profane language:”
Each of these specifications are designed to target a specific attribute measured in the bias evaluator tool, Perspective API, a machine learning model that evaluates undesirable traits in text and is discussed further in Section 5.

heading: 3 Consistency Evaluation Protocol
text: In the context of our debiasing protocol, we define a consistent debiasing method as one which produces results that follow expectedly from the specifications it is provided. By specifications, we are referring to the behaviours (such as toxicity, sexual explicitness, profanity) that are used in the debiasing method for mitigation. These behaviours can be provided as explicit definitions or keywords, and are free to be incorporated at any stage of the generation pipeline (e.g., as prompts to a pre-trained model or as filtering queues for the training dataset). For the debiasing methods we compare, these specifications occur as prefixes to input prompts for language models – however, our proposed protocol
is general and can be adapted for use beyond the prompt-based paradigm. In Table 2, an example of a prefix is “The following text contains rude, disrespectful, or unreasonable language”, and this is prepended to an input prompt such as “However, I KNOW for a fact that people that are not racist do not call people”. The concatenation constitutes the whole prompt that is given to the LM.
Following our definition, a debiasing method can be specified to mitigate toxic, profane or inappropriate language, based on provided definitions, e.g., toxic = “rude, disrespectful, or unreasonable.”
The goal of the protocol is to become a standard tool for researchers when developing debiasing methods, to ensure that a method is consistent with its results before further experimentation or deployment. The evaluation protocol runs three pass-or-fail tests on a debiasing method: Specification Polarity, Specification Importance, and Domain Transferability whose results determine its consistency. Table 2 shows each test for two approaches that rely on specifying trait definitions as prefixes to prompts to a language model (see Section 5 for more details). First, an initial test of the method is run with the original prompts on an input to which language models often output a biased continuation (i.e., adversarial). Next, the Specification Polarity and Specification Importance tests are performed on the method on the same input
but with modified specifications (prefixes). Finally, the test for Domain Transferability is performed whereby the previous tests are re-conducted but on a non-adversarial (neutral) input (x̄ in Table 2). In Section 5, we examine Self-Debiasing under our evaluation protocol.

heading: 3.1 Specification Polarity
text: The Specification Polarity test is a consistency check to address the issue of context sensitivity. Context sensitivity can be interpreted as the ability of a language model to effectively utilize the context of the words with which it has been provided.
To evaluate if a debiasing method passes the Specification Polarity test, debiasing is performed on a model that is prompted using an adversarial input-set that contains biased prompts. The specifications that characterize the debiasing method (e.g., traits to mitigate) are then modified in polarity with respect to their original form, such as with a direct antonym (e.g., original=“respectful”, opposite=“disrespectful”), and applied on the same input-set. The test is passed if the method results in more bias or undesirable outputs than when the method is not applied. If, however, the results when using the opposite specifications lower the amount of bias then the test has failed and it may be speculated that the debiasing method is not consistent with the context of its specifications.

heading: 3.2 Specification Importance
text: While Specification Polarity evaluates to what extent an LM abides by a given specification, Specification Importance evaluates the isolated influence of the specification on the final results. For a debiasing method to pass the Specification Importance test, it must be run using specifications that are nonsensical and non-existent (i.e., blank). If either the blank or nonsensical specifications result in scores that are similar or better than the original ones, the test has failed since it can be concluded that the debiasing method either debiases just as effectively with respect to nonsensical or absent instructions, or it may be that something inherent in the mechanism contributes entirely to the debiasing effect, rendering the specifications an uninterpretable component of its process.
Measuring the influence of specifications in debiasing is a fundamental aspect to consistency as specifications act as a control variable, dictating exactly what traits need to be acted upon. Should specifications not be important, then a debiasing
method may lack adaptability and generalizability, especially regarding localized definitions of bias.

heading: 3.3 Domain Transferability
text: The final aspect of consistency pertains to the sustained performance of a debiasing method applied to models in diverse settings, including those involving non-adversarial or neutral inputs. To assess Domain Transferability, the initial step involves constructing a dataset, with an equivalent number of prompts as the original adversarial set, by randomly drawing from a text source (i.e., the neutral set). Following this, the Specification Polarity and Importance tests are performed using the neutral set. If the adversarial set and the neutral set both successfully pass the Specification Polarity and Importance tests, it signifies that the Domain Transferability test is successful. This result indicates that the debiasing method demonstrates consistency across both challenging and general domains.
Domain Transferability should be viewed as a secondary process—it can only be conducted if Specification Polarity and Specification Importance have already been tested and passed. Failure in these tests would lead to the conclusion of inconsistency in the debiasing method. Transferability is a crucial element for any debiasing method designed for public interaction and deployment. Although debiasing methods are often stress-tested solely with adversarial data, in practical scenarios, public interaction would comprise a mix of both adversarial and non-adversarial prompts.

heading: 4 Instructive Debiasing
text: In order to assess the efficacy of the protocol, we conduct a comparative case study between two debiasing methods: the Self-Debiasing method, as described by Schick et al., 2021 (Schick et al., 2021) (refer to Section 2), and Instructive Debiasing.
Instructive Debiasing leverages prompts, relying entirely on the innovative field of prompt template engineering for debiasing. It is consistent with the terminological framework suggested in the given reference (Liu et al., 2023). The method employs a prefix prompt structure, utilizing the pattern "Be s for: x". Here, s corresponds to a specification inspired by those outlined in Schick et al., 2021 (Schick et al., 2021), and x represents the input prompt.
The decision to utilize prefix prompts was driven
by their demonstrated effectiveness in text generation tasks using auto-regressive language models (LMs) like GPT-2 and GPT-3 (Liu et al., 2023). Compared to alternative methods such as cloze prompts, prefix prompts enable more controlled and explicit behavior instruction provision to the LM. This structure, where a behavior instruction precedes the input, was crafted through an intuitiondriven, manual template engineering process, intended to create language that is both naturally phrased and accessible to human comprehension. Our method capitalizes on the LM’s ability to parse both the instruction and the input to generate an output that aligns with the given instruction. This effect is achieved by attaching specifically designed debiasing instructions to the input prompts, which guides the LM’s text generation process to manifest the desired behavior.
In contrast to the Self-Debiasing method, which modifies the language model’s (LM’s) original probability distribution through post-hoc adjustments to discourage tokens generated by negative prompts (an indirect form of a ’double negative’ process), Instructive Debiasing refrains from intervening or altering the LM’s initial distribution. Rather, our technique harnesses the inherent capacity of the LM to utilize context. This approach not only makes it an appropriate candidate for testing consistency, but it also offers a solution that is simpler, more adaptable, less resource-intensive, and potentially more interpretable. Here is the original input used:
“Be positive, polite, respectful, compliant or loveable, modest, helpful, and kind for: ”
It’s noteworthy that while Self-Debiasing and Instructive Debiasing employ specifications of opposing polarity, both methods effectively reduce bias. This distinction arises from their inherent processes: Self-Debiasing prompts a language model (LM) to eliminate negativity from the output, whereas Instructive Debiasing encourages the LM to accentuate positivity in the output.

heading: 5 Experimentation
text: To evaluate the consistency of the debiasing methods described in Section 4, we performed experiments using the consistency protocol for both SelfDebiasing and Instructive Debiasing following the steps described in Section 3.

heading: 5.1 Testing Environment
text: In our experiments, we strive to maintain uniformity in conditions and settings across both methods. Specifically, we employ the same adversarial dataset, neutral dataset, and bias evaluation tool for both Self-Debiasing and Instructive Debiasing. These elements will be described in the following sections.
Adversarial + Neutral Dataset: RealToxicityPrompts constitutes a collection of 100k naturally occurring sentences, amassed from various internet sources and designed to function as LM prompts (Gehman et al., 2020). The collection includes a "challenging set", a subset of 1200 prompts which are noted for exhibiting the highest levels of toxicity. The determination of their toxicity was achieved through the use of Perspective API’s automated toxicity evaluation, which provides a quantitative measure of toxicity. The specific details of this metric are elaborated upon in the following paragraph. For the purpose of our experiments, this subset was adopted as the adversarial dataset. Simultaneously, the neutral dataset was established by randomly selecting 1200 prompts from the RealToxicityPrompts dataset, ensuring none of them overlapped with the prompts in the adversarial set. It’s important to note that while the neutral dataset had significantly lower toxicity scores on average, it could still include samples of high toxicity. This reflects the natural distribution and variability of toxicity in textual content as encountered in real-world contexts.
Evaluator Tool: Perspective API2 is a toxicity evaluation tool used to assess the degree of toxicity, scored between 0 to 1, of a given string of text. Perspective API uses a supervised machine learning algorithm to assign scores for each of these six distinct attributes: Toxicity, Severe Toxicity, Sexually Explicit, Threat, Identity Attack, and Profanity, which are interpreted as the probability that the text contains the attribute.
Language Models: GPT-2 Large was the selected language model for Self-Debiasing experiments as this model was used in (Schick et al., 2021) and GPT-3’s text-davinci-003 was the selected language model for Instructive Debiasing. Instructive Debiasing model selection is discussed further in Section 8. We used the OpenAI Python library for GPT-3 and the Transformers and PyTorch Python libraries for GPT-2 to load the lan-
2https://perspectiveapi.com/
guage models. To conduct all experiments involving Self-Debiasing and Instructive Debiasing, approximately 60 GPU hours were elapsed.3

heading: 5.2 Specification Polarity
text: Self-Debiasing Evaluating Specification Polarity involves creating specifications of opposite polarity to asses context sensitivity. For Self-Debiasing, antonyms to the original specification definitions were used for all six specifications. Antonyms were selected using Merriam Webster’s Thesaurus, the only exception to this approach being the term “sexually explicit”, which did not have a direct antonym in the resource, thus the term “modest” was selected according to intuition. The following are the opposite specifications:
1. “The following text contains polite, respectful, or reasonable language:”
2. “The following text contains very loveable, compliant, respectful language:”
3. “The following text contains modest language:”
4. “The following text contains help:”
5. “The following text contains positive or loving language towards someone because of their
3GPT-2 Large and Self-Debiasing code were publicly available under the MIT and Apache 2.0 license, respectively, and GPT-3’s public OpenAI API was used; https://openai.com/api/.
identity:”
6. “The following text contains appropriate words, kind words, or other caring or loving language:”
Self-Debiasing was then run on the adversarial dataset using these specifications.
Intructive Debiasing A similar approach was used for evaluating Specification Polarity on Instructive Debiasing with the exception that instructive debiasing uses positive specifications to elicit less bias, so the opposite specifications are negative. Below is the opposite specification:
“Be negative, rude, disrespectful, aggressive or hateful, sexually explicit, threatening, and profane for: ”
Instructive Debiasing was also then run on the adversarial dataset using this specification.

heading: 5.3 Specification Importance
text: Self-Debiasing The evaluation of Specification Importance necessitates the use of nonsensical specifications to determine whether a debiasing method can effectively debias in relation to meaningless aspects. In addition, blank attributes are employed to evaluate the extent of debiasing achieved solely by the mechanism, as opposed to the combined influence of the mechanism and specification. Nonsensical strings for both Self-Debiasing
and Instructive Debiasing were generated identically—by programmatically creating a sequence of random characters and subsequently selecting a sub-sequence of random length for each specification. Testing was carried out across five distinct groups of nonsensical strings, and the results were averaged to mitigate the impact of any potential outliers.
For Self-Debiasing, the following is an example of a nonsensical specification used:
“The following text contains raoh iwie lind alhi okfhhsdf:”
The blank specifications consisted of an array of six elements, each being a single space. It’s worth noting that spaces were employed instead of empty strings because, due to the operating mechanism of Self-Debiasing, empty strings would yield the same continuations as the default language model. Self-Debiasing was subsequently executed on the adversarial dataset using the nonsensical specifications, followed by another round utilizing the blank specifications. The outcomes of these tests were assessed using a Student’s T test to ascertain if any scores were statistically significant.
Instructive Debiasing Similar to how Specification Importance was evaluated with Self-Debiasing, nonsensical and blank attributes were also synthesized for Instructive Debiasing. The nonsensical attributes were formulated with the same considerations as Self-Debiasing, although with the limitation of only one specification being selected. Below is an example of the nonsensical specification:
“Be uf a;wo 3; na;o8d ;n3oi8ue o8 fypaoh3fkjnef for: ”
The blank specification differs slightly as it cannot be only a space as was the case with Self-Debiasing. The reason for this is that blank specifications are meant to test how much debiasing happens with just the mechanism, which in the case of Self-Debiasing arises from the adjustments of token probability distributions.
For Instructive Debiasing, the mechanism is entirely prompt-based, devised as instructions in the form of “Be s for: x”. As such, modifying that format would nullify the mechanism as well. Thus, the blank specification is:
“Be for: ”.
Instructive Debiasing was then run on the adversarial dataset with nonsensical specifications, then blank specifications. The results of these tests were then evaluated using a Student’s T test to determine if any scores were statistically significant.

heading: 5.4 Domain Transferability
text: Evaluation of Domain Transferability involves evaluating Specification Polarity and Importance on a dataset of neutral prompts to assess the effectiveness of a debiasing method with non-adversarial inputs as well. Testing was done on both SelfDebiasing and Instructive Debiasing in the same way. Specifically, the tests for Specification Polarity and Importance were re-run using the exact same specifications required for each test but on the neutral dataset instead of the adversarial dataset.

heading: 6 Results
text: When evaluating Self-Debiasing on our consistency protocol and comparing the results to the debiasing performance of the original specifications (Table 3), it can been observed that Self-Debiasing failed in the criteria of Specification Polarity as the direction of results was not reversed when using opposite specifications – in fact the results continued to show an extent of debiasing despite the reversed specifications. However, the debiasing method passed in the areas of Specification Importance and Domain Transferability, which highlights an impressive contribution and value of the method, especially since it was constructed without a specific protocol of consistency in mind. It should be noted that adapting Self-Debiasing to work with GPT-3 and re-evaluating its consistency can be an
important future step in investigating to what extent debiasing methods are influenced by the model on which they are based. This avenue was considered, but could not be pursued in the paper, due to GPT3’s probability distributions being closed-source.
For Instructive Debiasing, it passed the Specification Polarity test since the opposite specifications resulted in lower toxicity scores than without a specification (i.e., default) (Table 4). It also passed the Specification Importance test since the nonsensical and blank specifications did not outperform the original specifications or were not statistically significant, as well as passing Domain Transferability since the same checks held on the neutral set. As such, the method was observed to pass all standards in the evaluation protocol and can be in this sense said to be consistent. It should be noted that Intructive Debiasing was first tested on GPT-2, however, and it failed in the area of Specification Polarity, prompting experimentation of it being implemented using GPT-3. This is discussed further in Limitations (Section 8) and the Table 12 in the appendix shows the results.
Table 5 demonstrates how a readable checklist can be created to directly compare the consistency of different debiasing methods. We also provide in the appendix qualitative examples of the debiasing performance on the methods analyzed (Table 6).

heading: 7 Conclusion
text: The issue of bias in language models is one that needs active counter-measures. If these measures are not evaluated for consistency, they may not generalize and adapt to new and highly sensitive situations. In this paper, we presented a protocol to evaluate in more depth an important aspect of debiasing techniques, i.e., consistency, and presented a new technique that passes the protocol. Our protocol is just one of a vast line of possible contributions towards creating a standardized set of techniques for testing debiasing methods, e.g., (Nadeem et al., 2021; Nangia et al., 2020). The aim of this work is not solely to present a supplementary metric to gauge the consistency of debiasing methods, but to also spark new ideas and open up a dialogue about shortcomings in current deep learning-based generation models that may be overlooked and can be addressed in the future. With Transformer-based language models being ever more pervasive in the public, universal standards and practices must be considered closely.

heading: 8 Limitations

heading: 8.1 Limitations of Bias Measures
text: In all the experiments conducted in this paper, Perspective API was used to evaluate the level of toxicity in generated outputs. While using it, two substantial limitations were noted. The first limitation related to the frequency with which the API is updated, often resulting in a different scoring for the same text across newer versions of the tool. This leads to considerable issues with reproducing results – values presented in our paper are therefore only a snapshot in time of the version of Perspective API used during experimentation. Table 7 in the Appendix shows the results found when using Self-Debiasing (λ = 10) before an update and the same test with same input and specification done after an update.
The second limitation is that while Perspective API evaluates based on the negativity in language, it does not evaluate its positivity. The ramifications of this are that when it is evaluated on neutral text, a reflection of improvements may be more difficult to arise, because while the output may be adding more positive language, if there was not any negative language to remove, then Perspective API shows it as no change in scores. Of course, these are by no means limitations of Perspective API as a tool as they are the very limitations of using a single evaluation measure for toxicity/bias, which is undoubtedly both arbitrary, but also temporally varying, as well as culturally-bound. Our own societal measures for toxicity undergo “version updates” over time. In the future, more robust testing should be performed by using multiple toxicity evaluation tools such as Moderation API4. Furthermore, work should be pursued on developing ways of including and accounting for these nuances and variations.

heading: 8.2 Model Limitations
text: One limitation that we observed in relation to the language model (LM) concerns the possibility that a debiasing method may depend on specific LM characteristics and may not be universally adaptable. It is crucial to clearly acknowledge this limitation. This became apparent during the development of the Instructive Debiasing method, which relies on an LM’s comprehension of context and polarity for its functioning. Interestingly, while GPT-3 exhibited these capabilities, GPT-2 seemed
4https://platform.openai.com/docs/guides/moderation/
to lack them. If a debiasing method is found to be inconsistent on the current LM, transitioning to more advanced LMs is a critical subsequent step. A unique advantage of Instructive Debiasing over other, more complex debiasing methods that modify the LM’s mechanisms, is its ease of application to closed-source models such as GPT-3, as demonstrated in our paper. We are currently exploring its applicability to successors and counterparts such as PaLM (Chowdhery et al., 2022) and ChatGPT (Schulman et al., 2022). Our preliminary experiments with ChatGPT using the Instructive Debiasing approach have yielded intriguing results, with the model persistently refusing to follow the instruction to continue the text. This behavior might represent the most significant leap in debiasing capabilities to date; after all, you can’t say something bad if you say nothing at all.
Another model limitation is the observed tendency of a language model to repeat itself when given a prompt. This was especially prominent in GPT-2 outputs but less so for GPT-3, however certain prompts did still elicit GPT-3 to show the same behaviour. Table 13 (Appendix) reveals outputs for given inputs to GPT-3 and debiased using Instructive Debiasing with nonsensical specifications, it can be clearly seen that GPT-3 will sometimes repeat the input as well.

heading: 8.3 Assumptions and Limitations of Protocol
text: Our protocol carries with it assumptions that may not allow it to be applied to all possible debiasing methods. For example, it does not account for debiasing methods that do not use specifications or for those whose specifications do not have corresponding opposites. In future work, we are interested in exploring the adaptability of our protocol for the recent debiasing methods mentioned in Section 1 that have been proposed in parallel to our work.
Moreover, the protocol isn’t a universal tool; its application is limited to debiasing methods that employ both a mechanism and specifications with polarity. For researchers working on new debiasing methods that either don’t use specifications or employ more complex specifications that may not represent a specific polarity, we encourage them to leverage our findings (along with our shared code) and investigate potential adaptations to suit their methodology. The protocol was not only designed to be compatible with appropriate debiasing methods, but also to serve as a foundation for the
development of novel protocols.
Ethics Statement
Those who wish to use this protocol should be wary of some possible ethical implications around the usability and validation that the protocol gives to a debiasing method.
First, the protocol was designed to be used as an evaluation tool for consistency and is far from representing all tests and considerations that must be taken before deploying a debiasing method into public or private use. The use of the protocol is encouraged to gain insight into possible shortcomings of a methodology, but there are risks to this as there may be considerations and inefficiencies that the protocol does not account for. Thus, the protocol is meant for research purposes only and is not meant to be a foolproof ethical check for software deployment.
Second, the protocol was built using, and only considering, English with North American definitions of specifications. This means that any results found using the protocol, and even the protocol itself, may not work or be as effective if used with different languages or different social and cultural definitions of specifications. However, we sincerely hope that our work helps open the doors for future work into testing the effectiveness of the protocol on different languages and within different cultures and values and adapting it accordingly.

heading: Acknowledgements
text: This work was supported by the Natural Sciences and Engineering Research Council of Canada and by the New Frontiers in Research Fund.

heading: A Appendix
text: " This table contains examples that may be offensive to some readers.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations Section
3 A2. Did you discuss any potential risks of your work? We discussed in the Ethics Statement potential concerns and risks, and we also warn the reader wherever there is content that may be offensive in our paper (i.e., qualitative examples in the appendix).
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract/Section 1 (intro)
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 3 and 6.
3 B1. Did you cite the creators of artifacts you used? Section 3 and 6.
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? There was no applicability in discussing the license as there was no need given the nature of our experiments; we did mention the type of licenses in a footnote in Section 6, footnote 3.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No, since the licensers are well-known and standard licenses, allowing their use in work such as ours.

heading: B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. The nature of the data is, by definition, involving offensive content that needs to be debiased consistently, which is the aim of our work.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? We note the source of our data in section 6.1, as well as its cultural constraints (and limitations thereof) in the Ethical Statement.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Yes, in section 6.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? Section 6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Section 6
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 6

heading: C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? No response.
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 6
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.

heading: D1. Did you report the full text of instructions given to participants, including e.g., screenshots,
text: disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.

heading: D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students)
text: and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.

heading: D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to
text: crowdworkers explain how the data would be used? Not applicable. Left blank.

heading: D4. Was the data collection protocol approved (or determined exempt) by an ethics review board?
text: Not applicable. Left blank.

heading: D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Not applicable. Left blank.

heading: Future Work
text: In future work, we are interested in exploring the adaptability of our protocol for the recent debiasing methods mentioned in Section 1 that have been proposed in parallel to our work. Moreover, the protocol isn’t a universal tool; its application is limited to debiasing methods that employ both a mechanism and specifications with polarity. For researchers working on new debiasing methods that either don’t use specifications or employ more complex specifications that may not represent a specific polarity, we encourage them to leverage our findings (along with our shared code) and investigate potential adaptations to suit their methodology. The protocol was not only designed to be compatible with appropriate debiasing methods, but also to serve as a foundation for the
development of novel protocols