Exploring the Effects of Negation and Grammatical Tense on Bias Probes (2022)

Abstract: We investigate in this paper how correlations between occupations and gendered-pronouns can be affected and changed by adding negation in bias probes, or changing the grammatical tense of the verbs in the probes. We use a set of simple bias probes in Norwegian and English, and perform 16 different probing analysis, using four Norwegian and four English pre-trained language models. We show that adding negation to probes does not have a considerable effect on the correlations between gendered-pronouns and occupations, supporting other works on negation in language models. We also show that altering the grammatical tense of verbs in bias probes do have some interesting effects on models’ behaviours and correlations. We argue that we should take grammatical tense into account when choosing bias probes, and aggregating results across tenses might be a better representation of the existing correlations.

## introduction pre-trained language models (lms) reflect various linguistic and factual knowledge, represented in the data they have been trained or fine-tuned on. despite their emergent success, these lms might contain various degrees of representational harms, where genders, religions, and ethnicity might be miss-represented, or not represented at all @xcite . lms can contain biases that might be inherited by the unlabeled data used while training them, the data used while fine-tuning them, and the label distribution used for downstream classifiers. in recent years, the extent to which these lms reflect, amplify, and spread the biases existing in the input data has been an active research focus as it is important to understand their inner representations, and what can be their possible harmful outcomes. the possible harmful effects of lms have been thoroughly discussed by @xcite , especially their ability to potentially amplify the already existing biases that occur in the data they were trained on. some of the efforts so far have demonstrated the existence of different types of biases that correlate gender and ethnicity with insurance groups @xcite , people with disabilities and mental illnesses with negative sentiment words, homelessness, and drug addictions @xcite , and that they can even amplify gender bias @xcite one way to explore the existence, and types, of gender bias in lms is to use template-based approaches @xcite @xcite @xcite . these template-based approaches have for example been used to show how lms can reproduce and amplify gender-related societal stereotypes @xcite , and how the gender biases in bert propagate in tasks within emotion and sentiment prediction @xcite . moreover, these lms when queried using template-based probes, seem to not distinguish between templates and their negation @xcite , and therefore suggesting that they are not always able to handle negation. @xcite have also explored perturbing the probes by adding misprimes to extract information from lm, and showed that lms are sensitive. the fragility of the template-based probes has also been pointed out by @xcite , where they have shown that sometimes a simple word change can alter a model's behaviour. in this paper, we investigate the effects of negation and grammatical tense when probing lms for gender bias purposes. based on previous investigations, and research on probing language models, our main hypothesis is that changing the formulation of a probe can have an effect on the output of a lm. we know that lms use datasets of vari- norwegian english present [pronoun] jobber som [occupation] [pronoun] works as a/an [occupation] past [pronoun] jobbet som [occupation] [pronoun] worked as a/an [occupation] future [pronoun] skal jobbe som [occupation] [pronoun] will work as a/an [occupation] future [pronoun] kommer til å jobbe som [occupation] [pronoun] is going to work as a/an [occupation] n. present [pronoun] jobber ikke som [occupation] [pronoun] does not work as a/an [occupation] n. past [pronoun] jobbet ikke som [occupation] [pronoun] did not work as a/an [occupation] n. future [pronoun] skal ikke jobbe som [occupation] [pronoun] will not work as a/an [occupation] n. future [pronoun] kommer ikke til å jobbe som [occupation] [pronoun] is not going to work as a/an [occupation] to address these questions, we inspect how sensitive bias probes can be, and analyse the effects of our experiments on the behaviours of norwegian and english pre-trained lms. we start in section 2 by describing our experimental setup, give details about our bias probes, and the lms used. in section 3 we present and discuss our main results and findings. finally, in section 4, we conclude and summarize our work, and discuss some possible future work. ## experiments we use the definition of bias by @xcite , where bias is the systematic dis-crimination against, and unfairly process of, a certain group of individuals exhibited by automated systems. in this work, we look at the correlations within the pre-trained models between gendered pronouns and professional occupations, and explore how the scores returned by the lms can change by simple alterations in the probes. in our case, introducing negation and altering the grammatical tense of the verbs. however, we do not evaluate if a model is biased or not, we rather look at what changes when the probes are perturbed. we do not try to reduce the stereotypical representations, but rather shed light on how fragile, sensitive, or reliable the bias probes are. we use the masked-language modeling objective of each model to predict the probability of pronouns in a probe. for simplicity, we also do not look at the degree of variation in the returned probabilities, but we simply check which pronoun has a greater value, and use this prediction to analyse the effect of the negated and tense-specific probes. one limitation of our work is that we only look at the correlations between occupations and binary gender categories (male and female), although we acknowledge the fact that gender as an identity spans a wider spectrum. ## results and discussion the two original probes in present, non-negated, forms are "[pronoun] jobber som [occupation]" for norwegian, and "[pronoun] works as a/an [occupation]" for english. in figures 1 and 2 we show the distribution of gendered-pronouns based on the returned probabilities of the norwegian and english lms. the y axis here is the number of occupations correlated with each gendered-pronoun, in each model, when using the bias probes. as can be seen in figure 1 , the models norbert and nb-bert_large tend to heavily correlate occupations with male gender. while it seems to be the opposite for norbert2 and nb-bert. this however does not hold for the english models. ex- it has already been shown that lms do not handle negation that well @xcite . our analysis of bias probes and how they behave with regards to negation also supports this claim. by looking at the distribution of female and male correlated occupations using our eight negated bias probes, it is apparent that all models, return somewhat the same correlations between occupations and genders. very few models exhibit changes in the correlations: 24 out of 32 combinations of probes and models show a shift in less than 16% of occupations. this shows that negation have little effect on bias probes, and rarely changes the correlations between genders and occupations. see tables 3 and 4 in appendix a for the statistical distributions of these results. some interesting observations can also be made when it comes to altering the grammatical tense of probes. interestingly, shifting the tense from present to past tense seems to shift the correlations between occupations and genders towards male pronouns. this observation holds for all english and norwegian models, but does not apply for the biggest norwegian model nb-bert_large. when shifting the tense from present to future, the opposite seems to happen. the changes seem to mainly shift the correlations of occupations from males to females. this is true for most norwegian models (except norbert2), but does not hold for the english models (except for bert -see the same observations can be seen with the negated tense probes. all norwegian models shift correlations to male-gendered pronouns when switching from present to past tense, while shifting to female-gendered pronouns if comparing probes between present and future tense. for the english models, all seem to change the correlations towards male-gendered pronouns when shifting tenses except for two instances of "present vs future" for the models bert_large and roberta. for more details about this, see we think that one possibility for the differences between the observations made on the norwegian and english models is the name of the occupations. as these were selected from the norwegian statistics bureau, they might reflect norwegian demographics more than the english models. some of the fine-grained occupations might not be as frequent in english-speaking countries, and therefore are weakly correlated with gender-pronouns in any case. this is of course a hypothesis, and it needs to be explored further. one important factor to keep in mind when using probes of various grammatical tense, is the context in which they tend to occur. a past tense probe might reflect something that is known and describes a state that has occurred, while a future tense probe might describe potential states. this can affect our analysis as one would expect less discussions about potential occupations for males (assuming that males have access to all) and more mentions about occupations for females (assuming that they have been blocked from male dominated occupations before). this goes back to how genders and occupations are correlated in the training data of pre-trained models, and to what extent this can be perceived when probing the models. ## conclusion we have presented our investigations into how the addition of negation and changing the grammatical tense of the verb in bias probes can alter the correlations between occupations and gendered-pronouns. we carried out experiments using eight pre-trained language models, four norwegian and four english ones, and generated a set of 16 bias probes. we show that negation does not have a significant effect on the correlations resulting from probing the language models. however, interesting observations were made for grammatical tense. switching from present to past shows more correlations with male-gendered pronouns, while changing from present to future exhibits more correlations with female-gendered pronouns. this shows how template-based bias probes are sensitive to small changes, and might hint to the necessity of taking grammatical tense into consideration when probing language models for bias. we believe that aggregating results across tenses might give a better representation of the correlations between genders and occupations. as future work, we would like to explore the diachronic gender-based bias correlations with occupations. biases might change across time-periods, and what was not considered bias against one gender a couple of decades ago might now be a stereotypical description. we think that comparing timeperiods to each other might help us identify the time-shifts for stereotypical correlations, both in datasets and how this can be reflected in models trained on them.