The Undesirable Dependence on Frequency of Gender Bias Metrics Based on Word Embeddings (2022)

Abstract: Numerous works use word embedding-based metrics to quantify societal biases and stereotypes in texts. Recent studies have found that word embeddings can capture semantic similarity but may be affected by word frequency. In this work we study the effect of frequency when measuring female vs. male gender bias with word embedding-based bias quantification methods. We find that Skip-gram with negative sampling and GloVe tend to detect male bias in high frequency words, while GloVe tends to return female bias in low frequency words. We show these behaviors still exist when words are randomly shuffled. This proves that the frequency-based effect observed in unshuffled corpora stems from properties of the metric rather than from word associations. The effect is spurious and problematic since bias metrics should depend exclusively on word co-occurrences and not individual word frequencies. Finally, we compare these results with the ones obtained with an alternative metric based on Pointwise Mutual Information. We find that this metric does not show a clear dependence on frequency, even though it is slightly skewed towards male bias across all frequencies.

## introduction word embeddings are one of the most commonly used techniques to measure semantic closeness between words in a corpus. in recent years, they have been widely used in computational social science applications to measure societal biases and stereotypes @xcite @xcite @xcite . for practical purposes, we consider bias to be the degree to which the language used to describe groups or things is different. bias is typically measured by computing the difference between the mean similarity of words of two context groups a and b with respect to a target word x: gender bias has long been one of the most studied biases with this method. in this context, a and b are usually defined as gendered nouns and pronouns @xcite . a representative example is @xcite , where the female vs. male bias of professions in historical corpora is found to correlate with the percentage of women employed in each profession over time. not as widely used as word embeddings, pointwise mutual information (pmi) is a metric of word similarity that can also be used to study biases @xcite @xcite . valentini et al. (2021) define the pmibased bias metric as bias @xmath0, a) -pmi(x, b), where pmi(x, y ) = log p (x, y ) p (x)p (y ) . p (x, y ) is the probability of co-occurrence between the word x with any one in y in a window of a predefined number of words, and p (x) and p (y ) are the probability of occurrence of the word x and any word in y , respectively. valentini et al. (2021) show that bias pmi can be expressed as bias @xmath1. that is, bias pmi can be interpreted as how much more likely it is to find words in x in the context of words in a than in the context of words in b, in a log scale. thus, it captures exclusively first-order associations and can be computed via maximum likelihood using co-occurrence counts from the text @xcite . some recent works have studied the relationship between word frequencies and word embeddings. in particular, embeddings seem to encode word frequency even after normalization @xcite , vector norm depends on word frequency @xcite , top principal component directions encode frequency in different ways @xcite and vectors of highfrequency and low-frequency words lie in different regions of the embedding space @xcite . these studies are nevertheless inconclusive in the sense that they do not determine clearly to what extent the association observed is caused by actual attributes of corpora or by undesirable properties of embedding training. hence, an answer to the origin of the relation between embeddings and frequency is yet to be found. what is more, the repercussions of this effect in applications relevant to computational social sciences such as bias quantification have not yet been explored. we make three main contributions. first, we show that frequency has an association with gender bias when measured with word embeddingbased metrics: both skip-gram with negative sampling (sgns) and glove-based bias metrics tend to detect male bias in high frequency words, while glove also yields female bias on average in lowfrequency words. second, we show that the dependence of the embedding-based gender bias on frequency holds when tokens in the corpus are randomly shuffled. this proves that the dependence on frequency is an artifact of the metric itself i.e. that embedding-based bias metrics can encode frequency spuriously. third, we find that the pmi-based gender bias metric does not present this frequency-based effect but is slightly skewed towards male bias across all frequency ranges. 1 our analyses are restricted to the english language and are based on a binary understanding of 1 code for the paper is available at https://github.com/ ftvalentini/embeddingsbiasfrequency gender (see limitations). ## the effect of frequency on gender bias our objective is to study the association between gender bias and frequency in the widely used embedding-based metrics and in the alternative pmi-based metric. therefore, in a first experiment, we analyze this in two pretrained word embeddings, glove @xcite and word2vec with sgns @xcite . we do this by studying the distribution of bias in different bins of frequency of words in the vocabulary. bias is computed with equation 1 with the female and male context words lists used in @xcite , and we refer to this as female bias or gender bias. for each frequency bin, we also compute the ratio between the mean and the sample standard deviation (sd). these are cohen's d effect sizes of the mean of each group under the null hypothesis of absence of bias on average @xcite . here we use it as a normalized magnitude of the deviation of the distribution from zero. we use this methodology to assess the association between gender bias and frequency hereinafter. see appendix b for further details. there is a clear association between gender bias we with the pretrained embeddings and target word frequency (figure 1 ). glove embeddings present a monotonic relationship between frequency and gender bias, such that the top 10 3.5 words tend to have male bias with large effect sizes, whereas less frequent words have mean female bias with medium to large effect sizes. in the sgns embeddings the effect is small and positive in less frequent words, but in the top 100 words there is a large shift towards male bias. even if there is literature which has studied the relationship between frequency and word vectors (see section 1), this result is still startling: a priori, we wouldn't expect the gender bias of words to correlate so strongly with frequency, because word similarity should be more closely related to semantics and co-occurrences in the training corpus than with the individual frequencies of words. to validate this behavior, we train glove and sgns embeddings from scratch with the english wikipedia and study the association between gender bias and word frequency. we compare this with the results obtained with bias pmi (equation 2). ## discussion and conclusion in this work we revealed the existence of a spurious frequency-based distortion in gender bias metrics based on the cosine similarity between word embeddings. both sgns and glove-based gender bias metrics tend to detect male bias in high frequency words, while glove also yields female bias on average in low frequency words. to determine whether this effect is indeed an undesirable artifact of the embedding-based metric we assessed the relation between gender bias and frequency in shuffled corpora, where words lose their context but keep their frequency. results reveal that the dependence on frequency is caused by the metric and does not originate from actual properties of the texts. this shows that popular gender bias measurements can detect bias even when there is none. additionally, we found that an alternative pmi-based bias metric does not show a clear dependence on frequency, even though it shows a slight tendency towards male bias. according to these results, we consider the pmi-based bias metric has an advantage over the embedding-based metrics, which adds to the advantages of interpretability and hypothesis testing @xcite . however, as pmi captures exclusively first-order associations and is unable to capture synonyms, it may be required to include several terms associated to the context words in order to measure some biases. male nouns and pronouns are usually more frequent than female ones in large corpora @xcite . for example, in the wikipedia corpus, he appears 11.8 million times, while the frequency of she is 3.5 million (refer to appendix b for the frequencies of the other gendered context words). the disparity in frequencies of male and female contexts is a type of bias in itself and can be measured by counting word occurrences. in contrast, the bias we study refers to the stereotyped contexts in which male and female entities are portrayed, and should be independent of individual word frequencies. when words are shuffled, the biases associated with the contexts of female and male context words are eliminated, but the disparities in frequencies are maintained. we propose that bias metrics capture this disparity in frequencies of female and male context words. in the case of the embedding-based metric, this hypothesis is supported by the existing evidence that embeddings encode word frequency in addition to semantics. we believe the random-shuffling experiment is general enough to show that the frequency effect would still exist with other word lists, types of biases and domains, as long as the frequencies of the context words differ. this result is important because the context words' frequencies are disregarded when measuring biases with embeddings. our findings have important implications for bias measurement applications, as they cast doubt on the reliability of widely used bias metrics when the frequencies of the words involved are very different. we believe that more effort should be put into designing new bias detection methods that do not suffer from this weakness.