On the Interpretability and Significance of Bias Metrics in Texts: a PMI-based Approach
Year: 2023

Abstract:
In recent years, word embeddings have been widely used to measure biases in texts. Even if they have proven to be effective in detecting a wide variety of biases, metrics based on word embeddings lack transparency and interpretability. We analyze an alternative PMIbased metric to quantify biases in texts. It can be expressed as a function of conditional probabilities, which provides a simple interpretation in terms of word co-occurrences. We also prove that it can be approximated by an odds ratio, which allows estimating confidence intervals and statistical significance of textual biases. This approach produces similar results to metrics based on word embeddings when capturing gender gaps of the real world embedded in large corpora.1

heading: 1 Introduction
text: Word embedding-based approaches have been used for detecting and quantifying gender, ethnic, racial, and other stereotypes present in corpora. While some research has focused on investigating biases by training embeddings on a specific corpus of interest (Garg et al., 2018; Kozlowski et al., 2019; Lewis and Lupyan, 2020; Charlesworth et al., 2021), others have employed pretrained word embeddings to assess potential biases inherent in the training corpus (Caliskan et al., 2017; Garg et al., 2018; DeFranza et al., 2020; Jones et al., 2020).
Though not as popular, Pointwise Mutual Information (PMI) is a measure of word similarity which has also been used to study biases (Gálvez et al., 2019; Bordia and Bowman, 2019; Aka et al., 2021).
1Code for the paper is available at https://github. com/ftvalentini/BiasPMI
However, the statistical properties and advantages of this measure as compared to the widely used word embeddings have not been studied yet.
In this article we study a PMI-based metric to measure bias in corpora and explain its statistical and interpretability benefits, which have been overlooked until now. Our contributions are as follows: (1) We show the PMI-based bias metric can be approximated by an odds ratio, which makes computationally inexpensive and meaningful statistical inference possible. (2) We provide evidence that methods based on GloVe, skip-gram with negative sampling (SGNS) and PMI produce comparable results when the biases measured in large corpora are compared to empirical information about the world. (3) We contend that the PMI-based bias metric is substantially more transparent and interpretable than the embedding-based metrics.
Scope: The detection and mitigation of bias in models is a research topic that is beyond the scope of this paper. Our paper’s contribution focuses on the measurement of bias in raw corpora (not models), which is a relevant task in Computational Social Science.

heading: 2 Background
text: Consider two sets of context words A and B, and a set of target words C. Textual bias measures quantify how much more the words of C are associated with the words of A than with those of B. Most metrics can be expressed as a difference between the similarities between A and C, on the one hand, and B and C, on the other:
Bias = sim(A,C)− sim(B,C) (1) 509
For instance, to estimate the female vs. male gender bias of occupations, context words are often gendered pronouns or nouns, e.g., A = {she, her, woman,..} and B = {he, him, man,...}; whereas C is usually considered one word at a time, estimating for each specific job (nurse, doctor, engineer, etc.) the relative association to A and B.
One particularly popular metric which uses word embeddings (WE) is that of Caliskan et al. (2017):
BiasWE = mean a∈A cos(va, vc)−mean b∈B cos(vb, vc)
std_dev x∈A∪B cos(vx, vc)
(2) where vi stands for the word embedding of word i and cos(vi, vj) is the cosine similarity between vectors.
Permutations tests that shuffle context words have been used to calculate the statistical significance of BiasWE (Caliskan et al., 2017; Charlesworth et al., 2021). These tests permute the words from A and B repeatedly and compute the bias metric in each iteration to simulate a null distribution of bias. The two-tailed p-value is calculated as the fraction of times the absolute value of bias from the null distribution is equal to or greater than the one observed (North et al., 2002).
With a similar re-sampling approach, bootstrap can also be performed (Garg et al., 2018). The bootstrap distribution is obtained by calculating the bias metric over many bootstrap samples from A and B, sampled separately for each group. The standard error of bias is then estimated as the sample standard deviation of the bootstrap distribution, and the quantiles of the distribution are used to obtain percentile confidence intervals (Davison and Hinkley, 1997).

heading: 3 Bias measurement with PMI
text: Here we introduce a bias metric that follows equation 1 but uses Pointwise Mutual Information (PMI) (Church and Hanks, 1990) as a measure of word similarity:
BiasPMI = PMI(A,C)− PMI(B,C) (3)
PMI measures the first-order association between two lists of words X and Y :
PMI(X,Y ) = log P (X,Y )
P (X)P (Y ) = log P (Y |X) P (Y ) ,
(4) where P (X,Y ) is the probability of co-occurrence between any word in X with any one in Y in a
window of words, and P (X) and P (Y ) are the probability of occurrence of any word in X and any word in Y , respectively. Equation 4 shows PMI can be expressed as the ratio between the probability of words in Y co-occurring with words in X , and the probability of words in Y appearing in any context.

heading: 3.1 Approximation of the PMI-based bias by log odds ratio
text: Combining equations 3 and 4, the PMI-based bias can be written as a ratio of conditional probabilities, which can be estimated via maximum likelihood using the co-occurrence counts from the corpus:
BiasPMI = log P (C|A) P (C|B) = log
fA,C fA,C+fA,nC
fB,C fB,C+fB,nC
,
(5) where fA,C and fB,C represent the number of times words in C appear in the context of words in A and B, respectively, and fA,nC and fB,nC represent how many times words not in C appear in the context of A and B, respectively. See contingency table in Appendix A for reference.
BiasPMI is not computable if fA,C = 0 or fB,C = 0. We address this by adding a small value ϵ to all co-occurrences in the corpus (Jurafsky and Martin, 2009).
For most practical applications, co-occurrences between words not in a group (most of the vocabulary) and a group of specific words are larger than the co-occurrences between two groups of specific words. More precisely:
fB,nC ≫ fB,C , fA,nC ≫ fA,C . (6)
Thus:
BiasPMI ≈ log fA,C fA,nC fB,C fB,nC ≈ log OR, (7)
where OR is the odds ratio. Therefore, parametric confidence intervals and hypothesis testing can be conducted for BiasPMI (details in Appendix B).

heading: 4 Experiments
text: To compare BiasPMI with BiasWE we replicate three experiments that compare the gender biases measured in texts with the ones from other datasets:
1. Occupations-gender (Caliskan et al., 2017): gender bias in text is compared to the percentage of women employed in a list of occupations in the U.S. Bureau of Labor Statistics in 2015.
2. Names-gender (Caliskan et al., 2017): for a list of androgynous names, gender bias in text is compared to the percentage of people with each name who are women in the 1990 U.S. census.
3. Norms-gender (Lewis and Lupyan, 2020): textual gender bias is compared to the Glasgow Norms, a set of ratings for 5,500 English words which summarize the answers of participants who were asked to rate the gender association of each word (Scott et al., 2019).
Details about these datasets are in Appendix C. We train GloVe, SGNS and PMI on two corpora: the 2014 English Wikipedia and English subtitles from OpenSubtitles (Lison and Tiedemann, 2016). We pre-process both corpora by converting all text to lowercase, removing non alpha-numeric symbols and applying sentence splitting, so that one sentence equates to one document. After preprocessing, the Wikipedia corpus is made up of 1.2 billion tokens and 53.9 million documents, whereas the OpenSubtitles corpus contains 2.4 billion tokens and 447.9 million documents. Refer to Appendix D for additional details about each corpus and to Appendix E for implementation details.
For each of the three settings, we assess the correlation between the dataset’s female metric and the female bias as measured by PMI (equation 5), and SGNS and GloVe (equation 2). Female bias refers to the bias metrics where A and B represent lists of female and male words, respectively.2 Positive values imply that the target word is more associated with female terms than with male ones.
We measure correlation with Pearson’s r. We also compute a weighted Pearson’s r, which takes into account the standard error of each bias estimate and reduces the influence of noisy estimates on the correlation. Finally, for each word in each experiment we compute confidence intervals and p-values for the null hypothesis of absence of bias.3
The aim of these experiments is not to find which method produces greater correlations in each task; it is rather to check whether BiasPMI produces similar results to the widely used BiasWE. If it does, it means our metric can extract trends from large corpora that correlate with gender stereotypes at least as well as embedding-based metrics can.
2A={female, woman, girl, sister, she, her, hers, daughter} and B={male, man, boy, brother, he, him, his, son} (Caliskan et al., 2017; Lewis and Lupyan, 2020).
3In the case of BiasWE, we apply bootstrap with 2,000 iterations and permutations with the all the possible combinations.

heading: 5 Results
text: Table 1 shows Pearson’s r weighted and unweighted coefficients for each of the eighteen experiments (three association tests in two corpora with three bias measures each). The scatter plots associated with the Wikipedia’s coefficients are available in Appendix F.1.
All in all, BiasPMI and BiasWE yield comparable results in these settings. There is no single method which consistently has the largest or lowest correlations.
Weights tend to either increase the correlation considerably or to make it slightly weaker. This implies that in these experiments, noisy textual bias estimates usually agree less with the gender bias in the validation datasets. However, this does not mean that for each individual bias estimate the standard errors of each method are mutually interchangeable or equally useful (see section 6.2).
In Figure 1 we compare the p-values of the permutation test of BiasWE with SGNS, with the pvalues of the log odds ratio test of BiasPMI for the Names-gender test conducted in Wikipedia. A Benjamini-Hochberg correction was applied to the p-values obtained by both methods to account for multiple comparisons (Benjamini and Hochberg, 1995). Appendix F.2 shows this example is consistent with the rest of the experiments.
In this example, only the word with the highest
BiasWE is significantly different from zero at a 0.10 significance level. In contrast, most words have a BiasPMI significantly different from zero, with the exception of some points with bias values close to zero. This is because the procedures that compute p-values for each type of metric capture essentially different types of variability (see section 6.2).

heading: 6 Discussion

heading: 6.1 Interpretability
text: Although there are studies on how word vector spaces are formed (Levy and Goldberg, 2014; Levy et al., 2015; Ethayarajh et al., 2019) and on the biases they encode (Bolukbasi et al., 2016; Zhao et al., 2017; Gonen and Goldberg, 2019), there is no transparent interpretation of the embedding-based bias metrics in terms of co-occurrences of words in the texts.
In contrast, BiasPMI can be expressed intrinsically in terms of conditional probabilities (equation 5). The bias is interpreted as the logarithm of how much more likely it is to find words in C in the context of words in A than in the context of words in B. For example, in the Wikipedia corpus the female BiasPMI of word nurse is 1.3172, thus,
P (nurse|A) P (nurse|B) = exp 1.3172 = 3.7330.
This means that it is 273.30% more likely to find the word nurse in the context of female words (A) than in the context of male words (B).
To the lack of interpretability of BiasWE contributes the fact that SGNS and GloVe can capture word associations of second order or higher
(Altszyler et al., 2018; Schlechtweg et al., 2019), whereas PMI is strictly a first-order association metric. When embeddings are used to measure biases, it is not possible to tell whether the results are due to widespread first-order co-occurrences or are derived from obscure higher-order co-occurrences (Brunet et al., 2019; Rekabsaz et al., 2021).
For instance, in OpenSubtitles, the BiasPMI of the word evil equals −0.25, indicating a higher likelihood of appearing in the context of male context words (B) compared to female ones (A). Conversely, BiasSGNS = 0.23. Even if this stands for female bias, it is difficult to understand the exact source of this result since it is influenced by second and higher-order co-occurrences. Moreover, in recent research we demonstrated that BiasWE can also yield misleading results by inadvertently capturing disparities in the frequencies of context words (Valentini et al., 2022).
Nevertheless, bias metrics that capture secondorder associations have the advantage of managing data sparsity. Since word embeddings can capture synonymy, when data is sparse it might not be necessary to include all related words to the concepts of interest in order to measure meaningful biases. In the case of our first-order metric, this problem must be addressed by increasing word lists with synonyms and forms of the words of interest.
To illustrate this, let’s consider the case of the words nourish and nurture, which have different frequencies in the Wikipedia corpus (700 and 3, 000, respectively). With BiasPMI, we obtain a bias of 0.33 for nurture (p-value < 10−4). However, if we had used its less frequent synonym nour-
ish instead, the BiasPMI would have been −0.10 and not statistically significant (p-value ≈ 0.66). Here we would not have been able to determine whether there is actually no bias or if there is insufficient data. This shows that it is generally advisable to include all pertinent synonyms and variations of the term whose bias we are trying to measure.

heading: 6.2 Statistical inference
text: The p-values, standard errors and confidence intervals of the logOR approximation are fundamentally different from the ones estimated for BiasWE through permutations and bootstrap. The uncertainty quantified for BiasPMI captures the variability of the underlying data generating process i.e. the one induced by the randomness of cooccurrence counts as random quantities. In contrast, the estimates for BiasWE only consider the variability across the sets of context words. This means that multiple words must be chosen so that inference can be conducted. In fact, whenever A and B are single-word lists, there is no way of estimating uncertainty for BiasWE with these methods, whereas it is perfectly feasible for BiasPMI.
As far as we know, we are the first to provide a simple and efficient way of evaluating the statistical significance of bias. This is especially important in Computational Social Science, for which it is useful to have not only a reliable metric to quantify stereotypes but also a reliable tool to measure uncertainty i.e. to know up to what degree the measured values might have been due to statistical fluctuation. Meaningful statistical tests and confidence intervals that capture the variability that really matters are therefore essential.

heading: 7 Conclusion
text: We presented a PMI-based metric to quantify biases in texts, which (a) allows for simple and computationally inexpensive statistical inference, (b) has a simple interpretation in terms of word cooccurrences, and (c) is explicit and transparent in the associations that it is quantifying, since it captures exclusively first-order co-occurrences. Our method produces similar results to the GloVe-based and SGNS-based metrics in experiments which compare gender biases measured in large corpora to the gender gaps of independent empirical data.
Limitations
We replicate three well-known experiments in the gender bias literature, where bias is measured according to a binary female vs. male view. This choice ignores other views of gender but eases the presentation of the frameworks.
We only use two corpora and three datasets which by no means capture the biases of all the people speaking or writing in the English language. Moreover, we don’t experiment with different corpus sizes, a more diversified set of corpora or more bias types. We hope to explore this in future work.
The hyperparameters of the models have not been varied, using their default values. This replicates the standard experimental setting used in the literature. Since there are no ground truths when measuring biases (that is, there are no annotations with the amount of bias of words in large corpora), hyperparameters are usually set to their default values.

heading: A Contingency table of co-occurrences
text: BiasPMI is computed with the co-occurrences between the groups of words A, B and C. These can be represented with the following contingency table:
This contains, for example, how many times words in A appear in the context of words in C (fA,C) and how many times they do not (fA,nC).
B Statistical inference for BiasPMI The distribution of the log odds ratio (equation 7) converges to normality (Agresti, 2003). Its 95% confidence interval is given by
CI95% ( BiasPMI ) = BiasPMI ± 1.96SE
with
SE =
√ 1
fA,C +
1
fB,C +
1
fA,nC +
1
fB,nC
≈ √ 1
fA,C +
1
fB,C
This last approximation considers condition 6. We can test the null hypothesis that the log odds ratio is 0 (absence of bias) with a standard Z-test, whereby the two-sided p-value is computed with 2P (Z < −|BiasPMI|/SE), where Z is a standard normal random variable.

heading: C Datasets
text: For the occupations-gender and names-gender experiments, the female proportions for names and occupations in the U.S. were extracted from the datasets provided by Will Lowe’s cbn R library 4, which contains tools for replicating Caliskan et al. (2017). We used the 50 names and 44 occupations available in this source.
The original Glasgow Norms comprise 5,553 English words. Individuals from the University of Glasgow were asked to measure the degree to which each word is associated with male or female behavior on a scale from 1 (very feminine) to 7 (very masculine). Following Lewis and Lupyan (2020), we average the norms of homonyms and compute 8− rating to flip the scale so that it represents femaleness according to human judgement. 4,668 words from the original list overlapped with OpenSubtitle’s vocabulary, and 4,642 words overlapped with the Wikipedia vocabulary.

heading: D Corpora
text: The Wikipedia corpus was built from the August 2014 dump, licensed under CC BY-SA 3.05. We removed articles with less than 50 tokens.
The OpenSubtitles corpus (Lison and Tiedemann, 2016) includes English subtitles from movies and TV shows and was built with the aid of the subs2vec Python package with MIT License (van Paridon and Thompson, 2021).

heading: E Model training
text: We ignore words with less than 100 occurrences, resulting in a vocabulary of 172,748 words for Wikipedia and 128,974 words for OpenSubtitles.
We use a window size of 10 in all models and apply "dirty" subsampling i.e. out-of-vocabulary tokens are removed before the corpus is processed into word-context pairs (Levy et al., 2015).
Word embeddings with 300 dimensions are trained with SGNS and GloVe. For SGNS we use the word2vec implementation of Gensim 4.1.2 licensed under GNU LGPLv2.1 (Řehůřek and Sojka, 2010) with default hyperparameters. GloVe is trained with the original implementation (Pennington et al., 2014) with version 1.2 (Apache License, Version 2.0) with 100 iterations. This version uses
4https://conjugateprior.github.io/cbn/ 5https://archive.org/download/
enwiki-20141208
by default additive word representations, in which each word embedding is the sum of its corresponding context and word vectors.
For PMI, we count co-occurrences with the GloVe module (Pennington et al., 2014) with version 1.2 and set the smoothing parameter ϵ to 0.5.
We ran all experiments on a desktop machine with 4 cores Intel Core i5-4460 CPU and 32 GB RAM. Training times were around 1 hour per epoch with SGNS and 5 minutes per iteration with GloVe. Co-occurrence counts used for PMI were obtained in around 20 minutes with GloVe.

heading: F Results
text: F.1 Experiments In Figures 2, 3 and 4 we display the scatter plots of the three experiments described in section 4 for the Wikipedia corpus. The findings for OpenSubtitles are qualitatively the same and we exclude the plots for simplicity.
The vertical axes represent the female vs. masculine bias measures based on PMI (left panels), GloVe (middle panels), and SGNS (right panels). Dashed lines represent linear regressions. In the second row, the bias standard error was taken into account as weights in the regression, and error bars are confidence intervals.
All unweighted and weighted correlation coefficients in Table 1 are significantly different from zero at the 0.0001 level.
F.2 p-values Figure 5 shows the corrected p-values for the gender bias of each word in the vertical axes vs. the value of the bias in the horizontal axes. p-values for SGNS and GloVe result from permutations tests whereas PMI uses the log odds ratio test. All p-values have been corrected with BenjaminiHochberg separately for each setting. The plots for OpenSubtitles are very similar and are excluded for simplicity.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: We hope to explore this in future work. The hyperparameters of the models have not been varied, using their default values. This replicates the standard experimental setting used in the literature. Since there are no ground truths when measuring biases (that is, there are no annotations with the amount of bias of words in large corpora), hyperparameters are usually set to their default values