Assessing the Reliability of Word Embedding Gender Bias Measures (2021)

Abstract: Various measures have been proposed to quantify human-like social biases in word embeddings. However, bias scores based on these measures can suffer from measurement error. One indication of measurement quality is reliability, concerning the extent to which a measure produces consistent results. In this paper, we assess three types of reliability of word embedding gender bias measures, namely test-retest reliability, inter-rater consistency and internal consistency. Specifically, we investigate the consistency of bias scores across different choices of random seeds, scoring rules and words. Furthermore, we analyse the effects of various factors on these measures’ reliability scores. Our findings inform better design of word embedding gender bias measures. Moreover, we urge researchers to be more critical about the application of such measures

## introduction despite their success in various applications, word embeddings have been shown to exhibit a range of human-like social biases. for example, @xcite find that both glove @xcite and skip-gram @xcite embeddings associate pleasant terms (e.g. love and peace) more with european-american names than with african-american names, and that they associate career words (e.g. profession and business) more with male names than with female names. various measures have been proposed to quantify such biases in word embeddings @xcite @xcite . these measures allow us to assess biases in word embeddings and the performance of bias-mitigation methods @xcite . they also enable us to study social biases in a new way complementary to traditional qualitative 1 our code is available at https://github.com/ nlpsoc/reliability_bias . and experimental approaches @xcite . a key challenge in developing bias measures is that social biases are abstract concepts that cannot be measured directly but have to be inferred from some observable data. this renders the resulting bias scores more prone to measurement errors. therefore, it is important to carefully assess these measures' measurement quality. @xcite also highlight similar measurement issues in the context of fairness research. in this paper, we focus on one aspect of measurement quality: reliability. it concerns the extent to which a measure produces consistent results. in particular, we investigate the reliability of word embedding gender bias measures. figure 1 illustrates how gender biases in word embeddings are typically measured, with measuring gender biases in the concept of arts as an example. to calculate the gender bias score of a target word w (e.g. poetry, a word that relates to the concept of interest), we need to specify a gender base pair (m, f ) (a gendered word pair, e.g. father/mother) and a scoring rule. a scoring rule is a function that takes the embedding vectors of w, m and f as input and returns a bias score bias m,f w as output. in practice, often an ensemble of gender base pairs are used. in this case, we aggregate (e.g. average) all the bias scores w.r.t. different gender base pairs to obtain an overall bias score for a target word. furthermore, multiple conceptually related target words (e.g. poetry, drama) may be specified to form a query q (e.g. arts). by aggregating individual bias scores of these target words, we can compute an overall bias score for a concept. clearly, the choice of target words, gender base pairs and scoring rules may influence the resulting bias scores. reliable measurements of word embedding gender biases thus require target words, gender base pairs and scoring rules that produce consistent results. in this work, by drawing from measurement theory, we propose a comprehensive approach ( §4) to evaluate three types of reliability for these different components: furthermore, we use multilevel regression to model the effects of various factors (e.g. word properties, embedding algorithms, training corpora) on the reliability scores of target words ( §5.5). our experiments show that word embedding gender bias scores are mostly consistent across different random seeds (i.e. high test-retest reliability) and across target words within the same query (i.e. high internal consistency). however, different scoring rules generally fail to agree with one another (i.e. low inter-rater consistency). moreover, word embedding algorithms have a large influence on the reliability of bias scores. contributions first, we connect measurement theory to word embedding bias measures. specifically, we propose a reliability evaluation framework for word embedding (gender) bias measures. second, we provide a comprehensive assessment of the reliability of word embedding gender bias measures. based on our findings, we urge researchers to be more critical about applying such measures. ## related work measuring gender biases in word embeddings has been receiving a growing amount of research interest in nlp. various gender bias measures have been proposed. they are based on different techniques, such as linear gender subspace identification @xcite @xcite , psychological tests @xcite , inference from nearest neighbours @xcite and regression @xcite . however, recent studies have raised concerns over the reliability of such measures. @xcite show that gender bias scores easily vary in their direction and magnitude when different forms (e.g. capitalisation) of target words or different gender base pairs are used. similarly, @xcite look into 178 different gender base pairs from previous works and find that the choice of gender base pairs can greatly impact bias measurements. they therefore urge future work to examine and document the choices of gender base pairs. moreover, d'amour et al. (2020) find that underspecification of models can lead to unstable contextualised word embedding bias scores. these findings call for a more systematic evaluation of the reliability of word embedding gender bias measures, which is the goal of our study. such measures' lack of reliability may partly stem from the fact that word embeddings themselves are often unstable, sensitive to choices of, for instance, word embedding algorithms @xcite @xcite , hyper-parameters @xcite @xcite and even random seeds @xcite @xcite during word embedding training. various word (embedding) attributes have been found to contribute to the instability of word embeddings, including part-of-speech tags @xcite , word frequency @xcite and word ambiguity @xcite . ## preliminaries in this section, we first review three popular scoring rules used for measuring word embedding gender biases ( §3.1). then, we introduce the conceptual framework of reliability and motivate its use in word embedding gender bias measurements ( §3.2). ## estimating reliability of word embedding gender bias measures in this section, we propose an evaluation framework to assess the reliability of word embedding gender bias measures. respectively, we present our operational definitions for test-retest reliability ( §4.1), inter-rater consistency ( §4.2) and internal consistency ( §4.3). see notation suppose we have s scoring rules, g gender base pairs and t target words. we train k word embedding models with the same hyper-parameters but k different random seeds. for each scoring rule, we calculate the bias score of each target word w.r.t each gender base pair, on each word embedding model. as a result, we get a four-dimensional bias scores matrix b of r s×g×t×k . for calculating inter-rater consistency and internal consistency, we average the gender bias scores derived from the k word embedding models. averaging these embedding models can partial out the influence of random seeds, and therefore lead to more accurate estimation of other types of reliability. in this way, we get another bias matrix b of r s×g×t . ## conclusion & discussion in this paper, we propose to leverage measurement theory to examine the reliability of word embedding bias measures. we find that bias scores are mostly consistent across different random seeds (i.e. high test-retest reliability), as well as across gender base pairs and target words within a query (i.e. high internal consistency). in contrast, the three scoring rules fail to agree with one another (i.e. low inter-rater consistency). furthermore, our regression results suggest that the consistency of bias scores across different random seeds are mostly influenced by various word-level features as well as the word embedding algorithm used. meanwhile, the bias scores of target words across different scoring rules are dominated by the word embedding algorithm used. we thus urge future studies to be more critical about applying such measures. nevertheless, our work has limitations. first, we only consider gender bias measures. future work should apply our reliability evaluation framework to other types of bias (e.g. racial bias). second, we focus on static word embeddings. future work should investigate the reliability of bias measures for contextualised embeddings. third, we do not address validity, the other crucial aspect of measurement quality. we thus call for future studies on the validity of word embedding bias measures. fourth, @xcite argue that intrinsic (word embeddings) biases sometimes fail to agree with extrinsic biases (measured in downstream tasks, e.g. coreference resolution). one potential research direction is to assess the reliability of extrinsic bias measurements as well, to shed further light on the disconnect between intrinsic and extrinsic biases. lastly, while icc and cronbach's alpha are established reliability estimators in many scientific disciplines, correct interpretation of their values is often challenging and requires both statistical and field-specific expertise @xcite . future work should address the appropriate use of these estimators and their limitations in the context of nlp research.