Understanding Gender Bias in Knowledge Base Embeddings (2022)

Abstract: Knowledge base (KB) embeddings have been shown to contain gender biases. In this paper, we study two questions regarding these biases: how to quantify them, and how to trace their origins in KB? Specifically, first, we develop two novel bias measures respectively for a group of person entities and an individual person entity. Evidence of their validity is observed by comparison with real-world census data. Second, we use the influence function to inspect the contribution of each triple in KB to the overall group bias. To exemplify the potential applications of our study, we also present two strategies (by adding and removing KB triples) to mitigate gender biases in KB embeddings.

## introduction gender biases have been shown to have noticeable presence in a wide range of nlp models. for example, we can observe that the word embedding of "engineer" is closer to "he" than "she" @xcite , and co-reference systems associate "surgeon" more with masculine pronouns than with feminine ones @xcite . these biases are brought to our models from training data by our algorithms. hence, besides revealing the existence of gender biases, it is important to quantify them and explain their origins in data. knowledge bases (kb, e.g. freebase, @xcite provide accessible organizations of human knowledge by the form of triples. each triple consists of a head entity, a relation, and a tail entity. for example, the fact that marie curie is a chemist is represented as marie curie, people.person.profession, chemist . kb embeddings encode these knowledge into dense vector representations. it is important to understand gender biases in kb embeddings for two major reasons. first, kb embeddings serve as sources of prior knowledge in many downstream nlp models (e.g. pre-trained language models, @xcite . clearly, if biases exist in kb embeddings, they can easily propagate into these models, and drive these models more biased. second, @xcite observe that word embeddings reflect biases in the training corpora, and hence our society. likewise, we suspect kb embeddings to reflect biases encoded in kbs, as also suggested by @xcite . in this paper, we propose two novel gender bias measures for kb embeddings, one for a group of person entities (group bias) and the other for an individual person entity (individual bias). furthermore, with influence function @xcite , we explain the origins of group bias at the fact triple level (i.e. how each triple in kb contribute to group bias). in practice, we use transe @xcite to demonstrate our methods, for its popularity and simplicity. nevertheless, most of our study can generalize to other embedding algorithms. specifically, we make four contributions. first, regarding a group of person entities with a shared relation-tail pair (e.g. of the same occupation), using correlation analyses, we measure their gender biases by the differences between different genders' link prediction errors. second, to understand the origins of the group bias, we use influence function to find its highlyinfluential triples in kb (i.e. triples that will change the bias most if being removed during training). third, regarding a single person entity, using counterfactual analyses, we develop a bias measure by measuring the change of the link prediction error when we keep everything else the same and perturb its gender. to avoid the intractable computational cost of re-training, we propose to use influence function to approximate the results. fourth, to further facilitate large-scale influence function based analyses, we derive a closed-form approximation of the hessian matrix of transe loss. we therefore improve the time complexity of computing influence function from o(n) (stochastic approximation) to o(1). moreover, in further analyses, we show that both group and individual bias correlate well with realworld biases. we argue that this suggests the validity of our bias measures. we also show the accuracy of our influence function approximation by comparing with the brute-force strategy (i.e., leave-one-out re-training). finally, to exemplify the applications of our study, we propose two simple de-biasing strategies, and demonstrate their effectiveness. ## preliminaries knowledge base kb is a set of structural human knowledge represented by triples @xmath0, r, t }, where h is a head entity, r is a relation type, and t is a tail entity. moreover, these triples form a graph with entities as nodes (denoted by e, where @xmath1. in this work, since we are particularly interested in person entities and their gender, we use s, r g , m or s, r g , f to represent a person s with gender male or female, where r g is the relation of gender. 1 transe the entities and relations in kb can be represented with embedding vectors. these embeddings can serve in many nlp task as a source of prior knowledge. in this work, we focus on the widely used transe @xcite . given a triple h, r, t , the key idea of transe is to make vectors of h, r and t close to each other in the sense of small link prediction error. concretely, transe embeddings are learned by minimizing a margin-based ranking loss, in this paper, we use freebase's @xcite subset fb5m @xcite as the kb for training transe embeddings and performing our analyses. see appendix a for detailed setup. influence function @xcite provides an efficient way to approximate each training sample's impact on correctly predicting a test sample. formally, let l(z, θ) be a convex loss function on a training set {z i } n @xmath2. the empirical risk minimizer (erm) is given by θ = arg min θ 1 n n @xmath3, θ). we are interested in a training sample z's impact on θ, with a weight of ε. in this case, the new erm is given by θz,ε = arg min θ moreover, we are interested in the change of the test performance, which is a function f of the test sample z test and the model parameter (lhs). by applying chain rule to f and equation 2, we can obtain the difference of test performance. formally, similarly, by splitting perturbation to first remove then add, we can also inspect the change of f when a training sample z is perturbed to z . denote θ -z,z ,ε = arg min θ 1 n n @xmath4, θ) -εl(z, θ) + εl(z , θ), and apply equation 3 twice, we obtain finally, besides single sample estimation, we are also interested in inspecting the influence of removing a group of training samples. in these cases, ## gender bias measures in this section, based on link prediction, we take two views to quantify gender biases in kb embeddings. first, using correlation analysis, we take a macro view to inspect gender biases of a group of person entities (e.g., how gender influences the overall occupation prediction accuracy of a group of engineer entities). second, under the framework of counterfactual analysis, we take a micro view to assess gender biases of an individual person entity (e.g., how a specific engineer entity's gender influences its occupation prediction accuracy). afterwards, we build connections between them. in this following, we adopt occupation prediction as our running example. the reason is two fold. first, among all of the relations connected with person entities, occupation relation has the highest coverage rate (i.e. connect with the most person entities). second, most previous relevant studies also focus on occupation. our choice makes it easier to perform comparative studies @xcite . ## related work various measures have been proposed to quantify gender biases in word embeddings @xcite @xcite . many of them are based on vector similarity (e.g. cosine similarity) between words, which matches the training objective of most word embedding al-gorithms (maximize the vector similarities between similar words, @xcite . moreover, @xcite suggest that word embedding can reflect biases in the training corpora and hence our society. recently, a few studies have explored gender biases in kbs and their embeddings. a pioneer study by @xcite investigates gender gap in wikidata across time, space, culture, occupation and language. a following study @xcite further analyzes the race and country of citizenship bias in kb regarding stem representation. moreover, @xcite analyze the potential bias issues in kbs from both data and schema viewpoints. @xcite propose a kb embedding bias measure based on the change of link prediction error after a one-step update towards male. @xcite and @xcite propose to use adversarial training objective to mitigate biases in kb embeddings. influence function is a commonly used technique in robust statistics @xcite . @xcite first use it to inspect each training point's influence on a neural network's prediction. a following study by @xcite investigate the accuracy of influence function on measuring the effect of removing a group of training samples, and show that its approximation has strong correla-tions with actual effects. afterwards, @xcite apply influence function as a differential bias measure to study gender bias in word embedding. moreover, @xcite use an simplification of influence function to perform adversarial attack on link prediction. ## conclusion and discussion in this paper, we study the gender biases in kb embeddings. first, we develop two bias measures to quantify biases: one from the group level and the other from the individual level. evidence of their validity are obtained in comparison with realworld biases. second, to understand the origins of biases, we adopt influence functions for triplelevel analysis and develop an efficient method for fast evaluation. the accuracy of this method is validated by comparing our approximation with group-truth changes after re-training. moreover, as examples of the potential applications of our findings, we propose two de-biasing strategies for kb embeddings and obtain promising performance. although we focus on freebase (fb5m) and transe in this paper, we note that our analyses are theoretically generalizable to other commonlyused kbs and embedding training algorithms. for instance, wikidata, another commonly-used kb, uses a different hierarchical structure to organize its data @xcite @xcite . however, it still loosely follows the triple structure used in freebase, and therefore can be pre-processed to fit in our analyses. also, because our bias measures and bias tracing methods are built on simple and generalizable definitions (i.e., differences between link predictions errors and influence function), they can naturally be adapted to other kb embedding algorithms @xcite @xcite . however, we recognize that such generalizations are not trivial efforts. take wikidata again for an instance, although a simple transformation is adequate for running the embedding algorithm, it is far from fully eliminating the differences between freebase and wikidata. for example, wikidata does not have an inverse predicate for each relation, and has a much smaller number of overall relations @xcite . such differences might have a large impact on the final results. also, to perform the same analyses with other embedding algorithms, we will need to develop algorithms to facilitate the computa-tion of their influence function (as lemma 1), too. therefore, we consider such generalizations to be promising future directions but out of the scope of our work.