On gender bias in word embeddings
By: Danielle Boccelli
Tags: AI, NLP, Data Science, Articial Intelligence, Language

On gender bias in word embeddings

What analogies can tell us about our language and our algorithms

The Natural Language Processing (NLP) group at Stanford University made publicly available the list of papers from their CS 384 seminar on Ethics and Social Issues in Natural Language Processing, and so I have been on a bit of a reading binge trying to learn more about this fascinating and important topic.

In this article, I want to explore the use of analogies for identifying biases in word embeddings by focusing on two papers on the topic: Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016) [1] and Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor (2020) [2]. The first, which I will refer to as “the paper on debiasing,” is from the Stanford NLP list, and the second, referred to as “the paper on fairness,” is available through the Computational Linguistics journal of MIT Press Journals.

But first things first.

What is a word embedding?

A word embedding is a vector representation of a word that can be used to convey the meaning of the word to a computer. Therefore, with a word embedding, an algorithm can take as input a numerical representation of a word, rather than simply relying on counts of words.

Word embeddings have been researched in some depth for machine learning applications, probably because they have some interesting (and perhaps unexpected) properties: (1) semantically similar words tend to have vectors that are close to each other in the vector space, and (2) the differences between word embeddings tend to produce vectors representing the difference in meaning between words (e.g., king − man + woman = queen).

Such differences between words can also be described using analogies (e.g., man is to (:) king as (::) woman is to (:) queen), and it seems as if many researchers have had their share of fun using word embeddings to fill in analogies (e.g., man : king :: woman : x). However, while it may be interesting to set up an analogy to see which word is selected by the algorithm to replace X, such research can find dangerous biases that exist within our language.

How can a word embedding be biased?

Word embeddings capture the meaning of a word based on its contextual applications in a corpus. For example, in the paper on debiasing, the word embedding for grandmother was found to be similar to the vector produced by adding the word embeddings for wise and gal, which shows that the word grandmother connotes wisdom, femininity, and personhood. However, while the word embedding for grandmother carries an unbiased gender association, other words, such as lovely (female bias) and brilliant (male bias), carry gender associations that are not necessarily linked to denotative meaning.

According to the paper on debiasing, word embeddings tend to maintain the gender stereotypes that pervade language through gender bias, which can be magnified when applied in certain models. Therefore, because the gender stereotypes are present in the training data, to correct for them is to acknowledge and correct for human bias more than for algorithmic bias, and so interestingly, by debiasing word embeddings, one aims to create algorithms that are more politically correct than the humans who collectively produced the corpus used for training (but more on this concept later).

The paper on debiasing uses analogy, as described above, to find gender bias in the word embeddings of occupational terms, such as those from the title, i.e., computer programmer (male bias) and homemaker (female bias), among other terms, and proposes a method for debiasing that involves identifying the gender subspace within the vector space.

I will not discuss the methodology in great detail, but with the method outlined in the paper, one can obtain analogies such as the following:

she : he :: nurse : physician

she : he :: giggle : chuckle

she : he :: blonde : burly

It is not difficult to understand why so many papers have been written on word embeddings and their applications: while they may magnify the less-than-ideal aspects of our society, as carried by our language, they produce interesting results that make sense based on our preexisting associations of words to their meanings, even if we are fully aware that our associations can be factually and politically incorrect.

How can a methodology be biased?

When I read the paper on debiasing, I was so swept up in how interesting the research seemed that I missed two key points of concern in the methodology. Luckily, I happened upon the article on fairness the very next day.

First, the article on fairness discusses analogies from a linguistic perspective and points out that “there is no single predefined term that ‘correctly’ completes the analogy [man : doctor :: woman : X]” at the semantic level. In other words, when asked to fill in a word for X, while there may be several that seem fitting, there is no correct answer. Therefore, we cannot rate the accuracy of the word selected by the algorithm, and any word that is selected, other than a word equivalent to doctor, will seem biased due to the nature of the experiment.

Second, the article states that, at the morphosyntactic level, X should be doctor. However, when the algorithm is looking for the word that best fits for X, doctor cannot be selected due to a constraint: the words completing both halves of the analogy must be different. Under this distinctness constraint, if one thinks about the optimal word for X, then the most obvious choice is probably nurse — not because men are doctors and women are nurses, but because doctors are people who care for patients and nurses are people who are not doctors and care for patients.

Further, the article goes on to show that, when the distinctness constraint is lifted, in this case and in many similar cases where biases were said to be found, the choice for completing the right side of the analogy was often the very word used to complete the left side of the analogy, which implies that the methodology, rather than the word embedding itself, may introduce bias.

Should word embeddings be biased?

While I am in full support of a burly woman’s right to chuckle, I believe that the necessity of debiasing depends on the practical application of the model, as it can be argued that some gender biases in language are more due to particularities in language usage that have naturally arisen than to socially stigmatizing stereotypes of any real importance.

For example, in certain text generation tasks, the use of debiased language models might impact the connotation of a statement due to particularities in usage. If a generated text refers to a woman as the debiased word handsome, then a reader might assume the woman is attractive in a masculine or non-traditional way, which differs from the connotation of the word when applied in the standard gendered way; there would be a similar difference in connotation if a man were to be referred to as pretty.

It is not my intention to either support or reject traditional ideals of beauty; however, because humans have preformed associations with words, if an algorithm generates text that does not adhere to those associations, then the text might may not capture some of the subtleties of natural language.

In the example above, the cost of referring to an attractive man as pretty is nowhere near as high as the cost of a gender-biased association in a task such as resume screening; however, I believe the scenario is still worth considering, especially as a way to reiterate that the practical application of the language model is important when deciding if gender bias is a necessary consideration.

Context-aware artificial intelligence

Before I go on, I want to be clear that I am in no way claiming that computer programming is an inherently male profession, nor am I claiming that homemaking is an inherently female undertaking, regardless of the ratio of men to women in those roles; however, if the reality at present is that more men than women work as computer programmers, and more woman than men work as homemakers, then perhaps a language model should not be debiased to erase these language artifacts.

Now, let us consider what I will call context-aware artificial intelligence (AI). For a context-aware AI to behave appropriately in a given context, it must have, not just a knowledge of how words are used, but a knowledge of the implications of not using a word in a way that is socially acceptable at the time of its use.

As such, a context-aware AI should know that certain roles are considered traditionally male roles, and perhaps rather than neutering words that carry implicit biases, the AI should be aware of these biases and use this awareness to inform its ability to, for example, generate text that is sensitive to these biases. In other words, a context-aware AI would have to learn what is socially acceptable (whether it then abides by social norms is another issue) based on recent materials, rather than all available text.

How this concept works mathematically, with word embeddings, I do not know, but it sure is fun to think about.