We test all examples on different embedding spaces built for English, using two settings for the analogy code: when all terms must be different (as in the original, widely used, implementation), and without this constraint, meaning that any word, including the input terms, can be returned. As far as we know, this is the first work that evaluates and reports analogies in an unrestricted fashion, since the analogy code is always used as is. Our experiments and results suggest that the mainstream examples as well as the use of the analogy task itself as a tool to detect bias should be revised and reconsidered. Warning This work does not mean at all to downplay the presence and danger of human biases in word embeddings. On the contrary: embeddings do encode human biases, and we believe that this issue deserves the full attention of the field. However, we also believe that overemphasising and specifically seeking biases to achieve sensational results is not beneficial. It is also not necessary: what we observe naturally is worrying and sensational enough. Rather, we should aim at transparency and experimental clarity so as to ensure the fairest and most efficient treatment of the problem.

Results are over 50% lower in the semantic set, and the drop is even more serious in the syntactic examples, with the exception of “nationality-adj". In the default evaluation set, and in the extended set proposed by  to cover additional linguistic relations, there are no word-pairs for which the gold target word is one of the three query words, in other words: A, B or C is the correct answer. Thus one might deem it a reasonable decision that the original analogy code does not let any of the original vectors to be returned. However, these conditions do exist, and this choice has consequences. The major consequence we observe is discussed in Section 4, and has to do with analogies affected by human bias. But even for the analogy types of Table 1, there are cases where this constraint is undesirable, due to homography. Additionally, there are other analogy types for which such constraint is utterly counterproductive, such as is-a or part-of relations.

Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor	Analogies such as man is to king as woman is to X are often used to illustrate the amazing power of word embeddings. Concurrently, they have also exposed how strongly human biases are encoded in vector spaces built on natural language. While finding that queen is the answer to man is to king as woman is to X leaves us in awe, papers have also reported finding analogies deeply infused with human biases, like man is to computer programmer as woman is to homemaker, which instead leave us with worry and rage. In this work we show that,often unknowingly, embedding spaces have not been treated fairly. Through a series of simple experiments, we highlight practical and theoretical problems in previous works, and demonstrate that some of the most widely used biased analogies are in fact not supported by the data. We claim that rather than striving to find sensational biases, we should aim at observing the data"as is", which is biased enough. This should serve as a fair starting point to properly address the evident, serious, and compelling problem of human bias in word embeddings.

In Table TABREF9 we report these examples, organised by the papers which discussed them, together with the returned term as reported in the paper itself, and the top two terms returned when using our modified code (1st and 2nd, respectively). Each example is tested over the same embedding space used in the corresponding paper.

Introduction	Word embeddings are distributed representations of texts which capture similarities between words. Beside improving a wide variety of NLP tasks, the power of word embeddings is often also tested intrinsically. Together with the idea of training word embeddings,  introduced the idea of testing the soundness of embedding spaces via the analogy task. Proportional analogies are equations of the form INLINEFORM0 , or simply A is to B as C is to D. Given the terms INLINEFORM1 , the model must return the word that correctly stands for INLINEFORM2 in the given analogy. A most classic example is man is to king as woman is to X, where the model is expected to return queen, by subtracting “manness" from the concept of king to obtain some general royalty, and then re-adding some “womanness" to obtain the concept of queen ( INLINEFORM3 ). Beside this kind of magical power, however, embeddings have been shown to carry worrying biases present in our society and thus encoded in language. Recent studies  ,  found that embeddings yield biased analogies such as the classic man is to doctor as woman is to nurse, or man is to computer programmer as woman is to homemaker.

Let women be doctors	One of the most well known analogies brought as example of human bias in word embeddings is man is to doctor as woman is to nurse  ,  . This heavily biased analogy reflecting gendered stereotypes in our society, is however truly meaningful only if the system were allowed to yield “doctor” (arguably the expected answer in absence of bias) instead of “nurse”, and it doesn't. But we know that the system isn't allowed to return this candidate, since the original analogy code rules out the possibility of returning as D any of the query terms INLINEFORM0 , making it impossible to obtain man is to doctor as woman is to doctor (where INLINEFORM1 ). This means that the bias isn't necessarily (or at least not only) in the representations themselves, rather in the way we query them. So, what do the embedding spaces actually tell if you let them return any word in the vocabulary? We took a selection of mainstream, striking examples from the literature on embedding bias, and tested them fairly, without posing any constraint on the returned term, exactly as we did for all analogies in Section SECREF3 .

In fact,  suggest that analogies are not quite reliable diagnostics for uncovering bias in word embeddings, since bias is anyway often encoded implicitly. As a side note, we would like to mention that in an earlier version of their paper,  accidentally searched for the inverse of the intended query, and still managed to find biased examples. This seems to be a further, strong, indication that strategies like this are not fully suitable to demonstrate the presence of bias in embeddings. If analogies might not be the most appropriate tool to capture certain relations, surely matters have been made worse by selecting results in order to prove (and emphasise) the presence of human bias. Using such sensational “party tricks"  is harmful, as they get easily propagated both in science itself  ,  ,  , even outside NLP and AI  , as well as in popularised articles of the calibre of Nature  . This is even more dangerous, because of the widened pool of readers, and because such readers are usually in no position to verify the reliability of such examples. In any case, anyone who constructs and uses analogies to uncover human biases must do this fairly and transparently, and be aware of their limitations.

Please, use analogies fairly, and with care	If we do not operate any manipulations on the returned vectors, neither by setting constraints nor by cherry-picking the output, we observe that in many cases, independently of the analogy type and the query terms, the model simply returns one of the input terms, and in particular INLINEFORM0 . Perhaps, this is a weakness of embeddings in modelling certain relations, or the analogy task as such is not apt at capturing them. Such observations relate to two points raised in previous work. First, the suggestive power of analogies should not be overestimated. It has been argued that what is observed through the analogy task might be mainly due to irrelevant neighborhood structure rather than to the vector offset that supposedly captures the analogy itself  ,  . Indeed,  have also shown that the 3cosadd method is not able to capture all linguistic regularities present in the embeddings. Interestingly, the analogy task has not been recently used anymore to evaluate the soundness of contextualised embeddings  ,  . Second, bias isn't fully captured anyway via the analogy task.

Constraining the output	What immediately stands out is that, bar a few exceptions, we do not obtain the term reported in the respective paper. One reason for this is that the model is now allowed to return the input vectors, and in most cases it does just that (especially INLINEFORM0 ). In Section SECREF3 , we saw how this heavily affects the results on the original analogy test, and we also discussed why it would nevertheless be beneficial to impose no restrictions on the returned answer. When analogies are used to study human bias, though, the problem is more serious: How can we claim the model is biased because it does not return doctor if the model is simply not allowed to return doctor? As a further constraint to the allowed output,  add an empirical threshold to Equation EQREF2 to ensure that terms that are too similar to INLINEFORM0 are excluded. Consequences are non-trivial. By not allowing the returned vector to be too close to the input vectors, this method basically skips potentially valid, unbiased answers until a potentially more biased answer is found.

Experimental details	For both word2vec  and gensim  we adapted the code so that the input terms of the analogy query are allowed to be returned. Throughout this article, we use two different embedding spaces. The first is the widely used representation built on GoogleNews  . The second is taken from  , and was trained on a Reddit dataset  . We test analogies using the code with and without modification, with the aim of showing the drawbacks and dangers of constraining (and selecting) the output of analogy queries to word embeddings. The analogies we use in this article come from three sources: the original analogy dataset proposed by  (Section SECREF3 ), a small selection of additional analogies to highlight the need to be able to return input vectors (Section SECREF3 ), and a collection of examples found in papers that address the problem of (human) biases in word embeddings (Section SECREF4 ). We follow  ,  and  by using 3cosadd to calculate the analogies, as shown in Equation EQREF2 :  DISPLAYFORM0  All the examples used in this article, plus any new query, can be tested on any of the embeddings in the original and modified analogy code, and through our online demo.

In this sense, it is admirable that  try to better understand their results by checking them against actual job distributions between the two genders. Aiming primarily at scientific discovery rather than sensational findings is a strict pre-requisite to truly understand how and to what extent embeddings encode and reflect the biases of our society, and how to cope with this, both socially and computationally.

Not all analogies are the same	The original, widely used, analogy test set introduced by  consists of two main categories: semantic analogies (Paris is to France as Tokyo is to Japan) and morpho-syntactic analogies (car is to cars as table is to tables). Within these, examples are classified in more specific sub-categories, as shown in the left column of Table TABREF5 . In the same table we report two scores based on the Google News embeddings as well as for the reddit embeddings from  . Under “orig" we report the score obtained using the original analogy code, and under “fair" we report the score yielded by our altered version, where the query terms ( INLINEFORM0 ) can be returned. The results show a drastic drop in performance for the fair setting. In most cases, this is because the second term is returned as answer (man is to king as woman is to king, thus INLINEFORM0 ), but in some cases it is the third term that gets returned (big is to bigger as cold is to cold, thus INLINEFORM1 ).

In Table TABREF12 , we report the top-10 candidates for asian is to engineer as black is to X in both the Reddit embeddings of  as well as GoogleNews, for completeness. Similarly, we now know that an unrestricted analogy search for man is to doctor as woman is to X returns doctor, but this does not provide a complete picture. Reporting the top-10 for this query as well as the top-10 for the inverted query (Table TABREF12 ) surely allows for a much better informed analysis rather than simply reporting doctor, or picking nurse.

It isn't necessarily the case that more distance corresponds to more bias, but it is usually the case that less distance is akin to less bias (for example, gynecologist is a less biased answer than nurse to the query man is to doctor as woman is to X).

While aiming at returning a vector distant enough from the input term might be desirable for some of the analogies, this threshold-based strategy is not fair when researching bias, as it potentially forces the exclusion of unbiased terms (in this case, after magnificent, one would find the following terms before encountering brilliant: marvelous, splendid, nice, fantastic, delightful, terrific, wonderful). In the example she is to sewing as he is to X., the threshold was strong enough to even exclude a potentially biased answer (woodworking).   also use the analogy test to demonstrate bias, starting from a pre-selection of terms to construct their queries from a variety of sources. In addition to using the original analogy code, thus missing out on what the actual returned term would be, they state that rather than reporting the top term, they hand-pick an example from the returned top-N words. While qualitatively observing and weighing the bias of a large set of returned answers makes sense, it can be misleading to cherry-pick and report very biased terms in sensitive analogies. At the very least, when reporting term-N, one should report the top-N terms to provide a more accurate picture.

Computer programmer or just programmer?	If the analogy really is a symptom of a biased vector space, we should find similar biases for synonyms or closely related words to the input word INLINEFORM0 . However, with computer_programmer for example, this does not seem to be the case. If we use the term programmer instead of computer_programmer, homemaker is not very close (165), while for coder (13,374), developer (26,117) and hacker (56,646) it does not even appear in the top 10,000. Also, when using white instead of the less frequent and more specialised (and in a way less parallel to black) caucasian in the analogy of black is to criminal as caucasian is to X, lawful is found at position 40 instead of 13. In a way, examples are always cherry-picked, but when making substantial claims on observed biases, the fact that the results we obtain are due to a carefully chosen word (rather than a similar one, possibly even more frequent), should not be overlooked.

Attempts at reducing bias, either via postprocessing  or directly in training  have nevertheless left two outstanding issues: bias is still encoded implicitly  , and it is debatable whether we should aim at removal or rather at transparency and awareness  ,  . With an eye to transparency, we took a closer look at the analogy structure. In the original proportional analogy implementation, all terms of the equation INLINEFORM0 are distinct  ,  . In other words, the model is forced to return a different concept than the original ones. Given an analogy of the form INLINEFORM1 , the model is not allowed to yield any term INLINEFORM2 such that INLINEFORM3 , or INLINEFORM4 , or INLINEFORM5 , since the code explicitly prevents this. While this constraint is helpful when all terms of the analogy are expected to be different, it becomes a problem, and even a dangerous artifact, when the terms could or even should be the same. We investigate this issue using the original analogy test set  , and examples from the literature.

First or twentieth is not the same	A closer look at the results makes things even more worrying. If the top answer yielded by our unrestricted code is one of the input vectors (e.g. doctor), the original code would not have shown it. It would have instead yielded what we obtain as our second answer. This is what we should see in the reported analogies. However, Table TABREF9 (column Index) shows that this is not always the case. The threshold method of  described in Section SECREF10 is the cause for this outcome in their examples, as vectors higher in the rank have been excluded as too close to the input vector. Unfortunately, while surely successful over standard, factual analogy cases, this strategy turns out to be essentially a way of selecting the output. For example, their strategy not only excludes lovely (input term), but also magnificent as a possible answer for she is to lovely as he is to X, since the vector for magnificent is not distant enough from the vector of input term lovely. As can be seen in Table TABREF9 , lovely and magnificent would be the first and second words returned otherwise. The term brilliant is only returned in 10th position by an unrestricted search.

Question: Which embeddings do they detect biases in?
Answer: Word embeddings trained on GoogleNews and Word embeddings trained on Reddit dataset