## 2 Related Work
In [28], a methodology based directly on word embeddings is proposed to differentiate gender bias associations (e.g., biased association between _receptionist_ and _female_) from associations of related concepts (e.g., between _queen_ and _female_); a neutralisation and equalisation debias process. The same debias process is used in [42], but in this work a new representation to detect the bias is proposed: the authors include a contextualisation step and create two subspaces of the attribute (e.g., gender), one for male and other for female. The contextualisation relies on a large and diverse set of sentences in which the bias attribute words (e.g., _he/she_, _man/twoman_) appear; for example, if in one subspace it is included a sentence containing _he_, the same sentence is included in the subspace for female but replacing _he_ by _she_. In addition, in this line, [29] proposes some metrics over word embeddings representations to measure the bias. In this work, they distinguish two different biases: (i) _implicit bias_, in which we only have sets of target terms with respect to which a bias is expected to exist in the embedding space (e.g., \(T1=\{physics;chemistry;experiment\}\) and \(T2=\{\mathit{poetry};dance;drama\}\) without any specification, one could expect in \(T1\) and \(T2\) a gender bias towards \(\{\mathit{man},\mathit{father},\mathit{woman},\mathit{girl}\}\}\); (ii) and _explicit bias_, in addition to sets \(T1\) and \(T2\), it is given one (e.g., \(A=\{\mathit{man},\mathit{father},\mathit{woman},\mathit{girl}\}\)) or more (e.g., two opposite attribute sets \(A1=\{\mathit{man},\mathit{father}\}\) and \(A2=\{\mathit{woman},\mathit{girl}\}\)) reference attribute sets. Some other recent works on bias again face the gender stereotypes. A Gender Stereotype Reinforcement (GSR) measure was proposed in [43] to quantify the extent to which a search engine responds to stereotypically gendered queries with documents containing stereotypical language. Recently, in [44], the authors have compared the efficacy of lexicon-based approaches and end-to-end machine learning-based approaches (in particular BERT); the obtained results showed that the latter is significantly more robust and accurate, even when trained by moderately sized datasets. Differently, in [45] it is used Natural Language Inference (NLI) as the mechanism for measuring stereotypes [45]. The idea is that invalid inferences about sentences can expose underlying biases, and based on that, it is possible to see how gender biases affect inferences related to occupations. For example, a gender-biased representation of the word _accountant_ may lead to a non-neutral prediction in which the sentence _The accountant ate a bagel_ is an entailment or contradiction of the sentences _The man ate a bagel_ and _The woman ate a bagel_; this could happen because of the gender bias concerning occupations. Therefore, the predictions of NLI on a set of entailment pairs that should be inherently neutral are used to compute the deviation from neutrality, which is assumed as the gender bias. A similar idea of [45] has been used in [6] but with a different perspective. In [6], the authors propose two different level tests for measuring bias: (i) the intrasentence test, in which there are a sentence describing the target group and a set of three attributes which correspond to a stereotype, an antistereotype, and an unrelated option; and (ii) the intersentence test, consisting of a first sentence containing the target group, a second sentence containing a stereotypical attribute of the target group, a third one containing an antistereotypical attribute, and lastly an unrelated sentence. These tests remain the aforementioned idea of [45] to use NLI to measure entailment, contradiction, or neutral inferences to quantify the bias. To evaluate their proposal, the authors of [6] collected a dataset (StereoSet) for measuring bias related to four domains: gender, profession, race, and religion. For this purpose, they use specific words to represent each social group.