# A Survey on Bias in Deep NLP
## 2 Defining Bias
### Definition of Bias in Language Modeling  
Once it is clear the semantic definition of the stereotyping bias, we can map that semantic identification down to word probabilities. This is straightforward, as a language model is nothing but a model able to compute a probability for a sequence of words \(P(w_{1},...,w_{m})\).  
Definition 3: A stereotyped language can be represented as the tuple \(<C,P,I,L,p_{p},P_{s},T_{p},T_{s}>\), which contains a stereotyped knowledge and two terminology sets: protected terms \(T_{p}\) and stereotyped terms \(T_{s}\). Protected terms \(T_{p}\) are those expressions (words or multi-words) in the vocabulary that can be unambiguously mapped to values of a protected property \(p_{p}\). Stereotyped terms \(T_{s}\) are those expressions (words or multi-words) in the vocabulary that can be unambiguously mapped to values of stereotyped properties \(P_{s}\).  
\(T_{s}\) is the set of words or terms that represent possible values of stereotype properties \(P_{s}\) (for example, \"high imagination\", \"low sensibility\", \"beauty\", \"rational mind\", etc.). Examples of expressions in \(T_{p}\) would be any term defining gender, like \"nurse\", \"actress\", \"woman\", \"girl\" or alike. Once the stereotype is defined at the semantic level, we can consider that if the probability of a sequence of words containing expressions in \(T_{s}\) on stereotyped properties \(P_{s}\) is significantly different according to the value of \(p_{p}\) of the referenced individual, then the model is biased.  
Now, we are ready for the final definition of stereotyping bias in language models.  
Definition 4: Let \(L_{s}=<C,P,I,L,p_{p},P_{s},T_{p},T_{s}>\) be the definition of a stereotyped language, stereotyping bias is defined as the distance \(d\) between probabilities \(d(P(w_{1},...,w_{m}|t_{p}^{i}))\), \(P(w_{1},...,w_{m}|t_{p}^{i}))\), with \(i\
eq j\) where \(t_{p}^{i}\) and \(t_{p}^{i}\) are the expressions for two different values of the protected property \(p_{p}\) and \(\exists w_{k}\in\{w_{1},...,w_{m}\}\) so that \(w_{k}\in T_{s}\).  
In other words, a language model is biased if distributions of probabilities of terms containing stereotyped expressions are different subject to existing protected expression priors. Following our simple example, the _stereotyped language_ could be defined as \(<C,P,I,L,\texttt{has-gender},\{\texttt{has-job}\},\{\texttt{girl},\texttt{ women},\texttt{[PERSON]},\texttt{man}\},\{\texttt{doctor},\texttt{nurse}\}>\).  
As key elements in the definition of a stereotype are \(p_{p},P_{s},T_{p}\) and \(T_{s}\), we can focus on these four components to characterize it. In Table 1, further examples in different bias domains are proposed.  
Now, consider this simple text:  
[PERSON] works as a nurse in the hospital. A man is the doctor.  
The definition is open to any kind of distance. If we select absolute difference, the stereotyping bias of the language model trained on the text above could be:  
\(|P(works,as,a,nurse|[PERSON])-P(works,as,a,doctor|[PERSON])|\)  
Another valid measure would be:  
\(|P(works,as,a,nurse|man)-P(works,as,a,doctor|man)|\)  
As you can see, different distances can be computed depending on the sequence or the prior value of the protected property considered. An appropriate evaluation of a language model would imply, therefore, a battery of expressions like the ones above, with protected expressions as priors and stereotyped expressions in the sequence, from which an average distance could be calculated.