# A Survey on Bias in Deep NLP
## 2 Defining Bias
### The Bias Problem in Machine Learning  
A concept that is intimately associated with bias is _fairness_. A system is considered to be \"fair\" when its outcomes are not discriminatory according to certain attributes, like gender or nationality. In machine learning evaluation, discrimination can be estimated looking at the confusion matrices for different protected groups. That is, we can compute confusion matrices and derived rates (positive rates, true positive rates, false positive rates and so on) for each subset of samples obtained as a segmentation of the full collection of samples on a certain feature (like \"gender\"). If these rates are far from being equal, that is a potential evidence of a prediction system with an \"unfair\" behavior, i.e., with a clear bias on how decisions are made depending on the values of that certain feature. Several measures have been proposed to study divergences among prediction rates over different population groups, and how to interpret them according to each system goal is now clearly identified [21]. From the large amount of biases derived from cognitive ones, about 24 are of interest in machine learning problems [5]. These latter two studies compile several measures that have been agreed in the analysis of the bias problem in machine learning systems, those measures are _demographic parity, equal opportunity, equal odds_ or _counterfactual fairness_, among others. Of course, these measures can be applied in many artificial intelligence subareas, like image recognition or natural language processing, Let us see the definition of one of them (demographic parity), as some elements can be transferred to our formal definition of bias in language modeling. Demographic parity states that all the groups resulting from the different values of a protected class (e.g., gender) should receive the same rate of positive outcomes [22]. For example, if the system decides to concede a scholarship with the same rate to people in both male and females groups, then system shows demographic parity. Let \(\hat{Y}\) be the predicted decision on whether a scholarship should be granted (\(\hat{Y}=1\)) or denied (\(\hat{Y}=0\)). Then, demographic parity can be defined as \(P(\hat{Y}=1|A=0)~{}=~{}P(\hat{Y}=1|A=1)\), which isequivalent to equal positive rates for both male and females \(PR(A=0)\ =\ PR(A=1)\). Here, \(\hat{Y}\) is the system prediction and \(A\) is the \"protected\" attribute/class. In our example, this is the gender and its possible values are 0 for male or 1 for female. Of course, this measure can be generalize to any protected class, like ethnicity or nationality. In that case, fairness is granted if positive rates for all possible population segments are equal. Where is bias here? It is right there, as the bias would be the deviation between groups resulting from different values of the protected attribute. Thus, the bias would be \(bias\ =\ |P(\hat{Y}=1|A=0)-P(\hat{Y}=1|A=1)|\), which is equal to \(bias\ =\ |PR(A=0)-Pr(A=1)|\) using demographic parity as estimator. _Equal opportunity_ is a good estimator of fairness. This one considers the equality between true positive rates. The rest of the measures are, as pointed out, variants on what we want to be equal from different scores. In general, fairness is computed over the distribution \(<X,A,Z,Y,\hat{Y}>\), referring \(X\) to samples, \(A\) to protected attribute, \(Z\) to rest of attributes, \(Y\) the true labels for those samples and \(\hat{Y}\) to the predicted labels by the model. This clear definition of fairness and how it is evaluated allows the introduction of correction mechanisms in the learning process, like those implemented in the FairTorch library ([[https://fairtorch.github.io/FairTorch/](https://fairtorch.github.io/FairTorch/)]([https://fairtorch.github.io/FairTorch/](https://fairtorch.github.io/FairTorch/)) accessed on 13 January 2020). This way of approaching bias correction is close to what is known as _statistical bias_, as we have seen.