# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Experimental Results
With respect to German, the results obtained for the different scenarios are analyzed and discussed in Appendix B.2. _Scenario 1: complete data debiasing._ In this scenario, we observe that GAN5 an preserves the accuracy of the dataset. More precisely, it increases the accuracy of the decision prediction on the sanitized dataset for all classifiers (cf. Figure 8, _Scenario 51_),compared to the original one which is 0.86, 0.84 and 0.78, respectively, for GB, MLP, and SVM. This increase can be explained by the fact that GAN5 an modifies the profiles to make them more coherent with the associated decision, by removing correlations between the sensitive attribute and the decision one. As a consequence, this sets the same decision to similar profiles in both the protected and the privileged groups. In fact, nearly the same distributions of decision attribute are observed before and after the sanitization but some record's decisions are shifted (\(7.56\%\pm 1.23\%\) of decision shifted in the sanitized whole set, \(11.44\%\pm 2.74\%\) of decision shifted in the sanitized sensitive group for \(\alpha=0.9875\)). Such decision shift could be explained by the similarity between those profiles to others with the opposite decisions in the original dataset. We also believe that the increase in accuracy is correlated with the drop of diversity. More precisely, if profiles become similar to each other, the decision boundary might be easier to find. Figure 7: Diversity and categorical damage on German. The discrimination is reduced as observed through _DemoParity_, \(EqOddGap_{1}\) and \(EqOddGap_{0}\), which all exhibit a negative slope. When correlations with the sensitive attribute are significantly removed (\(\alpha\geqslant 0.6\)), those metrics also significantly decrease. For \(\alpha=0.9875\), \(BER\geq 0.48\), \(yAcc=0.965\), \(DemoParity=0.0453\), \(EqOddGap_{1}=0.0286\) and \(EqOddGap_{0}=0.0062\) for GB; whereas as the original demographic parity gap and equalized odds gap are, respectively, \(DemoParity=0.16\), \(EqOddGap_{1}=0.083\)\(EqOddGap_{0}=0.060\). The performances are improved further for \(\alpha=0.9938\). In this situation, the results obtained are, respectively, \(BER\geq 0.48\), \(yAcc=0.973\), \(DemoParity=0.0185\), \(EqOddGap_{1}=0.0161\) and \(EqOddGap_{0}=0.0045\) (cf., Tables A1 and A2 in appendices for more details). In this setup, FairGan [23] achieves a BER of \(0.3862\pm 0036\) an accuracy of \(0.8247\pm 0.0115\) and a demographic parity of \(0.0354\pm 0.0206\), while FairGan+ [40] reached a protection of BER of \(0.3867\pm 0049\) an accuracy of \(0.817\pm 0.003\) and a demographic parity of \(0.014\pm 0.0065\). _Scenario 2: partial data debiasing._ Somewhat surprisingly, we observe an increase in accuracy for most values of alpha. The demographic parity also decreases while the equalized odds remains nearly constant (\(EqOddGap_{1}\), green line on Figure 8). Table 5 compare the results obtained to other existing work from the state-of-the-art. We include the classifier with the highest accuracy (MLP) and the one with the lowest one (SVM). From these results, we can observe that GANSan outperforms the other methods in terms of accuracy, but the lowest demographic parity is achieved with FairGan+ [40] (\(DemoParity=0.014\)). This is not surprising as this method is precisely tailored to reduce this metric. Our approach, as well as FairGan [23], do not perform well with the use of the original decisions (metric \(EqOddGap_{y}\)). We believe that these poor performances are due to the fact that correlations with original decisions have been removed from the dataset, thus making the new predictions not aligned with the original ones. We also observe that the demographic parity has been improved and our method provides one of the best results on this metric. FairGan+ [36] and MUBAL [36] achieve the best results on the equalized odds metrics as they have been specifically tailored to tackle these metrics.