# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Experimental Results
Even though our method is not specifically constrained to mitigate the demographic parity, we can observe that it significantly improve it. Thus, while partial data debiasing is not the best application scenario for our approach as the original decision might be correlated with the sensitive attribute, it still mitigates its effect to some extent. Figure 3:0 Accuracy (blue), demographic parity gap (orange) and equalized odds gap (true positive rate in green and false positive rate in red) computed for scenarios 1, 2, 3 and 4 (top to bottom), with the classifiers GB, MLP and SVM (left to right) on Adult dataset. The greater the value of \(\alpha\) the better the fairness. Using only the sanitized data \(\bar{A}\) (\(S1\), \(S2\)) increases the accuracy while a combination of the original (\(A\)) and sanitized data (\(\bar{A}\)) decreases it. _Scenario 3: building a fair classifier._ The sanitizer helps to reduce discrimination based on the sensitive attribute, even when using the original data on a classifier trained on the sanitized one. As presented in the third row of Figure 8, as we force the system to completely remove the unwarranted correlations, the discrimination observed when classifying the original unperturbed data is reduced. On the other hand, the accuracy exhibits here the highest negative slope with respect to all the scenarios investigated. More precisely, we observe a drop of 16% for the best classifier in terms of accuracy on the original set, which can be explained by the difference of correlations between \(A\) and \(Y\) and between \(\bar{A}\) and \(\bar{Y}\). As the fair classifiers are trained on the sanitized set (\(\bar{A}\) and \(\bar{Y}\)), the decision boundary obtained is not relevant for \(A\) and \(Y\). FairGan [23], which also investigated this scenario, achieves \(yAcc=0.82\) and \(DemoParity=0.0461\pm 0.0424\) whereas our GB classifier achieves \(yAcc=0.724\pm 0.038\) and \(DemoParity=0.111\pm 0.059\) for \(\alpha=0.9875\) and \(yAcc=0.725\pm 0.107\) and \(DemoParity=0.0598\pm 0.0422\) for \(\alpha=0.9938\). _Scenario 4: local sanitization._ On this setup, we observe that the discrimination is lowered as the \(\alpha\) coefficient increases. Similarly to other scenarios, the larger the correlations with the sensitive attribute are removed, the higher the drop of discrimination as quantified by the \(DemoParity\), \(EqOddGap_{1}\) as well as \(EqOddGap_{0}\), and the lower the accuracy on the original decision attribute. For instance, with GB we obtain \(yAcc=0.83\pm 0.039\), \(DemoParity=0.035\pm 0.022\) at \(\alpha=0.9875\) and \(yAcc=0.8240\pm 0.0352\), \(DemoParity=0.0114\pm 0.0061\) for \(\alpha=0.9938\) (the original values were \(yAcc=0.86\) and \(DemoParity=0.16\)). We have also evaluated the metric using sanitized decisions instead of the original ones We observed that the results significantly improve, especially for equalized odds. More precisely, the accuracy of GB increases to \(yAcc=0.8703\pm 0.0589\) for \(\alpha=0.9938\), while the equalized odds varies from \(EqOddGap_{1}=0.1646\pm 0.0927\) and \(EqOddGap_{0}=0.0853\pm 0.0319\) (original decision) to \(EqOddGap_{1}=0.0243\pm 0.0201\) and \(EqOddGap_{0}=0.0084\pm 0.0075\) (sanitized decision). As explained in scenario _S2_, this suggests that correlations with the original decisions are not preserved by the sanitization process (DemoParity remains unchanged as they only involve the predicted decisions, which is independent of the ground truth). Our observations highlights the possibility that GANSan can be used locally, thus allowing users to contribute to large datasets by sanitizing and sharing their information without relying on any third party, with the guarantee that the sensitive attribute GANSan has been trained for is removed. The drop of accuracy due to the local sanitization is 3.68% on GB (8% with MLP). Thus, for applications requiring a time-consuming training phase, using GANSan to sanitize profiles without retraining the classifier seems to be a good compromise.