# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Experimental Results
_General results on German._  
Similarly to Adult, the protection increases with \(\alpha\). More precisely \(\alpha=0\) (maximum reconstruction) achieves a fidelity of almost 0.96. The maximum protection of \(BER=0.5\) corresponds to a fidelity of 0.81 and a sensitive accuracy value of \(sAcc=0.76\). We can observe on Figure 5 that most values are concentrated on the \(sAcc=0.76\) plateau, regardless of the fidelity and the value of \(\alpha\). We believe this is due to the high disparity of the dataset. The fairness on German credit is initially quite high, being close to 0.33. Nonetheless, we can observe three interesting trade-offs on Figure 6, each located at a different shoulder of the Pareto front. These trade-offs are \(A\) (\(BER\approx 0.43,fid\approx 0.94\)), \(B\) (\(BER\approx 0.45,fid\approx 0.84\)) and \(C\) (\(BER\approx 0.5,fid\approx 0.81\)), each achievable with \(\alpha=0.6\) for the first one, and \(\alpha=0.9968\) for the rest. Figure 4: Boxplots of the quantitative analysis of sanitized datasets selected using \(HeuristicA\). These metrics are computed on the whole sanitized dataset. Modified records correspond to the proportion of records with categorical attributes affected by the sanitization. We review the diversity and the sanitization induced damage on categorical attributes in Figure 7. As expected, the diversity decreases with alpha, rendering most profiles identical with \(\alpha=1\). We can also observe some instabilities: higher \(\alpha\) values produce a shallow range of diversities (i.e., \(\alpha\geq 0.9\)) while smaller values have a higher range of diversities. Such instability is mainly explained by the size and the imbalance of the dataset, which does not allow the sanitizer to correctly learn the distribution (such phenomenon is common when training GANs with a small dataset). Nonetheless, most of the diversity results prove close to the original one, that is 0.51. The same trend is observed on the categorical attribute damage. For most values of \(\alpha\), the median damage is below or equal to 20%, meaning that we have to modify only two categorical columns in a record to remove unwanted correlations. For the numerical damage, most columns have a relative change lower than 0.5 for more than 70% of the dataset, regardless of the value of \(\alpha\). Only columns _Duration in month_ and _Credit amount_ have a higher damage. This is due to the fact that these columns have a very large range of possible values compared to the other columns (33 and 921), especially for column _Credit amount_ which also exhibits a nearly uniform distribution. Our reference points \(A\), \(B\) and \(C\) have a median damage close to 10% for \(A\) and 20% for both \(B\) and \(C\). The damage on categorical columns is also acceptable. To summarise our results, GANSan is able to maintain an important part of the dataset structure despite sanitization, making it usable for other analysis tasks. This is notably demonstrated by the lower damage and modifications, which preserve as much as possible of the original data values. Thus, results obtained on the sanitized dataset would therefore be close to those obtained on the original data, except on tasks involving the correlations with the sensitive attribute. Figure 5: Fidelity-fairness trade-off on German Credit. Each point represents the minimum possible \(sAcc\) of all the external classifiers. Figure 6: Fidelity-fairness trade-off on German Credit. Nonetheless, at the individual level, some perturbations might have a more fundamental impact on some profiles than on others. Future work will investigate the relationship between the characteristics of a profile and the damage introduced. For the different scenarios investigated hereafter, we fixed the values of \(\alpha\) to 0.9875 and 0.9938, which provides nearly a perfect level of sensitive attribute protection (respectively, 0.4897 and 0.4892) while leading to an acceptable damage on Adult (\(fid_{\alpha=0.9875}\approx 0.9464\) and \(fid_{\alpha=0.9938}\approx 0.9425\)).