# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Experimental Results
#### 5.3.1 Effect of Mixed Data Composition
These drops are explained by the fact that most original profiles have not been sanitized, as well as the reduction of correlations between the sanitized profiles and original decisions. In fact, the agreement \(agr_{mx\sim og}\) is maximal at those identical proportions. The high values of \(agr_{mx\sim sn}\) demonstrates that a classifier trained to predict decisions with the sanitized data and one trained on mixed data (as obtained with the local sanitization) behave similarly. Thus, we can expect both classifiers to achieve similar performances with respect to fairness metrics. We report in Figure 10 the accuracy of predicting the sensitive attribute (\(sAcc\)) on each group and each data version (_original_ and _sanitized_). As expected, the small proportion of the protected group causes the accuracy to increase with the proportion \(p\), especially when the profiles of the protected group are not sanitized _Prt-p_% or when then sanitization is applied on randomly selected profiles (_All-p_%). By looking at the classifier behavior on both the original and the sanitized parts of the mixed data, we can conclude that predicting the sensitive attribute could be viewed as two different operations: distinguishing the original from the sanitized profiles (which is easily performed as observed on proportions _Prt_-100% and _Prv_-100%) and distinguishing between the sanitized privileged and the sanitized protected profiles by leveraging on the additional information provided by the original profiles. We can also observe on Figure 10 that the impact of this additional information is highly dependent on the underlying  
Figure 10: Accuracy (mean) of Gradient Boosting (GB) trained to predict the sensitive attribute on Adult (mixed dataset). The accuracies on original data are both around 85% for the protected and the privileged group regardless of the decision type. On the sanitized data, it is around 68% for both groups using \(San\)\(y\) and around 71% using _Orig y_. Note that the standard deviation is less than 0.012. Figure 9: Agreement results (mean) of Gradient Boosting (GB) trained to predict the sanitized decision attribute on Adult Dataset. \(z\)\(\sim\)\(t\) (each column) refers to the agreement obtained when using GB, respectively, on data \(z\) and data \(t\), respectively: \(agr_{z\sim t}\). \(org\) denotes the original version while \(sun\) corresponds to sanitized profiles in the mixed data. Top: data (mixed, sanitized, or original) with original decisions, bottom: data with sanitized decisions (except the original dataset). Note that the standard deviation is less than 0.017.  
distribution. For the same proportion \(p\), keeping the original data from the privileged group (_Prv-p%_) has a limited impact on the accuracy compared to the increase due to the original data from the protected one (_Prt-p%_). Figure 11 displays the \(BER\) values obtained. Similary to the accuracy, the \(BER\) decreases with the increase of the proportion of original profiles. The higher this sampling proportion, the easier the prediction of the sensitive attribute becomes as it consists in distinguishing original from sanitized profiles (column _Miss_\(R\)_U.Vrs_ of Figure 11). From the miss prediction rate in each group (column _Miss_\(R\)_Gp_), we can see that the classifier tends to always predict the majority class when it cannot successfully distinguish between groups. Finally, we also observe the impact of the original decision, which contributes to the lowering of the BER values. We have also evaluated the positive rate (_Pos.Rate_), the true positives (\(Tp.Rate\)) and false positives (\(Fp.Rate\)) rates in each group (cf. Figure 12). The positive rate is used as the basis of the _demographic parity_ (_DemoParity_) computation, while the true positives and false positives rates are used to compute the gap in _equalized odds_\(EqOddGap_{y}\). The separate computation of these metrics allows us to observe the behavior of the classifier in each group.