# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Evaluation Scenarios  
Recall that GANSan  takes as input the whole original dataset (including the sensitive and the decision attributes) and outputs a sanitized dataset (_without_ the sensitive attribute) in the same space as the original one, but from which it is impossible to infer the sensitive attribute. In this context, the overall performance of GANSan  can be evaluated by analyzing the reachable space of points characterizing the trade-off between the fidelity \(fid\) to the original dataset and the fairness enhancement. More precisely, during our experimental evaluation, we will measure the fidelity between the original and the sanitized data, as well as the _diversity_, both in relation with the \(BER\) and \(sAcc\), computed on this dataset. However, in practice, our approach can be used in several situations that differ slightly from one another. In the following, we detail four scenarios that we believe are representing most of the possible use cases of GANSan. To ease the understanding, we will use the following notation: the subscript \(tr\) (respectively \(ts\)) will denote the data in the training set (respectively test set). For instance, \(\{A\}_{tr}\), \(\{Y\}_{tr}\), \(\{\bar{A}\}_{tr}\) or \(\{\bar{Y}\}_{tr}\) represent, respectively, the attributes of the original training set (not including the sensitive and the decision attributes), the decision in the original training set, the attributes of the sanitized training set and the decision attribute in the sanitized training set. Table 4 describes the composition of the training and the testings sets for these four scenarios. Scenario 1: complete data debiasingThis setting corresponds to the typical use of the sanitized dataset, which is the prediction of a decision attribute through a classifier. \begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Scenario**} & \multicolumn{2}{c}{**Train Set Composition**} & \multicolumn{2}{c}{**Test Set Composition**} \\ \cline{2-5}  & \(A\) & \(Y\) & \(A\) & \(Y\) \\ \hline _Baseline_ & Original & Original & Original & Original \\ _Scenario 1_ & Sanitized & Sanitized & Sanitized & Sanitized \\ _Scenario 2_ & Sanitized & Original & Sanitized & Original \\ _Scenario 3_ & Sanitized & Sanitized & Original & Original \\ _Scenario 4_ & Original & Original & Sanitized & Original \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Scenarios envisioned for the evaluation of GANSan. Each set is composed of either the original attributes or their sanitized versions, coupled with either the original or sanitized decision. The decision attribute is also sanitized as we assumed that the original decision holds information about the sensitive attribute. Here, we quantify the accuracy of prediction of \(\{\bar{Y}\}_{ts}\) as well as the discrimination represented by the _demographic parity_ (Equation (1)) and _equalized odds_ (Equation (2)). _Scenario 2: partial data debiasing._  
In this scenario, similarly to the previous one, the training and the test sets are sanitized with the exception that the sanitized decision in both these datasets \(\{\bar{A},\bar{Y}\}\) is replaced with the original one \(\{\bar{A},Y\}\). This scenario is generally the one considered in the majority of papers on fairness enhancement [24; 26; 34], the accuracy loss in the prediction of the original decision \(\{Y\}_{ts}\) between this classifier and another trained on the original dataset without modifications \(\{A\}_{tr}\) is a straightforward way to quantify the utility loss due to the sanitization. _Scenario 3: building a fair classifier._ This scenario was considered in [23] and is motivated by the fact that the sanitized dataset might introduce some undesired perturbations (e.g., changing the education level from Bachelor to PhD). Thus, a third party might build a fair classifier but still apply it directly on the unperturbed data to avoid the data sanitization process and the associated risks.