# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 5 Experimental Evaluation
### Experimental Results
#### 5.3.1 Effect of Mixed Data Composition  
In the local sanitization scenario, the user could possibly decide to sanitize their data or publish it unmodified. In this section, we assess the amount of protection and fairness obtained when some users decide not to sanitize their data, resulting in a dataset composed of original and sanitized data. In fact, some users might believe that the sanitization would reduce the advantage due to their group membership and would not sanitize their data in consequence. More precisely, we consider the following settings:  
* All. A random (i.e., regardless of their group membership) proportion of users did not use the sanitizer and instead submitted their profiles unmodified. * Prt. All users of the privileged group sanitized their respective profiles while some others from the protected group published their original profiles. * Prv. All members of the protected group deemed the application of the sanitization process useful while some users from the privileged group disregarded it. Thus, the dataset is composed of all sanitized profiles from the protected group and a mix of sanitized and original profiles from the privileged one. For these settings, we varied the proportion \(p\) (\(p\in\{25,50,75,100\}\)) of the original data composing each group (settings _Prt_ and _Prv_) or composing the dataset (setting _All_). For instance, in setting \(Prv\) with \(p=25\%\) (\(Prv\sim 25\%\)), the dataset is composed of all of the protected group sanitized profiles, and \(25\%\) of the privileged group data are unmodified. For each sanitized profile, we have carried out experiments using both the original (_Orig y_) and the sanitized decisions (_San y_). The resulting dataset (_mixed dataset_\(D_{mx}\)) is randomly split into a training and a testing set of size 70% and 30% of the total dataset. Furthermore, each experiment is repeated across the 10-fold cross-validation of the sanitization process. We computed the _agreement_ between a classifier trained on mixed data (\(C(D_{mx})\)) and the same classifier trained on the sanitized data (\(C(D_{sn})\)) to predict the decision: \(agr_{mx\sim sn}=Pr(C(D_{mx})=C(D_{sn}))\). We also computed the agreement obtained when training a classifier and predicting decisions using the mixed data and using the original one (\(agr_{mx\sim og}=Pr(C(D_{mx})=C(D_{og}))\)). In a nutshell, the _agreement_ quantifies how much a classifier behaves similarly in different contexts, by looking at the proportion of data points that received the same predicted decision across all contexts. A high \(agr_{mx\sim sn}\) indicates that the impact on the original data is limited, in which case the sanitization neither hinders the performance of the predictor nor disadvantages a particular group. As shown in Figure 9, agreements \(agr_{mx\sim sn}\) (second column of Figure 9) and \(agr_{mx\sim og}\) (first column of Figure 9) are above 85% for all proportions of mixed data, regardless of the decisions. More generally, we have observed that the use of original decisions (_Orig y_) results in a lower agreement than with sanitized ones (_San y_), because of the reduction of correlations between the data and the original decision. If original decisions are not transformed (by the sanitization) in relation to other attributes, they could still incorporate some form of unfairness (as observed with scenarios \(S1\) and \(S2\)). Sanitizing the privileged group data have the highest impact on both \(agr_{mx\sim gt}\) and \(agr_{mx\sim og}\) since it is the largest group of the dataset. This impact is also more pronounced with the original decisions. As a consequence, the highest agreement overall is achieved at \(Prt\)-25% with sanitized decisions (\(agr_{mx\sim gt}\) and \(San\)\(y\)). The agreement \(agr_{mx\sim gt}\) is the lowest at proportion _All_-75% when using the sanitized decisions and at proportion _Prv_-100% when using original ones.