# Local Data Debiasing for Fairness Based on Generative Adversarial Training
## 4 Adversarial Training for Data Debiasing
### Generative Adversarial Network Sanitization
```
1:Inputs: \(D=\{A,Y,S\}\), MaxEpochs, \(d_{\mathit{iter}}\), batchSize, \(\alpha\)
2:Output: \(S_{\mathit{an}}\), \(D_{\mathit{isc}}\) \(\triangleright\) Initialization
3:\(S_{\mathit{an}}\), \(D_{\mathit{isc}}\), \(D_{\mathit{disc}}=\mathsf{shuffle}(D)\)
4:\(\mathsf{iterations}=\frac{[D]}{\mathsf{batchSize}}\)
5:for\(\in\{1,..,\mathsf{MaxEpochs}\}\)do
6:for\(\in\{1,..,\mathsf{Iterations}\}\)do
7: Sample batch B of size batchSize from \(D\)
8:\(S_{\mathit{B}}\): extract S column from \(B\)
9:\(\{A,Y\}=S_{\mathit{an}}(B)\)
10:\(e_{A_{i}}=\frac{1}{\mathsf{batchSize}}\cdot\sum_{n=1}^{\mathsf{batchSize}}|A_{ i}^{n}-A_{i}^{n}|\) \(\triangleright\) Compute the reconstruction loss vector
11:\(\widehat{J}^{\mathit{San}}=(1-\alpha)\cdot(e_{A_{1}},e_{A_{2}},e_{A_{3}},..,e_{A_{4}},e_{Y})^{T}\)\(\triangleright\) compute the sensitive loss
12:\(d_{S}=\alpha*(\frac{1}{2}-\mathsf{BER}(D_{\mathit{isc}}(S_{\mathit{an}}(B),S_{B}))\)\(\triangleright\)concatenate the previously computed loss
13:\(\widehat{J}^{\mathit{San}}=\mathsf{concat}(\widehat{J}^{\mathit{San}},d_{S})\)for\(loss\in\widehat{J}^{\mathit{San}}\)do \(\triangleright\) Back-propagation using loss
14: Backpropagate \(loss\)
15: Update \(S_{\mathit{an}}\) weights
16:endfor
17:for\(\in\{1,..,d_{\mathit{iter}}\}\)do
18: Sample batch B of size \(batchSize\) from \(D_{\mathit{disc}}\)
19:\(S_{\mathit{B}}\): extract S column from \(B\)
20:\(d_{\mathit{disc}}=\text{MSE}(S_{\mathit{B}},D_{\mathit{isc}}(S_{\mathit{an}}(B)))\)
21: Backpropagate \(Loss\)
22: Update \(D_{\mathit{isc}}\) weights
23:endfor
24: Save \(S_{\mathit{an}}\) and \(D_{\mathit{isc}}\) states
25:endfor
```  
**Algorithm 1** GANSan Training Procedure  
Training objective of GANSan.Let \(\bar{S}\) be the prediction of \(S\) by the discriminator (\(\bar{S}=D_{\mathit{isc}}(S_{\mathit{an}}(D))\)). Its objective is to accurately predict \(S\), thus it aims at minimizing the loss \(J^{D_{\mathit{isc}}}(S,\bar{S})=d_{\mathit{disc}}(S,\bar{S})\). In practice in our work, we instantiate \(d_{\mathit{disc}}\) as the Mean Squared Error (MSE). Given the hyperparameter \(\alpha\) representing the desired trade-off between the fairness and the fidelity, the sanitizer minimizes a loss combining two objectives:  
\[J^{S_{\mathit{an}}}(D,\,S_{\mathit{an}},\,D_{\mathit{isc}})=\alpha*d_{S}(S, \bar{S})+(1-\alpha)*(d_{r}(D,\,S_{\mathit{an}}(D))) \tag{4}\]  
in which \(d_{s}\) is \(\frac{1}{2}-BER(D_{\mathit{isc}}(A,Y),s)\) on the sensitive attribute. The term \(\frac{1}{2}\) is due to the objective of maximizing the error of the discriminator (i.e., recall that the optimal value of the BER is 0.5). Concerning the reconstruction loss \(d_{r}\), we have first tried the classical Mean Absolute Error (MAE) and MSE losses. However, our initial experiments have shown that these losses produce datasets that are highly problematic in the sense that the sanitizer always outputs the same profile whatever the input profile, which protects against attribute inference but renders the profile unusable. Therefore, we had to design a slightly more complex loss function. More precisely, we chose not to merge the respective losses of these attributes (\(e_{A_{i}}=(1-\alpha)*|A_{i}-\bar{A}_{i}|\); \(\bar{A}_{i}\in\bar{A},i\in[1,d]\)), yielding a vector of attribute losses whose components are iteratively used in the gradient descent. Hence, each node of the output layer of the generator is optimized to reconstruct a single attribute from the representation obtained from the intermediate layers. The vector formulation of the loss is as follows: \(\vec{j}^{San}=(e_{A_{1}},e_{A_{2}},e_{A_{3}},...,e_{A_{d}},e_{Y},\alpha*d_{s}(S,S))^ {T}\) and the objective is to minimize all its components. The details of the parameters used for the training are given in Section 5.1. We want to point out that the sanitization process does not necessarily require the decision attribute as an input variable, thus while in this paper we chose to explicitly include it the sanitization can be carried out with or without this decision attribute.