## 3 Methodology
### Extended W2 reg for Multi-Class Classification
#### 3.3.1 Reformulating the Bias Mitigation Procedure for Multi-Class Classification  
The strategy proposed by [7] to mitigate undesired biases is to train optimal decision rules \(f_{\theta}\) by optimising Equation (3), where the 2-Wasserstein distance between the prediction distributions \(\mu_{\theta,0}\) and \(\mu_{\theta,1}\) (i.e., the distribution of the predictions \(f_{\theta}\) for observations in groups \(S=0\) and \(S=1\)) is given by Equation (2). As described in Section 3.1, the predictions \(f_{\theta}(x_{i})\) are now a vector of dimension \(K>2\) in a multi-class classification context (specifically, \(f_{\theta}(x)\in[0,1]^{K}\)). Their distributions \(\mu_{\theta,0}\) and \(\mu_{\theta,1}\) are then multivariate. In this context, Equation (2) does not hold, and another optimal transport metric such as the multivariate 2-Wasserstein distance or the Sinkhorn Divergence should be used [35]. Note that different implementations of these metrics exist and are compatible with our problem, e.g. those of [36; 37]. This, however, opens a critical issue related to the number of observations needed to reasonably penalise the differences between two multivariate point clouds, representing the observations in groups \(S=0\) and \(S=1\). If the dimension \(K\) of the compared data becomes large, the number of observations required to reasonably compare the point clouds at each gradient descent step explodes. This problem is very similar to the well-known _curse of dimensionality_ phenomenon in machine learning, where the amount of data needed to accurately generalise the predictions grows exponentially as the number of dimensions grows. This issue, therefore, lead us to think about which problem we truly need to solve when tackling undesired algorithmic bias in multi-class classification. From our application perspective, discrimination appears when there the prediction model \(f_{\theta}\) is significantly more accurate at predicting a specific output in one of the two groups represented by \(S\in\{0,1\}\). For instance, suppose that someone looks for _Software Engineer_ jobs and that an automatic prediction model \(f_{\theta}\) is used to recommend job candidates to an employer. For a given job candidate \(x_{i}\), the prediction model will return a set of \(K\) probabilities, each of them indicating whether \(x_{i}\) is recommended for the job class \(k\). Now, \(k\) will denote the class of jobs \(x_{i}\) is looking for, i.e., _Software Engineer_. The prediction model will be considered as unfair if male profiles are on average clearly more often recommended by \(f_{\theta}\) than female profiles, when an unbiased oracle would lead to equal opportunities, i.e.,  
\[|P(\hat{Y}^{k}=1|Y^{k}=1,S=1)-P(\hat{Y}^{k}=1|Y^{k}=1,S=0)|>\tau, \tag{6}\]  
where the left-hand term denotes the _True Positive Rate gap_ (TPRg) and \(\tau\) is a threshold above which the TPRg is considered as unfairly discriminating. As shown in Section 5, such situations can occur in automatic job profile recommendation systems using modern neural networks. Now that we have clarified the problem we need to tackle, we can reformulate the regularised multi-class model training procedure as follows:  
* We first trained and tested a non-regularised multi-class classifier \(f_{\theta^{k}}\). We denote it the _baseline classifier_. * We defined a threshold \(\tau\) under which all occupations with predictions \(k\in\{1,\dots,K\}\) should have a TPRg (see Equation (6)). We denote \(\{c_{1},\dots,c_{C}\}\) the classes for which this condition is broken, where each of these classes takes its values in \(\{1,\dots,K\}\). * We then retrained the multi-class classifier \(f_{\theta}\) with regularisation constraints on the classes \(\{c_{1},\dots,c_{C}\}\) only. The regularisation strategy will be developed below in Section 3.3.3.