## 3 Methodology
### General Notations  
**Input and output observations:** Let \((x_{i},y_{i})_{i=1,\ldots,n}\) be the training observations, where \(x_{i}\in\mathbb{R}^{p}\) and \(y_{i}\in\{0,1\}^{K}\) are the input and output observations, respectively. The value \(p\) represents the inputs dimension or, equivalently, the number of input variables. It can for instance represent a number of pixels if \(x_{i}\) is an image or a number of words in a text if \(x_{i}\) is a word embedding. The value \(K\) represents the output dimensions. In a binary classification context, i.e., if \(K=1\), the fact that \(y_{i}=0\) or \(y_{i}=1\) specifies the class of the observation \(i\). In a multi-class classification context, i.e., if \(K>1\), a common strategy consists of using one-hot vectors to encode the class \(c\) of observation \(i\): all values \(y_{i}^{k}\), \(k\in\{1,\ldots,K\}\) are equal to \(0\), except the value \(y_{i}^{c}\), which is equal to \(1\). We use this convention all along this manuscript. **Prediction model:** A classifier \(f_{\theta}\) with parameters \(\theta\) is trained so that the predictions \(\hat{y_{i}}\in\{0,1\}^{K}\) it indirectly makes based on the outputs \(f_{\theta}(x_{i})\in[0,1]^{K}\) are _on average_ as close as possible to the true output observations \(y_{i}\) in the training set. The link between the model outputs \(f_{\theta}(x_{i})\) and the prediction \(\hat{y_{i}}\) depends on the classification context: In binary classification, \(f_{\theta}(x_{i})\) is the predicted probability that \(\hat{y_{i}}=1\), so it is common to use \(\hat{y_{i}}=1_{f_{\theta}(x_{i})>0.5}\). Now, by using one-hot-encoded output vectors in multi-class classification, an output \(f_{\theta}(x_{i})=\left(f_{\theta}^{1}(x_{i}),f_{\theta}^{2}(x_{i}),\ldots,f_{ \theta}^{K}(x_{i})\right)\) represents the predicted probabilities that the observation \(i\) is in the different classes \(k\in\{1,2,\ldots,K\}\). As a consequence, \(\sum_{k}f_{\theta}^{k}(x_{i})=1\). More interesting for us, the predicted class is the one having the highest probability, so \(\hat{y_{i}}\) is a vector of size \(K\) with null values everywhere, except at the index \(\arg\max_{k}f_{\theta}^{k}(x_{i})\), where its value is \(1\). **Loss and empirical risk:** In order to train the classification model, the empirical risk \(\mathcal{R}\) is minimised with respect to the model parameters \(\theta\):  
\[\mathcal{R}(\theta):=\mathbb{E}[\ell(\hat{Y}:=f_{\theta}(X),Y)]\,, \tag{1}\]  
or, empirically, \(R(\theta)=\frac{1}{n}\sum_{i=1}^{n}\ell(\hat{y_{i}}:=f_{\theta}(x_{i}),y_{i})\), where the loss function \(\ell\) represents the price paid for the inaccuracy of the predictions. This optimisation problem is almost systematically solved by using variants of stochastic (or mini-batch) gradient descent [33] in the machine learning literature. **Sensitive variable:** An important variable in the field of _fair learning_ is the so-called _sensitive variable_, which we denote \(S\). This variable is often binary and distinguishes two groups of observations \(S_{i}\in\{0,1\}\). For instance, \(S_{i}=0\) or \(S_{i}=1\) can indicate that the person represented in observation \(i\) is either a male or a female. A widely used strategy to quantify that a prediction model is fair with respect to the variable \(S\) is to compare the predictions it makes on observations in the groups \(S=0\) and \(S=1\), using a pertinent _fairness metric_ (see the references of Section 2). From a mathematical point of view, this means that the difference between the distributions \((X,Y,\hat{Y})_{S=0}\) and \((X,Y,\hat{Y})_{S=1}\), quantified by the fairness metric, should be below a given threshold. Consider for instance a binary prediction case where \(\hat{Y_{i}}=1\) means that the individual \(i\) has access to a bank loan, \(\hat{Y_{i}}=0\) means that the bank loan is refused, and that \(S_{i}\) equal to \(0\) or \(1\) refers to the fact that the individual \(i\) is a male or a female.