Original: U NFAIR
Revised: U NFAIR

Original: GEOMETRIES : EXACTLY SOLVABLE DATA MODEL WITH FAIRNESS IMPLICATIONS
Revised: GEOMETRIES : EXACTLY SOLVABLE DATA MODEL WITH FAIRNESS IMPLICATIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation.
Revised: Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation.

Original: Marginalisation and iniquitous group representation are often traceable in the very data used for training, and may be reflected or even enhanced by the learning models.
Revised: Marginalisation and iniquitous group representation are often traceable in the very data used for training, and may be reflected or even enhanced by the learning models.

Original: In the present work, we aim at clarifying the role played by data geometry in the emergence of ML bias.
Revised: In the present work, we aim at clarifying the role played by data geometry in the emergence of ML bias.

Original: We introduce an exactly solvable high-dimensional model of data imbalance, where parametric control over the many bias-inducing factors allows for an extensive exploration of the bias inheritance mechanism.
Revised: We introduce an exactly solvable high-dimensional model of data imbalance, where parametric control over the many bias-inducing factors allows for an extensive exploration of the bias inheritance mechanism.

Original: Through the tools of statistical physics, we analytically characterise the typical properties of learning models trained in this synthetic framework and obtain exact predictions for the observables that are commonly employed for fairness assessment.
Revised: Through the tools of statistical physics, we analytically characterise the typical properties of learning models trained in this synthetic framework and obtain exact predictions for the observables that are commonly employed for fairness assessment.

Original: Despite the simplicity of the data model, we retrace and unpack typical unfairness behaviour observed on real-world datasets.
Revised: Despite the simplicity of the data model, we retrace and unpack typical unfairness behaviour observed on real-world datasets.

Original: We also obtain a detailed analytical characterisation of a class of bias mitigation strategies.
Revised: We also obtain a detailed analytical characterisation of a class of bias mitigation strategies.

Original: We first consider a basic loss-reweighing scheme, which allows for an implicit minimisation of different unfairness metrics, and quantify the incompatibilities between some existing fairness criteria.
Revised: We first consider a basic loss-reweighing scheme, which allows for an implicit minimisation of different unfairness metrics, and quantify the incompatibilities between some existing fairness criteria.

Original: Then, we consider a novel mitigation strategy based on a matched inference approach, consisting in the introduction of coupled learning models.
Revised: Then, we consider a novel mitigation strategy based on a matched inference approach, consisting in the introduction of coupled learning models.

Original: Our theoretical analysis of this approach shows that the coupled strategy can strike superior fairness-accuracy trade-offs.
Revised: Our theoretical analysis of this approach shows that the coupled strategy can strike superior fairness-accuracy trade-offs.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Machine Learning (ML) systems are actively being integrated in multiple aspects of our lives, from face recognition systems on our phones, to applications in the fashion industry, to high stake scenarios like healthcare.
Revised: Machine Learning (ML) systems are actively being integrated in multiple aspects of our lives, from face recognition systems on our phones, to applications in the fashion industry, to high stake scenarios like healthcare.

Original: Together with the advantages of automatising these processes, however, we must also face the consequences of their — often hidden — failures.
Revised: Together with the advantages of automatising these processes, however, we must also face the consequences of their — often hidden — failures.

Original: Recent studies Buolamwini & Gebru (2018); Weidinger et al.
Revised: Recent studies Buolamwini & Gebru (2018); Weidinger et al.

Original: (2021) have shown that these systems may have significant disparity in failure rates across the multiple sub-populations targeted in the application.
Revised: (2021) have shown that these systems may have significant disparity in failure rates across the multiple sub-populations targeted in the application.

Original: ML systems appear to perpetuate discriminatory behaviours that align with those present in our society Benjamin (2019);
Revised: ML systems appear to perpetuate discriminatory behaviours that align with those present in our society Benjamin (2019);

Original: Noble (2018); Eubanks (2018); Broussard (2018).
Revised: Noble (2018); Eubanks (2018); Broussard (2018).

Original: Discrimination over marginalised groups could originate at many levels in the ML pipeline, from the very problem definition, to data collection, to the training and deployment of the ML algorithm Suresh & Guttag (2021).
Revised: Discrimination over marginalised groups could originate at many levels in the ML pipeline, from the very problem definition, to data collection, to the training and deployment of the ML algorithm Suresh & Guttag (2021).

Original: Data represents a critical source of bias Perez (2019).
Revised: Data represents a critical source of bias Perez (2019).

Original: In some cases, the dataset can contain a record of a history of discriminatory behaviour, causing complex dependencies that are hardly eradicated even when the explicit discriminatory attribute is removed.
Revised: In some cases, the dataset can contain a record of a history of discriminatory behaviour, causing complex dependencies that are hardly eradicated even when the explicit discriminatory attribute is removed.

Original: In other cases (or even concurrently), the root of the discrimination can be found in the data collection process, and is related to the structural properties of the dataset.
Revised: In other cases (or even concurrently), the root of the discrimination can be found in the data collection process, and is related to the structural properties of the dataset.

Original: Heterogeneous representations of different sub-populations typically induce major bias in the ML predictions.
Revised: Heterogeneous representations of different sub-populations typically induce major bias in the ML predictions.

Original: Drug testing provides a historically significant example: substantial evidence Hughes (2007); Perez (2019) shows that the scarcity of data points corresponding to women individuals in drug-efficiency studies resulted in a larger number of side effects in their group.
Revised: Drug testing provides a historically significant example: substantial evidence Hughes (2007); Perez (2019) shows that the scarcity of data points corresponding to women individuals in drug-efficiency studies resulted in a larger number of side effects in their group.

Original: In spite of a vast empirical literature, a large gap remains in the theoretical understanding of the bias-induction mechanism.
Revised: In spite of a vast empirical literature, a large gap remains in the theoretical understanding of the bias-induction mechanism.

Original: A better theoretical grasp of this issue could help raise awareness and design more theoretically grounded and effective solutions.
Revised: A better theoretical grasp of this issue could help raise awareness and design more theoretically grounded and effective solutions.

Original: In this work, we aim to address this gap by introducing a novel synthetic data model, offering a controlled setting where data imbalances and the emergence of bias become more transparent and can be better understood.
Revised: In this work, we aim to address this gap by introducing a novel synthetic data model, offering a controlled setting where data imbalances and the emergence of bias become more transparent and can be better understood.

Original: To the best of our knowledge, the present study constitutes the first attempt to explore and exactly characterise by analytical means the complex phenomenology of ML fairness.
Revised: To the best of our knowledge, the present study constitutes the first attempt to explore and exactly characterise by analytical means the complex phenomenology of ML fairness.

Original: Summary of main results.
Revised: Summary of main results.

Original: We devise a novel synthetic model of data, the Teacher-Mixture (T-M), to obtain a theoretical analysis of the bias-induction mechanism.
Revised: We devise a novel synthetic model of data, the Teacher-Mixture (T-M), to obtain a theoretical analysis of the bias-induction mechanism.

Original: The geometrical properties of the model are motivated by common observations on the data structure in realistic datasets, concerning the coexistence of non-trivial correlations at the level of the inputs and between inputs and labels (some empirical observations can be found in appendix ?? ).
Revised: The geometrical properties of the model are motivated by common observations on the data structure in realistic datasets, concerning the coexistence of non-trivial correlations at the level of the inputs and between inputs and labels (some empirical observations can be found in appendix B).

Original: In particular, we focus on the role played by the presence of different sub-populations in the data, both from the point of view of the input distribution and from that of the labelling rule.
Revised: In particular, we focus on the role played by the presence of different sub-populations in the data, both from the point of view of the input distribution and from that of the labelling rule.

Original: Surprisingly, this simple structural feature is sufficient for producing a rich and realistic ML fairness phenomenology.
Revised: Surprisingly, this simple structural feature is sufficient for producing a rich and realistic ML fairness phenomenology.

Original: The parameters of the T-M can be tuned to emulate disparate learning regimes, allowing for an exploration of the impact of each bias-inducing factor and for an assessment of the effectiveness of a tractable class of mitigation strategies.
Revised: The parameters of the T-M can be tuned to emulate disparate learning regimes, allowing for an exploration of the impact of each bias-inducing factor and for an assessment of the effectiveness of a tractable class of mitigation strategies.

Original: In summary, in the present work we:
Revised: In summary, in the present work we:

Original: • Derive, through a statistical physics approach, an analytical characterisation of the typical performance of solutions of the T-M problem in the high-dimensional limit.
Revised: • Derive, through a statistical physics approach, an analytical characterisation of the typical performance of solutions of the T-M problem in the high-dimensional limit.

Original: The obtained learning curves are found to be in perfect agreement with numerical simulations in the same synthetic settings (as shown in the central panel in Fig. 1), and produce unfairness behaviours that are closely reminiscent of the results seen on real data.
Revised: The obtained learning curves are found to be in perfect agreement with numerical simulations in the same synthetic settings (as shown in the central panel in Fig. 1), and produce unfairness behaviours that are closely reminiscent of the results seen on real data.

Original: • Isolate the different sources of bias (shown in the left panel of Fig. 1) and evaluate their interplay in the bias-induction mechanism.
Revised: • Isolate the different sources of bias (shown in the left panel of Fig. 1) and evaluate their interplay in the bias-induction mechanism.

Original: This analysis also allows us to highlight how unfairness can emerge in settings where the data distribution is apparently balanced.
Revised: This analysis also allows us to highlight how unfairness can emerge in settings where the data distribution is apparently balanced.

Original: • Trace a positive transfer effect between the different sub-populations, which implies that, despite their distinctions, an overall similarity can be exploited for achieving better performance on each group.
Revised: • Trace a positive transfer effect between the different sub-populations, which implies that, despite their distinctions, an overall similarity can be exploited for achieving better performance on each group.

Original: • Analyse the trade-offs between the different definitions of fairness, by studying the effects of a sample reweighing mitigation strategy, which can be encompassed in the theoretical framework proposed in this work and thus characterised analytically.
Revised: • Analyse the trade-offs between the different definitions of fairness, by studying the effects of a sample reweighing mitigation strategy, which can be encompassed in the theoretical framework proposed in this work and thus characterised analytically.

Original: • Propose 
Revised: • Propose a model-matched mitigation strategy

Original: a model-matched mitigation strategy, where two coupled networks are simultaneously trained and can specialise on different sub-populations while mutually transferring useful information.
Revised: where two coupled networks are simultaneously trained and can specialise on different sub-populations while mutually transferring useful information.

Original: We analytically characterise its effectiveness, finding that with this method, in the T-M, the competition between accuracy and different fairness metrics becomes negligible.
Revised: We analytically characterise its effectiveness, finding that with this method, in the T-M, the competition between accuracy and different fairness metrics becomes negligible.

Original: Preliminary positive results are also reported on real data.
Revised: Preliminary positive results are also reported on real data

Original: Further related works.
Revised: Further related works.

Original: In the past decade, algorithmic fairness has been receiving growing attention, spurred by the increasing number of ML applications in highly consequential social and economic areas Datta et al.
Revised: In the past decade, algorithmic fairness has been receiving growing attention, spurred by the increasing number of ML applications in highly consequential social and economic areas Datta et al.

Original: ; Metz & Satariano (2020); Angwin et al. (2016).
Revised: ; Metz & Satariano (2020); Angwin et al. (2016).

Original: A central question in the field is on the proper mathematical definition of bias: the plethora of alternative fairness criteria includes measures of group fairness , e.g. statistical parity Corbett-Davies et al.
Revised: A central question in the field is on the proper mathematical definition of bias: the plethora of alternative fairness criteria includes measures of group fairness , e.g. statistical parity Corbett-Davies et al.

Original: (2017); Dwork et al.
Revised: (2017); Dwork et al.

Original: (2012); Kleinberg et al.
Revised: (2012); Kleinberg et al.

Original: (2016), disparate impact Calders & Verwer (2010);
Revised: (2016), disparate impact Calders & Verwer (2010);

Original: Feldman et al.
Revised: Feldman et al.

Original: ; Zafar et al. (2017b); Chouldechova (2017), equality of opportunity Hardt et al.
Revised: ; Zafar et al. (2017b); Chouldechova (2017), equality of opportunity Hardt et al.

Original: (2016), calibration within groups Kleinberg et al.
Revised: (2016), calibration within groups Kleinberg et al.

Original: , disparate mistreatment Zafar et al.
Revised: , disparate mistreatment Zafar et al.

Original: (2017a), as well as measures of individual fairness Speicher et al.
Revised: (2017a), as well as measures of individual fairness Speicher et al.

Original: ; Castelnovo et al.
Revised: ; Castelnovo et al.

Original: We focus on group fairness in the following, since it is well-defined also in the high-dimensional limit considered in our theoretical framework.
Revised: We focus on group fairness in the following, since it is well-defined also in the high-dimensional limit considered in our theoretical framework.

Original: Recent works have highlighted incompatibilities between some of these fairness measures Kleinberg et al.
Revised: Recent works have highlighted incompatibilities between some of these fairness measures Kleinberg et al.

Original: ; Corbett-Davies & Goel (2018); Barocas et al.
Revised: ; Corbett-Davies & Goel (2018); Barocas et al.

Original: (2019), e.g. calibration and error disparity Pleiss et al.
Revised: (2019), e.g. calibration and error disparity Pleiss et al.

Original: (2017), and their instability with respect to fluctuations in the training dataset Friedler et al.
Revised: (2017), and their instability with respect to fluctuations in the training dataset Friedler et al.

Original: ; Castelnovo et al.
Revised: ; Castelnovo et al.

Original: Our work is the first to allow an exact quantification of the intrinsic trade-offs between these notions of group-fairness.
Revised: Our work is the first to allow an exact quantification of the intrinsic trade-offs between these notions of group-fairness.

Original: A second major topic in the field of algorithmic fairness is that of bias mitigation.
Revised: A second major topic in the field of algorithmic fairness is that of bias mitigation.

Original: In this work, we focus on in-processing strategies Arrieta et al.
Revised: In this work, we focus on in-processing strategies Arrieta et al.

Original: (2020), where the training process is altered in order to include fairness as a secondary optimisation objective for the learning model.
Revised: (2020), where the training process is altered in order to include fairness as a secondary optimisation objective for the learning model.

Original: These methods range from including ad hoc regularisation terms to the loss function Kamishima et al.
Revised: These methods range from including ad hoc regularisation terms to the loss function Kamishima et al.

Original: ; Huang &
Revised: ; Huang &

Original: Vishnoi (2019), to formulating fair classification as a constrained optimisation problem and deriving reduction-based algorithms Agarwal et al.
Revised: Vishnoi (2019), to formulating fair classification as a constrained optimisation problem and deriving reduction-based algorithms Agarwal et al.

Original: (2018; 2019); Celis et al.
Revised: (2018; 2019); Celis et al.

Original: Other possible strategies include adversarial training Zhang et al.
Revised: Other possible strategies include adversarial training Zhang et al.

Original: (2018), where a fairness-arbiter model can drive learning towards a sough fairness criterion, and distributionally robust optimisation Słowik & Bottou (2021),
Revised: (2018), where a fairness-arbiter model can drive learning towards a sough fairness criterion, and distributionally robust optimisation Słowik & Bottou (2021),

Original: [Figure]
Revised: [Figure]

Original: where one accounts for worst-case unfairness scenarios the
Revised: where one accounts for worst-case unfairness scenarios the

Original: -populations data
Revised: -populations data

Original: In this work we analyse two simple schemes whose performance can be analytically traced in our framework.
Revised: In this work we analyse two simple schemes whose performance can be analytically traced in our framework.

Original: First, an approach Kamiran & Calders (2012); Plecko & Meinshausen (2020); Lum & Johndrow (2016) based on loss-reweighing according to the associated subgroup and label of each data point.
Revised: First, an approach Kamiran & Calders (2012); Plecko & Meinshausen (2020); Lum & Johndrow (2016) based on loss-reweighing according to the associated subgroup and label of each data point.

Original: Second, we propose –and analyse– a novel method based on the introduction of coupled learning models, which can be interpreted as a modification of the ”two naive Bayes” model in Calders & Verwer (2010).
Revised: Second, we propose –and analyse– a novel method based on the introduction of coupled learning models, which can be interpreted as a modification of the ”two naive Bayes” model in Calders & Verwer (2010).

Original: Alternative classes of debiasing approaches, which cannot be analysed within our framework, include pre-processing strategies Calmon et al.
Revised: Alternative classes of debiasing approaches, which cannot be analysed within our framework, include pre-processing strategies Calmon et al.

Original: ; Feldman et al.
Revised: ; Feldman et al.

Original: (2015), learning unbiased representations Zemel et al.
Revised: (2015), learning unbiased representations Zemel et al.

Original: (2013), and post-processing techniques based on Decision Theory and Causal
Revised: (2013), and post-processing techniques based on Decision Theory and Causal

Original: Reasoning Kamiran et al.
Revised: Reasoning Kamiran et al.

Original: ; Plecko & Meinshausen (2020).
Revised: ; Plecko & Meinshausen (2020).

Original:  M ODELLING D ATA I MBALANCE
Revised:  M ODELLING D ATA I MBALANCE

Original: The Teacher-Mixture model, sketched in Fig.
Revised: The Teacher-Mixture model, sketched in Fig.

Original: 1, combines aspects of two common modelling frameworks for supervised learning, namely the Gaussian-Mixture (GM) and the Teacher-Student (TS) setups.
Revised: 1, combines aspects of two common modelling frameworks for supervised learning, namely the Gaussian-Mixture (GM) and the Teacher-Student (TS) setups.

Original: The GM is a simple model of clustered input data, where each data point is sampled from one of a small set of –possibly overlapping– high-dimensional Gaussian distributions, while the
Revised: The GM is a simple model of clustered input data, where each data point is sampled from one of a small set of –possibly overlapping– high-dimensional Gaussian distributions, while the

Original: TS provides a simple model of input-label correlation, where the ground-truth labels are obtained from a random ”teacher” neural network and the ”student” learning model tries to reproduce similar outputs.
Revised: TS provides a simple model of input-label correlation, where the ground-truth labels are obtained from a random ”teacher” neural network and the ”student” learning model tries to reproduce similar outputs.

Original: While retaining analytical tractability, the novel T-M data model allows for a richer phenomenology than the previous models, retracing the main features of real data with multiple coexisting sub-populations.
Revised: While retaining analytical tractability, the novel T-M data model allows for a richer phenomenology than the previous models, retracing the main features of real data with multiple coexisting sub-populations.

Original: For simplicity, the results discussed in this paper will focus on the case of two groups, but the analysis could be extended to multiple sub-populations.
Revised: For simplicity, the results discussed in this paper will focus on the case of two groups, but the analysis could be extended to multiple sub-populations.

Original: More formally, we consider a synthetic dataset of n samples D = { x µ , y µ } nµ =1 , with x µ ∈ R d , y µ ∈ { 0 , 1 } .
Revised: More formally, we consider a synthetic dataset of n samples D = { x µ , y µ } nµ =1 , with x µ ∈ R d , y µ ∈ { 0 , 1 } .

Original: We define the O (1) ratio α = n/d
Revised: We define the O (1) ratio α = n/d

Original: and we refer to it as the data scarsity parameter.
Revised: and we refer to it as the data scarsity parameter.

Original: Each input vector is i.i.d.
Revised: Each input vector is i.i.d.

Original: sampled from a mixture of two symmetric Gaussians with variances ∆ = { ∆ + , ∆ − } , x ∼ N ( ± v / √ d, ∆ ± I d × d ) , with respective probabilities ρ and (1 − ρ ) .
Revised: sampled from a mixture of two symmetric Gaussians with variances ∆ = { ∆ + , ∆ − } , x ∼ N ( ± v / √ d, ∆ ± I d × d ) , with respective probabilities ρ and (1 − ρ ) .

Original: The shift vector v is a Gaussian vector with i.i.d.
Revised: The shift vector v is a Gaussian vector with i.i.d.

Original: entries with zero mean and variance 1 .
Revised: entries with zero mean and variance 1 .

Original: The 1 / √ d scaling corresponds to the high-noise noise regime, where the two Gaussian clouds are overlapping and hard to disentangle Mignacco et al.
Revised: The 1 / √ d scaling corresponds to the high-noise noise regime, where the two Gaussian clouds are overlapping and hard to disentangle Mignacco et al.

Original: ; Saglietti & Zdeborov ´ a (2022), e.g. as in the case of
Revised: ; Saglietti & Zdeborov ´ a (2022), e.g. as in the case of

Original: CelebA and MEPS shown in appendix ?? . The ground-truth labels, instead, are provided by two i.i.d.
Revised: CelebA and MEPS shown in appendix C. The ground-truth labels, instead, are provided by two i.i.d.

Original: Gaussian teacher vectors, namely W + T and W − T , with components of zero mean and variance 1.
Revised: Gaussian teacher vectors, namely W + T and W − T , with components of zero mean and variance 1.

Original: Each teacher produces labels for the inputs with the corresponding group-membership, namely y µ = sign (cid:0) W ± T · x µ ± + b ± T (cid:1) .
Revised: Each teacher produces labels for the inputs with the corresponding group-membership, namely y µ = sign (cid:0) W ± T · x µ ± + b ± T (cid:1) .

Original: The thresholds b ± T correspond to the teacher bias terms, included in the model to control the fraction of positive and negative samples within the two sub-populations.
Revised: The thresholds b ± T correspond to the teacher bias terms, included in the model to control the fraction of positive and negative samples within the two sub-populations.

Original: Overall, the geometric picture of the data distribution (a sketch in Fig. 1) is summarised by the following overlaps:
Revised: Overall, the geometric picture of the data distribution (a sketch in Fig. 1) is summarised by the following overlaps:

Original: [Equation]
Revised: [Equation]

Original: that respectively quantify the alignment of the teacher decision boundaries with respect to the shift vector, controlling the group-label correlation, and the overlap between the teacher vectors, controlling the correlation between labels assigned to similar inputs belonging to different communities.
Revised: that respectively quantify the alignment of the teacher decision boundaries with respect to the shift vector, controlling the group-label correlation, and the overlap between the teacher vectors, controlling the correlation between labels assigned to similar inputs belonging to different communities.

Original: Given the synthetic dataset D , we study the properties of a single-layer network  trained via empirical risk minimization (ERM) of the loss:
Revised: Given the synthetic dataset D , we study the properties of a single-layer network W trained via empirical risk minimization (ERM) of the loss:

Original: [Equation]
Revised: [Equation]

Original: where ℓ ( y, ˆ y ) is assumed to be convex, λ is an external parameter that regulates the intensity of the
Revised: where ℓ ( y, ˆ y ) is assumed to be convex, λ is an external parameter that regulates the intensity of the

Original: L 2 regularisation, and the index c µ ∈ { + , −} denotes the group membership of data point µ .
Revised: L 2 regularisation, and the index c µ ∈ { + , −} denotes the group membership of data point µ .

Original: In this work, we derive a theoretical characterisation of the asymptotics of this learning model and consider the possible implications from a ML fairness perspective.
Revised: Given this framework, we derive a theoretical characterisation of the asymptotics of this learning model and consider the possible implications from a ML fairness perspective.

Original: In particular, we aim at studying the role of data geometry and cardinality in the training of a fair classifier.
Revised: In particular, we aim at studying the role of data geometry and cardinality in the training of a fair classifier.

Revised: To quantify the level of bias in the predictions of the trained model, we need to choose a metric of fairness.

Revised: We will diffusely employ disparate impact (DI) Feldman et al.

Original: Note that the T-M has, at the same time, the advantage of being simple, allowing better understanding of the many facets ofML bias, and the disadvantage ofbeing simple, since some modelling assumptions might not reflect the complexity of real-world data.
Revised: (2015), a ML analogous of the 80% rule Commission et al. (1979), which allows a simple assessment of the over-specialisation of the classifier on one of the sub-populations. In principle, in the T-M there is no preferable realisation of the target attribute so we can adopt a symmetric version of DI = p (ˆ y =

Revised: y | +) /p

Revised: (ˆ y = y |− ) , defined as the ratio between test accuracy in sub-population + and sub-population − .

Revised: How to measure bias is itself an active line of research and the DI measure is imperfect.

Revised: In Sec.

Revised: 4 we compare with other metrics.

Revised: Note that the T-M has, at the same time, the advantage of being simple, allowing better understanding of the many facets of ML bias, and the disadvantage of being simple, since some modelling assumptions might not reflect the complexity of real-world data.

Original: For example, we ignore any type of correlation among the inputs other than the clustering structure.
Revised: For example, we ignore any type of correlation among the inputs other than the clustering structure.

Original: The goal of this modelling work continues a long tradition of research in statistical physics, which has shown that theoretical insights gained in prototypical settings can often be helpful to disentangle and interpret the complexity of real world behaviour.
Revised: The goal of this modelling work continues a long tradition of research in statistical physics, which has shown that theoretical insights gained in prototypical settings can often be helpful to disentangle and interpret the complexity of real world behaviour.

Original: Remark 

Revised: Remark

Original: By looking at the available degrees of freedom in the T-M, several possible sources of bias naturally emerge from the model:
Revised: By looking at the available degrees of freedom in the T-M, several possible sources of bias naturally emerge from the model:

Original: • the relative representation , ρ = n + / ( n + + n − ) , with n c the number of points in group c .
Revised: • the relative representation , ρ = n + / ( n + + n − ) , with n c the number of points in group c .

Original: • the group variance , ∆ c , determining the width of the clusters.
Revised: • the group variance , ∆ c , determining the width of the clusters.

Original: • the label frequencies , controlled through the bias terms b c .
Revised: • the label frequencies , controlled through the bias terms b c .

Original: • the group-label correlation , m c .
Revised: • the group-label correlation , m c .

Original: • the labelling rule similarity , q T , which measures the alignment between the two teachers, i
Revised: • the labelling rule similarity , q T , which measures the alignment between the two teachers, i

Original: e. the linear discriminators that assign the labels to the two groups of inputs.
Revised: e. the linear discriminators that assign the labels to the two groups of inputs.

Original: • the data scarcity , α , representing the ratio between dataset size and input dimension.
Revised: • the data scarcity , α , representing the ratio between dataset size and input dimension.

Original: Theoretical analysis in high-dimensions.
Revised: Theoretical analysis in high-dimensions.

Original: In principle, solving Eq. 2 requires finding the minimiser of a complex non-linear, high-dimensional, quenched random function.
Revised: In principle, solving Eq. 2 requires finding the minimiser of a complex non-linear, high-dimensional, quenched random function.

Original: Fortunately, statistical physics
Revised: Fortunately, statistical physics

Original: M´ezard et al.
Revised: M´ezard et al.

Original: (1987) showed that in the limit n, d → ∞ , n/d = α , a large class of problems, including the T-M model, becomes analytically tractable.
Revised: (1987) showed that in the limit n, d → ∞ , n/d = α , a large class of problems, including the T-M model, becomes analytically tractable.

Original: In fact, in this proportional high-dimensional regime, the behaviour of the learning model becomes deterministic and trackable due to the strong concentration properties of a narrow set of descriptors that specify the relevant geometrical properties of the ERM estimator.
Revised: In fact, in this proportional high-dimensional regime, the behaviour of the learning model becomes deterministic and trackable due to the strong concentration properties of a narrow set of descriptors that specify the relevant geometrical properties of the ERM estimator.

Original: The original high-dimensional learning problem can be reduced to a simple system of equations that depends on a set of scalar overlaps:
Revised: The original high-dimensional learning problem can be reduced to a simple system of equations that depends on a set of scalar overlaps:

Original: [Equation]
Revised: [Equation]

Original: representing the typical norm of the trained estimator, its magnetisation in the direction of the cluster 
Revised: representing the typical norm of the trained estimator, its magnetisation in the direction of the cluster centres, and its overlap with the two teachers of the T-M.

Original: Analytical result 

Revised: Analytical result

Original: In the high dimensional limit when n , d → ∞ at a fixed ratio α = n/d , the scalar descriptors Θ = { Q, m, R ± , δq } of the vector w obtained by the empirical risk minimisation of Eq.
Revised: In the high dimensional limit when n , d → ∞ at a fixed ratio α = n/d , the scalar descriptors Θ = { Q, m, R ± , δq } of the vector w obtained by the empirical risk minimisation of Eq.

Original: 2 with a convex loss, and their Lagrange multipliers ˆΘ = { ˆ Q, ˆ m, ˆ R ± , δ ˆ q } , converge to deterministic quantities given by the unique fixed point of the system:
Revised: 2 with a convex loss, and their Lagrange multipliers ˆΘ = { ˆ Q, ˆ m, ˆ R ± , δ ˆ q } , converge to deterministic quantities given by the unique fixed point of the system:

Original: [Equation]
Revised: [Equation]

Original: with:
Revised: with:

Original: [Equation]
Revised: [Equation]

Original: where c ∈ { + , −} ∼ Bernoulli( ρ ) , z ∼ N (0 , 1) , H ( · ) = 12 erfc( · / √ 2) is the Gaussian tail function, w is the solution of:
Revised: where c ∈ { + , −} ∼ Bernoulli( ρ ) , z ∼ N (0 , 1) , H ( · ) = 12 erfc( · / √ 2) is the Gaussian tail function, w is the solution of:

Original: [Equation]
Revised: [Equation]

Original: and the bias b implicitly solves the equation ∂ b e (Θ; ∆)
Revised: and the bias b implicitly solves the equation ∂ b e (Θ; ∆)

Original: = 0 .
Revised: = 0 .

Original: Note that this result was obtained through the non rigorous yet exact replica method from statistical physics M ´ ezard et al.
Revised: Note that this result was obtained through the non rigorous yet exact replica method from statistical physics M ´ ezard et al.

Original: (1987); Engel & Van den Broeck (2001); Zdeborov ´ a & Krzakala (2016).
Revised: (1987); Engel & Van den Broeck (2001); Zdeborov ´ a & Krzakala (2016).

Original: The derivation details are deferred to appendix ?? . We remark that several analytic results obtained through the replica method have been subsequently proved rigorously.
Revised: The derivation details are deferred to appendix D. We remark that several analytic results obtained through the replica method have been subsequently proved rigorously.

Original: In particular, the proofs presented by Thrampoulidis et al.
Revised: In particular, the proofs presented by Thrampoulidis et al.

Original: ; Mignacco et al.
Revised: ; Mignacco et al.

Original: ; Loureiro et al. (2021) in settings similar to the present one suggest that an extension for the T-M case could be derived.
Revised: ; Loureiro et al. (2021) in settings similar to the present one suggest that an extension for the T-M case could be derived.

Original: However, this is left for future work.
Revised: However, this is left for future work.

Original: In this manuscript, we verify the validity of our theory by comparison with numerical simulations, as shown e.g. in the central panel of Fig.
Revised: In this manuscript, we verify the validity of our theory by comparison with numerical simulations, as shown e.g. in the central panel of Fig.

Original: The obtained fixed point for the scalar descriptors Θ can be used to evaluate simple expressions for common model evaluation metrics, such as the confusion matrix or the generalisation error .
Revised: The obtained fixed point for the scalar descriptors Θ can be used to evaluate simple expressions for common model evaluation metrics, such as the confusion matrix or the generalisation error .

Original: Analytical result 

Revised: Analytical result

Original: In the same limit as in Analytical result 1, the entries of the confusion matrix, representing the probability of classifying as ˆ y an instance sampled from sub-population c with true label y , are given by:
Revised: In the same limit as in Analytical result 1, the entries of the confusion matrix, representing the probability of classifying as ˆ y an instance sampled from sub-population c with true label y , are given by:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where z ∼ N (0 , 1) and Heav( · ) is the Heaviside step function.
Revised: where z ∼ N (0 , 1) and Heav( · ) is the Heaviside step function.

Original: The generalization error, representing the fraction of wrongly labelled instances, can then be obtained as ϵ g =
Revised: The generalization error, representing the fraction of wrongly labelled instances, can then be obtained as ϵ g =

Original: E c (cid:104)(cid:80)
Revised: E c (cid:104)(cid:80)

Original: ˆ
Revised: ˆ

Original: y ̸
Revised: y ̸

Original: = y p
Revised: = y p

Original: (ˆ y | y ; c ) (cid:105) .
Revised: (ˆ y | y ; c ) (cid:105) .

Original: This second result provides us with a fully deterministic estimate of the accuracy of the trained model on the different data sub-populations.

Original: These scores will be used in the following sections to investigate the possible presence of bias in the classification output of the model.

Original: Note that theorems 1 and 2 allow for an extremely efficient and exact evaluation of the learning performance in the T-M, remapping the original high-dimensional optimisation problem onto a system of deterministic scalar equations that can be easily solved by recursion.

Original: I NVESTIGATING THE SOURCES OF BIAS

Original: With these analytical results at hand, we now turn to systematically investigating the effect of the sources of bias identified in remark 1, which potentially mine the design of a fair classifier.

Original: To

Original: 1.0 q T 0.
Revised: 1.0 q T 0.

Original: a cc u r a c y m +/ =0.0 m +/ =0.
Revised: a cc u r a c y m +/ =0.0 m +/ =0.

Original: m +/ =0.
Revised: m +/ =0.

Original: 1.0 m +/ 0.
Revised: 1.0 m +/ 0.

Original: a cc u r a c y =0.

Revised: a cc u r a c y =0.

Original: =0.
Revised: =0.

Original: =1.
Revised: =1.

Original: quantify the level ofbias in the predictions of the trained model, we need to choose a metric of fairness. Throughout this section, we employ disparate impact (DI) Feldman et al. (2015), a ML analogous of the 80% rule Commission et al.
Revised: This second result provides us with a fully deterministic estimate of the accuracy of the trained model on the different data sub-populations. These scores will be used in the following sections to investigate the possible presence of bias in the classification output of the model.

Original: (1979), which allows a simple assessment of the over-specialisation of  the classifier on one of the sub-populations. In principle, in  the T-M there is no preferable realisation ofthe target attribute so we can adopt a symmetric version of DI, defined as the ratio between test accuracy in sub-population + and sub-population − .
Revised: Note that theorems 1 and 2 allow for an extremely efficient and exact evaluation of the learning performance in the T-M, remapping the original high-dimensional optimisation problem onto a system of deterministic scalar equations that can be easily solved by recursion.  I NVESTIGATING THE SOURCES OF BIAS With these analytical results at hand, we now turn to systematically investigating the effect of the sources of bias identified in remark 1, which potentially mine the design of a fair classifier.

Original: We consider three separate experiments to summarise some distinctive features of the fairness behaviour in the T-M: namely, the impact of the correlation between the labelling rules and the group structure, the interplay between relative representation and group variance, and the positive transfer effect in the data-scarse regime.
Revised: We consider three separate experiments to summarise some distinctive features of the fairness behaviour in the T-M: namely, the impact of the correlation between the labelling rules and the group structure, the interplay between relative representation and group variance, and the positive transfer effect in the data-scarse regime.

Original: The parameters of the experiments, if not specified in the caption, are detailed in appendix ?? .
Revised: The parameters of the experiments, if not specified in the caption, are detailed in appendix E.1.

Original: Group-label correlation.
Revised: Group-label correlation.

Original: In the two left panels of Fig. 2, we consider a scenario where the labelling rules for the two groups are not perfectly aligned, i.e. WWW + T ̸ =
Revised: In the two left panels of Fig. 2, we consider a scenario where the labelling rules for the two groups are not perfectly aligned, i.e. WWW + T ̸ =

Original: WWW − T (and/or b + ̸ = b − ).
Revised: WWW − T (and/or b + ̸ = b − ).

Original: Note that in this case we have a clear mismatch between the learning model, a single linear classifier, and the true input-output structure in the data: the learning model cannot reach perfect generalisation for both sub-populations at the same time.
Revised: Note that in this case we have a clear mismatch between the learning model, a single linear classifier, and the true input-output structure in the data: the learning model cannot reach perfect generalisation for both sub-populations at the same time.

Original: For simplicity, we set an equal correlation between the two teacher vectors and the shift vector, m + = m − > 0 , and isolate the role of rule similarity q T .
Revised: For simplicity, we set an equal correlation between the two teacher vectors and the shift vector, m + = m − > 0 , and isolate the role of rule similarity q T .

Original: The first panel shows a phase diagram of the DI (DI < 1 indicating a lower accuracy on group + ), as function of the similarity of the teachers and the fraction of + samples in the dataset.
Revised: The first panel shows a phase diagram of the DI (DI < 1 indicating a lower accuracy on group + ), as function of the similarity of the teachers and the fraction of + samples in the dataset.

Original: As intuitively expected, the induced bias exceed the 80% rule when the labelling rules are misaligned and the group sizes are numerically unbalanced (small q T and ρ ).
Revised: As intuitively expected, the induced bias exceed the 80% rule when the labelling rules are misaligned and the group sizes are numerically unbalanced (small q T and ρ ).

Original: Indeed, in the cut displayed in the second panel, by lowering the group-label correlation m ± the gap between the measured accuracies on the two sub-populations becomes smaller.
Revised: Indeed, in the cut displayed in the second panel, by lowering the group-label correlation m ± the gap between the measured accuracies on the two sub-populations becomes smaller.

Original: However:
Revised: However:

Original: Remark 2 Even when q T = 1 and the task is solvable (i.e. the classifier can learn the input-output mapping), the trained model can still be biased.
Revised: Remark 2 Even when q T = 1 and the task is solvable (i.e. the classifier can learn the input-output mapping), the trained model can still be biased.

Original: This is shown in the two panels on the right of Fig.
Revised: This is shown in the two panels on the right of Fig.

Original: 2, where a large high-bias region (DI < 80 %) exists.
Revised: 2, where a large high-bias region (DI < 80 %) exists.

Original: In particular, the third panel shows the cause of this effect in the presence of a non-zero group-label correlation m ± , and 
Revised: In particular, the third panel shows the cause of this effect in the presence of a non-zero group-label correlation m ± , and in the fourth panel we see how this effect is more pronounced in the data-scarse regime. In all four panels, as ρ reaches 0 . 5 , the two sub-populations become equally represented and the classifier achieves the same accuracy for both. + 0. + 0.

Revised: + 0.

Original: Bias and variance.
Revised: Bias and variance.

Original: In Fig.
Revised: In Fig.

Original: 3, we plot the DI as a function of the group variances ∆ ± , for different values of the fraction of + samples.
Revised: 3, we plot the DI as a function of the group variances ∆ ± , for different values of the fraction of + samples.

Original: One finds that the model might need a disproportionate number of samples in the two groups to obtain comparable accuracies.
Revised: One finds that the model might need a disproportionate number of samples in the two groups to obtain comparable accuracies.

Original: We can see that:
Revised: We can see that:

Original: Remark 3 Balancing the group relative representation does not guarantee a fair training outcome.
Revised: Remark 3 Balancing the group relative representation does not guarantee a fair training outcome.

Original: + 0.
Revised: In fact, the quality of a group’s representation in the dataset can increase if the number of points is kept constant but the group variance is reduced. The blue regions in the first two panels indicate a higher accuracy for the minority group even if the dataset only contains 10% and 30% of samples belonging to it.

Original: + 0. + 0. In fact 
Revised: This exemplifies the fact that a very focused distribution (low ∆ ± ) actually requires less samples. The last panel ( ρ = 0 . 5 ) shows the scenario one would expect a priori : on the diagonal line the DI is balanced, but by setting ∆ + > ∆ − (or viceversa) one induces a bias in the classification.

Original: Positive transfer.
Revised: Positive transfer.

Original: If mixing different sub-populations in the same dataset can induce unfair behaviour, one could think of splitting the data and train independent models.
Revised: If mixing different sub-populations in the same dataset can induce unfair behaviour, one could think of splitting the data and train independent models.

Original: In Fig.
Revised: In Fig.

Original: 4, we show that a positive transfer effect Gerace et al.
Revised: 4, we show that a positive transfer effect Gerace et al.

Original: (2022) can yet be traced between the two groups when the rules are sufficiently similar.
Revised: (2022) can yet be traced between the two groups when the rules are sufficiently similar.

Original: Remark 4 The performance on the smaller group tends to further deteriorate if the dataset is split according to the sub-group structure.
Revised: Remark 4 The performance on the smaller group tends to further deteriorate if the dataset is split according to the sub-group structure.

Original: To clarify this point, we plot the DI as a function the data scarcity α , for several values of the rule similarity q T and at fixed ρ .
Revised: To clarify this point, we plot the DI as a function the data scarcity α , for several values of the rule similarity q T and at fixed ρ .

Original: We also compare the accuracies on each sub-population of a classifier trained on the full dataset and of a baseline classifier trained only on the respective data subsets ( + in the second figure, − in the third).
Revised: We also compare the accuracies on each sub-population of a classifier trained on the full dataset and of a baseline classifier trained only on the respective data subsets ( + in the second figure, − in the third).

Original: If the rules are sufficiently similar (large q T ), we can observe a positive transfer and using the dataset in his entirety leads to a performance and fairness improvement. 
Revised: If the rules are sufficiently similar (large q T ), we can observe a positive transfer and using the dataset in his entirety leads to a performance and fairness improvement. [Figure]

Original: As expected, positive transfer can be particularly useful in data-scarce regimes (small α ) and becomes ineffective or detrimental in large datasets (large α ), as shown in the last panel.
Revised: other sub-population is perfectly removed.

Original: [Figure]
Revised: [Figure]

Revised: As expected, positive transfer can be particularly useful in data-scarce regimes (small α ) and becomes ineffective or detrimental in large datasets (large α ), as shown in the last panel

Original: [Figure]
Revised: [Figure]

Original:  M ITIGATION STRATEGIES
Revised:  M ITIGATION STRATEGIES

Original: To assess the fairness of a ML model on a given data distribution [Figure] a plethora of different fairness criteria have been designed Speicher et al.
Revised: To assess the fairness of a ML model on a given data distribution, a plethora of different fairness criteria have been designed Speicher et al.

Original: ; Castelnovo et al.
Revised: ; Castelnovo et al.

Original: Appendix ?? presents a summary of the criteria considered in our analysis.
Revised: Appendix F presents a summary of the criteria considered in our analysis.

Original: Following the lines of Speicher et al.
Revised: Following the lines of Speicher et al.

Original: (2018), we aim to quantify exactly how far is a given trained model from meeting each of these criteria.
Revised: (2018), we aim to quantify exactly how far is a given trained model from meeting each of these criteria.

Original: Given a classification event E –specified by the criterion– and the group membership C , a natural measure of their independence is provided by the Mutual Information (MI):
Revised: Given a classification event E –specified by the criterion– and the group membership C , a natural measure of their independence is provided by the Mutual Information (MI):

Original: [Equation]
Revised: [Equation]

Original: Clearly, the fairness condition is completely verified only if the joint distribution factorises, i.e. P [ E, C ] =
Revised: Clearly, the fairness condition is completely verified only if the joint distribution factorises, i.e. P [ E, C ] =

Original: P [ E ] P [ C ] , and the mutual information goes to zero.
Revised: P [ E ] P [ C ] , and the mutual information goes to zero.

Original: This represents the impossibility of predicting the classification outcome of an unbiased model just from the group membership.
Revised: This represents the impossibility of predicting the classification outcome of an unbiased model just from the group membership.

Original: In the following, we consider two simple bias mitigation strategies that can be analysed within our analytical framework.
Revised: In the following, we consider two simple bias mitigation strategies that can be analysed within our analytical framework.

Original: The required generalisations of the replica results are detailed in appendix ?? .
Revised: The required generalisations of the replica results are detailed in appendix D.

Original: First, we study the de-biasing effect of a sample reweighing strategy where the relevance of each sample is varied based on its label and group membership Kamiran & Calders (2012); Plecko &
Revised: First, we study the de-biasing effect of a sample reweighing strategy where the relevance of each sample is varied based on its label and group membership Kamiran & Calders (2012); Plecko &

Original: Meinshausen (2020); Lum & Johndrow (2016).
Revised: Meinshausen (2020); Lum & Johndrow (2016).

Original: By adjusting the weights, one can indirectly minimise the MI relative to any given fairness measure.
Revised: By adjusting the weights, one can indirectly minimise the MI relative to any given fairness measure.

Original: We use the simultaneous quantitative predictions on the various metrics to assess the compatibility between different fairness definitions.
Revised: We use the simultaneous quantitative predictions on the various metrics to assess the compatibility between different fairness definitions.

Original: Then, we propose a theory-based mitigation protocol, along the lines of protocols used in the context of multi-task learning Rusu et al.
Revised: Then, we propose a theory-based mitigation protocol, along the lines of protocols used in the context of multi-task learning Rusu et al.

Original: Loss Reweighing.
Revised: Loss Reweighing.

Original: Recent literature shows that some fairness constraints cannot be satisfied simultaneously.
Revised: Recent literature shows that some fairness constraints cannot be satisfied simultaneously.

Original: ML systems are instead forced to accept trade-offs between them Kleinberg et al.
Revised: ML systems are instead forced to accept trade-offs between them Kleinberg et al.

Original: This sort of compromise is well-captured in the simple framework of the T-M model.
Revised: This sort of compromise is well-captured in the simple framework of the T-M model.

Original: Fig.shows, in form of phase diagrams, the MI measured with respect to the various fairness criteria while varying the two reweighing parameters, w 1 and w + , which up-weigh data points with true labeland in group + , respectively.
Revised: Fig.shows, in form of phase diagrams, the MI measured with respect to the various fairness criteria while varying the two reweighing parameters, w 1 and w + , which up-weigh data points with true labeland in group + , respectively.

Original: E.g., the loss term associated to a label 1 -group + sample will be weighed w + w 1 , while that of a label 0 -group − data point will receive weight (1 − w + )(1 − w 1 ) .
Revised: E.g., the loss term associated to a label 1 -group + sample will be weighed w + w 1 , while that of a label 0 -group − data point will receive weight (1 − w + )(1 − w 1 ) .

Original: By changing these relative weights one can force the model to pay more attention to some types of errors and re-establish a balance between the accuracies on the two sub-populations.
Revised: By changing these relative weights one can force the model to pay more attention to some types of errors and re-establish a balance between the accuracies on the two sub-populations.

Original: The redcrosses in the phase diagrams identify the points where the MI reaches its minimum value for each fairness metric. Notably, some minima are found to lie in different regions of the phase diagram (at the opposite extremes), and they often align only in correspondence of trivial classification, where
Revised: The red

Original: [Figure]
Revised: [Figure]

Original: is

Original: Coupled Networks.
Revised: Coupled Networks.

Original: The emergence of classification bias in the T-M could be lead back to the clear mismatch between the generative model of data and the learning model.
Revised: The emergence of classification bias in the T-M could be lead back to the clear mismatch between the generative model of data and the learning model.

Original: In order to move towards a matched inference setting, we need to enhance the learning model to account for the presence of multiple sub-populations and labelling rules.
Revised: In order to move towards a matched inference setting, we need to enhance the learning model to account for the presence of multiple sub-populations and labelling rules.

Original: This inspires a novel mitigation strategy – called coupled neural networks .
Revised: This inspires a novel mitigation strategy – called coupled neural networks .

Original: The strategy consists in the simultaneous training of multiple neural networks, each one seeing a different subset of the data associated with a different sub-population.
Revised: The strategy consists in the simultaneous training of multiple neural networks, each one seeing a different subset of the data associated with a different sub-population.

Original: The networks exchange information by means of an elastic penalty that mutually attracts them, and the intensity of this elastic interaction is obtained by cross-validation.
Revised: The networks exchange information by means of an elastic penalty that mutually attracts them, and the intensity of this elastic interaction is obtained by cross-validation.

Original: This approach is close in spirit to other methods already present in the literature Calders & Verwer (2010);
Revised: This approach is close in spirit to other methods already present in the literature Calders & Verwer (2010);

Original: Saglietti et al.
Revised: Saglietti et al.

Original: Zenke et al.
Revised: Zenke et al.

Original: Remark 

Revised: Remark

Original: The coupled neural networks method allows for higher expressivity and specialisation on the various sub-populations, while also encouraging a positive transfer between similarly labelled sub-groups, leading to better fairness-accuracy trade-offs
Revised: The coupled neural networks method allows for higher expressivity and specialisation on the various sub-populations, while also encouraging a positive transfer between similarly labelled sub-groups, leading to better fairness-accuracy trade-offs

Original: The first plot in Fig. 6, displaying the behaviour of the mutual information as a function of the coupling parameter for different fairness metrics, shows the key advantage of using this method.
Revised: The first plot in Fig. 6, displaying the behaviour of the mutual information as a function of the coupling parameter for different fairness metrics, shows the key advantage of using this method.

Original: We observe is a more robust consistency among the various fairness metrics: the positions of the different minima are now very close to each other.
Revised: We observe is a more robust consistency among the various fairness metrics: the positions of the different minima are now very close to each other.

Original: Moreover, the value of the coupling parameter achieving this agreement condition is also the one that minimises the gap in terms of test accuracy between the two sub-populations, as shown in the second plot of Fig.
Revised: Moreover, the value of the coupling parameter achieving this agreement condition is also the one that minimises the gap in terms of test accuracy between the two sub-populations, as shown in the second plot of Fig.

Original: 6, without hindering the performance on the larger group.
Revised: 6, without hindering the performance on the larger group.

Original: Notice that this result does not contradict the impossibility theorem Barocas et al.
Revised: Notice that this result does not contradict the impossibility theorem Barocas et al.

Original: (2019) which states that statistical parity, equal odds, and predicted parity cannot be satisfied altogether.
Revised: (2019) which states that statistical parity, equal odds, and predicted parity cannot be satisfied altogether.

Original: In fact, our result only concerns soft minimisation of each fairness metrics.
Revised: In fact, our result only concerns soft minimisation of each fairness metrics.

Original: In appendices ?? and ?? we provide additional results for this method and we discuss the effect of training the networks on data subsets that only partially correlate with the true group structure.
Revised: In appendices D and F.1 we provide additional results for this method and we discuss the effect of training the networks on data subsets that only partially correlate with the true group structure.

Original: Despite the fact that the T-M is just a data prototype, the positive agreement with real phenomenology suggests that this method could be effective also on real-world data.
Revised: Despite the fact that the T-M is just a data prototype, the positive agreement with real phenomenology suggests that this method could be effective also on real-world data.

Original: The remaining two plots in
Revised: The remaining two plots in

Original: Fig.
Revised: Fig.

Original: 6 show preliminary results of the performance of the coupled neural networks strategy in the realistic dataset from CelebA 1 .
Revised: 6 show preliminary results of the performance of the coupled neural networks strategy in the realistic dataset from CelebA 1 .

Original: We stress that although the method works significantly better in the synthetic framework, real data present more complex correlations that may hinder the effectiveness of the method.
Revised: We stress that although the method works significantly better in the synthetic framework, real data present more complex correlations that may hinder the effectiveness of the method.

Original: Therefore, an application of this technique on real settings requires caution.
Revised: Therefore, an application of this technique on real settings requires caution.

Original: A future research direction will be to understand the range of applicability of the coupled neural networks and, consequently, its limitations.
Revised: A future research direction will be to understand the range of applicability of the coupled neural networks and, consequently, its limitations.