Original: E NFORCING FAIRNESS IN PRIVATE FEDERATED LEARNING VIA THE MODIFIED METHOD OF DIFFERENTIAL MULTIPLIERS
Revised: E NFORCING FAIRNESS IN PRIVATE FEDERATED LEARNING VIA THE MODIFIED METHOD OF DIFFERENTIAL MULTIPLIERS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Federated learning with differential privacy, or private federated learning, provides a strategy to train machine learning models while respecting users’ privacy.
Revised: Federated learning with differential privacy, or private federated learning, provides a strategy to train machine learning models while respecting users’ privacy.

Original: However, differential privacy can disproportionately degrade the performance of the models on under-represented groups, as these parts of the distribution are difﬁcult to learn in the presence of noise.
Revised: However, differential privacy can disproportionately degrade the performance of the models on under-represented groups, as these parts of the distribution are difﬁcult to learn in the presence of noise.

Original: Existing approaches for enforcing fairness in machine learning models have considered the centralized setting, in which the algorithm has access to the users’ data.
Revised: Existing approaches for enforcing fairness in machine learning models have considered the centralized setting, in which the algorithm has access to the users’ data.

Original: This paper introduces an algorithm to enforce group fairness in private federated learning, where users’ data does not leave their devices.
Revised: This paper introduces an algorithm to enforce group fairness in private federated learning, where users’ data does not leave their devices.

Original: First, the paper extends the modiﬁed method of differential multipliers to empirical risk minimization with fairness constraints, thus providing an algorithm to enforce fairness in the central setting.
Revised: First, the paper extends the modiﬁed method of differential multipliers to empirical risk minimization with fairness constraints, thus providing an algorithm to enforce fairness in the central setting.

Original: Then, this algorithm is extended to the private federated learning setting.
Revised: Then, this algorithm is extended to the private federated learning setting.

Original: The proposed algorithm, FPFL , is tested on a federated version of the Adult dataset and an “unfair” version of the
Revised: The proposed algorithm, FPFL , is tested on a federated version of the Adult dataset and an “unfair” version of the

Original: FEMNIST dataset.
Revised: FEMNIST dataset.

Original: The experiments on these datasets show how private federated learning accentuates unfairness in the trained models, and how FPFL is able to mitigate such unfairness.
Revised: The experiments on these datasets show how private federated learning accentuates unfairness in the trained models, and how FPFL is able to mitigate such unfairness.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Machine learning requires data to build models.
Revised: Machine learning requires data to build models.

Original: This data is often private and reside in users’ devices.
Revised: This data is often private and resides on users’ devices.

Original: Federated learning (FL) (McMahan et al., 2017) is a strategy where multiple users (or other entities) collaborate in training a model under the coordination of a central server or service provider.
Revised: Federated learning (FL) (McMahan et al., 2017) is a strategy where multiple users (or other entities) collaborate in training a model under the coordination of a central server or service provider.

Original: In FL, devices share only data statistics (e.g. gradients of the model computed from their data) with the server, and therefore the users’ data never leaves their devices.
Revised: In FL, devices share only data statistics (e.g. gradients of the model computed from their data) with the server, and therefore the users’ data never leaves their devices.

Original: However, these statistics can still leak information about the users (see Shokri et al.
Revised: However, these statistics can still leak information about the users (see Shokri et al.

Original: ; Fredrikson et al.
Revised: ; Fredrikson et al.

Original: (2015) for practical attacks recovering the users’ identity and reconstructing the users’ face images).
Revised: (2015) for practical attacks recovering the users’ identity and reconstructing the users’ face images).

Original: To protect against this leakage, differential privacy (DP) can be provided.
Revised: To protect against this leakage, differential privacy (DP) can be provided.

Original: Differential privacy offers a mathematical guarantee on the maximal amount of information any attacker can obtain about a user after observing the released model.
Revised: Differential privacy offers a mathematical guarantee on the maximal amount of information any attacker can obtain about a user after observing the released model.

Original: In practice, often (Bhowmick et al., 2018;
Revised: In practice, often (Bhowmick et al., 2018;

Original: McMahan et al., 2018a;b; Truex et al., 2019; Granqvist et al., 2020)
Revised: McMahan et al., 2018a;b; Truex et al., 2019; Granqvist et al., 2020)

Original: this guarantee is achieved by adding a certain amount of noise to the individuals’ statistics.
Revised: this guarantee is achieved by adding a certain amount of noise to the individuals’ statistics.

Original: In this paper, federated learning with differential privacy will be referred to as private federated learning (PFL).
Revised: In this paper, federated learning with differential privacy will be referred to as private federated learning (PFL).

Original: The models trained with PFL are often neural networks, since they work well in many use cases, and turn out to be resistant to the noise added for DP.
Revised: The models trained with PFL are often neural networks, since they work well in many use cases, and turn out to be resistant to the noise added for DP.

Original: These models have been successfully trained with
Revised: These models have been successfully trained with

Original: PFL for tasks like next word prediction or speaker veriﬁcation (McMahan et al., 2017; Granqvist et al., 2020).
Revised: PFL for tasks like next word prediction or speaker veriﬁcation (McMahan et al., 2017; Granqvist et al., 2020).

Original: However, they are prone to perpetuating societal biases existing in the data (Caliskan et al., 2017) or to discriminate against certain groups even when the data is balanced (Buolamwini &
Revised: However, they are prone to perpetuating societal biases existing in the data (Caliskan et al., 2017) or to discriminate against certain groups even when the data is balanced (Buolamwini &

Original: Gebru, 2018).
Revised: Gebru, 2018).

Original: Moreover, when the training is differentially private, degradation in the performance of these models disproportionately impacts under-represented groups (Bagdasaryan et al., 2019).
Revised: Moreover, when the training is differentially private, degradation in the performance of these models disproportionately impacts under-represented groups (Bagdasaryan et al., 2019).

Original: More speciﬁcally, the accuracy of the model on minority groups is deteriorated to a larger extent than the accuracy for the majority groups.
Revised: More speciﬁcally, the accuracy of the model on minority groups is deteriorated to a larger extent than the accuracy for the majority groups.

Original: In the realm of FL, there has been some research studying how to achieveindividual fairness. This ensures that the performance of the model is similar among devices (Hu et al., 2020; Huang et al.,  Li et al., 2019; 2021).
Revised: In the realm of FL, there has been some research studying how to achieve that the performance of the model is similar among devices (Hu et al., 2020; Huang et al., 2020; Li et al., 2019; 2021).

Original: However, this notion of fairness falls short in terms of protecting users from under-represented groups falling foul of disproportionate treatment.
Revised: However, this notion of fairness falls short in terms of protecting users from under-represented groups falling foul of disproportionate treatment.

Original: For example, Castelnovo et al.
Revised: For example, Castelnovo et al.

Original: (2021, Section 3.6) show that a model that performs well on the majority of the users will have a good score on individual fairness metrics, even if all the users suffering from bad performance belong to the same group.
Revised: (2021, Section 3.6) show that a model that performs well on the majority of the users will have a good score on individual fairness metrics, even if all the users suffering from bad performance belong to the same group.

Original: Conversely, there is little work proposing solutions to enforce group fairness, i.e. that the performance is similar among users from different groups.
Revised: Conversely, there is little work proposing solutions to enforce group fairness, i.e. that the performance is similar among users from different groups.

Original: The current work in this area  (Du et al., 2021) is devoted to  a speciﬁc measure of fairness (demographic parity) when using logistic regression models.
Revised: Current work in this area is either limited to reducing demographic parity in logistic regression (Du et al., 2021) or to minimizing the largest loss across the different groups with a minimax optimization (Mohri et al., 2019). Concurrently to this paper, Cui et al. (2021) proposed an approach to enforce group fairness in FL for general models and metrics.

Original:  Moreover, all such prior workfocuses on non-private FL, and therefore does not consider the adverse affects that DP has on the models trained with PFL.
Revised: However, this approach does not consider DP. Moreover, most prior work does not consider the adverse effects that DP has on the models trained with private federated learning, or is restricted to logistic regression models (Abay et al., 2020).

Original: On the other hand, work studying the trade-offs between privacy and group fairness proposes solutions that are either limited to simple models such as linear logistic regression (Ding et al., 2020), require an oracle (Cummings et al., 2019), scale poorly for large hypothesis classes (Jagielski et al., 2019), or only offer privacy protection for the variable determining the group (Tran et al., 2021;
Revised: On the other hand, work studying the trade-offs between privacy and group fairness proposes solutions that are either limited to simple models such as linear logistic regression (Ding et al., 2020), require an oracle (Cummings et al., 2019), scale poorly for large hypothesis classes (Jagielski et al., 2019), or only offer privacy protection for the variable determining the group (Tran et al., 2021;

Original: Rodr´ıguez-G´alvez et al., 2021).
Revised: Rodr´ıguez-G´alvez et al., 2021).

Original: Furthermore, the aformentioned work only considers the central learning paradigm, and the techniques are not directly adaptable to FL.
Revised: Furthermore, the aforementioned work only considers the central learning paradigm, and the techniques are not directly adaptable to FL.

Original: For all the above reasons, in this paper, we propose an algorithm to train neural networks with
Revised: For all the above reasons, in this paper, we propose an algorithm to train neural networks with

Original: PFL while enforcing group fairness.
Revised: PFL while enforcing group fairness.

Original: We pose the problem as an optimization problem with fairness constraints and extend the modiﬁed method of differential multipliers (MMDM)
Revised: We pose the problem as an optimization problem with fairness constraints and extend the modiﬁed method of differential multipliers (MMDM)

Original: (Platt & Barr, 1987) to solve such a problem with FL and DP.
Revised: (Platt & Barr, 1987) to solve such a problem with FL and DP.

Original: Hence, the resulting algorithm (i) is applicable to any model that can be learned using stochastic gradient descent (SGD) or any of its variants, (ii) can be tailored to enforce the majority of the group fairness metrics, (iii) can consider any number of attributes determining the groups, and (iv) can consider both classiﬁcation and regression tasks.
Revised: Hence, the resulting algorithm (i) is applicable to any model that can be learned using stochastic gradient descent (SGD) or any of its variants, (ii) can be tailored to enforce the majority of the group fairness metrics, (iii) can consider any number of attributes determining the groups, and (iv) can consider both classiﬁcation and regression tasks.

Original: The paper is structured as follows: in Section 2, we review the background on differential privacy, private federated learning, the MMDM algorithm, and group fairness; in Section 3, we present our approach for fair private federated learning; in Section 4, we describe our experimental results; and in Section 5, we conclude with a summary and interpretation of our ﬁndings.
Revised: The paper is structured as follows: in Section 2, we review the background on differential privacy, private federated learning, the MMDM algorithm, and group fairness; in Section 3, we present our approach for fair private federated learning; in Section 4, we describe our experimental results; and in Section 5, we conclude with a summary and interpretation of our ﬁndings.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: In this section, we review several aspects on the theory of private federated learning, the MMDM algorithm, and group fairness that are necessary to develop and understand the proposed algorithm.
Revised: In this section, we review several aspects on the theory of private federated learning, the MMDM algorithm, and group fairness that are necessary to develop and understand the proposed algorithm.

Original: 2. F EDERATED L EARNING
Revised: 2. F EDERATED L EARNING

Original: The federated learning setting focuses on learning a model that minimizes the expected value of a loss function (cid:96) using a dataset d = ( z 1 , z 2 , . . .
Revised: The federated learning setting focuses on learning a model that minimizes the expected value of a loss function (cid:96) using a dataset d = ( z 1 , z 2 , . . .

Original: , z n ) ∈ D of n samples distributed across K users.
Revised: , z n ) ∈ D of n samples distributed across K users.

Original: This paper will consider models parametrized by a ﬁxed number of parameters w ∈ R p and differentiable loss functions (cid:96) , which includes neural networks.
Revised: This paper will consider models parametrized by a ﬁxed number of parameters w ∈ R p and differentiable loss functions (cid:96) , which includes neural networks.

Original: Since the distribution of the data samples Z is unknown and each user k is the only owner of their private dataset d k , the goal of FL is to ﬁnd the parameters w (cid:63) that minimize the loss function across the samples of the users.
Revised: Since the distribution of the data samples Z is unknown and each user k is the only owner of their private dataset d k , the goal of FL is to ﬁnd the parameters w (cid:63) that minimize the loss function across the samples of the users.

Original: That is, deﬁning L ( d k , w ) :
Revised: That is, deﬁning L ( d k , w ) :

Original: = (cid:80)
Revised: = (cid:80)

Revised: z

Original: [Equation]
Revised: [Equation]

Original:  ( z , w ) , to solve
Revised: k (cid:96) ( z , w ) , to solve

Original: [Equation]
Revised: [Equation]

Original: This way, the parameters w can be learned with an approximation of SGD.
Revised: This way, the parameters w can be learned with an approximation of SGD.

Original: McMahan et al.
Revised: McMahan et al.

Original: (2017) suggest that the central server iteratively samples m users; they compute and send back to the server an approximation of the model’s gradient ∇ w L ( d k , w ) ; and ﬁnally the server updates the parameters as dictated by gradient descent w ← w − 1 n (cid:80)
Revised: (2017) suggest that the central server iteratively samples m users; they compute and send back to the server an approximation of the model’s gradient ∇ w L ( d k , w ) ; and ﬁnally the server updates the parameters as dictated by gradient descent w ← w − 1 n (cid:80)

Original: k
Revised: k

Original: η ∇ w L ( d k , w ) , where η is the learning rate.
Revised: η ∇ w L ( d k , w ) , where η is the learning rate.

Original: If the users send back the exact gradient the algorithm is known as FederatedSGD .
Revised: If the users send back the exact gradient the algorithm is known as FederatedSGD .

Original: A generalisation of this, FederatedAveraging , involves users running several local epochs of mini-batch SGD and sending up the difference.
Revised: A generalisation of this, FederatedAveraging , involves users running several local epochs of mini-batch SGD and sending up the difference.

Original: However, the method in this paper relies on an additional term in the loss function, so it will extend FederatedSGD .
Revised: However, the method in this paper considers an optimization with fairness constraints, requiring the communication of extra information in each iteration, and therefore will extend FederatedSGD .

Original: 2. P RIVATE FEDERATED LEARNING
Revised: 2. P RIVATE FEDERATED LEARNING

Original: Private federated learning combines federated learning with differential privacy.
Revised: Private federated learning combines federated learning with differential privacy.

Original: Differential privacy (Dwork et al., 2006; 2014) provides a mathematical guarantee by restricting the amount of information that a randomized function (cid:109) of a dataset d leaks about any individual.
Revised: Differential privacy (Dwork et al., 2006; 2014) provides a mathematical guarantee by restricting the amount of information that a randomized function (cid:109) of a dataset d leaks about any individual.

Original: More precisely, it ensures that the probability that the output of the function (cid:109) on the dataset d belongs to any set of outcomes (cid:79) is close to the probability that the output of the function (cid:109) applied to a neighbouring dataset d (cid:48)
Revised: More precisely, it ensures that the probability that the output of the function (cid:109) on the dataset d belongs to any set of outcomes (cid:79) is close to the probability that the output of the function (cid:109) applied to a neighbouring dataset d (cid:48)

Original: (i.e. equal to d but where all samples from any individual are removed or replaced) belongs to said set of outcomes.
Revised: (i.e. equal to d but where all samples from any individual are removed or replaced) belongs to said set of outcomes.

Original: Formally, a randomized function (cid:109) is ( (cid:15), δ ) -differentially private if
Revised: Formally, a randomized function (cid:109) is ( (cid:15), δ ) -differentially private if

Original: [Equation]
Revised: [Equation]

Original: where (cid:15) and δ are parameters that determine the strength of the guarantee.
Revised: where (cid:15) and δ are parameters that determine the strength of the guarantee.

Original: Two useful properties of
Revised: Two useful properties of

Original: DP are that that it composes (i.e. the combination of DP functions is DP) and that post-processing does not undo its privacy guarantees (Dwork et al., 2014).
Revised: DP are that that it composes (i.e. the combination of DP functions is DP) and that post-processing does not undo its privacy guarantees (Dwork et al., 2014).

Original: These properties ensure that the modiﬁcations that we will make to FederatedSGD will keep the algorithm differentially private.
Revised: These properties ensure that the modiﬁcations that we will make to FederatedSGD will keep the algorithm differentially private.

Original: Typically, the functions of interest are not differentially private.
Revised: Typically, the functions of interest are not differentially private.

Original: For this reason, they are privatized with “privacy mechanisms”, which apply a random transformation to the output of the function on the dataset.
Revised: For this reason, they are privatized with “privacy mechanisms”, which apply a random transformation to the output of the function on the dataset.

Original: A standard mechanism is the Gaussian mechanism, which adds Gaussian noise to the function’s output, calibrating the noise variance with the privacy parameters ( (cid:15), δ ) and the maximum individual’s contribution to the function’s output measured by the (cid:96) 2 norm.
Revised: A standard mechanism is the Gaussian mechanism, which adds Gaussian noise to the function’s output, calibrating the noise variance with the privacy parameters ( (cid:15), δ ) and the maximum individual’s contribution to the function’s output measured by the (cid:96) 2 norm.

Original: In the context of FederatedSGD , the functions to privatize are the gradient estimates sent to the server.
Revised: In the context of FederatedSGD , the functions to privatize are the gradient estimates sent to the server.

Original: The individual’s contribution is the gradient computed on their data.
Revised: The individual’s contribution is the gradient computed on their data.

Original: This gradient is “normclipped” (Abadi et al., 2016), i.e. it is scaled down so its (cid:96) 2 norm is at most a predetermined value C .
Revised: This gradient is “normclipped” (Abadi et al., 2016), i.e. it is scaled down so its (cid:96) 2 norm is at most a predetermined value C .

Original: Then, Gaussian noise is added to the sum of contributions, which is sent to the server.
Revised: Then, Gaussian noise is added to the sum of contributions, which is sent to the server.

Original: Applying the privacy mechanism to a sum of contributions is a form of central DP.
Revised: Applying the privacy mechanism to a sum of contributions is a form of central DP.

Original: An alternative is local DP (Dwork et al., 2014), where the privacy mechanism is applied to the statistics before leaving the device.
Revised: An alternative is local DP (Dwork et al., 2014), where the privacy mechanism is applied to the statistics before leaving the device.

Original: However, models trained with local DP often suffer from low utility (Granqvist et al., 2020), and are thus not considered in this paper.
Revised: However, models trained with local DP often suffer from low utility (Granqvist et al., 2020), and are thus not considered in this paper.

Original: Central DP can involve trust in the server that updates the model, or a trusted third party separated from the model, which is how DP was originally formulated.
Revised: Central DP can involve trust in the server that updates the model, or a trusted third party separated from the model, which is how DP was originally formulated.

Original: This paper instead assumes the noisy sum is computed through secure multiparty computation (Goryczka & Xiong, 2015), such as secure aggregation (Bonawitz et al., 2017).
Revised: This paper instead assumes the noisy sum is computed through secure multiparty computation (Goryczka & Xiong, 2015), such as secure aggregation (Bonawitz et al., 2017).

Original: This is a cryptographic method that ensures the server only sees the sum of contributions.
Revised: This is a cryptographic method that ensures the server only sees the sum of contributions.

Original: A crucial limitation of secure aggregation is that it can only compute sums, so the novel method introduced in Section 3 is restricted to working only on sums of individuals’ statistics.
Revised: A crucial limitation of secure aggregation is that it can only compute sums, so the novel method introduced in Section 3 is restricted to working only on sums of individuals’ statistics.

Original: 2. T HE M ODIFIED M ETHOD OF D IFFERENTIAL M ULTIPLIERS
Revised: 2. T HE M ODIFIED M ETHOD OF D IFFERENTIAL M ULTIPLIERS

Original: Ultimately, our objective is to ﬁnd a parametric model that minimizes a differentiable loss function and respects some fairness constraints.
Revised: Ultimately, our objective is to ﬁnd a parametric model that minimizes a differentiable loss function and respects some fairness constraints.

Original: Thus, this subsection reviews a constrained differential optimization algorithm, the modiﬁed method of differential multipliers (MMDM)
Revised: Thus, this subsection reviews a constrained differential optimization algorithm, the modiﬁed method of differential multipliers (MMDM)

Original: (Platt & Barr, 1987).
Revised: (Platt & Barr, 1987).

Original: This algorithm tries to ﬁnd a solution to the following constrained optimization problem:
Revised: This algorithm tries to ﬁnd a solution to the following constrained optimization problem:

Original: [Equation]
Revised: [Equation]

Original: where φ : R p
Revised: where φ : R p

Original: → R is the function to minimize, g ( w ) =
Revised: → R is the function to minimize, g ( w ) =

Original: ( g 1 ( w ) , g 2 ( w ) , . . .
Revised: ( g 1 ( w ) , g 2 ( w ) , . . .

Original: , g r ( w )) is the concatenation of r constraint functions, and { w ∈ R p : g ( w ) = 0 } is the solution subspace.
Revised: , g r ( w )) is the concatenation of r constraint functions, and { w ∈ R p : g ( w ) = 0 } is the solution subspace.

Original: The algorithm consists of solving the following set of differential equations resulting from the Lagrangian of (P1) with an additional quadratic penalty c g ( w ) 2 using gradient descent/ascent.
Revised: The algorithm consists of solving the following set of differential equations resulting from the Lagrangian of (P1) with an additional quadratic penalty c g ( w ) 2 using gradient descent/ascent.

Original: This results in an algorithm that applies these updates iteratively:
Revised: This results in an algorithm that applies these updates iteratively:

Original: [Equation]
Revised: [Equation]

Original: where λ ∈ R r is a Lagrange (or dual) multiplier, c ∈ R + is a damping parameter, η is the learning rate of the model parameters, and γ is the learning rate of the Lagrange multiplier.
Revised: where λ ∈ R r is a Lagrange (or dual) multiplier, c ∈ R + is a damping parameter, η is the learning rate of the model parameters, and γ is the learning rate of the Lagrange multiplier.

Original: Intuitively, these updates gradually fulﬁll (P1).
Revised: Intuitively, these updates gradually fulﬁll (P1).

Original: Updating the parameters ∇ w φ ( w ) and ∇ w g ( w ) respectively enforce function minimization and constraint’s satisfaction.
Revised: Updating the parameters ∇ w φ ( w ) and ∇ w g ( w ) respectively enforce function minimization and constraint’s satisfaction.

Original: Then, the multiplier λ and the multiplicative factor c g ( w ) control how strongly the constraints’ violations are penalized.
Revised: Then, the multiplier λ and the multiplicative factor c g ( w ) control how strongly the constraints’ violations are penalized.

Original: A desirable property of MMDM is that for small enough learning rates η, γ and large enough damping parameter c , there is a region comprising the vicinity of each constrained minimum such that if the parameters’ initialization is in that region and the parameters remain bounded, then the algorithm converges to a constrained minimum (Platt & Barr, 1987).
Revised: A desirable property of MMDM is that for small enough learning rates η, γ and large enough damping parameter c , there is a region comprising the vicinity of each constrained minimum such that if the parameters’ initialization is in that region and the parameters remain bounded, then the algorithm converges to a constrained minimum (Platt & Barr, 1987).

Original: Intuitively, the term c g ( w ) ∇ w g ( w ) enforces a quadratic shape on the optimization search space to the neighbourhood of the solution subspace, and this local behavior is stronger the larger the damping parameter c .
Revised: Intuitively, the term c g ( w ) ∇ w g ( w ) enforces a quadratic shape on the optimization search space to the neighbourhood of the solution subspace, and this local behavior is stronger the larger the damping parameter c .

Original: Thus, the algorithm converges to a minimum if the parameters’ initialization is in this locally quadratic region.
Revised: Thus, the algorithm converges to a minimum if the parameters’ initialization is in this locally quadratic region.

Original: 2. G ROUP F AIRNESS
Revised: 2. G ROUP F AIRNESS

Original: In this subsection, we mathematically formalize what group fairness means.
Revised: In this subsection, we mathematically formalize what group fairness means.

Original: To ease the exposition, we describe this notion in the central setting, i.e. where the data is not distributed across users.
Revised: To ease the exposition, we describe this notion in the central setting, i.e. where the data is not distributed across users.

Original: Let us consider a dataset d = ( z 1 , z 2 , . ..
Revised: Consider a dataset d = ( z 1 , z 2 , . .

Original: , z n ) of n samples, where each sample z i belongs toa group a i∈ A .
Revised: z n ) of n samples. The dataset is partitioned into groups from A such that each sample z i belongs to group a i .

Original: Group fairness considers how differently a model treats the samples belonging to each group.
Revised: Group fairness considers how differently a model treats the samples belonging to each group.

Original: Many fairness metrics can be written in terms of the similarity of the expected value of a function of interest f of the model evaluated on the general population d with that on the population of each group d a = { z i ∈
Revised: Many fairness metrics can be written in terms of the similarity of the expected value of a function of interest f of the model evaluated on the general population d with that on the population of each group d a = { z i ∈

Original: d : a i = a } (Agarwal et al., 2018; Fioretto et al., 2020).
Revised: d : a i = a } (Agarwal et al., 2018; Fioretto et al., 2020).

Original: That is, if we consider a supervised learning problem, where z i = ( x i , y i , a i ) and where the output of the model is an approximation ˆ y i of y
Revised: That is, if we consider a supervised learning problem, where z i = ( x i , y i , a i ) and where the output of the model is an approximation ˆ y i of y

Original: i , we say a model is fair if
Revised: i , we say a model is fair if

Original: [Equation]
Revised: [Equation]

Original:  Most of the group fairness literature focuses on the case of binary classiﬁers, i.e. ˆ y i ∈ { 0 , 1 } , and on the binary group case, i.e. A = { 0 , 1 } .
Revised: Y )] for all a ∈ A . [Equation] Most of the group fairness literature focuses on the case of binary classiﬁers, i.e. ˆ y i ∈ { 0 , 1 } , and on the binary group case, i.e. A = { 0 , 1 } .

Original: However, many of the fairness metrics can be extended to general output spaces Y and categorical groups A .
Revised: However, many of the fairness metrics can be extended to general output spaces Y and categorical groups A .

Original: It is common that the function f is the indicator function 1 of some logical relationship between the random variables, thus turning (4) to an equality between probabilities.
Revised: It is common that the function f is the indicator function 1 of some logical relationship between the random variables, thus turning (4) to an equality between probabilities.

Original: As an example, we describe two common fairness metrics that will be used later in the paper.
Revised: As an example, we describe two common fairness metrics that will be used later in the paper.

Original: For a comprehensive survey of different fairness metrics and their inter-relationships, please refer to (Verma & Rubin, 2018; Castelnovo et al., 2021).
Revised: For a comprehensive survey of different fairness metrics and their inter-relationships, please refer to Verma & Rubin (2018); Castelnovo et al.

Original: False negative rate (FNR) parity (or equal opportunity)
Revised: False negative rate (FNR) parity (or equal opportunity)

Original: (Hardt et al., 2016)
Revised: (Hardt et al., 2016)

Original: This fairness metric is designed for binary classiﬁcation and binary groups.
Revised: This fairness metric is designed for binary classiﬁcation and binary groups.

Original: It was originally deﬁned as equal true positive rate between the groups, which is equivalent to an equal FNR between each group and the overall population.
Revised: It was originally deﬁned as equal true positive rate between the groups, which is equivalent to an equal FNR between each group and the overall population.

Original: That is, if we let f ( X , Y, ˆ Y ) =
Revised: That is, if we let f ( X , Y, ˆ Y ) =

Original: E [ 1 ( ˆ Y = 0)
Revised: E [ 1 ( ˆ Y = 0)

Original: | Y
Revised: | Y

Original: = 1] then (4) reduces to
Revised: = 1] then (4) reduces to

Revised: P

Original: [Equation]
Revised: [Equation]

Original: This is usually a good metric when the target variable Y is something positive such as being granted a loan or hired for a job, since we want to minimize the group disparity of misclassiﬁcation among the individuals deserving such a loan or such a job (Hardt et al., 2016; Castelnovo et al., 2021).
Revised: This is usually a good metric when the target variable Y is something positive such as being granted a loan or hired for a job, since we want to minimize the group disparity of misclassiﬁcation among the individuals deserving such a loan or such a job (Hardt et al., 2016; Castelnovo et al., 2021).

Original: Accuracy parity (or overall misclassiﬁcation rate) (Zafar et al., 2017)
Revised: Accuracy parity (or overall misclassiﬁcation rate) (Zafar et al., 2017)

Original: This fairness metric is also designed for binary classiﬁcation and binary groups.
Revised: This fairness metric is also designed for binary classiﬁcation and binary groups.

Original: Nonetheless, it applies well to general tasks and categorical groups.
Revised: Nonetheless, it applies well to general tasks and categorical groups.

Original: In this case, if we let f ( X , Y, ˆ Y ) = 1 ( ˆ Y = Y ) then (4) reduces to
Revised: In this case, if we let f ( X , Y, ˆ Y ) = 1 ( ˆ Y = Y ) then (4) reduces to

Original: P [ ˆ Y = Y
Revised: P [ ˆ Y = Y

Original: |
Revised: |

Original: A = a ] = P [ ˆ Y
Revised: A = a ] = P [ ˆ Y

Original: =
Revised: =

Original: Y ] for all a ∈ A .
Revised: Y ] for all a ∈ A .

Original: This is usually a good metric when there is not a clear positive or negative semantic meaning to the target variable, and also when this variable is not binary.
Revised: This is usually a good metric when there is not a clear positive or negative semantic meaning to the target variable, and also when this variable is not binary.

Original:  A N A LGORITHM FOR F AIR AND P RIVATE F EDERATED L EARNING (FPFL)
Revised:  A N A LGORITHM FOR F AIR AND P RIVATE F EDERATED L EARNING (FPFL)

Original: In this section, we describe the proposed algorithm to enforce fairness in PFL of parametric models learned with SGD.
Revised: In this section, we describe the proposed algorithm to enforce fairness in PFL of parametric models learned with SGD.

Original: First, we describe an adaptation of the MMDM algorithm to enforce fairness in standard central learning.
Revised: First, we describe an adaptation of the MMDM algorithm to enforce fairness in standard central learning.

Original: Then, we extend that algorithm to PFL.
Revised: Then, we extend that algorithm to PFL.

Original: 3. A DAPTING THE MMDM A LGORITHM TO E NFORCE F AIRNESS
Revised: 3. A DAPTING THE MMDM A LGORITHM TO E NFORCE F AIRNESS

Original: Consider a dataset d = ( z 1 , z 2 , . .
Revised: Consider a dataset d = ( z 1 , z 2 , . .

Original: z n ) of n samples, where each sample z i belongs toa group a i∈ A .
Revised: z n ) of n samples. The dataset is partitioned into groups from A such that each sample z i belongs to group a i .

Original: Consider also a supervised learning setting where z i =
Revised: Then, let us consider also a supervised learning setting where z i =

Original: ( x i , y i , a i ) , the model’s output
Revised: ( x i , y i , a i ) , the model’s output

Original: ˆ y i approximates y i , and the model is parametrized by the parameters w ∈ R p .
Revised: ˆ y i approximates y i , and the model is parametrized by the parameters w ∈ R p .

Original: Finally, assume we have no information about the variables’ ( X , Y, A ) distribution apart from the available samples.
Revised: Finally, assume we have no information about the variables’ ( X , Y, A ) distribution apart from the available samples.

Original: We concern ourselves with the task of ﬁnding the model’s parameters w (cid:63) that minimize a loss function (cid:96) across the data samples and enforce a measure of fairness on the model.
Revised: We concern ourselves with the task of ﬁnding the model’s parameters w (cid:63) that minimize a loss function (cid:96) across the data samples and enforce a measure of fairness on the model.

Original: That is, we substitute the expected values of the fairness constraints in (4) with empirical averages in (P2):
Revised: That is, we substitute the expected values of the fairness constraints in (4) with empirical averages:

Original: [Equation]
Revised: [Equation]

Original: n (cid:48) [Equation] (cid:12)(cid:12)(cid:12)(cid:12) ≤ α, for all a ∈ A , (P2) where L ( d, w ) :
Revised: where L ( d, w ) :

Original: = (cid:80) z ∈ d (cid:96) ( z , w ) , F ( d (cid:48) , w ) :
Revised: = (cid:80) z ∈ d (cid:96) ( z , w ) , F ( d (cid:48) , w ) :

Original: = (cid:80) z ∈ d (cid:48)
Revised: = (cid:80) z ∈ d (cid:48)

Original: f ( z , w ) , d (cid:48) is a subset of d that varies among different fairness metrics, n (cid:48) is the number of samples in d (cid:48) , d (cid:48) a is the set of samples in d (cid:48)
Revised: f ( z , w ) , d (cid:48) is a subset of d that varies among different fairness metrics, n (cid:48) is the number of samples in d (cid:48) , d (cid:48) a is the set of samples in d (cid:48)

Original: such that a i = a , n (cid:48) a is the number of samples in d (cid:48) a , f is the function employed for the fairness metric deﬁnition, and α is a tolerance threshold.
Revised: such that a i = a , n (cid:48) a is the number of samples in d (cid:48) a , f is the function employed for the fairness metric deﬁnition, and α is a tolerance threshold.

Original: The subset d (cid:48) is chosen based on the function f used for the fairness metric: if the fairness function does not involve any conditional expectation (e.g. accuracy parity), then d (cid:48)
Revised: The subset d (cid:48) is chosen based on the function f used for the fairness metric: if the fairness function does not involve any conditional expectation (e.g. accuracy parity), then d (cid:48)

Original: =
Revised: =

Original: d ; if, on the other hand, the subset involves a conditional expectation, then the d (cid:48) is the subset of d where that condition holds, e.g. when the fairness metric is FNR parity d (cid:48)
Revised: d ; if, on the other hand, the subset involves a conditional expectation, then the d (cid:48) is the subset of d where that condition holds, e.g. when the fairness metric is FNR parity d (cid:48)

Original: = { ( x , y, a ) ∈
Revised: = { ( x , y, a ) ∈

Original: d : y = 1 } .
Revised: d : y = 1 } .

Revised: We exclude the case where the conditioning in the expectation is ˆ Y , thus excluding predictive parity as a target fairness metric.

Original: Note that the constraints are not strict, meaning that there is a tolerance α for how much the function f can vary between certain groups and the overall population.
Revised: Note that the constraints are not strict, meaning that there is a tolerance α for how much the function f can vary between certain groups and the overall population.

Original: The reason for this choice is twofold.
Revised: The reason for this choice is twofold.

Original: First, it facilitates the training since the solution subspace is larger.
Revised: First, it facilitates the training since the solution subspace is larger.

Original: Second, it is known that some fairness metrics, such as FNR parity, are incompatible with DP and non-trivial accuracy.
Revised: Second, it is known that some fairness metrics, such as FNR parity, are incompatible with DP and non-trivial accuracy.

Original: However, if the fairness metric is relaxed, fairness, accuracy, and privacy can coexist (Jagielski et al., 2019;
Revised: However, if the fairness metric is relaxed, fairness, accuracy, and privacy can coexist (Jagielski et al., 2019;

Original: Cummings et al., 2019).
Revised: Cummings et al., 2019).

Original: This way, we may re-write (P2) in the form of (P1) to solve the problem with MMDM.
Revised: This way, we may re-write (P2) in the form of (P1) to solve the problem with MMDM.

Original: To do so we let φ ( w ) =
Revised: To do so we let φ ( w ) =

Original: L ( d, w ) /n and g ( w ) =
Revised: L ( d, w ) /n and g ( w ) =

Original: ( g 0 ( w ) , g 1 ( w ) , . .
Revised: ( g 0 ( w ) , g 1 ( w ) , . .

Original: . , g |A|− 1 ( w )) , where g a ( w ) = (cid:26) h a ( w ) if h a ( w ) ≥ otherwise and h a ( w ) : = (cid:12)(cid:12)(cid:12) F ( d (cid:48) [Equation] w ) n (cid:48)
Revised: . , g |A|− 1 ( w )) , where g a ( w ) = (cid:26) h a ( w ) if h a ( w ) ≥ otherwise and h a ( w ) :=

Original: − F

Original: [Equation]

Original: d (cid:48)

Original: a

Original: [Equation]
Revised: [Equation]

Original: w

Original: [Equation]

Original: n (cid:48)

Original: [Equation]

Original: (cid:12)(cid:12)(cid:12) − α.

Original: Therefore, the parameters are updated according to
Revised: Therefore, the parameters are updated according to

Original: [Equation]
Revised: [Equation]

Original: where we note that ∇ w g ( w )
Revised: where we note that ∇ w g ( w )

Original: = ( ∇ w g 0 ( w ) , ∇ w g 1 ( w ) , . .
Revised: = ( ∇ w g 0 ( w ) , ∇ w g 1 ( w ) , . .

Original: . , ∇ w g |A|− 1 ( w )) and ∇ w g a ( w [Equation] =[Equation]
Revised: . , ∇ w g |A|− 1 ( w )) and ∇ w g a ( w ) =

Original: sign

Original: [Equation]
Revised: [Equation]

Original: otherwise .
Revised: otherwise .

Original: [Equation]
Revised: [Equation]

Original: [Equation] Now, the fairness-enforcing problem (P2) can be solved with gradient descent/ascent or mini-batch stochastic gradient descent/ascent, where instead of the full dataset d , d (cid:48) , and d (cid:48) a , one considers batches b , b (cid:48) , and b (cid:48) a (or subsets) of that dataset.
Revised: Now, the fairness-enforcing problem (P2) can be solved with gradient descent/ascent or mini-batch stochastic gradient descent/ascent, where instead of the full dataset d , d (cid:48) , and d (cid:48) a , one considers batches b , b (cid:48) , and b (cid:48) a (or subsets) of that dataset.

Original: Moreover, it can be learned with DP adapting the
Revised: Moreover, it can be learned with DP adapting the

Original: DP-SGD algorithm from (Abadi et al., 2016), where a clipping bound and Gaussian noise is included in both the network parameters’ and multipliers’ individual updates.
Revised: DP-SGD algorithm from (Abadi et al., 2016), where a clipping bound and Gaussian noise is included in both the network parameters’ and multipliers’ individual updates.

Original: Nonetheless, there are a series of caveats of doing so.
Revised: Nonetheless, there are a series of caveats of doing so.

Original: First, the batch size | b (cid:48) | should be large enough to, on average, have enough samples of each group a ∈ A so that the difference (cid:0) F ( b (cid:48) , w ) | b (cid:48) | −
Revised: First, the batch size | b (cid:48) | should be large enough to, on average, have enough samples of each group a ∈ A so that the difference (cid:0) F ( b (cid:48) , w ) | b (cid:48) | −

Original: [Equation]
Revised: [Equation]

Original: a
Revised: a

Original: [Equation]
Revised: [Equation]

Original: w ) | b (cid:48) a | (cid:1) is well estimated.
Revised: | b [Equation] a | (cid:1) is well estimated.

Original: Second, in many situations of interest such as when we want to enforce FNR parity or accuracy parity, the function f employed for the fairness metric is not differentiable and thus ∇ w F ( d (cid:48) , w ) does not exist.
Revised: Second, in many situations of interest such as when we want to enforce FNR parity or accuracy parity, the function f employed for the fairness metric is not differentiable and thus ∇ w F ( d (cid:48) , w ) does not exist.

Original: To solve this issue, we resort to estimate the gradient ∇ w F ( d (cid:48) , w ) using a differentiable estimation of the function aggregate F ( d (cid:48) , w ) ; see Appendix A.1 for the details on these estimates in the above situations of interest.
Revised: To solve this issue, we resort to estimate the gradient ∇ w F ( d (cid:48) , w ) using a differentiable estimation of the function aggregate F ( d (cid:48) , w ) ; see Appendix A.1 for the details on these estimates in the above situations of interest.

Original: We conclude this section noting the similarities and differences of this work and (Tran et al., 2021).
Revised: We conclude this section noting the similarities and differences of this work and (Tran et al., 2021).

Original: Even though their algorithm is derived from ﬁrst principles on Lagrangian duality, it is ultimately equivalent to an application of the basic method of differential multipliers (BMDM) to solve a problem equivalent to (P2) when α = 0 .
Revised: Even though their algorithm is derived from ﬁrst principles on Lagrangian duality, it is ultimately equivalent to an application of the basic method of differential multipliers (BMDM) to solve a problem equivalent to (P2) when α = 0 .

Original: Nonetheless, the two algorithms differ in three main aspects:
Revised: Nonetheless, the two algorithms differ in three main aspects:

Original: BMDM vs MMDM: BMDM is equivalent to MMDM when c
Revised: BMDM vs MMDM: BMDM is equivalent to MMDM when c

Original: = 0 , that is, when the effect of making the neighbourhood of the solution subspace quadratic is not present.
Revised: = 0 , that is, when the effect of making the neighbourhood of the solution subspace quadratic is not present.

Original: Moreover, the guarantee of achieving a local minimum that respects the constraints does not hold for c = 0 unless the problem is simple (e g. quadratic programming).
Revised: Moreover, the guarantee of achieving a local minimum that respects the constraints does not hold for c = 0 unless the problem is simple (e.g. quadratic programming).

Original: How they deal with impossibilities in the goal of achieving perfect fairness together with privacy and accuracy.
Revised: How they deal with impossibilities in the goal of achieving perfect fairness together with privacy and accuracy.

Original: Tran et al.
Revised: Tran et al.

Original: (2021) include a limit λ max to the Lagrange multiplier to avoid ﬂoating point errors and reaching trivial solutions, which in our case is taken care by the tolerance α , which, in contrast to λ max , is an interpretable parameter.
Revised: (2021) include a limit λ max to the Lagrange multiplier to avoid ﬂoating point errors and reaching trivial solutions, which in our case is taken care by the tolerance α , which, in contrast to λ max , is an interpretable parameter.

Original: This paper uses the exact expression in (9) for the gradient ∇ w (cid:12)(cid:12)(cid:12) F ( d (cid:48) , w ) n 
Revised: This paper uses the exact expression in (9) for the gradient ∇ w (cid:12)(cid:12)(cid:12) F ( d (cid:48) , w ) n (cid:48)

Original: [Table]
Revised: [Equation]

Original: , while Tran et al.
Revised: , while Tran et al.

Original: (2021) ignore the sign of the difference and use 
Revised: (2021) ignore the sign of the difference and use (cid:12)(cid:12)(cid:12)

Original: [Table]
Revised: [Equation]

Original:  In the next subsection, we extend the algorithm to PFL, which introduces two new differences with (Tran et al., 2021).
Revised: , w ) n (cid:48) [Equation] In the next subsection, we extend the algorithm to PFL, which introduces two new differences with (Tran et al., 2021).

Original: Firstly, the privacy guarantees will be provided for the individuals, and not only to the group to which they belong; and secondly, the algorithm will be tailored to FL.
Revised: Firstly, the privacy guarantees will be provided for the individuals, and not only to the group to which they belong; and secondly, the algorithm will be tailored to FL.

Original: 3. E XTENDING THE A LGORITHM TO P RIVATE F EDERATED L EARNING
Revised: 3. E XTENDING THE A LGORITHM TO P RIVATE F EDERATED L EARNING

Original: In the federated learning setting we now consider that the dataset d is distributed across K users such that each user maintains a local dataset d k with n k samples.
Revised: In the federated learning setting we now consider that the dataset d is distributed across K users such that each user maintains a local dataset d k with n k samples.

Revised: This setting applies to both crossdevice and cross-silo FL, since each sample contains the group information a i .

Original: Nonetheless, as in the central setting, the task is to ﬁnd the model’s parameters w (cid:63) that minimize the loss function across the data samples of the users while enforcing a measure of fairness to the model.
Revised: Nonetheless, as in the central setting, the task is to ﬁnd the model’s parameters w (cid:63) that minimize the loss function across the data samples of the users while enforcing a measure of fairness to the model.

Original: That is, to solve (P2).
Revised: That is, to solve (P2).

Original: To achieve this goal, we might ﬁrst combine the ideas from FederatedSGD (McMahan et al., 2017) and the previous section to extend the developed adaptation of MMDM to FL.
Revised: To achieve this goal, we might ﬁrst combine the ideas from FederatedSGD (McMahan et al., 2017) and the previous section to extend the developed adaptation of MMDM to FL.

Original: Our adaptation of MMDM cannot be combined with FederatedAveraging ; for more details see Appendix A.2.
Revised: Our adaptation of MMDM cannot be combined with FederatedAveraging ; for more details see Appendix A.2.

Original: In order to perform the model updates from (8), the central server requires the following statistics:
Revised: In order to perform the model updates from (8), the central server requires the following statistics:

Original: [Equation]
Revised: [Equation]

Original: However, some of these statistics can be obtained from the others: F ( d (cid:48) , w )
Revised: However, some of these statistics can be obtained from the others: F ( d (cid:48) , w )

Original: = (cid:80) a ∈A F ( d [Equation] a , w ) , ∇ w F ( d (cid:48) , w ) =
Revised: = (cid:80) a ∈A F ( d (cid:48) a , w ) , ∇ w F ( d (cid:48) , w ) =

Original: (cid:80) a ∈A ∇ w F ( d (cid:48) a , w ) , and n (cid:48) =
Revised: (cid:80) a ∈A ∇ w F ( d (cid:48) a , w ) , and n (cid:48) =

Original: (cid:80) a ∈A n (cid:48) a .
Revised: (cid:80) a ∈A n (cid:48) a .

Original: Moreover, as mentioned in the previous section, one might use a sufﬁciently large batch b of the data instead of the full dataset for each update.
Revised: Moreover, as mentioned in the previous section, one might use a sufﬁciently large batch b of the data instead of the full dataset for each update.

Original: Therefore, we consider an iterative algorithm where, at each iteration, the central server samples  m users S that report a vector with the sufﬁcient statistics for the update, that is 
Revised: Therefore, we consider an iterative algorithm where, at each iteration, the central server samples a cohort of m users S that report a vector with the sufﬁcient statistics for the update, that is [Equation]

Original: [Equation]
Revised: [Equation]

Original: This way, if we deﬁne the batch b as the sum of the m users’ local datasets b := (cid:80)
Revised: This way, if we deﬁne the batch b as the sum of the m users’ local datasets b := (cid:80)

Original: k ∈ S
Revised: k ∈ S

Original: d k and the batch b (cid:48) analogously, then the aggregation of each user’s vectors results in
Revised: d k and the batch b (cid:48) analogously, then the aggregation of each user’s vectors results in

Original: [Equation]
Revised: [Equation]

Original:  which contains all the sufﬁcient statistics for the parameters’ update.
Revised: 12) which contains all the sufﬁcient statistics for the parameters’ update.

Original: Finally, the resulting algorithm, termed Fair PFL or FPFL and described in Algorithm 1, inspired by the ideas from (McMahan et al., 2018b; Truex et al., 2019; Granqvist et al., 2020; Bonawitz et al., 2017) guarantees the users’ privacy by (i) making sure the aggregation of the users’ sent vectors is done securely with secure aggregation (Bonawitz et al., 2017), and (ii) clipping each users’ vectors with a clipping bound C and employing the Gaussian mechanism to the sum of the clipped vectors (lines 13 and 15).
Revised: Finally, the resulting algorithm, termed Fair PFL or FPFL and described in Algorithm 1, inspired by the ideas from (McMahan et al., 2018b; Truex et al., 2019; Granqvist et al., 2020; Bonawitz et al., 2017) guarantees the users’ privacy by (i) making sure the aggregation of the users’ sent vectors is done securely with secure aggregation (Bonawitz et al., 2017), and (ii) clipping each users’ vectors with a clipping bound C and employing the Gaussian mechanism to the sum of the clipped vectors (lines 13 and 15).

Original: The variance of the Gaussian noise is calculated according to the reﬁned moments accountant privacy analysis from (Wang et al., 2019), taking into account the number of iterations
Revised: The variance of the Gaussian noise is calculated according to the reﬁned moments accountant privacy analysis from (Wang et al., 2019), taking into account the number of iterations

Original: T , the cohort size m , the total number of users (or population size) K , and the privacy parameters (cid:15) and δ .
Revised: T , the cohort size m , the total number of users (or population size) K , and the privacy parameters (cid:15) and δ .

Original: Note that the post-processing property of DP keeps the algorithm private even if the received noisy vector v is processed to extract the relevant information for the model’s update.
Revised: Note that the post-processing property of DP keeps the algorithm private even if the received noisy vector v is processed to extract the relevant information for the model’s update.

Original: Batch size.
Revised: Batch size.

Original: This algorithm can also be employed using only a fraction of the user’s data in each update, i.e. using a batch b k of their local dataset d k .
Revised: This algorithm can also be employed using only a fraction of the user’s data in each update, i.e. using a batch b k of their local dataset d k .

Original: Nonetheless, it is desirable (i) to delegate as much computation to the user as possible and (ii) to use as much users’ data as possible to have a good approximation of the performance metrics F ( d (cid:48) a , w ) , which are needed to enforce fairness.
Revised: Nonetheless, it is desirable (i) to delegate as much computation to the user as possible and (ii) to use as much users’ data as possible to have a good approximation of the performance metrics F ( d (cid:48) a , w ) , which are needed to enforce fairness.

Original:  E XPERIMENTAL R ESULTS
Revised:  E XPERIMENTAL R ESULTS

Original: Algorithm number iterations the bound, and

Original: Server Executes: w 0 , λ 0 ← InitializeParameters () σ ← CalculateNoiseScale ( K, m, ε, δ, T ) for each iteration t = 1 , 2 , . .

Original: , T do

Original: S t ← (random set of m users) v t ←

Original: SecureAggregation ( w t − 1 , S t , C, σ ) λ t ← UpdateMultiplier

Original: v t , w t − 1 ) w t ← UpdateParameters ( v t , λ t ) end

Original: SecureAggregation ( w , S, C, σ ) : for each user k in S do v k ← UserStatistics ( k, w ) v k ← v k · min (cid:8) 1 , C / (cid:107) v k (cid:107) 2 (cid:9) end v ← (cid:80) k ∈ S v k + N (cid:0) 0 , C 2 · σ 2 (cid:1) return v

Original: UserStatistics ( k, w ) : v ← (cid:104) ∇ w L ( w , d k ) , (cid:2) F ( w , d (cid:48) ka ) (cid:3) a ∈A , (cid:2) ∇ w F ( w , d (cid:48) ka ) (cid:3) a ∈A , (cid:2) n (cid:48) ka (cid:3) a ∈A (cid:105) return v

Original: We study the performance of the algorithm in twodifferent classiﬁcation tasks.
Revised: We study the performance of the algorithm in two classiﬁcation tasks.

Original: The ﬁrst task is a binary classiﬁcation based on some demographic data from the publicly available Adult dataset (Dua & Graff, 2017).
Revised: The ﬁrst task is a binary classiﬁcation based on some demographic data from the publicly available Adult dataset (Dua & Graff, 2017).

Original: The fairness metric considered for this task is FNR parity.
Revised: The fairness metric considered for this task is FNR parity.

Original: The second task is a multi-class classiﬁcation where there are three different attributes.
Revised: The second task is a multi-class classiﬁcation where there are three different attributes.

Original: This task considers accuracy parity as the fairness metric and uses a modiﬁcation of the publicly available FEMNIST dataset (Caldas et al., 2018), where only digits are considered and the classes are digits written with a black pen on a white sheet, digits written with a blue pen on a white sheet, and digits written with white chalk on a blackboard (see Figure 1).
Revised: This task considers accuracy parity as the fairness metric and uses a modiﬁcation of the publicly available FEMNIST dataset (Caldas et al., 2018), where only digits are considered and the classes are digits written with a black pen on a white sheet, digits written with a blue pen on a white sheet, and digits written with white chalk on a blackboard (see Figure 1).

Original: For the ﬁrst task we ﬁrst compare the performance of the MMDM algorithm to vanilla SGD centrally.
Revised: For the ﬁrst task, we ﬁrst compare the performance of the MMDM algorithm with BMDM, Tran et al. (2021), and vanilla SGD centrally and without privacy.

Original: After that, for both tasks, we conﬁrm how FederatedSGD deteriorates the performance of the model for the under-represented classes when clipping and noise (DP) are introduced.
Revised: After that, for both tasks, we conﬁrm how FederatedSGD deteriorates the performance of the model for the under-represented classes when clipping and noise (DP) are introduced.

Original: Finally, we demonstrate how FPFL can, under the appropriate circumstances, level the performance of the model across the groups without largely decreasing the overall performance of the model.
Revised: Finally, we demonstrate how FPFL can, under the appropriate circumstances, level the performance of the model across the groups without largely decreasing the overall performance of the model.

Original: We do not directly compare against Tran et al.

Original: (2021), since they do not provide privacy protection on the whole data.

Original: In all our experiments, the fairness metrics are deﬁned as the maximum difference of the value of a performance measure between the general testing data and each of the groups described by that sensitive attribute.
Revised: In all our experiments, the fairness metrics are deﬁned as the maximum difference of the value of a performance measure between the general testing data and each of the groups described by that sensitive attribute.

Original: Moreover, the privacy parameters are ( (cid:15), δ ) =
Revised: Moreover, the privacy parameters are ( (cid:15), δ ) =

Original: (2 , 1 /K ) , where K is the population size.
Revised: (2 , 1 /K ) , where K is the population size.

Original: All experimental details and additional experiments are in Appendix B.
Revised: All experimental details and additional experiments are in Appendix B.

Original: 4. R ESULTS ON THE A DULT DATASET
Revised: 4. R ESULTS ON THE A DULT DATASET

Original: We start our experiments comparing the performance and fairness of a model trained with vanilla
Revised: We start our experiments comparing the performance and fairness in the central, non-federated, nonprivate setting.

Original: SGD and the MMDM adaptation. We trained a shallow network  with these algorithms, where we tried to enforce FNR parity with a tolerance α = 0 . 
Revised: We trained a shallow network (consisting of one hidden layer with 10 hidden units and a ReLU activation function) with these algorithms, where we tried to enforce FNR parity with a tolerance α = 0 . For MMDM, the damping parameter is c = 2 . For Tran et al.

Original:  The results after 1000 iterations are displayed in Table 1, which also shows the gap in other common measures of fairness such as the equalized odds, the demographic parity, or the predictive parity, see e.g. (Castelnovo et al., 2021).
Revised: (2021), the multiplier threshold is λ max = 0 . The results after 1,000 iterations are displayed in Table 1, which also shows the gap in other common measures of fairness such as the equalized odds, the demographic parity, or the predictive parity, see e.g. Castelnovo et al.

Original: The MMDM algorithm reduces the FNR gap from 7% to the targeted 2%, while  reducing the accuracy of the model by less than 0.5%, thus succeeding in its objective.
Revised: Compared to Federated SGD, the MMDM algorithm reduces the FNR gap from 7% to 0.5%, while hardly reducing the accuracy of the model.

Original: Similarly, the gap in the equalized odds, which is a stronger fairness notion than the FNR parity, also decreases from around 7% to 3%.
Revised: Similarly, the gap in the equalized odds (EO), which is a stronger fairness notion than the FNR parity, also decreases from around 7% to 3%.

Original: Moreover, the demographic parity  gap, which considers the probability of predicting one or the other target class, also improves.
Revised: Moreover, the demographic parity (DemP) gap, which considers the probability of predicting one or the other target class, also improves.

Original: In terms of predictive parity, which uses the precision as the performance function, the MMDM algorithm did not improve the parity among groups.
Revised: In terms of predictive parity (PP), which uses the precision as the performance function, the MMDM algorithm did not improve the parity among groups.

Original: [Table]
Revised: The BMDM algorithm, which lacks a quadratic term of the fairness function, performs even slightly better in the central scenario. However, the algorithm from Tran et al. (2021), an approximation to BMDM as discussed in Section 3.1, gives only a small improvement.

Original: [Table]
Revised: Therefore, we continue our analysis to the federated and private settings only with the BMDM and MMDM algorithms.

Original: The second experiment is to study how clipping and DP decreases the performance for the underrepresented groups, thus increasing the fairness gap on the different fairness metrics.
Revised: The second experiment is to study how federated learning, clipping, and DP decrease the performance for the under-represented groups, thus increasing the fairness gap on the different fairness metrics.

Original: For that, we trained the sameshallow network with FederatedSGD and versions of this algorithm where only clipping was performed and where both clipping and DP where included.
Revised: For that, we trained the same network with FederatedSGD and versions of this algorithm where only clipping was performed and where both clipping and DP where included.

Original: They were trained with a cohort size of m = 200 for T = 1000 iterations and the model with best training cohort accuracy was selected.
Revised: They were trained with a cohort size of m = 200 for T = 1 , 000 iterations and the model with best training cohort accuracy was selected.

Revised: The privacy parameters are ( (cid:15), δ ) =

Revised: (2 , 5 · 10 − 5 ) .

Original: The results  are displayed in Table 2.
Revised: The results [Table] [Table] are displayed in Table 2.

Original: Theshallow network becomes less fair under all the metrics considered, compared with the central setting.
Revised: The network becomes less fair under all the metrics considered, compared with the central setting.

Original: Moreover, the introduction of clipping largely increases the unfairness of the models reaching more than a 16% gap in FNR.
Revised: Moreover, the introduction of clipping largely increases the unfairness of the models reaching more than a 16% gap in FNR.

Original: The addition of DP  does not have a larger effect on the unfairness of the models.
Revised: The addition of DP requires an increase in cohort size from 200 to 1 , 000 , but does not have a larger effect on the unfairness of the models (see Tablein Appendix B.1 for the details).

Original: These observations are in line with (Bagdasaryan et al., 2019), where they note that the under-represented groups usually have the higher loss gradients and thus clipping affects them more than the majority groups.
Revised: These observations are in line with (Bagdasaryan et al., 2019), where they note that the under-represented groups usually have the higher loss gradients and thus clipping affects them more than the majority groups.

Original: After that, we repeated the above experiment with FPFL .
Revised: After that, we repeated the above experiment with FPFL and BMDM with DP.

Original: FPFL  converged to a solution faster than FederatedSGD , and thus the models were trained for only T = 250 iterations.
Revised: FPFL and BMDM with DP converged to a solution faster than FederatedSGD , and thus the models were trained for only T = 250 iterations.

Original: Here the model with the best training cohort accuracy while respecting the fairness condition on the training cohort data was selected.
Revised: Here the model with the best training cohort accuracy while respecting the fairness condition on the training cohort data was selected.

Original: Note that the fairness condition is evaluated with noisy statistics, so a model may be deemed as fair while slightly violating the desired constraints.
Revised: Note that the fairness condition is evaluated with noisy statistics, so a model may be deemed as fair while slightly violating the desired constraints.

Original: The results are also included in Table 2 to aid the comparison.
Revised: The results are also included in Table 2 to aid the comparison.

Original: We note how, similarly to the central case, the model trained with FPFL manages to enforce the fairness constraints while keeping a similar accuracy.
Revised: We note how, similarly to the central case, the model trained with FPFL manages to enforce the fairness constraints while keeping a similar accuracy.

Original: Then, clipping does not seem to affect largely the performance of FPFL since it compensates the gradient loss clipping with the fairness enforcement. Finally, the addition of noise toguarantee DP is not a concern to the shallow network. However, further experiments reveal that the performance can deteriorate for deeper networks (c.f. Appendix B.1). This is largely due to the fact that  the noise is large enough so that sign of the constraints’ gradient, see (9), is sometimes mistaken.
Revised: In contrast, BMDM seems to not be able to ﬁnd network weights that respect the fairness conditions, as expected by the less favorable (“less convex”) loss surface around these solutions.

Original: In practice, this can be mitigated with larger cohort sizes, which reduce the relative noise thus increasing the signal-to-DP noise ratio in the training (c.f. Appendix B.1).
Revised: Moreover, clipping does not seem to affect largely the performance of FPFL since it compensates the gradient loss clipping with the fairness enforcement.

Original: 4. R ESULTS ON THE MODIFIED FEMNIST DATASET
Revised: 4. R ESULTS ON THE UNFAIR FEMNIST DATASET

Original: We start our experiments conﬁrming again the hypothesis and ﬁndings from Bagdasaryan et al.
Revised: We start our experiments conﬁrming again the hypothesis and ﬁndings from Bagdasaryan et al.

Original: (2019) that clipping and DP disproportionately affect under-represented groups.
Revised: (2019) that clipping and DP disproportionately affect under-represented groups.

Original: For that, we trained a convolutional network  with FederatedSGD and versions of this algorithm where only clipping was performed and where both clipping and DP were included. 
Revised: For that, we trained a convolutional network (consisting of 2 convolution layers with kernel of size 5 × 5 , stride of 2, ReLU activation function, and 32 and 64 ﬁlters respectively) with FederatedSGD and versions of this algorithm where only clipping was performed and where both clipping and DP were included. For the FEMNIST experiments, the privacy parameters are ( (cid:15), δ ) = (2 , 2 . 5 · 10 − 4 ) and the damping parameter is c = 20 .

Original: They were trained with a cohort size of m = 100 for T = 2000 iterations and the model with best training cohort accuracy was selected.
Revised: They were trained with a cohort size of m = 100 for T = 2 , 000 iterations and the model with best training cohort accuracy was selected.

Original: The results are displayed in Table 3.
Revised: The results are displayed in Table 3.

Original: Similarly to before, clipping increases the accuracy gap from 13% to almost 17%.
Revised: Similarly to before, clipping increases the accuracy gap from 13% to almost 17%.

Original: In this case, since the number of users K is small, the DP noise is large compared to the users’ statistics.
Revised: In this case, since the number of users K is small, the DP noise is large compared to the users’ statistics.

Original: Therefore, theaccuracy drops from more than 94% with clipping to 80.7% when DP is also used, and the accuracy gap increases to more than 40%. [Table]
Revised: Therefore, the

Original: [Table]
Revised: [Table]

Original: The second experiment tests whether FPFL can remedy the unfairness without decreasing accuracy too much.
Revised: The second experiment tests whether FPFL can remedy the unfairness without decreasing accuracy too much.

Original: We trained the same convolutional network, again for T = 2000 iterations, and selected the model with the best training cohort accuracy that respected the fairness condition on the training cohort.
Revised: We trained the same convolutional network, again for T = 2 , 000 iterations, and selected the model with the best training cohort accuracy that respected the fairness condition on the training cohort.

Original: When DP noise is not included, FPFL reduces the accuracy gap with FederatedSGD by around 9% while keeping the accuracy within 1%.
Revised: When DP noise is not included, FPFL reduces the accuracy gap with FederatedSGD by around 9% while keeping the accuracy within 1%.

Original: We note how, as before, clipping largely does not affect the ability of FPFL to enforce fairness.
Revised: We note how, as before, clipping largely does not affect the ability of FPFL to enforce fairness.

Original: However, note that since the data is more non-i.i.d.
Revised: However, note that since the data is more non-i.i.d.

Original: than before (i.e. there are more differences between the distribution of each user)
Revised: than before (i.e. there are more differences between the distribution of each user)

Original: the models that are deemed fair in the training cohort may not be as fair in the general population, and now we see a larger gap between the desired tolerance α = 0 .
Revised: the models that are deemed fair in the training cohort may not be as fair in the general population, and now we see a larger gap between the desired tolerance α = 0 .

Original: 04 and the obtained accuracy gap from FPFL without noise (0.047 and 0.
Revised: 04 and the obtained accuracy gap from FPFL without noise (0.047 and 0.

Original: without and with clipping).
Revised: without and with clipping).

Original: When DP is used, the noise is too large for
Revised: When DP is used, the noise is too large for

Original: FPFL to function properly and often the sign of the constraints’ gradient, see (9), ﬂips.
Revised: FPFL to function properly and often the sign of the constraints’ gradient, see (9), ﬂips.

Original: Note that in the estimation of the performance function, i.e. F ( d a , w ) /n
Revised: Note that in the estimation of the performance function, i.e. F ( d a , w ) /n

Original: a , both the numerator and denominator are obtained from a noisy vector, thus making the estimators more sensitive to noise than the estimators for plain FederatedSGD .
Revised: a , both the numerator and denominator are obtained from a noisy vector, thus making the estimators more sensitive to noise than the estimators for plain FederatedSGD .

Revised: This is due to two main factors: (i) the larger the model, the larger the aggregation of the gradients for each weight, and thus more noise needs to be added; and (ii) the smaller the amount of users, the larger the noise that needs to be added.

Original: For this reason, we considered the  scenario where the population, used for calculating the DP noise, is 100 and 1000 times larger, which is a conservative assumption for federated learning deployments (Apple, 2017).
Revised: For this reason, we considered the hypothetical scenario where the population, used for calculating the DP noise, is 100 and 1,000 times larger, which is a conservative assumption for federated learning deployments (Apple, 2017).

Original: Then, we repeated the experiment with DP FederatedSGD and FPFL and increased the cohort size to m = 2000 .
Revised: Then, we repeated the experiment with DP FederatedSGD and FPFL and increased the cohort size to m = 2 , 000 .

Original: In this scenario, DP FederatedSGD maintained an accuracy gap of more than 15% while FPFL reduced this gap to less than a half in both cases.
Revised: In this scenario, DP FederatedSGD maintained an accuracy gap of more than 15% while FPFL reduced this gap to less than a half in both cases.

Original: Nonetheless, the accuracy still decreases slightly, with a reduction of around 5% and 2% respectively compared with DP FederatedSGD .
Revised: Nonetheless, the accuracy still decreases slightly, with a reduction of around 5% and 2% respectively compared with DP FederatedSGD .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we studied and proposed a solution to the problem of group fairness in private federated learning.
Revised: In this paper, we studied and proposed a solution to the problem of group fairness in private federated learning.

Original: For this purpose, we adapt the modiﬁed method of multipliers (MMDM)
Revised: For this purpose, we adapt the modiﬁed method of multipliers (MMDM)

Original: (Platt & Barr, 1987) to empirical loss minimization with fairness constraints, producing an algorithm for enforcing fairness in central learning.
Revised: (Platt & Barr, 1987) to empirical loss minimization with fairness constraints, producing an algorithm for enforcing fairness in central learning.

Original: Then, we extend this algorithm to private federated learning.
Revised: Then, we extend this algorithm to private federated learning.

Original: Through experiments in the Adult (Dua & Graff, 2017) and a modiﬁed version of the FEMNIST (Caldas et al., 2018) datasets, we ﬁrst conﬁrm previous knowledge that DP disproportionately affects performance on under-represented groups (Bagdasaryan et al., 2019), with the further observation that this is true for many different fairness metrics, and not just for accuracy parity.
Revised: Through experiments in the Adult (Dua & Graff, 2017) and a modiﬁed version of the FEMNIST (Caldas et al., 2018) datasets, we ﬁrst conﬁrm previous knowledge that DP disproportionately affects performance on under-represented groups (Bagdasaryan et al., 2019), with the further observation that this is true for many different fairness metrics, and not just for accuracy parity.

Original: The proposed FPFL algorithm is able to remedy this unfairness even in the presence of DP.
Revised: The proposed FPFL algorithm is able to remedy this unfairness even in the presence of DP.

Original: Limitations.
Revised: Limitations and future work.

Original: The FPFL algorithm is more sensitive to DP noise than other algorithms for PFL.
Revised: The FPFL algorithm is more sensitive to DP noise than other algorithms for PFL.

Original: This requires increasing the cohort size or ensuring that enough users take part in training.
Revised: This requires increasing the cohort size or ensuring that enough users take part in training.

Original: Still, for the experiments, the number of users required is still lower (more than an order of magnitude) than the usual amount of users available in deployments of federated learning settings (Apple, 2017).
Revised: Still, for the experiments, the number of users required is still lower (more than an order of magnitude) than the usual amount of users available in deployments of federated learning settings (Apple, 2017).

Revised: In the future, we intend to scale the algorithm to larger models and empirically integrate it with faster training methods like Federated Averaging (McMahan et al., 2017).

Original: A A DDITIONAL DETAILS ABOUT FPFL A. D IFFERENTIABLE ESTIMATES OF THE FUNCTION AGGREGATES
Revised: A A DDITIONAL DETAILS ABOUT FPFL A. D IFFERENTIABLE ESTIMATES OF THE FUNCTION AGGREGATES

Original: As mentioned in Section 3.
Revised: As mentioned in Section 3.

Original: [Equation]
Revised: [Equation]

Original: there are many situations of interest when the function f employed to describe the fairness metric is not differentiable.
Revised: there are many situations of interest when the function f employed to describe the fairness metric is not differentiable.

Original: In these cases
Revised: In these cases

Original: [Equation]
Revised: [Equation]

Original: the gradient ∇ w F ( d (cid:48) , w ) does not exist and we resort to estimate the gradient ∇ w F ( d (cid:48) , w ) using a differentiable estimation of the function aggregate F ( d (cid:48) , w ) .
Revised: the gradient ∇ w F ( d (cid:48) , w ) does not exist and we resort to estimate the gradient ∇ w F ( d (cid:48) , w ) using a differentiable estimation of the function aggregate F ( d (cid:48) , w ) .

Original: As an example
Revised: As an example

Original: [Equation]
Revised: [Equation]

Original: when trying to enforce FNR parity or accuracy parity (as is the case in our experiments from Section 4 and Appendix B) the employed estimates are:
Revised: when trying to enforce FNR parity or accuracy parity (as is the case in our experiments from Section 4 and Appendix B) the employed estimates are:

Original: • Enforcing FNR parity on a neural network ψ w (with a Sigmoid output activation function) in a binary classiﬁcation task.
Revised: • Enforcing FNR parity on a neural network ψ w (with a Sigmoid output activation function) in a binary classiﬁcation task.

Original: We note that given an input x i , the raw output of the network ψ w ( x i ) is an estimate of the probability that ˆ y i = 1 .
Revised: We note that given an input x i , the raw output of the network ψ w ( x i ) is an estimate of the probability that ˆ y i = 1 .

Original: Hence, the function aggregate can be estimated as
Revised: Hence, the function aggregate can be estimated as

Original: [Equation]
Revised: [Equation]

Original: • Enforcing accuracy parity on a neural network ψ w (with softmax output activation function) in a multi-class classiﬁcation task.
Revised: • Enforcing accuracy parity on a neural network ψ w (with softmax output activation function) in a multi-class classiﬁcation task.

Original: We note that given an input x
Revised: We note that given an input x

Original: i , the raw j th output of the network ψ w ( x i )
Revised: i , the raw j th output of the network ψ w ( x i )

Original: j is an estimate of the probability that ˆ y i =
Revised: j is an estimate of the probability that ˆ y i =

Original: j .
Revised: j .

Original: Hence, if y i the one-hot encoding vector of y i , the function aggregate can be estimated as
Revised: Hence, if y i the one-hot encoding vector of y i , the function aggregate can be estimated as

Original: [Equation]
Revised: [Equation]

Original: A. W HY IS F EDERATED A VERAGING INCOMPATIBLE WITH FPFL ?
Revised: A. W HY IS F EDERATED A VERAGING INCOMPATIBLE WITH FPFL ?

Original: The proposed algorithm extends the MMDM algorithm from Section 3.1 to FL adapting
Revised: The proposed algorithm extends the MMDM algorithm from Section 3.1 to FL adapting

Original: FederatedSGD , where each user uses all their data, computes the necessary statistics for a model update, and sends them to the third-party.
Revised: FederatedSGD , where each user uses all their data, computes the necessary statistics for a model update, and sends them to the third-party.

Original: A natural question could be why is not FederatedAveraging adapted instead.
Revised: A natural question could be why is not FederatedAveraging adapted instead.

Original: That is, to perform several stochastic gradient descent/ascent updates of the model’s parameters w and the
Revised: That is, to perform several stochastic gradient descent/ascent updates of the model’s parameters w and the

Original: Lagrange multipliers λ , and send the difference of the updated and the original version, i.e. to send the vector v k :=
Revised: Lagrange multipliers λ , and send the difference of the updated and the original version, i.e. to send the vector v k :=

Original: [ w k − w , λ k − λ ] .
Revised: [ w k − w , λ k − λ ] .

Original: This way, the size of the communicated vector would be reduced from ( |A| + 1) p + 2 |A| to just p + |A| and a larger part of the computation would be done locally, increasing the convergence speed.
Revised: This way, the size of the communicated vector would be reduced from ( |A| + 1) p + 2 |A| to just p + |A| and a larger part of the computation would be done locally, increasing the convergence speed.

Original: Moreover, the clipping bound could be reduced, thus decreasing the noise necessary for the DP guarantees.
Revised: Moreover, the clipping bound could be reduced, thus decreasing the noise necessary for the DP guarantees.

Original: Unfortunately, for the proposed MMDM algorithm, this option could lead to catastrophic effects.
Revised: Unfortunately, for the proposed MMDM algorithm, this option could lead to catastrophic effects.

Original: Imagine for instance a situation where each user only has data points belonging to one group, say a = 0 or a = 1 .
Revised: Imagine for instance a situation where each user only has data points belonging to one group, say a = 0 or a = 1 .

Original: Then in their local dataset the general population is equivalent to the population of their group and thus F ( d k (cid:48) , w ) =
Revised: Then in their local dataset the general population is equivalent to the population of their group and thus F ( d k (cid:48) , w ) =

Original: F ( d ka (cid:48) , w ) , implying that locally g ( w ) = 0 .
Revised: F ( d ka (cid:48) , w ) , implying that locally g ( w ) = 0 .

Original: Therefore, the
Revised: Therefore, the

Original: Lagrange multipliers will never be locally updated and the weights updates will be equivalent to those updates without considering the fairness constraints.
Revised: Lagrange multipliers will never be locally updated and the weights updates will be equivalent to those updates without considering the fairness constraints.

Original: That is, using this approach one would (i) recover the same algorithm than standard PFL and (ii) communicate a vector of size p + |A| instead of size p .
Revised: That is, using this approach one would (i) recover the same algorithm than standard PFL and (ii) communicate a vector of size p + |A| instead of size p .

Original: B E XPERIMENTAL DETAILS AND ADDITIONAL EXPERIMENTS B. D ETAILS OF THE EXPERIMENTS AND ADDITIONAL EXPERIMENTS ON A DULT B.1. D ETAILS OF THE EXPERIMENTS
Revised: A. T HE ALGORITHM

Revised: The employed algorithm is detailed in algorithm 1. It is separated into three main parts: Server :

Revised: The server initializes the network w 0 and Lagrange multiplier λ 0 parameters

Revised: and also calculates the noise scale σ using Wang et al.

Revised: (2019)’s privacy analysis before training.

Revised: Then, for each iteration t it samples m users S t and asks for the noisy aggregation of their statistics v t .

Revised: Then it updates the Lagrange multiplier λ t and the network weights w t following (8). 2.

Revised: Virtual secure third

Revised: -party (obtained via secure multi-party computation) :

Revised: The third party asks for each users’ statistics v k and clips them with the clipping bound C .

Revised: After that, it aggregates the clipped statistics and privatizes the aggregation with the Gaussian mechanism using the previously calculated by the server noise scale σ .

Revised: Users :

Revised: Each user calculates the relevant statistics, see (11), and sends them to the secure virtual third-party

Revised: Server Executes: w 0 , λ 0 ← InitializeParameters () σ ← CalculateNoiseScale ( K, m, ε, δ, T ) for each iteration t = 1 , 2 , . .

Revised: , T do

Revised: S t ← (random set of m users) v t ← SecureAggregation ( w t − 1 , S t , C, σ )

Revised: λ t ← UpdateMultiplier ( v t , w t − 1 ) w t ← UpdateParameters ( v t , λ t ) end

Revised: SecureAggregation ( w , S, C, σ ) : // Performed by the virtual secure third-party for each user k in S do v k ← UserStatistics ( k, w ) v k ← v k · min (cid:8) 1 , C / (cid:107) v k (cid:107) 2 (cid:9) end v ← (cid:80) k ∈ S v k + N (cid:0) 0 , C 2 · σ 2 (cid:1) return v

Revised: UserStatistics ( k, w ) : // Performed by the user v ← (cid:104) ∇ w L ( w , d k ) , (cid:2) F ( w , d (cid:48) ka ) (cid:3) a ∈A , (cid:2) ∇ w F ( w , d (cid:48) ka ) (cid:3) a ∈A , (cid:2) n (cid:48) ka (cid:3) a ∈A (cid:105) return v

Revised: B E XPERIMENTAL DETAILS AND ADDITIONAL EXPERIMENTS B. D ETAILS OF THE EXPERIMENTS AND ADDITIONAL EXPERIMENTS ON A

Revised: DULT

Revised: B.1. D ETAILS OF THE EXPERIMENTS

Original: Adult dataset, from the UCI Machine Learning Repository (Dua & Graff, 2017).
Revised: Adult dataset, from the UCI Machine Learning Repository (Dua & Graff, 2017).

Original: This dataset consists of 32,561 training and 16,281 testing samples of demographic data from the US census.
Revised: This dataset consists of 32,561 training and 16,281 testing samples of demographic data from the US census.

Original: Each datapoint contains various demographic attributes.
Revised: Each datapoint contains various demographic attributes.

Original: Though the particular task, predicting individuals’ salary ranges, is not itself of interest, this dataset serves as a proxy for tasks with inequalities in the data.
Revised: Though the particular task, predicting individuals’ salary ranges, is not itself of interest, this dataset serves as a proxy for tasks with inequalities in the data.

Original: A reason this dataset is often used in the literature on ML fairness is that the fraction of individuals in the higher salary range is 30% for the men and only 10% for the women.
Revised: A reason this dataset is often used in the literature on ML fairness is that the fraction of individuals in the higher salary range is 30% for the men and only 10% for the women.

Original: The experiments in this paper will aim to stop this imbalance from entering into the model by balancing the false negative rate (Castelnovo et al., 2021).
Revised: The experiments in this paper will aim to stop this imbalance from entering into the model by balancing the false negative rate (Castelnovo et al., 2021).

Original: Federated Adult dataset.
Revised: Federated Adult dataset.

Original: To generate a version of the Adult dataset suitable for federated learning, it must be partitioned into individual contributions.
Revised: To generate a version of the Adult dataset suitable for federated learning, it must be partitioned into individual contributions.

Original: In the experiments, differential privacy will be guaranteed per contribution.
Revised: In the experiments, differential privacy will be guaranteed per contribution.

Original: In this paper, the number of datapoints per contribution is Poissondistributed with mean of 2.
Revised: In this paper, the number of datapoints per contribution is Poissondistributed with mean of 2.

Revised: Therefore, the number of users is K ≈ n/ 2 ≈ 16 , 280 . 5 .

Original: Privacy and fairness parameters.
Revised: Privacy and fairness parameters.

Original: For all our experiments, we considered the privacy parameters (cid:15) =
Revised: For all our experiments, we considered the privacy parameters (cid:15) =

Original: and δ =1 /K ≈ 5 · 10 − 5  and the fairness tolerance α = 0 .
Revised: and δ = 5 · 10 − 5 < 1 /K and the fairness tolerance α = 0 .

Original: Data pre-processing.
Revised: Data pre-processing.

Original: The 7 categorical variables were one-hot encoded and the 6 numerical variables where normalized with the training mean and variance.
Revised: The 7 categorical variables were one-hot encoded and the 6 numerical variables where normalized with the training mean and variance.

Original: There is an underlying assumption that these means and variances can be learned at a low privacy cost.
Revised: There is an underlying assumption that these means and variances can be learned at a low privacy cost.

Original: Hence, to be precise, the private models are ( (cid:15) 

Revised: Hence, to be precise, the private models are ( (cid:15)

Original: + 2 , 5 · 10 − 5 ) -DP, where (cid:15) 0 is a small constant representing the privacy budget for learning said parameters for the normalization.
Revised: + 2 , 5 · 10 − 5 ) -DP, where (cid:15) 0 is a small constant representing the privacy budget for learning said parameters for the normalization.

Original: Models considered.
Revised: Models considered.

Original: We experimented with two different fully connected networks.
Revised: We experimented with two different fully connected networks.

Original: The ﬁrst network, from now on the shallow network, has one hidden layer with 10 hidden units and a ReLU activation function.
Revised: The ﬁrst network, from now on the shallow network, has one hidden layer with 10 hidden units and a ReLU activation function.

Original: The second network, henceforth the deep network, has three hidden layers with 16, 8, and 8 hidden units respectively and all with ReLU activation functions.
Revised: The second network, henceforth the deep network, has three hidden layers with 16, 8, and 8 hidden units respectively and all with ReLU activation functions.

Original: Both networks ended with a fully connected layer to a ﬁnal output unit with a Sigmoid activation function.
Revised: Both networks ended with a fully connected layer to a ﬁnal output unit with a Sigmoid activation function.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: For all the experiments, the learning rate was η = 0 .
Revised: For all the experiments, the learning rate was η = 0 .

Original: 1 for the network parameters and γ = 0 .
Revised: 1 for the network parameters and γ = 0 .

Original: 01 for the Lagrange multipliers.
Revised: 01 for the Lagrange multipliers.

Original: The damping parameter was c = 2 .
Revised: The damping parameter was c = 2 .

Original: The batch size for the experiments learned centrally was n batch =
Revised: The batch size for the experiments learned centrally was n batch =

Original: 400 and the cohort sized studied for the federated experiments were m = 200 and m = 1000 .
Revised: 400 and the cohort sized studied for the federated experiments were m = 200 and m = 1 , 000 .

Original: Finally, the clipping bounds for the shallow and the deep networks were, respectively and depending if the training algorithm was PFL or FPFL ,
Revised: Finally, the clipping bounds for the shallow and the deep networks were, respectively and depending if the training algorithm was PFL or FPFL ,

Original: C = 1 . 3 or C = 2 and C = 2 or C = 2 . 4 .
Revised: C = 1 . 3 or C = 2 and C = 2 or C = 2 . 4 .

Original: These hyper-parameters where not selected with a private hyper-parameter search and were just set as an exemplary conﬁguration.
Revised: These hyper-parameters where not selected with a private hyper-parameter search and were just set as an exemplary conﬁguration.

Revised: For Tran et al.

Revised: (2021), we chose the multiplier threshold λ max = 0 . 05 by sweeping over multiple orders of magnitude and ﬁnding the optimal accuracy and FNR gap.

Original: If one desires to ﬁnd the best hyper-parameters, one can do so at an additional privacy budget cost following e.g. (Abadi et al., 2016, Appendix D).
Revised: If one desires to ﬁnd the best hyper-parameters, one can do so at an additional privacy budget cost following e.g. Abadi et al. (2016, Appendix D), Liu & Talwar (2019).

Original: B.1. A DDITIONAL EXPERIMENTS
Revised: B.1. A DDITIONAL EXPERIMENTS

Original: We replicated all the experiments with the shallow network from Section 4.1 with the deep network instead.
Revised: We replicated all the experiments with the shallow network from Section 4.1 with the deep network instead.

Original: The results for these experiments on the central and federated setting are displayed in Table 4 and Table 5, respectively.
Revised: The results for these experiments on the central and federated setting are displayed in Table 4 and Table 5, respectively.

Original: The results obtained with the deep network are almost identical to those of the shallow network in the central setting without DP noise.
Revised: The results obtained with the deep network are almost identical to those of the shallow network in the central setting without DP noise.

Original: The ﬁrst difference with the results for the shallow network is that the performance and fairness of the deep network does not change much when going from the central to the federated setting without DP noise nor clipping.
Revised: The ﬁrst difference with the results for the shallow network is that the performance and fairness of the deep network does not change much when going from the central to the federated setting without DP noise nor clipping.

Original: The shallow network, however, becomes less fair under all the metrics considered.
Revised: The shallow network, however, becomes less fair under all the metrics considered.

Original: The results for federated learning with clipping do not differ much between the shallow and deep networks.
Revised: The results for federated learning with clipping do not differ much between the shallow and deep networks.

Original: In both cases we see how FederatedAveraging with clipping deteriorates the fairness of the model with all the measures considered.
Revised: In both cases we see how FederatedAveraging with clipping deteriorates the fairness of the model with all the measures considered.

Original: Moreover, they also suggest how FPFL can mitigate this problem, maintaining similar levels of accuracy (in this case, even higher) while keeping the fairness below the fairness tolerance.
Revised: Moreover, they also suggest how FPFL can mitigate this problem, maintaining similar levels of accuracy (in this case, even higher) while keeping the fairness below the fairness tolerance.

Original: Another contrast with the shallow network appears in the comparison of PFL and FPFL .
Revised: Another contrast with the shallow network appears in the comparison of algorithms with and without differential privacy.

Original: In this case, even though both models maintain the fairness metrics below the tolerance level, the accuracy performance of the deep network is more deteriorated than the shallow network, with a gap in performance between the model trained with PFL and FPFL of around 4%.
Revised: With DP, training does not reliably converge.

Original: This is largely due to the fact that the noise is large enough so that sign of the constraints’ gradient, see (9), is sometimes mistaken.
Revised: This is partly due to the fact that the noise is large enough so that sign of the constraints’ gradient, see (9), is sometimes mistaken.

Original: For this reason, we repeat the experiments with PFL and FPFL with a larger cohort size, m = 1000 , to see if a smaller relative noise would aid the training with PFL or with FPFL .
Revised: For this reason, we repeat the experiments with PFL and FPFL with a larger cohort size, m = 1 , 000 , to see if a smaller relative noise would aid the training with PFL or with FPFL .

Original: The results with PFL were almost identical, with similar levels of accuracy and unfairness.
Revised: The results with PFL were almost identical, with similar levels of accuracy and unfairness.

Original: On the other hand, the larger signal-to-DP noise ratio helped the models trained with FPFL to keep models with the desired levels of FNR gap and lower unfairness measured with any other metric.
Revised: On the other hand, the larger signal-to-DP noise ratio helped the models trained with FPFL to keep models with the desired levels of FNR gap and lower unfairness measured with any other metric.

Original: Moreover, the accuracy of the models, that now work better for the under-represented group, is in fact slightly higher than for the models trained with PFL.
Revised: Moreover, the accuracy of the models, that now work better for the under-represented group, is in fact slightly higher than for the models trained with PFL.

Revised: B. D ETAILS OF THE EXPERIMENTS ON THE U NFAIR FEMNIST

Revised: FEMNIST dataset (Caldas et al., 2018).

Revised: This dataset is an adaptation of the Extended MNIST dataset (Cohen et al., 2017), which collects more than 800,000 samples of digits and letters distributed across 3,550 users.

Revised: The task considered is to predict which of the 10 digits or 26 letters (upper or lower case) is depicted in the image, so it is a multi-class classiﬁcation with 62 possible classes.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Shallow
Revised: Shallow

Original: FL 0.
Revised: BMDM-FL 0.

Original: Deep
Revised: Shallow

Original: FFL 0.
Revised: FFL 0.

Original: Shallow
Revised: Deep

Original: FFL 0.
Revised: FL 0.

Revised: Deep

Revised: BMDM-FL 0.

Original: Deep
Revised: Deep

Revised: FFL 0.

Revised: Shallow

Original: FL + Clip 0.
Revised: FL + Clip 0.

Revised: Shallow

Revised: BMDM-FL + Clip 0.

Original: Shallow
Revised: Shallow

Original: FL + Clip 0.
Revised: FFL + Clip 0.

Original: Deep
Revised: Deep

Original: FFL + Clip 0.
Revised: FL + Clip 0.

Original: Shallow
Revised: Deep

Original: FFL + Clip 0.
Revised: BMDM-FL + Clip 0.

Original: Deep
Revised: Deep

Original: PFL 0.
Revised: FFL + Clip 0.

Original: Shallow
Revised: Shallow

Original: PFL 0.
Revised: PFL 0.

Original: Deep
Revised: Shallow

Original: FPFL 0.
Revised: BMDM-PFL 0.

Original: Shallow
Revised: Shallow

Original: FPFL 0.
Revised: FPFL 0.

Original: m =

Original: Deep
Revised: Deep

Original: PFL 0.
Revised: PFL 0.

Original: Shallow

Original: PFL 0.

Original: Deep
Revised: Deep

Original: FPFL 0.
Revised: BMDM-PFL 0.

Original: Shallow
Revised: Deep

Original: FPFL 0.
Revised: FPFL 0.

Original: FEMNIST dataset (Caldas et al., 2018). This dataset is an adaptation of the Extended MNIST dataset (Cohen et al., 2017), which collects more than 800,000 samples of digits and letters distributed across 3,550 users. The task considered is to predict which of the 10 digits or 26 letters (upper or lower case) is depicted in the image, so it is a multi-class classiﬁcation with 62 possible classes.
Revised: [Table]

Original: Unfair FEMNIST dataset.
Revised: Unfair FEMNIST dataset.

Original: We considered the FEMNIST dataset with only the digit samples.
Revised: We considered the FEMNIST dataset with only the digit samples.

Original: This restriction consists of 3,383 users spanning 343,099 training and 39,606 testing samples.
Revised: This restriction consists of 3,383 users spanning 343,099 training and 39,606 testing samples.

Original: The task now is to predict which of the 10 digits is depicted in the image, so it is a multi-class classiﬁcation with 10 possible classes.
Revised: The task now is to predict which of the 10 digits is depicted in the image [Table] so it is a multi-class classiﬁcation with 10 possible classes.

Original: Since the dataset does not contain clear sensitive groups, we artiﬁcially create three classes (see Figure 1):
Revised: Since the dataset does not contain clear sensitive groups [Table] we artiﬁcially create three classes (see Figure 1):

Original: • Users that write with a black pen in a white sheet.
Revised: • Users that write with a black pen in a white sheet.

Original: These users represent the ﬁrst (lexicographical) 45% of the users, i e. (cid:98) 0 .
Revised: These users represent the ﬁrst (lexicographical) 45% of the users, i.e. (cid:98) 0 .

Original: 45 · 3 , 383 (cid:99) = 1 , 522 users.
Revised: 45 · 3 , 383 (cid:99) = 1 , 522 users.

Original: These users contain 146, (42.7%) training and 16,689 (42.1%) testing samples.
Revised: These users contain 146, (42.7%) training and 16,689 (42.1%) testing samples.

Original: The images belonging to this group are unchanged.
Revised: The images belonging to this group are unchanged.

Original: • Users that write with a blue pen in a white sheet.
Revised: • Users that write with a blue pen in a white sheet.

Original: These users represent the second (lexicographical) 45% of the users,
Revised: These users represent the second (lexicographical) 45% of the users,

Original: i
Revised: i

Original: e. 1,522 users as well.
Revised: e. 1,522 users as well.

Original: These users contain 159,902 (46.6%) training and 18,672 (47.1%) testing samples. 
Revised: These users contain 159,902 (46.6%) training and 18,672 (47.1%) testing samples. The

Original: The images belonging to this group are modiﬁed making sure that the digit strokes are
Revised: images belonging to this group are modiﬁed making sure that the digit strokes are

Original: blue instead of black.
Revised: blue instead of black.

Original: • Users that write with white chalk in a blackboard.
Revised: • Users that write with white chalk in a blackboard.

Original: These users represent the last remaining 10% of the users, i e. 339 users.
Revised: These users represent the last remaining 10% of the users, i.e. 339 users.

Original: These users contain 36,643 (10.7%) training and 4, (10.7%) testing samples.
Revised: These users contain 36,643 (10.7%) training and 4, (10.7%) testing samples.

Original: The images belonging to this group are modiﬁed making sure that the digit strokes are white and the background is black.
Revised: The images belonging to this group are modiﬁed making sure that the digit strokes are white and the background is black.

Original: Moreover, to make the task more unfair, we simulated the blurry effect that chalk leaves in a blackboard.
Revised: Moreover, to make the task more unfair, we simulated the blurry effect that chalk leaves in a blackboard.

Original: With this purpose, we added Gaussian blurred noise to the image, and then we blended them with further Gaussian blur.
Revised: With this purpose, we added Gaussian blurred noise to the image, and then we blended them with further Gaussian blur.

Original: To be precise, if x is the image normalized to [0 , 1] , the blackboard effect is the following.
Revised: To be precise, if x is the image normalized to [0 , 1] , the blackboard effect is the following.

Original: [Equation]
Revised: [Equation]

Original: 15) where ξ 1 ∼ N (0 , I ) is Gaussian noise of the size of the image, κ 1 and κ 2 are Gaussian kernels 1 with standard deviation 1 and 2, respectively, and (cid:126) represents the convolution operation.
Revised: where ξ 1 ∼ N (0 , I ) is Gaussian noise of the size of the image, κ 1 and κ 2 are Gaussian kernels 1 with standard deviation 1 and 2, respectively, and (cid:126) represents the convolution operation.

Original: Moreover, the images are rotated 90 degrees, simulating how the pictures were taken with the device in horizontal mode due to the usual shape of the blackboards.
Revised: Moreover, the images are rotated 90 degrees, simulating how the pictures were taken with the device in horizontal mode due to the usual shape of the blackboards.

Original: Privacy and fairness parameters.
Revised: Privacy and fairness parameters.

Original: For all our experiments, we considered the privacy parameters (cid:15) =
Revised: For all our experiments, we considered the privacy parameters (cid:15) =

Original: and δ =1 /K ≈ 2 . 5 · 10 − 4  .
Revised: and δ = 2 . 5 · 10 − 4 < 1 /K .

Original: However, for the last experiment, we consider the hypothetical scenario where we had a larger number of users K ← 100 K and K ← 1000 K , thus decreasing the privacy parameter to δ ← δ/ 100 and δ ← δ/ 1000 and reducing the added noise in the analysis from (Wang et al., 2019).
Revised: However, for the last experiment, we consider the hypothetical scenario where we had a larger number of users K ← 100 K and K ← 1 , 000 K , thus decreasing the privacy parameter to δ ← δ/ 100 and δ ← δ/ 1 , 000 and reducing the added noise in the analysis from (Wang et al., 2019).

Original: Model considered.
Revised: Model considered.

Original: We experimented with a network with 2 convolution layers with kernel of size 5 × 5 , stride of 2, ReLU activation function, and 32 and 64 ﬁlters respectively.
Revised: We experimented with a network with 2 convolution layers with kernel of size 5 × 5 , stride of 2, ReLU activation function, and 32 and 64 ﬁlters respectively.

Original: These layers are followed by a fully connected layer with 100 hidden units and a ReLU activation function, and a fully connected output layer with 10 hidden units and a Softmax activation function.
Revised: These layers are followed by a fully connected layer with 100 hidden units and a ReLU activation function, and a fully connected output layer with 10 hidden units and a Softmax activation function.

Original: From now on, this model will be referred as the convolutional network.
Revised: From now on, this model will be referred as the convolutional network.

Original: Hyperparameters.
Revised: Hyperparameters.

Original: For all the experiments the learning rate was η = 0 .
Revised: For all the experiments the learning rate was η = 0 .

Original: 1 for the network parameters and γ = 0 .
Revised: 1 for the network parameters and γ = 0 .

Original: for the Lagrange multipliers.
Revised: for the Lagrange multipliers.

Original: The damping parameter was c = 20 .
Revised: The damping parameter was c = 20 .

Original: Note that we set a larger damping parameter to increase the strength to which we want to enforce the constraints, given that the task is harder than before.
Revised: Note that we set a larger damping parameter to increase the strength to which we want to enforce the constraints, given that the task is harder than before.

Original: The cohort sizes considered were m = 100 and m = 2000 .
Revised: The cohort sizes considered were m = 100 and m = 2 , 000 .

Original: Finally, the clipping bound for the convolutional network was C = 250 if the training algorithm was PFL and C = 350 if it was FPFL .
Revised: Finally, the clipping bound for the convolutional network was C = 250 if the training algorithm was PFL and C = 350 if it was FPFL .

Original: As previously, these hyper-parameters where not selected with a private hyper-parameter search and were just set as an exemplary conﬁguration.
Revised: As previously, these hyper-parameters where not selected with a private hyper-parameter search and were just set as an exemplary conﬁguration.

Original: If one desires to ﬁnd the best hyper-parameters, one can do so at an additional privacy budget cost following e.g. (Abadi et al., , Appendix D)
Revised: If one desires to ﬁnd the best hyper-parameters, one can do so at an additional privacy budget cost following e.g. (Abadi et al., 2016, Appendix D).