Fair Federated Learning via Bounded Group Loss
Authors: Shengyuan Hu
Steven Wu
Virginia Smith
Venue: Submitted to ICLR 2023 (2023)
Keywords: Federated Learning
Group Fairness

Abstract: In federated learning, fair prediction across protected groups is an important constraint for many applications. Unfortunately, prior work studying group fair federated learning lacks formal convergence or fairness guarantees. In this work we propose a general framework for provably fair federated learning. In particular, we explore and extend the notion of Bounded Group Loss as a theoretically-grounded approach for group fairness. Using this setup, we propose a scalable federated optimization method that optimizes the empirical risk under a number of group fairness constraints. We provide convergence guarantees for the method as well as fairness guarantees for the resulting solution. Empirically, we evaluate our method across common benchmarks from fair ML and federated learning, showing that it can provide both fairer and more accurate predictions than baseline approaches.

Under review as a conference paper at ICLR 2023
FAIR FEDERATED LEARNING VIA BOUNDED GROUP LOss
Anonymous authors Paper under double-blind review
ABSTRACT
Fair prediction across protected groups is an important constraint for many fed- erated learning applications. However, prior work studying group fair federated learning lacks formal convergence or fairness guarantees. In this work we pro- pose general framework for provably fair federated learning. particular, we explore and extend the notion of Bounded Group Loss as theoretically -grounddd approach for group fairness. Using this setup, we propose a scalable federated optimization method that optimizes the empirical risk under a number of group fairness constraints. We provide convergence guarantees for the method as well as faimess guarantees for the resulting solution. Empirically, across common benchmarks from fair MI. and federated learning, showing that it can provide both fairer and more accurate predictions than baseline approaches.
evaluate method
INTRODUCTION
Group fairness aims to mitigate unfair biases against certain protected demographic groups (e.g. race, gender, age) in the use of machine learning. Many methods have been proposed incorpo- rate group fairness constraints in centralized settings (e.g., Agarwal et al., 2018; Feldman et al., 2015; Hardt et al., 2016; Zafar et al., 2017a). However, there is a lack of work studying these approaches in the context of federated learning (FL), training paradigm where a model is fit to data generated by set of disparate data silos, such as a network of remote devices or collection of organizations (Kairouz et al., 2019; Li et al., 2020; McMahan et al., 2017). Mirroring concerns around fairness in non-federated settings, many FL. applications similarly require performing fair prediction across protected groups. Unfortunately, approaches to each client in a federated network in isolation may be inaccurate due to heterogeneity across clients-filligg failing to produce fair model across the entire population (Zeng et al., 2021).
show in Figure 1, naively applying existing
Several recent works have considered ad- dressing this issue by exploring specific forms of group fairess in FL (c.g.. Chu et al., 2021; Cui et al., 2021; Du et al., 2021; Papadaki et al., 2022; Rodríguez- Gálvez et al., 2021; Zeng et al., 2021). Despite promising empirical performance. these prior works lack formal guarantees surrounding the resulting fairness of the so- lutions (Section 2), which is problematic as it is unclear how the methods may perform in real-world Fl. deployments.
2ETL Siebal DG.
PPTL Local
D
Cruu
Server
0.2725
Error rabe (1 Accuracy
Figure Naively applying fair learning method locally each client might be problematic. Left: Due to data hetero- geneity in FL. data distributions conditioned on each protected attribute (shown in different colors) may differ across clients. Fair FL aims learn model that provides fair prediction on the entire data distribution. Right: Empirical results (ACS dataset) verify that training with local fairness constraints alone induces higher error and worse fairness than using global faimess constraint.
In this work we provide formulation and method for group fair FL that can provably satisfy global faimess constraints. Com- mon group fairness notions that aim to achieve equal prediction quality between any two protected groups (e.g., Demographic Parity, Equal Opportunity (Hardt et al., 2016)) are difficult to provably satisfy while simultaneously finding a model with high utility. Instead, we consider different fairness notion known Bounded Group Loss (BGL) (Agarwal et al., 2019), which aims to promote worst group's 's performance, to capture these common group faimess criteria. As show, a benefit of this approach is that in addition to

---

Under review as a conference paper at ICLR 2023
having practical advantages in terms of fairness-utility trade-offss (Section 5), it maintains smoothness and convexity properties that can equip our solver with favorable theoretical guarantees.
Based on our group fairness formulation, we then provide a scalable method (PFFL) to solve the proposed objectives via federated saddle point optimization. Theoretically, we provide convergence guarantees for the method as well as fairess and generalization guarantees for the resulting solutions. Empirically, we demonstrate the effectiveness of our approach on common benchmarks from fair machine learning and federated learning. We summarize our main contributions below:
We propose a novel fair federated learning framework for range of group faimess notions. Our framework models the fair FL problem as a saddle point optimization problem and leverages variations of Bounded Group Loss (Agarwal et al., 2019) capture common forms of group fairness. We also extend BGL to consider a new faimess notion called Conditional Bounded Group Loss (CBGL), which may be of independent interest and utility in non-federated settings.
We propose a scalable federated optimization method for our group fair FL framework. We provide regret bound analysis for our method under convex ML objectives to demonstrate formal convergence guarantees. Further, we provide fairness and generalization guarantees on the model for variety of fairness notions.
Finally, we evaluate our method on common benchmarks used in fair machine learning and federated learning. In all settings, we find that our method can significantly improve model fairness compared to baselines without sacrificing model accuracy. Additionally, even though we do not directly optimize classical group fairness constraints (e.g., Demographic Parity, Equal Opportunity), we find that our method can still provide comparable/better fairness-utility trade-offs relative to existing approaches when evaluated on these metrics.
2 BACKGROUND AND RELATED WORK
Fair Machine Learning. Algorithmic fairness in machine learning aims to identify and correct bias in the learning process. Common approaches for obtaining faimess include pre-procsssing methods that rectify the features or raw data to enhance fairness (Calmon et al., 2017: Feldman al., 2015; Zemel et al., 2013); post-processing methods that revise the prediction score for trained model (Dwork et al., 2018; Hardt et al., 2016; Menon & Williamson, 2018); and in-processing methods that directly modify the training objective/solver to produce a fair predictor (Agarwal et al., 2018; 18; 2019; Woodworth et al., 2017; Zafar et al., 2017a;b). Most existing methods in fair ML rely on using a centralized dataset to train and evaluate the model. As shown in Figure I, in the federated setting where data is privately distributed across different data silos, directly applying these methods locally only ensures fairness for each silo rather than the entire population. Developing effective and efficient techniques for fair FL is thus an important area of study.
Fair Federated Learning. In FL, definitions of fairness may take many forms. A commonly studied notion of fairness representation parity (Hashimoto et al., 2018), whose application in FL requires the model's performance across all clients to have small variance (Donahue & Kleinberg, 2021: ; Li et al., 2019a; 2021; Mohri et al., 2019; Yue et al., 2021). In this work we instead focus on notions of group fairness, in which every data point in the federated network belongs to some (possibly) protected group, and we aim to find a model that doesn't introduce bias towards any group.
Recent works have proposed various objectives for group fairness in federated learning. Zeng et al. (2021) proposes bi-level optimization objective that minimizes the difference between each group's loss while finding an optimal global model. Similarly, several works propose using a constrained optimization problem that aims to find the best model subject to an upper bound on the group loss difference (Chu et al., 2021; Cui et al., 2021; Du et al., 2021; Rodriguez- from these approaches, our method focuses on a faimess constraint based on upperbounding the loss of each group with a constant rather than the loss difference between any two groups. More closely related to our work, Papadaki et al. (2022) weighs the empirical loss given each group by a trainable vector À and finds the best model for the worst case X. Though similar to method for < = O, this approach fails to achieve both strong utility and fairness performance under non-convex loss functions (see Section 5). Zhang al. (2021) also propose similar objective to learn model with unified fairness. Among these works, Zeng et al. (2021) and Cui et al. (2021) also provide simplified convergence and fairness guarantees for their method. However, these works lack formal analyses around the convergence for arbitrary convex loss functions as well as the behavior of the fairness constraint over the true data distribution. Ours is the first work we are aware to provide such guarantees in the context of group fair federated learning.
et al., 2021). Different
2

---

Under review as a conference paper at ICLR 2023
3 FAIR FEDERATED LEARNING VIA BOUNDED GROUP LOSS
In this section we first formalize the group fair federated learning problem and fairness-aware objective solving this problem (Section 3.1). We then provide several examples of group faimess based on the notion of BGL and show how incorporate them into our framework (Section 3.2).
3.1 SETUP: GROUP FAIR FEDERATED LEARNING
Many applications of FL. require treatment of data belonging protected groups (e.g., race, gender, age). This is particularly common in applications of cross-silo Fl., where we may wish to produce a model that fairly treats individuals from various demographic groups across a collection of data silos (c.g. hospitals, schools, financial institutions) (Chu et al., 2021; Cui et al., 2021; Vaid et al., 2021).
FL Setup. Following standard federated learning scenarios (McMahan et al., 2017), we con- sider a network with K different clients. Each client k E [K] has access to training data sampled from the true data distribution DR. where z; is an obser- vation, E Y is the label, ag E A is the protected attribute. Let the hypothesis class be H and for any model h H. and define the loss function data (I, a) to be 1(b(xz), v). Federated learning applications typically aim to solve:
Dr {(x;; Vi, d=)),= .m, ,,,
min F(h) min (aa)) [(h()))))
(1)
REH
MEH
In practice, D3 is estimated by observing {(zi, yi, a:) a.))a==i....... ma. and we solve the empirical risk:
my
min
F(h)
min WEH
(h(Ix..).
hEH
K
mk
For simplicity, we define fa( Further, we assume h is parameterized by a vector RP where is the number of parameters. We will use F(w) and (w) to represent F(h) and fa(h) intermittently in the remainder of the paper.
my E ((((zR,), 3k.x) as the local objective for client k.
Fairness via Constrained Optimization. When centralized dataset is available, standard approach to learn a model that incorporates fairness criteria is to solve a constrained optimization problem where one tries find the best model subject some fairness notion (Agarwal et al., 2019; Barocas et al., 2019). Following this formulation, we formalize a similar learning problem in the federated setting, solving:
min F(h) subject to R(h)= S.
where R(h), < E R² encodes the constraint set on h. For instance, the z-th constraint could be written as R. (h) I Se where Ça is a fixed constant. This formulation is commonly used to satisfy group faimesss notions such as equal opportunity, equalized odds (Hardt et al., 2016), and minimax group fairness (Diana et al., 2021).
To solve the constrained optimization problem 3, a common method is to use Lagrangian multipliers. In particular, let À E R. be dual variable and assume À has || |/1 at most B. The magnitude of B could be viewed as the regularization strength for the constraint term. Objective equation 3 can then be converted into the following saddle point optimization problem:
min max AER JABBB
G(w; A) = (F))) + +
(Main Objective)
where the q-th index of r encodes the q-th constraint from R (i.c. [w( := (w,w) - Cu) and / is a fixed constant. In other words, the objective finds the best model under the scenario where the fairness constraint is most violated (i.e., the regularization term is maximized).
There are two steps needed in order to provide meaningful utility and faimess guarantees for the model found by solving Main Objective: (1) showing that it is possible to recover a solution close to the 'optimal" solution, (2) providing
upper bound for both the risk (F(w)) and the faimesss constraint (r(w)) given this solution. To formally define what an 'optimal' solution is, in this work we aim to identify constraints that satisfy the following key assumption:
Assumption (Convexity of G). Assume that G(w; >) is convex in for any fixed X.
3

---

Under review as a conference paper at ICLR 2023
Remark. In particular, since G is linear in X, given fixed wo. can find solution to the problem max G(we; X), denoted as sri.e. G[wo; X* 2 G(wy; A) for all x. When G is convex in w, argue that given fixed Ag. there exists w" that satisfies w² arg min G(w; Xo), i.c. G(w"; Ao) S G(w; Ao) for all w. Therefore, (w", ; X²) 2) is a saddle point of G(3 +), which is denoted as the optimal solution in our setting.
3.2 FORMULATING FAIR FL: BOUNDED GROUP LOSS AND VARIANTS
Many prior works in fair federated learning consider instantiating R(h) in equation that bounds the difference between any two groups' losses, a common technique used to enforce group fairness notions such equalized odds and demographic parity (e.g.. Chu al., 2021; Cui et al., 2021; Zeng et al., 2021). Unfortunately, this results in G(w; X) becoming nonconvex in w, thus violating our Assumption O. This nonconvexity is problematic as it increases the likelihood that a solver will find a local minima that either does not satisfy the faimess constraint or achieves poor utility. Instead of enforcing equity between the prediction quality of any two groups, in this work we explore using a constraint based on Bounded Group Loss (BGL) (Agarwal et al., 2019) which promotes worst group's prediction quality and propose new variants that can retain convexity assumptions while satisfying meaningful fairness notions. In particular, we explore three instantiations of group fairness constraints R(hh) that satisfy Assumption 0 below.
constraint
Instantiation 1 (Bounded Group Loss), We begin by considering fairness via the Bounded Group Loss (defined below), which was originally proposed by Agarwal et al. (2019). Different from applying Bounded Group Loss in centralized setting, BGL in the context of federated learning requires that for any group a E A. the average loss for all data belonging to group a is below certain threshold. As we discuss in Section 4 this (along with general constraints of FL such as communication) necessitates the development of novel solvers and analyses for the objective.
Definition 1 (Agarwal et al. (2019)). classifier h satisfies Bounded Group Loss (BGI.) at level ( under distribution D if for all E A, we have E [l(((z),, y) = a] S.
In practice, we could define empirical bounded group loss constraint at level & under the empirical Ek=1
distribution D- = K
Dr to be
k=
K 1/ma Σ ((((Z).). - S/K
(h) Σ 1=1
Fa. (h)
S..
k=1
Benefits of BGL. BGL ensures that the prediction quality on any group reaches certain threshold. Compared standard loss parity constraints that aim to equalize the losses across all protected groups (e.g. overall accuracy equity (Dieterich et al., 2016)), BGL has two main advantages. First, G(w; A) preserves convexity in w, as long as the loss function 1 itself convex. In contrast, loss parity constraints are generally non-convex even if / is convex. Second, when the prediction difficulties are uneven across groups, loss parity may force an unnecessary drop of accuracy on some groups just to equalize all losses (Agarwal et al., 2019). In contrast, the criterion of BGL can avoid such undesirable effects.
Instantiation 2 (Conditional Bounded Group Loss). In some applications one needs stronger fairness notion beyond ensuring that no group's loss too large. For example, in the scenario of binary classification, a commonly used fairness requirement is equalized true positive rate or false positive rate (Hardt et al., 2016). In the context of optimization for arbitrary loss functions, a natural substitute is equalized true / false positive loss. In other words, any group's 's loss conditioned on positively negatively labeled data should be equalized. Therefore, similar to BGL., we propose a novel fairness definition known as Conditional Bounded Group Loss (CBGL) defined below:
Definition 2. A classifier h satisfies Conditional Bounded Group Loss (CBGL) for y E Y at level by under distribution D if for all a E A, we have E [[(((z), 4) = a, Y = y) Σ Cy-
In practice, we could define empirical Conditional Bounded Group Loss constraint level KylWEY under D to be
K
K Σ
Ta.y (h) == Σ 1=1
((az)) (h)
1/ma.y
Σ
((h(ZR.)), - Su/KK S S0.
1=1
4

---

Under review as a conference paper at ICLR 2023
Note that satisfying CBGL for all Y is strictly harder problem than satisfying BGL alone. In fact, we can show that a classifier that satisfies CBGL at level Kylyey also satisfies BGL at level By--p [Syl where Pa be the probability density of labels for all data from group a.
Relationship between CBGL and Equalized Odds. For binary classification tasks in centralized settings, a common faimess notion is Equalized Odds (EO) (Hardt et al., 2016), which requires the True/False Positive Rate to be equal for all groups. Our CBGL definition can be viewed a relaxation of EO. Consider a binary classification example where Y = {0, 1). Let the loss function 2 be the 0-1 loss. CBGL requires classifier h to satisfy Pr(h(x) = Y = yo: = al I Syo for all E A and yo E Y. EO requires Pr(h(z)) = Y y0, A = a] to be the same for all a E A given fixed yo., which may not be feasible the hypothesis class H is not rich enough. Instead of requiring equity of each group's TPR/FPR, CBGL only imposes an upper bound for each group's ' TPR/FPR. Similar to the comparison between BGL and loss parity, CBGL offers more flexibility than EO since it does not force artificial increase on FPR or FNR when a prediction task one of the protected groups is much harder. In addition, for applications where logistic regression or DNNs are used (e.g.. CV, NLP), it is uncommon use the 0-1 loss in the objective. Thus, CBGL can provide relaxed notion of fairness for more general loss functions whose level of fairness can be flexibly tuned.
Instantiation 3 (MinMax Fairness). Recently Papadaki et al. (2022) proposed a framework called FedMinMax by solving an agnostic fair federated learning framework where the weight is applied to empirical risk conditioned on each group. Note that using BGL as the fairness constraint, our framework could reduce to FedMinMax as a special case by setting 3 = 0, B = 1 and (= =.
Definition 3. Use the same definition of ra (h) as we had in Instantiation FedMinMax (Papadaki el al., 2022) aims to solve for the following objecfive: ming max m {=R|___A .|Al[==1 EaEa (h).
Note that a key property of FedMinMax is the constant ( used to upper bound the per group loss is set to O. From constrained optimization view, the only feasible solution that satisfies all fairness constraints for this problem is model with perfect utility performance since requiring all losses to be smaller than is equivalent to having all of them to be exactly 0. Such a property limits the ability to provide fairness guarantees for FedMinMax. Fixing B and Ç also limits its empirical performance on the relation between fairness and utility, as we will show later in Appendix F.
4 PROVABLY FAIR FEDERATED LEARNING
In this section, we first propose Provably Fair Federated Learning (PFFL), scalable solver for Main Objective, presented in Algorithm 1. We provide formal convergence guarantees for the method in Section 4.2. Given the solution found by PFFL, in Section 4.3 we then demonstrate the fairness guarantee for different examples of fairness notions defined in Section 3 (BGL, CBGL).
4.1 ALGORITHM
To find a saddle point for Main Objective, we follow the scheme from Freund & Schapire (1997) and summarize our solver for fair FL in Algorithm (full algorithm description in Appendix A). Our algorithm is based off of FedAvg (McMahan et al., 2017), a common scalable approach in federated learning. Intuitively, the method alternates between two steps:
(l) given a fixed X, optimize our regularized objective F(w) + ( over w;; (2) given a fixed w. optimize the fairness violation term X (((w)) over X.
While Agarwal et al. (2019) also follows similar recipe to ensure BGL, our method needs to overcome additional challenges in the federated settings. In particular, the method in Agarwal et al. (2019) optimizes w by performing exact best response, which is in general in feasible when data for distributed data sets. Our method overcomes this challenge by applying a gradient-descent-assee style optimization process that utilizes the output of a FL learning algorithm as an approximation for the best response. In Algorithm 1, we provide an example in which the first step is achieved by using AedAvg to solve min,, F(w) + X" X(((w) (L 4-12). Note that solving this objective does not require the FedAvg solver, any algorithm that learns a global model in FL could be used to find a certain w given X. After we obtain a global model from a federated training round, we use exponentiated gradient descent to update X, following Alg 2 in Agarwal et al. (2019). This completes one training round. At the end of training, we calculate and return the average iterate as the fair global model.
Note that the ultimate goal to solve for Main Objective is to find a w such that it minimizes the empirical risk subject r(w) < 0. Therefore, at the end of training, our algorithm checks whether
5

---

Under review as a conference paper at ICLR 2023
the resulting model tü violates the faimess guarantee by at most some constant error M+22 where M is the upper bound for the empirical risk and v/ is the upper bound provided in Equation 5 (L 16-20). We will show in the Lemma that this is always true when there cxists a solution w² for Problem 3. However, it is also worth noting that the Problem 3 is not always feasible. For example when we set Ç 0, requiring r(w) S O is equivalent to requiring the empirical risk given any group E A is non positive, which is only feasible when the loss is O for every data in the dataset. In this case, our algorithm will simply output nall if the fairness guarantee is violated by an error larger than M+22,
Privacy aspect of PFFL Compared to FedAvg. our solver communicates losses conditioned on each group in addition to model updates. This is common in prior works that solve a min-max optimization problem in federated learning (Hou et al., 2021; Zeng et al., 2021). We note that our method could be easily extended to satisfy example-level DP for FL by performing DP-SGD locally for each client. Our algorithm also gives natural client-level DP and LDP variants. In particular, we can compute via trusted server the average loss each client, which sufficient statistics to update X.
4.2 CONVERGENCE GUARANTEE
Different from Agarwal et al. (2019), while our algorithm handles arbitrary convex losses in federated setting by replacing the best response with the FedAvg output, we want to show that after running finitely many rounds, how close our solution is to the actual best response. In this section, we provide regret bound style analysis for PFFL algorithm. To formally measure the the distance between the solution found by our algorithm and the optimal solution, we introduce u-approximate saddle point generalization of saddle point (See Remark in Section 3.1) defined below:
Definition 4. (w, 2) is a aasav-apppal v-approximate saddle point of G if
G(ū, X) S G(w, 2) G( X) 2 G( A) - for all
for all
(4)
As an example, the optimal solution (w= X+) is a O-approximate saddle point of G. To show convergence, we first introduce some basic assumptions below: Assumption 1. Let fxe be p-strongly convex and L-smooth for all = K.
Assumption 2. Assume the stochastic gradient of Jk has bounded variance: EE| V Ji ct) (ie("()) < of. all k 11 ,K.
Assumption 3. Assume the stochastic gradient of fi is uniformly bounded: [ fu (w : [/) G for all
all k L....., K.
These are common assumptions used when proving the convergence for FedAvg (e.g., Li et al., 2019b). Now we present our main theorem of convergence:
Theorem 1 (Informal Convergence Guarantee). Let Assumption 1-3 hold. Define k L
max Sk, J} and step size nQQ (+B))u(+1)) and assume ||T||x S p. Letting w = ET 1== w',
x = ET EET X we have;
Blog(Z+1) 11) + nop2B,
Σ max G(w; >) min G(w; X) S T 7 C+
(5)
w
no ET
where C is a constant.
The upper bound in Equation 5 consists of two parts: (1) the error for the FedAvg process to obtain w which is term of order (log T/T); (2) the error for the Exponentiated Gradient Ascent process to obtain which converges to a noise ball given fixed rjo- Following Theorem 1, we could express the solution of Algorithm 1 as p-approximate saddle point of G by picking appropriate doo and T:
Corollary 2. Let 170 = B and T 2 u(y+1)-2nd 2xC
4p2 1² logi (+1)(+1))
2kC(y 1)). then
EE
(w, X) is a u-approximate saddle point of G.
We provide detailed proofs for both Theorem I and Corollary 2 in Appendix C. Different from the setting in prior FedAvg analyses (e.g., Li et al., 2019b), in our case the outer minimization problem changes as À gets updated. Therefore, our analysis necessitates considering more general scenario where the objective function could change over time.
4.3 FAIRNESS GUARANTEE
In the previous section, we demonstrated that our Algorithm could converge and find a v- approximate saddle point of the objective G. In this section, we further motivate why we care

---

Under review as a conference paper at ICLR 2023
about finding r-approximate saddle point. The ultimate goal for our algorithm is to: (1) learn a model that produces fair predictions on training data, and (2) more importantly, produces fair predictions on test data, i... data from federated clients not seen during training.
Before presenting the formal fairness and generalization guarantees, we state the following additional assumption, which is common assumption for showing the generalization guarantee using the Rademacher complexity generalizations bound (Mohri et al., 2019).
Assumption 4. Let F and F be upper bounded by constant M.
We first show the fairness guarantee on the training data.
Lemma (Empirical Faimesss Guarantee). Assume there exists w satisfies r(w") S Oz, we have
maxr, (axr,( M + 2v
(6)
B
Lemma characterizes the upper bound for the worst fairness constraint evaluated on the training data. Given a fixed v, one could increase B to obtain a stronger fairness guarantee, i.e. a smaller upper bound. Combining this with Corollary 2, can be seen that when B is large, additional exponentiated gradient ascent rounds are required to achieve stronger fairness.
Next we formalize the fairness guarantee for the entire true data distribution. Define the true data distribution to be D K Σ Dr. We would like to formalize how well our model is evaluated on the true distribution D as well as how well the fairness constraint is satisfied under D. This result is presented below in Theorem 3.
Theorem (Full Fairness and Generalization Guarantee). Let Assumption 1-4 holds and (w, X) a v-approximate saddle point of G. Then with probability - 6, either there doesn't exist solution for Problem 3 and Algorithm returns null or Algorithm returns vū satisfies
F(ū) F(w)) 2v + 49 (H)
2M
V 2mn
log (og(2/8),
K
T5(1) M+22 + GenrJ
where w is solution for Problem and Gen, is the generalization error.
The first part for Equation characterizes how well our model performs over the true data distribution compared to the optimal solution. As number of clients K increases, we achieve smaller generalization error. The second part for Equation 7 characterizes how well the fairness constraints are satisfied over the true data distribution. Note that the upper bound could be viewed the sum of empirical fairness violation and a generalization error. Based on our fairness notions defined in Section 3.2, we demonstrate what generalization error is under different faimess constraints
Proposition 1 (r encodes BGL at level (). There are in total |A] fairness constraints, one for each group. Define the weighted rademacher complexity for group as R, (H)
E ES. , SuPnEh EkE I Ean.=o Th.d (M(T),,) BR.)) In this scenario, we have:
M K log(21A1/8).
Gen,, (H) Ma
Note that the fairness constraint for group under true distribution in Equation 7 is upper bounded by For any group ag with sufficient data, i.e., map is large, the BGL constraint with respect to group ag under DD has stronger formal fairness guarantee compared any group with less data. is also worth noting that this generalization error grows as the number of clients K grows. Recall that the generalization error becomes smaller when K grows; combing the two results together provides us a tradeoff between fairness notion of BGL and utility over the true data distribution in terms of K.
V
Proposition 2 (r encodes CBGL at level [Sulprr). There are in total |A||Y|| fairness constraints, one for each group and label. Define the weighted rademacher complexity for group a conditioned on
y Ru,y (H) E,, "s." [supheR Σx=1 mayy Eor. T (h(*),)). v) where ma,y is the number of all examples from group with label y. In this scenario, we have:
M.
Genr,fa.) = 2Ra +
log(2)A[|V1/0))
ma.y

---

Under review as a conference paper at ICLR 2023
Coleb
Celet4
Feshi
Fashig
FFLLth Local
1314
PPL wtt Local CDC.
4335
035
Wested festop
men with fidta
1:41
2IM
2044
Eror rate
Accurncy
rote Accuracy)
Emor rate 11 Accurscy)
ACS Employment
ACS Employment
ACS Emaloyment
11.ma
Weihted toclug
We phou ractop
Waypeee rekiee m91 with nen--akkl CRO.
ev, with reeu
1.30
0230
12ay
0211
2218 care case site saon c.dw Eile tate
0341 vase tao ear sats waee Accuracy) COMPAS
COMPAS
Reciig
Fastro
Feshi
Welented
Hos Droun Loxe
deiul LSG.
FFF. Wit G cas. FFF. with Laca CtD.
H11 win loca d6s
IViL with Lacal CRG.
1200
0.64
6.64
5:20
0200
620 Frror rate 11 Atraaree))
Enjr rute [L. Accuracy!
Error rate 11 Accurscy)
BGL
CBGL Y
CBGL Y
for for (column 3) on CelebA (row 1), ACS Employment (row 2), and COMPAS (row 3). Interestingly. we find all settings that our proposed method (PPFL with Global BGL) not only enables a flexible fairness/utility trade-off, but can in fact achieve both stronger fairness and better utility (lower error) than baselines.
Figure 2: Experimental results for using BGL (column 1), CBGL.
= 1 (column 2), and CBGL. for Y 0
Similar to Proposition 1, in order to achieve strong fairesss guarantees for any specific constraint on the true data distribution, we need a sufficient number of samples associated with that constraint.
We provide details and proof for Theorem 3 in Appendix D. Different from the analysis performed in Agarwal et al. (2019), we analyze the generalization behaviour in federated setting where we introduce the generalization bound as a function of number of clients K. We then further formally demonstrate the tension between utility and fairnsss performance evaluated on the true data distribution induced by K, which has not been studied previously to the best of our knowledge.
5 EXPERIMENTS
We evaluate PFFL (Algorithm 1) empirically on ProPublica COMPAS, dataset commonly studied in fair ML (Angwin et al., 2016; Zeng et al., 2021); the US-wide ACS PUMS data, a recent group fairness benchmark dataset (Ding al., 2021); and CelebA (Caldas al., 2018), common federated learning dataset. We compare our method with training vanilla FedAvg model in terms of both fairness and utility in Section 5.1, and explore performance relative baselines that aim to enforce other faireess notions (Demographic Parity and Equal Opportunity) in Section 5.2.
Setup. For all experiments, we evaluate the accuracy and the empirical loss for each group test data that belongs to all the silos of our fair federated learning solver. We consider COMPAS Recidivism prediction with gender protected attribute, the ACS Employment task (Ding et al., 2021) with race as protected attribute, and CelebA (Caldas et al., 2018) with gender as a protected attribute. To reflect the federated setting, we use heterogeneous data partitions to create data silos. ACS Employent is naturally partitioned into 50 states; COMPAS and CelebA are manually partitioned in non-lII manner into collection of data silos. A detailed description of datasets, models, and partition schemes can be found in Appendix B.
5.1 FAIRNESS-UTIUE
TRADE-OFFS FOR ALGORITHM
We first explore how test error rate differs as a function maximum group loss using our Algorithm
1. To be consistent with our method and theoretical analysis, we exclude the protected attribute a; for each data as a feature for learning the predictor. For each dataset, we evaluated PFFL with BGL; CBGL for Y = 1; and CBGL for Y = 0. For each method we evaluate, given fixed number of training iterations E and T, we finetune B and & and evaluate both test error rate and test loss each
8

---

Under review as a conference paper at ICLR 2023
CONNNS
Emrpitteent
Enpplypeent
644
NAMS
Ni.
ICP.
Emar rate D1 Accenol
End tAGTL Adurecy)
500 CA4 Kil
140 Accorapy
(a) Demographic Parity (b) Equal Opportunity
(c) Demographic Parity
(d) Equal Opportunity
Figure 3: Comparison between PFFL and three different prior works on COMPAS and ACS for ADP and AEO. Although PPFL not directly designed to optimize Demographic Parity/Equal Opportunity,
that outperforms the baseline of FedAvg, and performs comparably / better than prior works that were designed for these objectives.
group. Given certain test error rate, select the hyperparameter pair (B, that yields the lowest maximum group loss. We show the relation between test accuracy vs. max group loss in Figure 2. In particular, we compare our fairness-aware FL methods with two baseline methods: vanilla FedAvg and FedAvg trained on loss weighted by groups. In FL, applying fair training locally at each data silo and aggregating the resulting model may not provide strong population the same fairness definition (Zeng et al., 2021). Hence, we also explore the relationship between test accuracy and max group loss under local BGL and global BGL constraints.
faimess guarantees with
On all datasets, there exists a natural tradeoff' between error rate and the fairness constraint: when model achieves stronger fairness (smaller max group loss), the model tends to have worse utility (higher error rate). However, in all scenarios, our method not only yields a model with significantly smaller maximum group loss than vanilla FedAvg, but also achieves higher test accuracy than the baseline FedAvg which is unaware of group fairness. Meanwhile, for all datasets and fairness metrics, as expected, PFFL with Global BGL achieves improved fairness-utiliig tradeoffs relative to PFFL with Local BGL. Therefore, our PFFL with Global faimess constraint framework yields model where utility can coexist with fairness constraints relying on Bounded Group Loss.
5.2 BGL/CBL EVALUATED ON OTHER FAIRNESS NOTIONS
Beyond BGL and CBGL, there are other fairesss notions commonly used in the fair machine learning literature. For example, several works in group fair FL have proposed optimizing the difference between every two groups' losses (possibly conditioned on the true label) with the aim of achieving Demographic Parity (or Equal Opportunity) (Chu et al., 2021; Cui et al., 2021; Hardt et al., 2016; Zeng et al., 2021). Formally, consider the case where the protected attribute set A- = {0, 1}. Define APP = [Pr(A((X) = 1/A = 0) -rr(((X)- = 1/A = 1)] and Ago = ( = 1|A = 0, = 1) - Pr(h(X)- 1/A - 1,Y 1)J. These works aim to train model such that we could achieve small ADP or small Ago. depending on the fairness constraint selected during optimization. As discussed in Section 3.2, CBGL could be viewed as a more general definition of Equal Opportunity and Equalized Odds. In this section, we compare our method with FedFB (Zeng et al., 2021), FedFair (Chu et al., 2021), and FCFL (Cui et al., 2021), all of which aim to optimize ADP and AgO. We evaluate ADP and AgO for all approaches Employment, with results shown in Figure 3. Similar to Figure 2, we only show the points lying on the pareto frontier for our method. Although PFFL with BGL and CBGL was not directly designed for this fairness criteria (i.e., it does not directly enforce the loss or prediction parity of two groups' losses to be close), we that our method is still able to outperform training FedAvg baseline, and in fact performs comparably or better than prior methods (which were designed for this setting).
COMPAS and ACS
6 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK
In this work, we propose a fair learning objective for federated settings via Bounded Group Loss. We then propose a scalable federated solver to find an approximate saddle point for the objective. Theoretically. we provide convergence and fairness guarantees for our method. Empirically, we show that our method can provide high accuracy and fairness simultaneously across tasks from fair MI. and federated learning. In addition to strong empirical performance, ours is the first work we are aware of to provide formal convergence and fairness/generalizatin convex loss functions. In future work we are interested in investigating additional benefits that could be provided by using our framework, including applications in non- federrred settings. Finally, similar to prior works in group fair FL. method communicates additional parameters beyond standard non-fair FL (c.g., via FedAvg); studying and mitigating the privacy risks of such communications in the context of fair federated learning would be interesting direction of future work.
guarantees for group fair FL with general

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning. PMLR, 2018.
Alekh Agarwal, Miroslav Dudik, and Zhiwei Steven Wu. Fair regression: Quantitative definitions and reduction-based algorithms. In International Conference on Machine Learning. PMLR, 2019.
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. In Ethics of Data and Analytics, pp. 254-264. Auerbach Publications, 2016.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Faimnsss and machine learning, fairmlbook. org. URL: http:///ww. fairmlhook. org, 2019.
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings, https://leaf cmu eduu. arXiv preprint arXiv: 1xXv::/812000997 2018.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush
Varshney. Optimized pre-processing for discrimination prevention. Advances in neural information processing systems, 2017.
Lingyang Chu, Lanjun Wang. Yanjie Dong, Jian Pei, Zirui Zhou, and Yong Zhang. Fedfair: Training fair models in cross-silo federated learning. arXiv preprint arXiv:2 (00.::200.00662., 2021.
Sen Cui, Weishen Pan, Jian Liang, Changshui Zhang, and Fei Wang. Addressing algorithmic disparity and performance inconsistency in federated learning. Advances in Neural Information Processing Systems, 34, 2021.
Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, and Aaron Roth. Minimax group fairness: Algorithms and experiments. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 66-76, 2021.
William Dieterich, Christina Mendoza, and Tim Brennan. Compas risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc, 7(4), 2016.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt, Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 2021.
Kate Donahue and Jon Kleinberg. Models of fairness in federated learning. arXiv preprint arXiv:2112.00811 2021.
Wei Du, Depeng Xu, Xintao Wu, and Hanghang Tong. Fairness-aware agnostic federated learning. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 181-189. SIAM, 2021.
Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. Decoupled classifiers for group-fair and efficient machine learning. In Conference on fairness, accountability and transparency. PMLR, 2018.
Yahya H Ezzeldin, Shen Yan, Chaoyang He, Emilio Ferrara, and Salman Avestimehr. Fairfed: Enabling group fairness in federated learning. arXiv preprint 1 10.00857, 2021.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 2015.
Yoav Freund and Robert E. Schapire. A decision-theorettc generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-13) 1997.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 2016.
10

---

Under review as a conference paper at ICLR 2023
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairesss without demographics in repeated loss minimization. In International Conference on Machine Learning.
PMLR. 2018.
Charlie Hou, Kiran K Thekumparampil, Giulia Fanti, and Sewoong Oh. Efficient algorithms for federated saddle point optimization. arXiv preprint arXiv:2 2021.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, al. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 2019.
Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. Fair resource allocation in federated learning. arXiv preprint arXiv:1905. 10497, 2019a.
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50-6) 60, 2020.
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, pp. 6357-6368. PMIR, 2021.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang. and Zhihua Zhang. On the convergence of fedavg on non-iid data, arXiv preprint arXiv: arXiv: /99. 2019b.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication -eftiiioneff learning of deep networks from decentralized data. In Artificial Intelli- gence and Statistics, pp. 1273 1282. PMLR, 2017.
Aditya Krishna Menon and Robert C Williamson. The cost of fairness in binary classification. In Conference Fairness, Accountability and Transparency. PMLR, 2018.
Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. Agnostic federated learning. In Interna- fional Conference on Machine Learning. PMLR, 2019.
Afroditi Papadaki, Natalia Martinez, Martin Bertran, Guillermo Sapiro, and Miguel Rodrigues. Minimax demographic group fairness in federated learning. arXiv preprint arXiv:2201.88.8 083044 2022.
Borja Rodríguez-Gāäver Filip Granqvist, Rogier van Dalen, and Mat Seigel. Enforeing fairness in private federated learning via the modified method of differential multipliers. arXiv preprint arXiv:2109.00 08604, 2021.
Shai Shalev-Shwartz et al. Online learning and online convex optimization. Foundations and trends in Machine Learning, 1924 2011.
Akhil Vaid, Suraj K Jaladanki, Jie Xu, Shelly Teng, Arvind Kumar, Samuel Lee, Sulaiman Somani, Ishan Paranjpe, Jessica De Freitas, Tingyi Wanyan, et al. Federated learning of electronic health records to improve mortality prediction in hospitalized patients with covid-19: Machine learning approach. JMIR medical informatics, 9(1), 2021.
Blake Woodworth, Suriya Gunasekar, Mesrob I Ohannessian, and Nathan Srebro. Learning non- discriminatory predictors. In Conference on Learning Theory. PMLR, 2017.
Xubo Yue, Maher Nouiehed, and Raed Al Kontar. Gifair-fl: An approach for group and individual fairness federated learning. arXiv preprint arXiv:2108.02744 2021.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fair- ness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference world wide web, 7a.

---

Under review as a conference paper at ICLR 2023
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics. PMLR, 2017b.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning. PMLR, 2013.
Yuchen Zeng, Hongxu Chen, and Kangwook Lee. Improving faireess via federated learning, arXiv preprint 1 2021.
Fengda Zhang, Kun Kuang, Yuxuan Liu, Chao Wu, Fei Wu, Jiaxun Lu, Yunfeng Shao, and Jun Xiao. Unified group fairness federated learning. arXiv preprint 1.iv:2/11,049960 2021.
12


Reviews:
[{"review_id": "vrz4ALzf6v", "reviewer": "Reviewer_juWF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper discusses the use of bounded group loss (BGL) in the context of fair federated learning. The core objective is to learn a classifier that satisfies some predefined constraints on the expected loss function of each group on average across all clients. It is expected that the client-conditional data distribution may differ between clients.\n\nThe paper further assumes that the bounded loss function is convex w.r.t. the model parameters, and proposes a distributed saddle point optimization algorithm to recover a model that minimizes empirical risk subject to these group constraints. A convergence proof for the algorithm is also provided\n", "summary_of_the_review": "The proposed method is sensible, but there is limited novelty, and uncertainty if the method is not equivalent to existing approaches as it stands.", "strength_and_weaknesses": "Strengths:\n\nThe paper allows the direct specification of target group-specific loss minimums, which are a natural way of specifying group constraints. The resulting algorithm is simple and well grounded, and the resulting fairness-utility tradeoffs seem comparable to existing approaches.\n\nWeaknesses:\n\nI think the novelty of BGL is limited, the proposed objective is equivalent to FedMinMax, which is discussed on the paper,  with an additional static bias added to the group-conditional loss terms. The formulation as a saddle point presented in this paper is not fully compatible with the bounds on the Lagrangian weights presented in Eq 2, and Algorithm 1 makes it seem like the weights asymptotically reach the norm bound B, which begs the question on why is that a separate optimization parameter from \\beta in Eq 2.\n\nThe theoretical comparisons with existing methods, in particular group DRO, which uses a similar exponential weight update rule for lambda and thresholds on the group losses (albeit with an additional relu), and FedMinMax, which in its paper formulation has a lower bound constraint on lambda  which makes both objectives theoretically equivalent, is sorely lacking.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "qCWD74A5gA", "reviewer": "Reviewer_ScJT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a fair learning objective for federated settings via Bounded Group Loss. The authors propose a scalable federated solver to find an approximate saddle point for the objective. Theoretically, they provide convergence and fairness guarantees for the method. Empirically, they show that their method can provide high accuracy and fairness simultaneously across tasks from fair ML and\nfederated learning.\n\n", "summary_of_the_review": "The paper proposes a fair learning objective for federated settings via Bounded Group Loss. The selection of Bounded Group Loss is not very well motivated. The writing is ok but lacks some literature search for federated saddle point optimization. Overall, I do not recommend acceptance.\n", "strength_and_weaknesses": "Strength:\n- The topic of fair federated learning is relevant.\n- The technical idea of reducing to federated saddle point optimization is natural.\n\nWeaknesses:\n- The selection of Bounded Group Loss is not very well motivated. To my understanding, this selection is due to its convexity instead of its practical interest. The advantage of the proposed algorithm in Bounded Group Loss is not surprising since the baselines are not defined for this fairness measure.\n- Lack of discussion on techniques. The reduction to federated saddle point optimization is natural, and hence I expect to see a comparison with federated saddle point optimization algorithms in the literature. For example, [Hou et al., Efficient Algorithms for Federated Saddle Point Optimization, 2021] and [Shen et al., FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation, 2021].", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_Tri2qBuy9", "reviewer": "Reviewer_SyJQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a group federated learning algorithm with formal convergence/BGL guarantees.", "summary_of_the_review": "See my comments above.", "strength_and_weaknesses": "-Privacy leakage/secrecy of the proposed algorithm is not discussed. It is unclear how much privacy leakage occurs. The proposed algorithm requires the exchange of additional information \"r\" per round, so one *must* discuss the implication of this. Federated learning algorithms with frequent synchronization/additional data exchange may be subject to an unacceptable level of information leakage, making it no different from sharing all the local datasets with every client. To avoid this kind of pitfall, one must clearly discuss the privacy aspect of new algorithms.\n\n-BGL seems to be ok if it's used for regression or used for classification without any preferred labels. ACS Emp./COMPAS are the tasks with preferred outputs, so I am not sure if using BGL makes sense in these settings.\n\n-Some of the experimental results look weird. In most of the experiments, FedAvg, which should minimize the error rate better than any other constrained counterparts, does not seem to achieve the lowest error rate. Sometimes it performs the worst among all tested methods. Why?\n\n-How are the hyperparameters chosen for the baseline methods?\n\n-Some missing baselines. Also, please add the performance of the standard reduction method with central data (upper bound). \n\n[1] FairFed: Enabling Group Fairness in Federated Learning https://arxiv.org/abs/2110.00857\n[2] GIFAIR-FL: A Framework for Group and Individual Fairness in Federated Learning https://arxiv.org/abs/2108.02741", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "LeoSPxOL8b", "reviewer": "Reviewer_KBz5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies fair federated learning (FL) where each group of clients are guaranteed to have the same loss function value upper bound. Theoretical guarantees for both model convergence (for convex loss functions) and fairness are provided, together with experimental results that support the claims.", "summary_of_the_review": "This work is technically sound given the premise of the fairness formulation. However, this premise does not seem to satisfy the core principle of fairness -- treating all groups fairly.", "strength_and_weaknesses": "Strength:\n+ Given bounded group loss (BGL) as the fairness criterion, this paper develops a FL solution that optimizes empirical risks for different groups under absolute loss constraints. \n+ Theoretical analysis of convergence and fairness guarantee is provided.\n\nWeakness:\n- The biggest issue of this work is the fairness criterion that, in my opinion, violates the core principle of fairness. The BGL definition in Def. 1 uses an absolute average loss threshold for every (restricted) group $a \\in A$. The authors' argument to move away from the equity between different groups and move towards such absolute, one-size-for-all quality threshold is highly flawed. First, the authors argue that this makes the problem convex, but such consideration is based on whether the authors can technically solve the problem as opposed to what is the right metric for fairness. Second, the argument that this metric \"boosts the worst group's prediction quality\" is solely focused on one group's performance, as opposed to the fairness **among all groups**. Fundamentally, fairness must consider all stakeholders. Last but probably the most importantly, using a single threshold could essentially lead to one group (the so-called worst group) barely meeting this threshold, while other groups achieve significantly smaller loss values. The authors consider this case to be fair, but in reality, such prediction outcome discrepancy can still lead to unfair decision making. Additionally, choosing such threshold $\\zeta$ becomes non-trivial, as it significantly affects BGL. Having the algorithm fairness depend critically on hyperparameter tuning is dangerous and should be avoided when possible. As the whole work is built on such absolute threshold, I believe this is a critical issue that should be addressed.\n- As the authors stated in the Remark, $G$ is linear in $\\lambda$ and convex in $w_0$. I'm not sure why the optimal solution has to be defined w.r.t. a saddle point instead of the true optimum.\n- The novelty of this work is limited given the state of the art. The algorithmic approach described in Sec. 4.1 is a standard alternative optimization procedure. The technique used in the convergence analysis in Sec. 4.2 is very similar to the prior approach, e.g., in Li et al. 2019b. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject