Towards Multi-level Fairness and Robustness on Federated Learning
Abstract
Federated learning (FL) has emerged as an important machine learning paradigm where a global model is trained based on the private data from distributed clients.
However, federated model can be biased due to the spurious correlation or distribution shift over subpopulations, and it may disproportionately advantage or disadvantage some of the subpopulations, leading to the problem of unfairness and non-robustness.
In this paper, we formulate the problem of multi-level fairness and robustness on FL to train a global model performing well on existing clients, different subgroups formed by sensitive attribute(s), and newly added clients at the same time.
To solve this problem, we propose a unified optimization objective from the view of federated uncertainty set with theoretical analyses.
We also develop an efficient federated optimization algorithm named Federated
Mirror Descent Ascent with Momentum Acceleration (FMDA-M) with convergence guarantee.
Extensive experimental results show that FMDA-M outperforms the existing FL algorithms on multilevel fairness and robustness.
1. Introduction
Federated learning (FL) has emerged as an important machine learning paradigm where distributed clients (e.g., a large number of mobile devices or several organizations) collaboratively train a shared global model while keeping private data on clients (McMahan et al., 2017).
However, federated model can be biased because of possible spurious correlation and distribution shift over data subpopulations.
As a result, the model performance may degrade significantly on some data subpopulations and bring the problem
Published at the ICML
Invariance [Figure] [Figure] 𝒄′ of unfairness and non-robustness, which becomes an increasing concern, especially in some high-stakes scenarios such as loan approvals, healthcare, etc (Kairouz et al., 2019).
How to train an unbiased federated model with fair and robust performance is of paramount importance, and has become an important research theme in recent years (Mohri et al., 2019;
Li et al., 2019a; Wang et al., 2021).
It is unfair that the federated model disproportionately advantages or disadvantages some of the clients, since the purpose of clients participating in FL is to get a better model ( client-level fairness , Fig. 1(a)).
Motivated by it, recent research mainly focus on encouraging the federated model to have similar performance over different clients (Mohri et al., 2019;
Li et al., 2019a; Wang et al., 2021).
However, a federated model trained by client-level method may still suffer from ethical issues in real applications due to neglect of fairness at other levels.
For example, consider a scenario that several banks (clients) participate in FL to collaboratively train a loan approval model.
Different banks will have different customer demographic compositions (formed by sensitive attribute(s), such as gender).
Consider a federated model trained by client-level method, which can treat different banks fairly.
Although the model may perform well on male subpopulation, it is also unfair that the model cannot make accurate decision for female subpopulation ( attribute-level fairness , Fig. 1(b)).
Moreover, the fairness of federated model on those banks that newly participate in FL ( agnostic distribution fairness , Fig. 1(c)) is also not guaranteed.
A model violating any of the above fairness may lead to serious ethical problems, which naturally leads us to ask: Can we propose a unified FL framework to train a federated model achieving client-level, attribute-level and agnostic distribution fairness at the same time ?
In fact, a model with good fairness (e.g. a model with similar performance among subpopulations) yet low performance is meaningless.
The overall performance is a common metric, but in many real applications (e.g., medical diagnosis and credit evaluations), we are more concerned about FL model robustness, i.e. the worst-performing subpopulation.
In this paper, we focus on the problem of multi-level fairness and robustness on FL to train a federated model performing well on all subpopulations including existing clients, the subgroups formed by sensitive attribute(s), and newly added clients at the same time.
To address this problem, we propose a unified risk towards fair and robust FL from the view of federated uncertainty set (Delage & Ye, 2010;
Wiesemann et al., 2014; Duchi & Namkoong, 2017).
Theoretically, we prove that the proposed unified risk provides an upper bound for both client-level and attribute-level risks, which helps to deal with complex distribution shifts and thus guarantee fairness and robustness at multiple levels simultaneously.
We also develop an efficient federated optimization algorithm named Federated Mirror Descent Ascent with Momentum Acceleration (FMDA-M) to optimize the proposed risk with convergence guarantee.
Empirically, the advantages of our proposed FMDA-M method, in terms of multi-level fairness and robustness which refers to the worst performance in groups, are demonstrated under different kinds of distribution shift on three real-world datasets.
2. Problem Formulation 2.1. Preliminary on Federated Learning
Suppose that there are N clients in FL and each client i ∈ { 1 , 2 , . .
, N } is associated with a local dataset D ci = { ( x ci, 1 , y ci, 1 ) , . . .
, ( x ci,n ci , y ci,n ci ) } , where n ci is the sample size of client i .
Let D
=
{ D c 1 , · · · , D cN } be the full dataset with sample size n = (cid:80)
Ni =1 n ci .
Let P ci and P denote the data-generating distribution of each client data D ci and whole data D over X × Y , respectively.
In general, the basic goal of FL is to learn a global model with parameters θ ∈ Θ that performs well on distribution P (in terms of average performance) without accessing the private data of clients.
2.2. Multi-level Fairness and Robustness on FL
In this section, we first define metrics and introduce three levels in FL setting.
Then we formulate the problem of multi-level fairness and robustness on FL.
2.2.1. F AIRNESS AND R OBUSTNESS M ETRICS
Suppose that the full dataset D is divided into M groups:
D =
{ D g 1 , D g 2 , . . .
, D gM } .
We let P gi denote the datagenerating distribution of D gi .
We first define Disparity of a FL model across groups { D gi | i = 1 , 2 , . . .
, M } as:
[Equation]
where Acc ( D gi ) is the predictive accuracy on group D gi , and
Avg Acc = 1 M (cid:80) Mi
=1 Acc ( D gi ) .
In this paper, following the difference principle on distributive justice and stability (Rawls, 2001), we view the performance of federated model as the resource which is supposed to be allocated into groups fairly.
Specifically, we define fairness by Disparity , and the smaller Disparity , the fairer of a FL model.
Besides, we are also concerned about FL model robustness.
Robustness in FL refers to the performance of the worst group with the following definition: Robustness
[Equation]
In this paper, we focus on improving both the fairness (in terms of Disparity ) and Robustness of the FL model.
2.2.2. M ULTIPLE L EVELS IN F EDERATED S ETTING
Suppose that the groups { D g 1 , D g 2 , . . .
, D gM } are formed by a given sensitive variable S .
Note that the sensitive variable
S can be defined flexibly, and different sensitive variables
S correspond to different problems.
In this paper, we focus on the following three common cases: 1) client level : S specified as the index of existing clients; 2) attribute level :
S specified as sensitive attribute; 3) agnostic distribution :
S specified as the index of (potential) newly added clients with agnostic distribution.
We argue that a model violating any of the above fairness/robustness definitions may lead to serious ethical problems in reality, which motivates us to achieve multi-level fairness and robustness simultaneously.
2.2.3. M ULTI - LEVEL F AIRNESS AND R OBUSTNESS
Now we propose a novel and meaningful problem as below:
Problem 1 ( Multi-level Fairness and Robustness on FL ) .
Let the sensitive variable S be specified as existing client index c , the protected attribute(s) a , and newly added client index ad , respectively, then the dataset D can be split into groups { D ck | k
= 1 , 2 , . .
, M c } , { D ak | k = 1 , 2 , . . .
, M a } , and { D adk | k
= 1 , 2 , . .
, M ad } , respectively.
The task is to learn a federated model with small Disparity and large
Robustness on { D ck } , { D ak } , and { D adk } simultaneously.
3. Unified Risk and Federated Optimization
In this section, we first propose a unified risk to guarantee multi-level fairness and robustness on FL.
Then we develop an efficient federated optimization algorithm for it.
3.1. Unified Risk for Multi-level Fair and Robust FL
The essential issue of the existing single-level methods (including client level and attribute level) lies in the cases they consider are not bad enough, so that they cannot deal with complex distribution shifts or spurious correlation.
From the view of distributionally robust optimization (DRO), we should construct a wide enough uncertainty set that not only contains the client level and attribute level, but also contains the worse cases to help to adapt to newly added clients.
As one possible way, we specify S as the combination of the client index and the given sensitive attribute(s).
Specifically, we divide the local dataset D ci of client i into subgroups
D ci =
{ D ui, 1 , D ui, 2 , . . .
, D ui,M i } and consider the potential distribution shifts over them, where M i is the number of subgroups on client i .
Suppose that the samples of D ui,k are drawn from the distribution P ui,k .
Then we define:
[Equation]
where M = (cid:80) Ni =
1 M i is the total number of subgroups and λ ui,k is the weight of k -th subgroup on client i .
3.2. Tractable and Efficient Federated Optimization
To optimize the proposed risk, we first introduce the empirical risk on k -th group of client i defined as f i,k
( θ ) :=
E ( x,y ) ∼ ˆ P ui,k [ ℓ ( θ ; ( x, y ))] , where ˆ P ui,k is the empirical distribution over samples of group D ui,k .
Then, with the techniques of DRO (Wiesemann et al., 2014; Duchi &
Namkoong, 2017; Namkoong & Duchi, 2016), the problem of minimizing the risk in Eq.
(3) can be rewritten as: min
[Equation]
To solve this minimax problem, we can alternately optimize model parameters θ and weights λ u .
Instead of gradient ascent, we adopt mirror ascent method to update λ u .
It is impractical to naively extend the mirror method to federated setting, since frequent communication is not allowed in FL.
So we use an estimation technique to execute mirror update.
Besides, we choose the negative entropy function h ( x ) = (cid:80) n i =1 ( x i ln x i − x i ) for simplified calculation.
Then we can update λ u as the following:
[Equation]
where r is global communication round, E is the number of local iterations, γ is stepsize, and v i,k is loss of subgroup D ui,k .
The proposed update rule makes our algorithm more practical by significantly reducing computation complexity and communication cost.
Since communication costs are the principal constraint in FL (McMahan et al., 2017), we explore to further improve the convergence rate of the above federated algorithm by leveraging momentum acceleration techniques (Nesterov, 1983;
Li et al., 2017; Ochs, 2018).
Specifically, we additionally update model parameters as below:
[Equation]
and update group weights according to the following rule:
[Equation]
where β θ and β λ are momentum coefficients.
The second terms of step (6) and step (7) are momentum terms, which contains historical gradient information and helps speed up the convergence of our algorithm.
The details of Federated Mirror Descent Ascent with Momentum Acceleration (FMDA-M) are in Appendix.
4. Theoretical Analysis
In this section, we provide theoretical analysis for our proposed unified risk and convergence guarantee for FMDA-M.
Theorem 4.1.
Let ˆ P ci , ˆ P ai and ˆ P ui,k be the empirical distributions over samples of local dataset D ci , D ai , and group
D ui,k respectively, ˆ Q c := { (cid:80) Ni =1 λ ci ˆ P ci : λ c ∈ ∆ N − 1 } be the client-level uncertainty set, ˆ Q a :=
{ (cid:80)
Ai =1 λ ai ˆ P ai : λ a ∈ ∆ A − 1 } be the attribute-level uncertainty set, and ˆ Q u :=
{ (cid:80) Ni =1 (cid:80)
M i
k
=1 λ ui,k ˆ P ui,k : λ u ∈ ∆ M − 1 } be the unified uncertainty set.
We have
[Equation]
Moreover, let ˆ R client ( θ ) , ˆ R attribute ( θ ) and ˆ R unified ( θ ) be the empirical risks based on uncertainty sets ˆ Q c , ˆ Q a and ˆ Q u respectively.
We have ˆ
[Equation]
( θ ) , ˆ R attribute ( θ ) ≤ ˆ R unified ( θ ) .
Furthermore, assume that ∃ i and k , and j ( j ̸ = i ) s.t.
the attribute of samples from D ui,k is same as D uj,k
but ˆ P ui,k ̸
= ˆ P uj,k , and ˆ P ui,k ̸
= ˆ P cl for ∀ l , then we have
[Equation]
See Appendix for the proof.
Theorem 4.1 shows that both client-level and attribute-level uncertainty sets are subsets of our proposed unified uncertainty set.
Therefore, our proposed risk (3) provides an upper bound for both clientlevel risk and attribute-level risk, thereby optimizing it can guarantee client-level fairness and attribute-level fairness simultaneously.
Actually, our proposed risk also considers the worse cases that the set union of client-level uncertainty set and attribute-level uncertainty set does not contain, which helps to deal with more complex distribution shifts and adapt to those newly added clients with agnostic distributions.
Theorem 4.2.
Let R ui,j ( θ ) :
= E ( x,y ) ∼ P ui,j [ ℓ ( θ ; ( x, y ))] be a risk defined on D ui,j , λ u ∈ ∆ M − 1 be the group weights, M be the total number of groups, ¯ R u ( θ ) be the average of group risks, d i,j :=
( R ui,j ( θ ) − ¯ R u ( θ )) 2 and
Var( R ui,j ( θ )) :
= 1 M (cid:80) Ni =1 (cid:80)
M i j =1 d
i,j be the variance of group risks.
If ∥ M λ u − 1 ∥ 22 ≤ min i,j { (cid:80) Ni =1 (cid:80) Mij =
1 d i,j d i,j } , then there exists a constant C > 0 such that
[Equation]
See Appendix for the proof.
The theorem above shows that our proposed risk R unified can be viewed as a combination of the average risk that helps improve the average performance and the variance term that encourages the model to have a uniform performance across different subgroups.
Theorem 4.3.
Suppose that each function f i,k is convex and
L-smooth, global function F is linear in λ and L-smooth and the gradient w.r.t.
θ and λ , model parameters θ , the variance of stochastic gradient method w.r.t.
θ and λ are bounded.
If we optimize (4) using FMDA-M algorithm with local iterations E =
O ( T 14 ) , learning rate for model parameters
η
= O
( T − 12 ) and stepsize for group weights γ
= O ( T − 12 ) , and then it holds that
[Equation]
O ( T − 12 ) .
See Appendix for the proof.
Here we give the convergence rate of the proposed FMDA-M algorithm in convex setting.
5. Experiments
In this section, we validate the effectiveness of our method on a real-world, large-scale, and challenging dataset.
Additional experiment results and discussion are in Appendix.
5.1. Experimental Setup
Federated Dataset.
We use ACS (Ding et al., 2021), which is collected from US Census surveys and consists of more than 1,500,000 samples.
The goal is to predict whether an individual earns greater than 50,000 US dollars a year.
We consider 50 states as clients in FL and choose gender as the sensitive attribute.
We also evaluate different methods on
Fashion-MNIST (Xiao et al., 2017), Digit-Five (Xu et al., 2018; Peng et al., 2019; Zhao et al., 2020), and Adult (Dua & Graff, 2017) datasets, and the details are in Appendix.
Evaluation Metrics.
We evaluate models’ fairness and robustness on attribute level, client level, and agnostic distribution.
In each kind of fairness, we use Disparity in Eq.
[Table]
to measure the degree of fairness, and use Robustness in
Eq.
(2) to measure the robustness.
Besides, we also report
Equalized Odds (EO) at attribute level.
The average accuracy of the models are similar/comparable for all algorithms, and reported in appendix.
Baselines.
We compare the proposed FMDA-M algorithm with the following baselines: (i)
FedAvg (McMahan et al., 2017):
FedAvg is a commonly used algorithm in FL, which minimizes an average risk.
(ii) q-FFL (Li et al., 2019a)
(client level).
(iii) TERM (Li et al., 2020)
(client level).
(iv)
DRFA (Deng et al., 2020) (client level, an improvement on the AFL (Mohri et al., 2019)).
(v) FADE (Hong et al., 2021) (attribute level).
(vi) Individual-level Algorithm (denoted as IndA for convenience): We use the same algorithm as
FMDA-M to solve the individual-level problem with objective (13) as a compared baseline.
Results and Discussion.
As shown in Table 1, our proposed
FMDA-M outperforms all baselines in terms of multi-level fairness and robustness on FL.
For client level and attribute level, we find that FMDA-M also outperforms single-level baselines.
We think the reason is that we adopts mirror method, which prevents the weights from being too hard to guarantee convergence stability.
For agnostic distribution, FMDA-M also perform well due to the wider federated uncertainty set.
We note that the performance gap between attribute-level method and FMDA-M on clientlevel fairness/robustness is not large, because the biased among clients may not strong enough.
We also find FMDAM achieves low EO, since EO can be viewed as a relaxed version of our fairness notion if S is specified as the combination of target label and sensitive attribute.
Individual method is ideal but impractical since it is usually leads to the over pessimism problem in practice.
6. Conclusion
In this paper, we formulate the goal of multi-level fairness and robustness on FL, which is to achieve client-level, attribute-level and agnostic distribution fairness and robustness simultaneously.
To achieve it, we propose a unified risk based on DRO and develop an efficient FMDA-M algorithm.
Both theoretical analysis and experimental results demonstrate the effectiveness of our method.
7. Acknowledgements
This work was supported by the National Natural Science
Foundation of China (U19B2043, U19B2042), Zhejiang
Innovation Foundation(2019R52002) and Program of Zhejiang Province Science and Technology (2022C01044)
A. Related Work
Fairness in Machine Learning.
Fairness in ML has attracted much attention, which can be divided into two branches: individual fairness and group fairness (Zemel et al., 2013; Awasthi et al., 2020; Binns, 2020).
Individual fairness encourages the models to treat similar individuals similarly (Biega et al., 2018; Sharifi-Malvajerdi et al., 2019; Mukherjee et al., 2020), while group fairness requires that the model treats different groups equally (Dwork et al., 2012; 2018).
Here we mainly focus the latter one, which is typically defined via some protected attribute(s) and a metric such as statistical parity, equalized odds (Hardt et al., 2016), and predictive parity (Chouldechova, 2017).
In this paper, we consider how to learn a fair global model that achieves a uniform performance across groups in FL.
Federated Learning and Fairness.
FL has received much attention as an important distributed learning paradigm (McMahan et al., 2017).
The scope of federated learning studies is broad, which includes statistical challenges (Zhao et al., 2018;
Yu et al., 2020), privacy protection (Truex et al., 2019), systematic challenges (Koneˇcn`y et al., 2016b;a; Suresh et al., 2017), fairness (Mohri et al., 2019; Li et al., 2019a), etc (Kairouz et al., 2019;
Yang et al., 2019).
The existing studies of fairness on FL can be divided into three categories: performance fairness across clients (Li et al., 2019a;
Mohri et al., 2019; Deng et al., 2020; Li et al., 2021), model fairness defined on sensitive attributes (Du et al., 2021) and incentive mechanism (Kang et al., 2019; Zhan et al., 2020).
The most relevant work is (Cui et al., 2021), which aims to address algorithm disparity and performance inconsistency in client level.
However, they focus on the client level and train a federated model with similar performance and fairness metrics among different client, while the problem we study involves multiple levels.
Specifically, the attribute-level fairness they define is a local fairness notion (in each client), while ours is a global fairness notion.
Besides, we also pay attention to out-of-distribution fairness.
In this paper, we focus on the performance fairness across clients (including existing clients and newly added clients) and sensitive attribute(s).
Distributionally Robust Optimization.
There has been a surge of interest in distributionally robust optimization (DRO), which can deal with distribution shifts by considering a potential distribution set around the original distribution and optimizing the worst case (Delage & Ye, 2010; Wiesemann et al., 2014).
There are mainly two definitions of distance between distributions in DRO: f -divergences and Wasserstein distance .
The former method is effective when the support of the distribution is fixed (Duchi & Namkoong, 2017; Namkoong & Duchi, 2016; Duchi & Namkoong, 2021), while
Wasserstein distance-based DRO considers the potential distributions with different supports and allows robustness to unseen data, but is difficult to optimize (Sinha et al., 2018; Esfahani & Kuhn, 2018; Liu et al., 2021).
Recently, some studies about group DRO have emerged (Hu et al., 2018; Sagawa et al., 2020; Oren et al., 2019), which considers the distribution shifts over groups.
In this paper, we extend DRO to FL setting for unified group fairness.
B. Discussion of Fairness Notions
Note that there are a lot of fairness notions.
We argue that different notions correspond to different practical meanings, and there is no universally accepted fairness notion.
Then we briefly discuss the relationship between our fairness notions and others.
First, different from Equalized Odds (EO) and so on, our fairness notion naturally supports multi-attribute & multi-class settings, which is more practical .
Then, for the binary setting, if we specify the sensitive variable S as sensitive attribute, our notion will degrade to Accuracy Parity.
If we specify the S as the combination of target label and sensitive attribute, EO can be viewed as a relaxed version of our notion.
Therefore, our notion is general and flexible .
We recommend setting S based on expert knowledge in practice.
C. Discussion of Individual-level Approach
Here we discuss an ideal but impractical way for unified group fairness by treating each sample as a group.
Then, the framework will degenerate to an individual-level fairness method with risk:
[Equation]
where P is the data-generating distribution of full dataset D , f : R + → R is a convex function with f (1) = 0 , D f ( Q ∥ P )
= (cid:82) ( X , Y ) f
(cid:16) dQdP (cid:17) dP is f-divergence between distribution Q and P defined on ( X , Y ) and r is radius of uncertainty set Q ind .
Note that the individual-level method is more capable of modeling the agnostic distribution by constructing a wide uncertainty set around distribution P .
Intuitively, it may help to guarantee unified group fairness.
Unfortunately, the uncertainty set defined on individual level is usually overwhelmingly large leading to the over pessimism problem in practice (Hu et al., 2018; Sagawa et al., 2020; Liu et al., 2021).
In fact, our proposed unified uncertainty set Q u can be considered as a subset of Q ind by imposing some structural constrains.
Hence, By contrasting with risk (13), our proposed risk (3) provides a relatively tight upper bound for both client-level risk and attribute-level risk, and help to overcome this pessimism.
D. FMDA-M Algorithm
The details of our FMDA-M algorithm are summarized in Algorithm 1.
FMDA-M consists of two main steps in each round: update of model parameters (lines 2 to 15 in Algorithm 1) and update of group weights (lines 16 to 23 in Algorithm 1).
Algorithm
Input
Server samples t ′ from rE + 1 to ( r + 1) E uniformly 4:
Server broadcasts θ ( rE ) and λ ui,k ( r ) to corresponding client i ∈
U ( r ) 5: for client
i ∈
U ( r ) do 6:
Set local model parameters θ ( rE ) i
= θ ( rE ) 7: for t = rE, rE + 1 , ..., ( r + 1) E − 1 do 8:
Select a group D ui,k with probability λ ui,k ( r ) /λ ci ( r ) 9:
Sample data ξ ( t ) i from D ui,k uniformly 10:
Update model: θ ( t +1) i
= θ ( t ) i − η ∇ l (cid:16) θ ( t ) i ; ξ ( t ) i (cid:17) 11: end for 12: end for 13:
Client i ∈
U ( r ) sends θ (( r +1) E ) i and θ ( t ′ )
i to the server 14:
Server computes: (cid:101) θ ( r +1) E
= 1 K (cid:80) i ∈
U ( r ) θ (( r +1) E ) i 15:
Server updates global model parameters with momentum: θ ( r +1) E
= (cid:101) θ ( r +1) E
+ β θ ( (cid:101)
θ ( r +1) E − (cid:101) θ rE ) 16:
Server computes: θ ( t ′ )
= 1 K (cid:80) i ∈
U ( r ) θ ( t ′ )
i 17:
Server samples a subset of clients U ( r ) ∗ ⊂
[ N ] with size of K uniformly 18:
Server broadcasts θ ( t ′ ) to client
i ∈
U ( r ) ∗ 19: for client i ∈ U ( r ) ∗ do do 20:
Compute loss v ( r ) i,k of model θ ( t ′ ) on a minibatch of each group D ui,k 21: end for 22:
Server computes: ( (cid:101) λ
ui,k ) ( r +1)
=
( λ ui,k ) ( r ) e γEv
[Equation]
Server updates global weights with momentum: ( λ u ) ( r +1) =
( λ u ) ( r ) + β λ
(( (cid:101) λ u ) ( r +1) − ( λ u ) r ) 24: end for 25: return θ ( T ) , λ ui,k
( R ) ( k = 1 , 2 , . . .
, M i , i = 1 , 2 , . . .
, N )
E. Proof E.1. Proof of Theorem 4.
Proof.
For ∀ Q ∈ ˆ Q c , it can be expressed as follows:
[Equation]
where n ui,k is the sample size of group D ui,k , n ci is the sample size of local dataset D ci , (cid:80)
Ni
λ ci = 1 and λ ci ≥ 0 , i = 1 , 2 , . .
N .
If we let λ ui,k := λ ci n ui,k n ci , it is obvious that (cid:80) Ni =1 (cid:80)
M i
k
=1 λ ci n ui,k n ci =
and λ ci n ui,k n ci ≥ 0 , i = 1 , 2 , . .
N , k = 1 , 2 , . .
, M i , and thus Q ∈ ˆ Q u .
So we have
[Equation]
Note that the feasible set (uncertainty set) of client-level empirical risk ˆ R client is a subset of that of group-level empirical risk ˆ R group , so we have ˆ R client (
[Equation]
( θ
[Equation]
[Equation]
Similarly, we also have
[Equation]
(17) and ˆ
[Equation]
Let the assumption in Theorem 4.2 hold: ∃ i and k , and j ( j ̸ = i ) s.t.
the attribute of samples from D ui,k is same as D uj,k
but ˆ P ui,k ̸
= ˆ P uj,k , and ˆ P ui,k ̸
= ˆ P cl for ∀ l .
Note that
ˆ P ui,k / ∈ ˆ Q c , ˆ P ui,k / ∈
ˆ Q a and ˆ P ui,k ∈
ˆ Q u .
Therefore, we have
[Equation]
E.2. Relationship between Unified Uncertainty Set ˆ Q u and Individual-level Uncertainty Set ˆ Q ind
Our proposed unified uncertainty set ˆ Q u can be considered as a subset of ˆ Q ind imposing some structural constrains.
Compared with individual-level risk, our proposed group-based unified risk provides a relatively tight upper bound for both client-level risk and attribute-level risk.
We give theoretical analysis as the following:
Proof.
For ∀ Q ∈ ˆ Q u , it can be expressed as follows:
[Equation]
Without loss of generality, if we define convex function f ( t ) as f ( t ) :
= t · log t , then we have
[Equation]
So if (cid:80) Ni =1 (cid:80)
M i
k
=1 λ ui,k log( nn ui,k λ ui,k ) ≤ r , we have Q ∈ ˆ Q ind and thus ˆ Q u ⊆
ˆ Q ind .
Note that the feasible set (uncertainty set) of group-level unified empirical risk ˆ R unified is a subset of that of individual-level empirical risk ˆ R ind , so we have
[Equation]
E.3. Proof of Theorem 4.
Proof.
Recall that the group-level risk R group ( θ ) is defined as
[Equation]
With the techniques of distributional robustness optimization (Wiesemann et al., 2014; Duchi & Namkoong, 2017; Namkoong & Duchi, 2016), the problem of minimizing the risk in Eq.
(24) can be rewritten as:
[Equation]
Inspired by (Duchi & Namkoong, 2017), we introduce an instrumental variable u defined as
[Equation]
[Equation]
where λ g =
( λ g 1 , λ g 2 , . . .
, λ M ) and u = ( u 1 , u 2 , . . .
, u M ) .
Then the objective function of Eq.
(25) can be rewritten as
[Equation]
With Cauchy–Schwarz inequality [Equation] we have
[Equation]
The equality is attained if and only if
[Equation]
Recall that u i = λ
gi − 1 M , which requires that for ∀ i ,
[Equation]
[Equation]
If ∥ M λ g − 1 ∥ 22 ≤ min i { (cid:80) Mi =1 d i d i } , then for ∀ i , we have (cid:115)
∥ u ∥ 22 · d i (cid:80) Mi =1 d i ≤
M , (31) and thus Eq.
(30) holds.
E.4. Proof of Theorem 4.
We analyze the convergence rate of Algorithm 1 by bounding the error ε T defined as:
[Equation]
[Equation] where T is the number of total iterations, ¯ θ ( T ) is the average of global model parameters of T iterations and ¯ λ u ( T ) is the average of group weights of T iterations.
Before it, we first introduce some technical lemmas.
Lemma E.1.
The stochastic gradient u ( t ) defined as
[Equation]
33) is unbiased, and its variance is bounded, which implies:
[Equation]
where B > 0 is a constant bound.
Proof.
The stochastic gradient u ( t ) is unbiased due to the fact that we sample the groups according to λ u ( ⌊ tE ⌋ ) .
The variance term is due to the assumption in Theorem 5.1.
Inspired by (Li et al., 2019b), we introduce the gradient dissimilarity Γ defined as
[Equation]
where f i ( θ ) is the local objective of client i .
Lemma E.2.
Define δ ( t ) :
= 1 K (cid:80) i ∈
U
⌊ tE ⌋ )
∥ θ ( t ) i − θ ( t ) ∥ 22 .
For FMDA-M, the expected average squared norm distance of local models θ ( t ) i , i ∈ U
( ⌊ tE ⌋ ) and θ ( t ) is bounded as follows:
[Equation]
where expectation is taken over sampling of devices at each iteration.
Proof.
Considering rE ≤ t ≤ ( r + 1) E , we have:
[Equation]
Using Jensen’s inequality
[Equation]
we have:
[Equation]
Then we sum the above equation over t = rE to ( r + 1) E to get:
[Equation]
Now we sum the above equation over r = 0 to R − 1 , and we have:
[Equation]
Lemma E.3.
For FMDA-M, under the same conditions as in Theorem 5.1, for all θ , we have:
[Equation]
Proof.
According to the stochastic gradient method
[Equation]
we have
[Equation]
[Equation]
We first bound the second term in Eq.
(43) by the properties of smoothness and convexity:
[Equation]
Then we bound the third term in Eq.
(43) as follows:
[Equation]
By plugging Eq.
(44), Eq.
(45) and Eq.
(51) back to Eq.
(43), we have:
[Equation]
[Equation]
Lemma E.4.
The stochastic gradient at λ u generated by Algorithm 1 is unbiased, and its variance is bounded, which implies:
[Equation]
where B > 0 is a constant bound.
Proof.
The stochastic gradient at λ u is unbiased due to we sample the groups uniformly.
The variance term is due to the assumption in Theorem 5.1.
Lemma E.5.
For FMDA-M, under the assumption of Theorem 5.1, assuming the function h is an α -strongly convex function, then the following holds true for any λ u ∈ ∆ m − 1 :
[Equation]
Proof.
The proof is based on the update rule of mirror ascent and others are similar to Lemma E.1.
Note that we can pick the right function h to make the assumption hold.
Now we can give the proof of Theorem 5.1.
Proof.
With above lemmas, we can prove the Theorem 5.1.
By the convexity of global function w.r.t.
θ and its linearity in terms of λ u , we have:
[Equation]
We first bound the first term in Eq.
Sum the last term in Eq.
(43) over t = 0 to T − 1 to get: T
[Equation]
[Equation]
Then we plug Lemma E.1 and E.2 into Lemma E.3 and sum over t = 1 to T to get:
[Equation]
To bound the second term in Eq.
(50), plugging Lemma E.4 into Lemma E.
[Equation]
we have:
[Equation]
Taking max over λ u , min over θ , we have
[Equation]
Plugging in E
= O ( T 14 ) ,
η
=
O
( T − 12 ) , and γ =
O ( T − 12 ) , we complete the proof.
[Equation]
[Figure]
(f) D5: Weakly Non-IID
[Figure]
(g) [Figure] : Strongly Non-IID
[Figure]
h) D5: Extremely Non-IID
A
C 295
C
C C
A
A A 147/148/ 295 590/ 737/6662/1179/ 1199/ 9455/  3000/
[Figure]
{+(i) Adult: IID A C 148 
 C+}
C C A
A A 147/148/ 295 590/ [Figure] F. Additional Experimental Results F.1. Additional Federated Datasets.
(1) Fashion-MNIST (FM) dataset (Xiao et al., 2017): FM is a classical image classification dataset containing 60, training examples with 10 categories.
For FM, we set the target label as the sensitive attribute and consider label distribution shift across clients.
As shown in Figure 2, we consider 4 different degrees of Non-IID settings: (a) IID, (b) weakly Non-IID, (c) strongly Non-IID, (d) extremely Non-IID.
We run our algorithm and compared baselines on FM dataset with logistic regression model.
(2) Digit-Five (D5) dataset (Xu et al., 2018; Peng et al., 2019; Zhao et al., 2020): D5 includes digit images with 10 categories sampled from 5 domains.
For D5, we set the domain (i.e. data collection source) as the sensitive attribute and consider 4 different degrees of feature distribution shift across clients.
We use a 2-layer CNN with a linear classifier.
(3) UCI Adult dataset(Dua & Graff, 2017):
Adult is a census dataset with 32,561 examples, and each sample has 14 features (including race, gender and so on) and a target label indicating whether the income is greater or less than $50K.
For Adult, we set gender and income as the protected attributes and consider 4 different degrees of unbalance setting (i.e. the amount of data varies greatly across both clients and attributes), which is hard to avoid in real FL scenario.
We use a logistic regression model to predict the income.
[Table]
F .2. Additional Training Distribution Setting
Fashion-MNIST.
For Fashion-MNIST, we set the target label as the sensitive attribute, which can take on 10 values.
As shown in the first row of Figure 2, we consider label distribution shift across clients and split the training dataset intoclients in 4 manners: (a) IID, (b) weakly Non-IID, (c) strongly Non-IID, (d) extremely Non-IID.
Digit-Five.
For Digit-Five, we set the domain (i.e. data collection source) as the sensitive attribute, which can take onvalues.
As shown in the second row of Figure 2, we consider feature distribution shift across clients and split the training dataset into 5 clients in 4 manners: (e) IID, (f) weakly Non-IID, (g) strongly Non-IID, (h) extremely Non-IID.
Adult.
For Adult, we set the combination of gender and income (target label) as the sensitive attribute, which can take on 4 values: high-income male, low-income male, high-income female, low-income female.
As shown in the third row of Figure 2, we consider both label distribution shift , feature distribution shift and unbalance across clients and split the training dataset into 4 clients in 4 manners: (i) IID, (j) weakly Non-IID, (k) strongly Non-IID, (l) extremely Non-IID.
F.3. Hyper-parameter Setting
Fashion-MNIST.
We use a logistic regression model to predict the target of images.
We set the number of local iterations
E = 10 , the number of rounds R = 500 , batchsize is 50, learning rate for model parameters
η =1e-2, stepsize for group weights γ =1e-2 and momentum coefficients β θ = β λ = 0 .
Digit-Five.
We use a 2-layer CNN with a linear classifier to classify the images.
We set the number of local iterations
E = 10 , the number of rounds R = 500 , batchsize is 50, learning rate for model parameters η =2e-2 and stepsize for group weights γ =2e-2 and momentum coefficients β θ = β λ = 0 .
Adult.
We use a logistic regression model to predict the income.
We set the number of local iterations E = 10 , the number of rounds R = 500 , batchsize is 50, learning rate for model parameters
η =1e-2 and stepsize for group weights γ =1e-2 and momentum coefficients β θ = β λ = 0 .
F.4. Results of Attribute-level Fairness
We evaluate the attribute-level fairness of models trained by FMDA-M and compared baselines.
The results are reported in
Table 2.
From the results, we observe that FMDA-M outperforms baselines on three datasets, in terms of both the metric
Disparity and Robustness .
As we analyzed in the previous section, the client-level method is not flexible enough to deal with distribution shifts over attributes, and the individual-level method is too conservative to perform well in practice.
By constructing an appropriate uncertainty set, FMDA-M achieves good performance which is very similar to centralized DRO, even in Non-IID settings.
The results on Adult dataset demonstrate that the unbalance of dataset is a great challenge for training a fair model, especially
[Table]
[Figure]
[Table]
[Figure]
in FL where the data distribution of each client is unknown to others.
We find that the overfitting appears in FedAvg,
DRFA and individual-level method due to the jumbo sample size of the low-income male group.
By contrast, our proposed
FMDA-M samples from each subgroup according to the group weights, thus overcoming this challenge.
Note that FMDA-M also outperforms DRFA in extremely Non-IID setting, where the unified group fairness optimized in our
FMDA-M is exactly the client-level fairness optimized in DRFA.
The reason for this is that DRFA uses naive gradient ascent method to update weights and the projection operator usually leads to the very hard weights, which may affect the stability of algorithm.
By contrast, our algorithm adopts Eq.
(5) to generate smoother weights
and it helps the model to converge.
F.5. Results of Client-level Fairness
We evaluate the client-level fairness of models trained by different algorithm and report the results in Table 3.
We observe that FMDA-M is able to guarantee the accuracy of the worst-performing client and decrease Disparity in most training distribution settings.
We also find that, unlike other methods of which the performance decreases significantly with increasing degree of Non-IID, FMDA-M shows extremely stable performance under various settings, which is thanks to the weights update rule (5) we adopt.
We note that Robustness of FMDA-M is slightly lower than FedAvg and DRFA in IID setting, because the distributions of clients are very similar and the Robustness will degrade to average accuracy, which is in line with the optimization objective of FedAvg and DRFA.
Indeed, our FMDA-M significantly improves client-level fairness in Non-IID settings (more challenging and more common in reality), though occasionally with a small performance sacrifice in IID setting.
F.6. Results of Agnostic Distribution Fairness
To evaluate the agnostic distribution fairness, we simulate the newly added clients as follows: we train each federated model in one of the the training distribution settings (e.g., IID setting), but test under other three settings (e.g., weakly Non-IID, strongly Non-IID and extremely Non-IID settings) where the distributions of clients are different and agnostic from the existing clients.
The results of agnostic distribution fairness are shown in Figure 3.
We find that our FMDA-M outperforms compared baselines in terms of both Robustness and Disparity in most cases, which illustrates that our FMDA-M is better adapted to new distributions.
As we state before, the proposed FMDA-M considers a larger uncertainty set but with appropriate degrees of freedom, so the model trained by FMDA-M can deal with kinds of new distributions.
However, the resulting model can be overly pessimistic when the radius of uncertainty set is too large.
Besides, the individual-level method is hard to optimize, and that is why the individual-level method does not perform well.
F.7. Average Performance
Table 4 shows that the average accuracy of model over groups and clients are similar for all algorithms on Fashion-MNIST and Digit-Five.
On Adult, FMDA-M improves the average performance over groups, while the average accuracy over clients of the FMDA-M is slightly lower than the compared baselines.
There is a serious imbalance in Adult dataset.
Specifically, as shown in the third row of Figure 2, the sample size of low-income male group is much larger than high-income female group.
Our FMDA-M focuses on the performance of minority group.
As we see in the previous experiments, our FMDA-M improves the accuracy of the minority group by about 50% (Table 2).
And it is inevitable that a bit of performance of majority group will be lost.
There is a large amount of data of low-income male group on clients, and thus the average accuracy over clients will go down.
Indeed, our FMDA-M significantly improves attribute-level fairness and shows extremely stable performance on agnostic distribution, though occasionally with a small average performance sacrifice.
F.8. Efficiency and Ablation Study
To evaluate the efficiency of our FMDA-M algorithm and demonstrate how it works, we run the following algorithms on
Fashion-MNIST dataset in extremely Non-IID setting: (i) DRFA
(Deng et al., 2020):
DRFA algorithm is proposed to solve the min-max problem in federated setting.
(ii) FMDA.
(iii) FMDA-M ( β = β θ = β λ = 0 . 3 ).
(iv) FMDA-M ( β = β θ = β λ = 0 . 5 ).
Note that the above algorithms share the same optimization objective.
We report the learning curves of models in terms of attribute-level robustness and fairness over 300 rounds of communications, as shown in Figure 4.
Results in other settings are in the appendix.
The results show that the proposed FMDA outperforms DRFA in terms of convergence rate.
The most likely reason is that
FMDA adopt mirror ascent based on Bregman divergence, instead of projection operation based on Euclidean distance, to update the group weights, which prevents the weights from being too hard to guarantee convergence stability.
We observe that FMDA-M is more efficient and can achieve the same level as others with fewer number of communication rounds, because the momentum term can modify the direction of the current gradients to accelerate convergence.
To demonstrate how the momentum term works, we do further experiments on Fashion-MNIST dataset.
By denoting the parameters of convergent federated model as θ opt and denoting the parameters of current federated model as θ current , we can define the optimal update direction as d opt = θ current − θ opt .
Then we can get the cosine similarity between d opt and current gradient g , and the cosine similarity between d opt and the gradient modified by momentum d mod .
The results are shown in Figure 6.
We observe that the gradient modified by momentum d mod is more similar to the optimal update direction as d opt , which means that the momentum term can correct the gradients and accelerate the convergence.
[Figure]
[Table]
[Figure] [Figure]