H ANDLING C OVARIATE S HIFTS IN F EDERATED L EARNING WITH G ENERALIZATION G UARANTEES
Anonymous authors
Paper under double-blind review
A BSTRACT
Covariate shift across clients is a major challenge for federated learning (FL).
This work studies the generalization properties of FL under intra-client and interclient covariate shifts.
To this end, we propose Federated Importance-weighteD
Empirical risk Minimization (FIDEM) to optimize a global FL model, along with new variants of density ratio matching methods, aiming to handle covariate shifts.
These methods trade off some level of privacy for improving the overall generalization performance.
We theoretically show that FIDEM achieves smaller generalization error than classical empirical risk minimization under some certain settings.
Experimental results demonstrate the superiority of FIDEM over federated averaging (McMahan et al., 2017) and other baselines, which would open the door to study FL under distribution shifts more systematically
I NTRODUCTION
Federated learning (FL) (Li et al., 2020; Kairouz et al., 2021; Wang et al., 2021) is an efficient and powerful paradigm to collaboratively train a shared machine learning model among multiple clients, such as hospitals and cellphones, without sharing local data.
Existing FL literature mainly focuses on training a model under the classical empirical risk minimization (ERM) paradigm in learning theory, with implicitly assuming that the training and test data distributions of each client are the same.
However, this stylized setup overlooks the specific requirements of each client.
Statistical heterogeneity is a major challenge for FL, which has been mainly studied in terms of non-identical data distributions across clients, i.e., inter-client distribution shifts (Li et al., 2020; Kairouz et al., 2021;
Wang et al., 2021).
Even for a single client, the distribution shift between training and test data, i.e., intra-client distribution shift, has been a major challenge for decades (Wang & Deng 2018; Kouw & Loog 2019, and references therein).
For instance, scarce disease data for training and test in a local hospital can be different.
To adequately address the statistical heterogeneity challenge in FL, we need to handle both intra-client and inter-client distribution shifts under stringent requirements in terms of privacy and communication costs.
We focus on the overall generalization performance on multiple clients by considering both intraclient and inter-client distribution shifts.
There exist three major challenges to tackle this problem: 1) how to modify the classical ERM to obtain an unbiased estimate of an overall true risk minimizer under intra-client and inter-client distribution shifts; 2) how to develop an efficient density ratio estimation method under stringent privacy requirements of FL; 3) are there theoretical guarantees for the modified ERM under the improved density ratio method in FL?
We aim to address the above challenges in our new paradigm for FL.
For description simplicity, in our problem setting, we focus on covariate shift, which is the most commonly used and studied in theory and practice in distribution shifts (Sugiyama et al., 2007; Kanamori et al., 2009; Kato & Teshima, 2021; Uehara et al., 2020; Tripuraneni et al., 2021; Zhou & Levine, 2021).
1 To be specific, for any client k , covariate shift assumes the conditional distribution p tr k ( y | x ) =
p te k
( y | x ) :
= p ( y | x ) remains the same; while marginal distributions p tr k ( x ) and p te k
( x ) can be arbitrarily different, which gives rise to intra-client and inter-client covariate shifts .
Handling covariate shift is a challenging issue, especially in federated settings (Kairouz et al., 2021).
To this end, motivated by Sugiyama et al.
(2007) under the classical covariate shift setting, we propose Federated Importance-weighteD Empirical risk Minimization (FIDEM), that considers covariate shifts across multiple clients in FL.
We show that the learned global model under intra/inter-client covariate shifts is still unbiased in terms of minimizing the overall true risk, i.e.,
FIDEM is consistent in FL.
To handle covariate shifts accurately, we propose a histogram-based density ratio matching method (DRM) under both intra/inter-client distribution shifts.
Our method unifies well-known DRMs in FL, which has its own interest in the distribution shift community for ratio estimation (Zadrozny, 2004; Huang et al., 2006; Sugiyama et al., 2007; Kanamori et al., 2009; Sugiyama et al., 2012; Zhang et al., 2020; Kato & Teshima, 2021).
To fully eliminate any privacy risks , we introduce another variant of FIDEM, termed as Federated Independent ImportanceweighteD Empirical risk Minimization (FIIDEM).
It does not require any form of data sharing among clients and preserves the same level of privacy and same communication costs as those of baseline federated averaging (FedAvg) (McMahan et al., 2017).
An overview of FIDEM is shown in Fig.
T ECHNICAL CHALLENGES AND CONTRIBUTIONS
Learning on multiple clients in FL under covariate shifts via importance-weighted ERM is challenging due to multiple data owners with own learning objectives, multiple potential but unpredictable train/test shift scenarios, privacy, and communication costs (Kairouz et al., 2021).
To be specific, 1) It is non-trivial to control privacy leakage to other clients while estimating ratios and relax the requirement to have perfect estimates of the
supremum over true ratios, which is a key step for nonnegative BD (nnBD) DRM.
Our work is the first step towards handling inter/inter-client distribution shifts in FL; 2) It is challenging to obtain per-client generalization bounds for a general nnBD DRM with multiple clients and imperfect estimates of the supremum due to intra/inter-client couplings in ratios.
Note that, even if we have access to perfect estimates of density ratios, it is still unclear whether importanceweighted ERM results in smaller excess risk compared to classical ERM.
Our work gives an initial attempt by providing an affirmative answer for ridge regression; 3) While well-established benchmarks for multi-client FL have been used, they are usually designed in a way that each client’s test samples are drawn uniformly from a set of classes.
However, we believe this might not be the case in real-world applications and then design realistic experimental settings in our work.
To address those technical challenges we
• Algorithmically propose an intuitive framework to minimize average test error in FL, design efficient mechanisms to control privacy leakage while estimating ratios (FIDEM) along with a privacy-preserving and communication-efficient variant (FIIDEM), and improve nnBD DRM under FL without requiring perfect knowledge of the supremum over true ratios.
• Theoretically establish generalization guarantees for general nnBD DRM with multiple clients under imperfect estimates of the supremum, which unifies a number of DRMs, and show benefits of importance weighting in terms of excess risk decoupled from density ratio estimation through bias-variance decomposition.
• Experimentally demonstrate more than 16% overall test accuracy improvement over existing FL baselines when training ResNet-18 (He et al., 2016) on CIFAR10 (Krizhevsky) in challenging imbalanced federated settings in terms of data distribution shifts across clients.
In
conclusion, we expand the concept and application scope of FL to
a general
setting under intra
inter
client covariate shifts
provide an
in-depth
theoretical
understanding of
learning
with
FIDEM
via a
general
DRM and experimentally validate the utility of the proposed framework.
We
hope
that our
work opens the door to a
new FL paradigm
1. R ELATED WORK
In this section, we overview a summary of related work.
See Appendix B for complete discussion.
The current FL literature largely focuses on minimizing the empirical risk, under the same training/test data distribution assumption over each client (Li et al., 2020; Kairouz et al., 2021;
Wang et al., 2021).
In contrast, we focus on learning under both intra-client and inter-client covariate shifts.
Communication-efficient, robust, and secure aggregations can be viewed as complementary technologies, which can be used along with FIDEM to improve FIDEM’s scalability and security while addressing overall generalization.
Shimodaira (2000) introduced covariate shift where the input train and test distributions are different while the conditional distribution of the output variable given the input variable remains unchanged.
Importance-weighted ERM is widely used to improve generalization performance under covariate shift (Zadrozny, 2004; Sugiyama & Müller, 2005;
Huang et al., 2006; Sugiyama et al., 2007; Kanamori et al., 2009; Sugiyama et al., 2012; Fang et al., 2020; Zhang et al., 2020; Kato & Teshima, 2021).
Sugiyama et al.
(2012) proposed a Bregman divergence-based
DRM, which unifies various DRMs.
Kato & Teshima (2021) proposed a non-negative Bregman divergence-based DRM when using deep neural networks for density ratio estimation.
Our work largely differs from Kato & Teshima (2021) in our problem setting that allows multiple clients, algorithm design to estimate different ratios across clients while controlling privacy leakage, and theoretical analyses to show the benefit of importance weighting in generalization.
C OVARIATE SHIFT AND FIDEM FOR FL
We first provide the problem setting under intra/inter client covariate shifts, and then describe the proposed FIDEM as an unbiased estimate in terms of minimizing the overall true risk 2 .
2. P ROBLEM SETTING
Let X ⊆
R
d
x be a compact metric space, Y ⊆ R d y , and K be the number of clients in an FL setting.
Let S k
= { ( x tr k,i , y tr k,i ) } n tr k i =
denote the training set of client k with n tr k samples drawn i.i.d from an unknown probability distribution p tr k on X × Y .
The test data of client k , is drawn from another unknown probability distribution p te k on X × Y .
Under the covariate shift setting
(Sugiyama et al., 2007; Kanamori et al., 2009; Kato & Teshima, 2021; Uehara et al., 2020; Tripuraneni et al., 2021;
Zhou & Levine, 2021), the conditional distribution p tr k ( y | x ) =
p te k
( y | x ) :
= p ( y | x ) is assumed to be the same for all k , while p tr k ( x ) and p te k
( x ) can be arbitrarily different, which gives rise to intra-client and inter-client covariate shifts .
We consider supervised learning where the goal is to find a hypothesis h w : X → Y , parameterized by w ∈ R d e.g., weights and biases of a neural network, such that h w ( x ) (for short h ( x ) ) is a good approximation of the label y ∈ Y corresponding to a new sample x ∈ X .
Let ℓ : X × Y → R + denote a loss function.
In our FL setting, the true (expected) risk of client k is given by R k ( h w ) =
E ( x , y ) ∼ p te k ( x , y )
[ ℓ ( h w ( x ) , y )] .
2. FIDEM FOR FL UNDER COVARIATE SHIFT
For a scenario with K clients, we first focus on minimizing R l ( l ∈
[ K ] ) under intra/inter-client covariate shifts, i.e., p tr k ( x ) ̸
= p te l ( x ) for all k .
We then formulate FIDEM to minimize the average test error over K clients under covariate shifts by optimizing a global model under our FL setting.
FIDEM for one client.
Under p tr k ( x ) ̸ = p te l ( x ) ∀ k , FIDEM focusing on minimizing R l is given by: min w ∈ R d
[Equation]
In Appendix C, we elaborate on four special cases of the above scenario, i.e., p tr k ( x ) ̸ = p te l ( x ) ∀ k , focusing on one client under various covariate shifts and formulate their FIDEM’s.
Proposition 1.
Let l ∈
[ K ] .
FIDEM in Eq.
(2.1) is consistent.
i.e., the learned function converges in probability to the optimal function in terms of minimizing R l .
See Appendix C for the proof.
Proposition 1 implies that, under intra/inter-client covariate shifts,
FIDEM outputs an unbiased estimate of a true risk minimizer of client l .
In Appendix C.1, we show usefulness of importance weighting under no intra-client covariate shifts but inter-client covariate shifts, which is a special and important case of our setting.
Building on Eq.
(2.1) that aims to minimize R l , we now formulate FIDEM to minimize the average test error over all clients and explain its costs and benefits for federated settings.
FIDEM for K clients.
Let w be the global model.
For K clients under intra/inter-clinet covariate shifts, FIDEM minimizes the average test error over all clients and is formulated as:
[Equation]
where
[Equation]
Each client requires an estimate of a ratio in the form of sum test over own train.
We emphasize that F k ( w ) should not be viewed as the local loss function of client k .
Our formulation FIDEM is meant to minimize the overall test error over all clients given intra/inter-client covariate shifts.
To solve FIDEM, we employ the stochastic gradient descent (SGD) algorithm for T iterations starting from an initial parameter w 0 : w t +1 =
w t − η t (cid:80) Kk =1 g k ( w t ) where η t > 0 is the step size, g k ( w t ) is an unbiased estimate of ∇ w F k ( w t ) , and w T is the output.
Under no covariate shift, both FIDEM and classical ERM result in the same solution , which is a minimizer of the overall empirical risk .
The main difference happens under intra-client and interclient covariate shifts.
In those challenging settings, FIDEM’s solution is an unbiased estimate of a minimizer of the overall true risk , while the solution of ERM minimizes the overall empirical risk .
2. P RIVACY , C OMMUNICATION , AND C OMPUTATION IN FL
Privacy and communication efficiency are major concerns in FL (Kairouz et al., 2021).
We elaborate on them and introduce another variant of FIDEM with the same guarantees and costs as FedAvg.
Communication/computational costs and security benefits.
Compared to classical ERM, the communication/computational overhead of FIDEM is negligible.
4 To solve FIDEM, client k should compute an unbiased estimate of the weighted gradient ∇ w F k ( w t ) , which requires a single backward pass at a single parameter w = w t .
Hence, given the ratios, there is no extra computational/communication overhead compared to classical ERM.
Clients compute the ratios in parallel.
In Appendix E, we provide a concrete example and show that the number of communication bits needed during training in standard FL is usually many orders of magnitudes larger than the size of samples shared for estimating the ratios.
To further reduce communication costs of density ratio estimation and gradient aggregation, compression methods such as quantization, sparsification, and local updating rules, can be used along with FIDEM on the fly .
More importantly, due to importance weighting , g k ( w ) can be arbitrarily different from an unbiased stochastic gradient of classical ERM for client k , i.e., 1 n tr k (cid:80) n tr k i =1 ∇ w ℓ ( h w ( x tr k,i ) , y tr k,i ) .
The formulation FIDEM makes it impossible for an adversary to apply gradient inversion attack and obtain private training data of clients (Zhu et al., 2019).
In particular, the attacker cannot find the vanilla (stochastic) gradients and reconstructs data unless the attacker has a perfect knowledge of the ratio r k ( x )
= (cid:80) Kl =1 p te l ( x ) /p
tr k ( x ) .
Privacy.
Given { r k ( x ) } Kk =1 , FIDEM efficiently minimizes the overall test error over all clients in a privacy-preserving manner.
To estimate those ratios, if clients can tolerate some level of privacy leakage, clients send unlabelled samples x te l,j for l ∈ [ K ] and j ∈ [ n te ] from their test distributions.
To control privacy leakage to other clients, we propose that the server randomly shuffles these unlabelled samples before broadcasting to clients.
In Appendix Q, we discuss an alternative method instead of sending original unlabelled samples and discuss its limitations.
To fully eliminate any privacy risks compared to classical ERM, clients may opt to minimize the following surrogate objective, which we name Federated Independent Importance-weighteD
Empirical risk Minimization (FIIDEM) :
[Equation]
The formulation FIIDEM preserves the same level of privacy and same communication costs as those of classical ERM, e.g., FedAvg.
However, to exploit the entire data distributed among all clients and achieve the optimal global model in terms of overall test error, clients need to compromise some level of privacy and share unlabelled test samples with the server.
Hence, in this paper, we focus on the original objective in FIDEM.
R ATIO ESTIMATION FOR FL UNDER COVARIATE SHIFT
To solve FIDEM, client k should have access to an accurate estimate of this ratio
[Equation]
Ratio estimation is a key step for importance weighting (Sugiyama et al., 2007; 2012).
The discrepancy between the true ratio r ∗ k for client k in Eq.
(3.1) and the estimated one r k using our ratio model can be measured by E p tr k
[BD f ( r ∗ k ( x ) ∥
r k ( x ))] where the Bregman divergence (BD) associated with a strictly convex f leads to BD-based DRMs (Kato & Teshima, 2021;
Kiryo et al., 2017):
Definition 1 (Bregman 1967) .
Let B f ⊂ [0 , ∞ ) and f : B f → R be a strictly convex function with bounded gradient.
The BD associated with f from ˜ z
to z is given by BD f (˜ z ∥ z )
=
f
(˜ z ) − f ( z ) − ∇ f ( z )(˜ z − z ) .
Note that BD f (˜ z ∥ z ) is a convex function w.r.t.
˜ z ; however, it is not necessarily convex w.r.t.
z .
Motivated by Kato & Teshima (2021); Kiryo et al.
(2017), we propose a new histogram-based DRM (HDRM) for FL with multiple clients.
HDRM overcomes the over-fitting issue (Kiryo et al., 2017;
Kato & Teshima, 2021) while providing an estimate for the upper bound r k = sup x ∈X tr r ∗ k ( x ) , which is a key step for non-negative BD (nnBD) DRM.
We now extend nnBD DRM to FL settings.
3. E XTENSION OF NN BD DRM TO FL
We assume that p tr k ( x tr ) > 0 for k ∈
[ K ] and all x tr ∈ X tr ⊆ X with X te ⊆
X tr , i.e., we need a common data domain with strictly positive train density, which is a common assumption (Kanamori et al., 2009; Kato & Teshima, 2021).
Let H r ⊂ { r : X → B f } denote a hypothesis class for our ratios r k , e.g., neural networks with a given architecture.
Our goal is to estimate r k by minimizing the discrepancy E p tr k [BD f ( r ∗ k
( x ) ∥ r k ( x ))] , which leads to BD-based DRM for FL and is formulated in
Appendix D.2.
Let { x tr k,i } n tr k i
=
and { x te l,j } n te j =1 denote unlabelled samples drawn i.i.d from distributions p tr k and p te l , respectively, for l ∈ [ K ] .
Standard BD-based DRM is shown to suffer from an over-fitting issue where − 1 n te (cid:80) n te j =1 ∇ f ( r k ( x te l,j )) diverges if there is no lower bound on this term (Kiryo et al., 2017; Kato & Teshima, 2021).
To resolve this issue in FL, we consider non-negative BD (nnBD)
DRM for client k , i.e., min r k ∈H r ˆ E + f ( r k ) where ˆ E
[Equation]
=
[Equation] [Equation]
te n te (cid:88) j =
K (cid:88) l = ℓ 2 ( r k ( x te l,j )) , (3.2)
ReLU( z ) = max { 0 , z } , 0 < C k < 1 r k , r k = sup x ∈X tr r ∗ k ( x ) , ℓ 1 ( z ) = ∇ f ( z ) z − f ( z ) , and ℓ 2 ( z )
=
C ( ∇ f ( z )
z − f ( z )) −∇ f
( z ) .
Intuitively, ReLU is used for non-negativity and 0 < C k < 1 r k acts as a regularization parameter.
Substituting different f ’s into Eq.
(3.2) leads to different variants of nnBD, which covers previous work (Basu et al., 1998;
Hastie et al., 2001; Gretton et al., 2009;
Nguyen et al., 2010; Kato et al., 2019).
We provide explicit expressions of those variants for client k in Appendix H. In this work, we focus on f ( z ) =
( z − 1)
2leading to the well-known least-squares importance fitting (LSIF) variant of nnBD for client k .
3. E STIMATION OF THE UPPER BOUND r k
Estimating r k = sup x ∈X tr r ∗ k ( x ) is a key step for nnBD DRM.
For a single train and test distribution, it is shown that overestimating r leads to significant performance degradation (Kato & Teshima, 2021, Section 5).
Kato & Teshima (2021) considered C as a hyper-parameter, which can be tuned.
However, obtaining an efficient estimate of r k is desirable, in particular when training a deep model.
Here we propose a histogram-based method for estimation of r k .
Let B ⊂ X tr , and assume p tr k and p te l are continuous for l ∈ [ K ] .
Since B is connected and
Lebesgue-measurable with finite measure, by applying intermediate value theorem (Russ, 1980), there exist ˜ x
tr and ˆ x te such that Pr { X tr k ∈ B} = p tr k (˜ x tr )Vol( B ) and (cid:80) Kl
=1 Pr { X te l ∈ B} = (cid:80) Kl
=1 p te l ( ˆ x te )
Vol( B ) where Vol( B ) =
(cid:82) x ∈B d x .
We note that sup x ∈B r ∗ k ( x ) ≤ sup x ∈B (cid:80) K l =1 p te l ( x ) inf x ∈B p tr k ( x ) and (cid:80)
Kl
=1 p te l (ˆ x te ) p tr k (˜ x tr ) ≤ sup
[Table]
[Equation]
[Table]
[Equation]
To estimate r k , we first partition X tr into M bins where for each bin B m , if there exists some x tr k,i ∈ B m , then we define ˜ r k,m := (cid:80)
Kl
=
Pr { X te l ∈B m } Pr { X tr k ∈B
[Equation]
≃
[Equation]
for m ∈
[ M ] .
Otherwise, ˜ r k,m = 0 .
Finally, we propose to use C k
= 1˜
r k where ˜ r k = max { ˜ r k, 1 , · · · , ˜ r k,M } .
Convergence of ˜ r k to r k is established in Appendix G.
Furthermore, for high-dimensional data, an efficient implementation of HDRM using k -means clustering is provided in Appendix G.
In HDRM, K clients estimate their ratios in parallel.
To be specific, clients first share unlabelled test samples with the server.
The server returns the randomly shuffled pool of samples to all clients.
Then clients find C k ’s in parallel.
Given C k ’s, clients estimate their corresponding ratios in parallel.
To handle high-dimensional data samples and deep ratio estimation models, we adopt a variant of SGD.
For client k , we divide unlabelled samples { x tr k,i } n tr k i =1 and { x te l,j } n te j =1 for l ∈
[ K ] into N k batches { x tr k,n,i } B tr k i
=1 and { x te k,n,j } B te k j
=1 for n ∈ [ N k ] .
Client k first computes 1 B tr k (cid:80) B tr k i =1 ℓ 1 ( r k ( x tr k,n,i ))
−
KC
[Table]
If it becomes negative, then we apply a gradient ascent step to increase this term.
We may also opt to apply 1 -norm or 2 -norm regularizations.
The details of the HDRM algorithm are shown in Algorithm 1 in Appendix A.
T HEORETICAL GUARANTEES
To address learning on multiple clients in FL, it is essential to obtain per-client generalization bounds for a general nnBD DRM with imperfect estimates of r k ’s.
Even if we have access to perfect estimates of density ratios, it is still unclear the usefulness of importance weighting.
In this section, we firstly study the high-probability generalization guarantees of nnBD DRM under imperfect estimate of r k in terms of BD risk.
We then show the benefit of importance weighting in term of excess risk through a refined bias-variance decomposition on a ridge regression problem.
Theorem 1, Lemma 1, Theoremare proved in Appendix I, Appendix L, and Appendix M, respectively.
4. G ENERALIZATION ERROR IN TERMS OF BD RISK
We establish a high-probability bound on the generalization error of nnBD DRM with an arbitrary f for client k in terms of BD risk given by
[Equation]
where ˜ E k := E p tr k − C k (cid:80) Kl
=1 E p te l .
Our bound for client k depends on the Rademacher complexity (Shalev-Shwartz & Ben-David, 2014) of the hypothesis class for our density ratio model
H r ⊂ { r : X → B f } w.r.t.
client k train distribution p tr k and all client’s test distributions p te l for l ∈ [ K ] .
Let R pn ( H ) denotes the Rademacher complexity of function class H w.r.t.
distribution p , formally defined in Appendix A.
We first make the following assumptions on ℓ 1 ( z ) =
∇ f ( z ) z − f ( z ) and ℓ 2 ( z )
=
C ( ∇ f ( z )
z − f ( z ))
− ∇ f ( z ) .
Assumption 1 (Basic assumptions on ℓ 1 and ℓ 2 ) .
We assume 1) sup z ∈B f max i ∈{ 1 , 2 } | ℓ i ( z ) | < ∞ ; 2) ℓ 1 is L 1 -Lipschitz and ℓ 2 is L 2 -Lipschitz on X ; 3) inf r ∈H r ˜ E k
[ ℓ 1 ( r k ( x ))] > 0 for k ∈ [ K ] .
The first two assumptions are satisfied if inf { z | z ∈ B f } > 0 for commonly used loss functions, e.g., unnormalized Kullback– Leibler and logistic regression.
The third assumption is mild, commonly used in DRM literature (Kiryo et al., 2017; Lu et al., 2020; Kato & Teshima, 2021).
Theorem 1 (Generalization error bound for client k ) .
Let f be a strictly convex function with bounded gradient.
Denote ∆ ℓ := sup z ∈B f max i ∈{ 1 , 2 } | ℓ i ( z ) | , ˆ r k :=
arg min r k ∈H r ˆ E
+ f ( r k ) and r ∗ k :=
arg min r k ∈H r E f ( r k ) where ˆ E + f and E f are defined in Eqs.
(3.2) and (4.1) , respectively.
Suppose that ℓ 1 and ℓ 2 satisfy Assumption 1, then for any 0 < δ < 1 , with probability at least 1 − δ :
[Equation]
where Υ = ∆ 2 ℓ (1 /n tr k + C 2 k K/n te ) .
Remark 1 .
Theorem 1 provides generalization guarantees for a general nnBD DRM in a federated setting under a strictly convex f with bounded gradient.
We make the following remarks.
1) Our results are general to cover various ratio models.
For example, in Corollary 1 of Appendix J, we consider neural networks with depth L and bounded Frobenius norm ∥ W i ∥ F ≤ ∆ W i and establish explicit generalization error bounds for client k in O (cid:0) √ L (cid:81)
[Equation]
+ (cid:113)
[Equation]
If the additional error due to estimation of r k with HDRM in Section 3 using M bins is considered, it leads to O ( K ∆ ℓ (cid:0) 1 M + (cid:113)
Mn tr k (cid:1) ) under mild assumptions.
Refer to Appendix K for details.
Our error bound increases with K due to the structure of BD risk.
Note that K is in a constant order.
Our goal is to show that nnBD DRM is guaranteed to generalize in a general federated setting.
4. E XCESS RISK AND BENEFIT OF FIDEM
In this section, we aim to demonstrate the benefit of importance weighting in term of excess risk through bias-variance decomposition.
We consider the classical least squares problem, a good starting point to understand the superiority of FIDEM over ERM with generalization guarantees.
We consider the single client setting K = 1 for the ease of description, and our results can be extended to the multiple clients setting.
Let ( x , y ) denote the (test) data sampled from an unknown probability measure ρ .
The least squares problem is to estimate the true parameter θ ∗ , which is assumed to be the unique solution that minimizes the population risk in a Hilbert space H :
L ( θ ∗ ) =
min θ ∈H L ( θ ) where L ( θ ) :
= 12 E ( x ,y ) ∼ ρ
[( y − θ ⊤ x )] 2 .
Moreover, we have L ( θ ∗ ) =
σ 2 ϵ corresponding to the noise level.
For an estimate θ found by a learning algorithm such as ridge regression, its performance is measured by the expected excess risk , E [ L ( θ )] − L ( θ ∗ ) , where the expectation is over the random noise, randomness of the algorithm, and training data.
In the following, we consider two settings: random-design setting and fixed-design settings where the training data matrix is random and given, respectively.
Bias variance decomposition.
We need the following noise assumption for our proof.
Assumption 2 (Dhillon et al.
2013; Zou et al.
2021, bounded variance) .
Let ϵ :
= y − θ ⊤∗ x .
We assume that E [ ϵ ] = 0 and E [ ϵ 2 ] = σ 2 ϵ .
We have the following lemma on the bias-variance decomposition of the ridge regression FIDEM estimate in the random-design setting.
Lemma 1.
Let X ∈ R n × d be the training data matrix.
Let W = diag
( w 1 , . . . ,
w n ) with w i = p te ( x i ) /p
tr ( x i ) for i ∈ [ n ] , ˆ θ be the regularized least square estimate with importance weighting: ˆ θ = arg min θ (cid:80) ni
=
w i ( θ ⊤ x i − y i ) 2 + λ ∥ θ ∥ 22 where λ is the regularization parameter.
Denote θ ∗ be the true estimate, then the excess risk can be decomposed as the bias B and the variance V :
E [ L (ˆ θ )] − L ( θ ∗ ) =
B + V , with
[Equation]
where Σ W ,λ := X ⊤ WX
+ λ I , and Σ te =
E x
[ xx ⊤ ] .
Note that the expectation is taken over the randomness of the training data matrix X and label noise.
Remark 2 .
Our results in Lemma 1 hold under the fixed-design setting where the training data are given (Dhillon et al., 2013; Hsu et al., 2012), by omitting the expectations from B and V .
One-hot case.
To theoretically prove that FIDEM outperforms ERM in non-trivial settings, we start from the one-hot case, along the lines of Zou et al.
(2021), and strictly show that, under which level of covariate shift, the excess risk of FIDEM is always smaller than the classical ERM.
To be specific, in the one-hot case, every training data x is sampled from the set of natural basis { e 1 , e 2 , . .
. , e d } according to the data distribution given by Pr { x = e i } =
λ i
where 0 < λ i ≤and (cid:80) i λ i
= 1 .
The class of one-hot least square instances is characterized by the following problem set: (cid:8) ( θ ∗ ; λ 1 , . . .
, λ d ) : θ ∗ ∈ H , (cid:80) i λ i = 1 (cid:9) .
It is not difficult to show that the population second momentum matrix is Σ tr = E
[ x i x ⊤ i ] = diag( λ 1 , . . .
, λ d ) for i ∈ [ n ] .
Similarly, we assume that each test data follows the same scheme but with different probabilities
Pr { x = e i } = λ ′ i , and hence, we have Σ te = diag( λ ′ 1 , . . .
, λ ′ d ) .
This is a relatively simple setting, which admits covariate shift.
Take { µ 1 , µ 2 , . . . , µ d } as the eigenvalues of X ⊤ X .
Since x i can only take on natural basis, the eigenvalue µ i can be understood as the number of training data that equals e i .
For notational simplicity, we rearrange the order of the training data following the decreasing order of the ratio, such that the i -th sample x i corresponds to the ratio w i as the exact i -th largest value.
Theorem 2.
Let ˆ θ be the estimate of FIDEM, θ v be the classical ERM, and ξ i :=
λ λ + µ i .
Under the fixed-design setting in the one-hot case, label noise assumption, and data correlation assumption, if the ratio w i :
=
p te ( x i ) /p
tr ( x i ) satisfies
[Equation]
then we have R (ˆ θ ) ⩽ R ( θ v ) .
Remark 3 .
We have the following remarks:
The condition (4.3) is equivalent to (cid:113) λ
′ i
λ i ∈ (cid:16) 0 , 1+ √ 1+4 ξ i(cid:17) , which requires the training and test data to behave similarly in terms of eigenvalues, avoiding significant differences under distribution shifts for learnability.
Other metrics, e.g., similarity on eigenvectors (Tripuraneni et al., 2021) also coincide with the spirit of our assumption.
The ratio matrix is W ∈ R n × n .
However, we only need its top d eigenvalue,
i
e., the top- d ratios.
In particular, the last n − d ratios have no effect on the final excess risk.
This makes our algorithm robust to noise and domain shift.
For the special case by taking the ratio as w i := (cid:113) λ ′
i
λ i , we have
[Equation]
which implies that the ratio can be regarded as an implicit regularization (Zou et al., 2021)
E XPERIMENTAL EVALUATION
In this section we illustrate conditions under which FIDEM is favored over both Federated Averaging without ratio estimation (FedAvg) (McMahan et al., 2017) and FIIDEM.
In all experiments, we use a LeNet (LeCun et al., 1989) with cross entropy loss and compute standard deviations overindependent executions .
Due to the page limit, the implementation details are in Appendix O.
[Table] [Table]
Target shift
We consider the case of target shift where the label distribution p ( y ) changes but the conditional distribution p ( x | y ) remains invariant.
We split the 10-class Fashion MNIST dataset between 5 clients and simulate a target shift by including different fractions of examples from each class across the training data and test data.
We further consider the separable case in order to compute the exact ratio for FIDEM and FIIDEM in closed form.
The specific distribution and the construction of the ratio can be found in Appendix O.1.
The results in Table 1 illustrate that
FIIDEM can outperform FedAvg on average while preserving the same level of privacy.
By relaxing the privacy slightly the proposed FIDEM improves on FedAvg uniformly across all clients.
Even though the proportions of the classes have been artificially created, we believe that this demonstrates a realistic scenario where clients have a different fraction of samples per class.
Additional experiments using larger models on the CIFAR10 dataset under a challenging target shift setting can be found in
Appendix O.1 where FIDEM is observed to improve uniformly over FedAvg.
Covariate shift.
We now focus on covariate shift, where p ( x ) undergoes a shift while p ( y | x ) remains unchanged.
For this setting, we extend the Colored MNIST dataset in Arjovsky et al.
to the multi-client setting.
The dataset is constructed by first assigning a binary label 0 to digits from 0-4 and label 1 for digits between 5-9.
The label is then flipped with probability 0 .
25 to make the dataset non-separable.
A spurious correlation is introduced by coloring the digits according to their assigned labels and then flipping the colors according to a different probability for each distribution (see Appendix O.2).
For this experimental setup, we introduce an idealized scheme, which ignores the color and thus the spurious correlation, i.e., provides an upper bound, and is referred to as Grayscale .
FIDEM outperforms both baselines in terms of the average accuracy even in a two-client setting.
FIDEM is also close to Grayscale upper bound that by construction ignores the spurious correlations.
C ONCLUSIONS AND FUTURE WORK
In this work, we focus on FL under both intra-client and inter-client distribution shifts and propose
FIDEM to improve the overall generalization performance.
We establish high-probability generalization guarantees for a general DRM method in a federated setting.
We further show the benefit of importance weighting in term of excess risk through bias-variance decomposition in a ridge regression problem.
Our theoretical guarantees indicate how FIDEM can provably solve a learning task under distribution shifts.
We experimentally evaluate FIDEM under both label shift and covariate shift cases.
Our experimental results validate that under certain covariate and target shifts, the proposed method can learn the task, while baselines such as vanilla federated averaging fails to do so.
We anticipate that our methods to be applicable in learning from e.g., medical data, where there might be arbitrary skews on the distribution.
In addition, we believe our study can further encourage the investigation of distribution shifts in FL, as this is a critical subject for learning across clients.
A C ONTENT OF THE APPENDIX
Notation: We use E [ · ] to denote the expectation and ∥ · ∥ to represent the Euclidean norm of a vector.
We use lower-case bold font to denote vectors.
Sets and scalars are represented by calligraphic and standard fonts, respectively.
We use [ n ] to denote { 1 , · · · , n } for an integer n .
We use ≲ to ignore terms up to constants and logarithmic factors.
The appendix is organized as follows:
• Definition of Rademacher complexity, examples of f for BD-based DRM, and the steps of HDRM in Algorithm 1 are provided in Appendix A. • Complete related work is provided in Appendix B. • FIDEM with a focus on minmizing R 1 are provided in Appendix C. • Details of density ratio estimation are provided in Appendix D. • Communication costs of FIDEM and FIIDEM are analyzed in in Appendix E. • UKL, LR, and PU variants of nnBD are provided in Appendix F. • Convergence of ˜
r and k -means clustering for HDRM are provided in Appendix G. • UKL, LR, and PU variants of nnBD for multiple clients are provided in Appendix H.
• The proof of the core Theorem 1 exists in Appendix I. • Generalization bounds for multi-layer perceptron and multiple clients are established in Appendix J. • Additional error due to estimation of r k with HDRM is analyzed in Appendix K. • Lemma 1 is proved in Appendix L. • Theorem 2 is proved in Appendix M. • A counterexample under which FIDEM cannot outperform ERM is provided in Appendix N. • Additional experimental details are included in Appendix O. • Computational complexity of Algorithm 1 is analyzed in Appendix P. •
The limitations of our work are described in Appendix Q.
[Table]
Samples {{
[Table]
Kk
[Table] [Table]
regularization
[Table]
Ratio model parameters { θ
[Table]
} Kk =
[Table]
1 for k = 1 to K (in parallel) do
[Table]
Send n te samples to the server
[Table]
Server randomly shuffles
[Table]
broadcasts
[Table]
{ x te l
[Table]
j
[Table]
n
[Table]
=
[Table]
Kl =1 to clients ;
[Table]
for k = 1 to K (in parallel) do
[Table]
[Equation]
Estimate C k =
[Equation]
˜
[Equation] [Equation]
r k,M
[Equation]
; 7 for t = 1 to T do for k = 1 to K (in parallel) do for n
= 1 to N k do
if 1 B tr k (cid:80) B tr k i =1 ℓ
[Equation]
( r k ( x tr k,n,i )
[Equation]
−
[Equation]
(cid:80)
[Equation]
te k j =1 ℓ 1 ( r k ( x te k,n,j )) ≥ 0 then
[Equation]
else
[Equation] [Equation]
Definition 2.
Let n ∈ Z
+
and p be a distribution, S = { x 1 , · · · , x n } be i.i.d.
random variables drawn from p
[Equation]
and H be a function class.
The Rademacher complexity of H w.r.t.
p is given by:
[Equation]
[Table]
where { σ i } ni =1 are Rademacher variables uniformly chosen from {− 1 , 1 } .
B R ELATED WORK
Federated learning.
One well-known method in FL is FedAvg (McMahan et al., 2017).
FedAvg and its variants are extensively studied in optimization with a focus on communication efficiency and partial participation of clients while preserving privacy.
Indeed, a host of techniques, such as gradient quantization, sparsification, and local updating rules, have been proposed to improve communication efficiency in FL (Alistarh et al., 2017).
Furthermore, robust and secure aggregation schemes have been also proposed to provide robustness against trainingtime attacks launched by an adversary, and to compute aggregated values without being able to inspect the clients’ local models and data, respectively (Li et al., 2020;
Kairouz et al., 2021; Wang et al., 2021).
Taken together, these work largely focus on minimizing the empirical risk in the optimization objective, under the same training/test data distribution assumption over each client.
Differences across clients are handled using personalization methods based on heuristics and currently do not have a statistical learning theoretical support (Smith et al., 2017; Khodak et al., 2019; Li et al., 2021b).
In contrast, we focus on learning and overall generalization performance under both intra-client and inter-client distribution shifts.
Communication-efficient, robust, and secure aggregations can be viewed as complementary technologies, which can be used along with our proposed FIDEM method to improve the generalization performance.
In our setting, clients can also all participate in every training iteration, such as cross-silo FL.
We note that (Hanzely et al., 2020; Gasanov et al., 2022) focus on minimizing the empirical risk , under the same training/test data distribution assumption over each client.
Our formulation in FIDEM does not require specific assumptions on function F k ’s for k ∈ [ K ] to provide an unbiased estimate of true risk minimizer.
Under strong convexity and smoothness assumptions w.r.t.
model parameters, similar optimal algorithms to those proposed in (Hanzely et al., 2020; Gasanov et al., 2022) will be optimal for FIDEM.
Different from recent FL work by Duan et al.
(2021) and Li et al.
(2021c), our work introduces new
FIDEM formulation and shows statistical consistency .
Importance-weighted ERM and density ratio matching.
Density ratio estimation is an important step in various machine learning problems such as learning under covariate shift, learning under noisy labels, anomaly detection, two-sample testing, causal inference, change-pint detection, and classification from positive and unlabelled data (Qin, 1998; Shimodaira, 2000; Cheng & Chu, 2004;
Keziou & Leoni-Aubin, 2005; Sugiyama et al., 2007; Kawahara & Sugiyama, 2009; Smola et al., 2009; Hido et al., 2011; Kanamori et al., 2011; Sugiyama et al., 2011; Yamada et al., 2011; Reddi et al., 2015; Liu & Tao, 2015; Kato et al., 2019; Fang et al., 2020; Uehara et al., 2020; Zhang et al., 2020; Kato & Teshima, 2021).
In particular, covariate shift has been observed in real-world applications including brain-computer interfacing, emotion recognition, human activity recognition, spam filtering, and speaker identification (Bickel & Scheffer, 2007; Li et al., 2010; Yamada et al., 2010; Hachiya et al., 2012; Jirayucharoensak et al., 2014).
Shimodaira (2000) introduced covariate shift where the input train and test distributions are different while the conditional distribution of the output variable given the input variable remains unchanged.
Importance-weighted ERM is widely used to improve generalization performance under covariate shift (Zadrozny, 2004;
Sugiyama &
Müller, 2005; Huang et al., 2006; Sugiyama et al., 2007; Kanamori et al., 2009; Sugiyama et al., 2012; Fang et al., 2020; Zhang et al., 2020; Kato & Teshima, 2021).
Zhang et al.
(2020) proposed a one-step approach that jointly learns the predictive model and the corresponding weights in one optimization problem.
Sugiyama et al.
(2012) proposed a Bregman divergence-based DRM, which unifies various DRMs.
Kato & Teshima (2021) proposed a non-negative Bregman divergence-based
DRM to resolve the overfitting issue when using deep neural networks for density ratio estimation.
While this line of work focuses on DRM with a single train and test distributions, we consider a federated setting with multiple clients in this paper.
Domain adaptation.
Distribution shifts between a source and a target domain have been a prominent problem in machine learning for several decades (Kouw & Loog, 2019; Wang & Deng, 2018).
The premise behind such shifts is that data is frequently biased, and this results in distribution shifts that can be estimated by assuming some (unlabelled) knowledge of the target distribution.
The following two categories of domain adaptation methods are most closely related to our work: a) sample-based, and b) feature-based methods.
In feature-based methods, the goal is to find a transformation that maps the source samples to target samples (Ganin et al., 2016; Bousmalis et al., 2017; Das & Lee, 2018; Damodaran et al., 2018).
Contrary to feature-based methods, sample-based methods aim at minimizing the target risk through data in the source domain.
Importance weighting
[Table]
is often used in sample-based methods (Shimodaira, 2000; Jiang & Zhai, 2007; Baktashmotlagh et al., 2014).
However, the focus on domain adaptation has been mainly to adapt to a single target distribution, not the overall generalization performance on multiple clients, which is addressed in this paper.
Statistical generalization and excess risk bounds.
Understanding generalization performance of learning algorithms is one essential topic in modern machine learning.
Typical techniques to establish generalization guarantees include uniform convergence by Rademacher complexity (Bartlett, 1998), and its variants (Bartlett et al., 2005), bias-variance decomposition (Geman et al., 1992; Adlam & Pennington, 2020), PAC-Bayes (McAllester, 1999), and stability-based analysis (Bousquet &
Elisseeff, 2002; Shalev-Shwartz et al., 2010).
Our work employs the first two techniques to analyze our density ratio estimation method in a federated setting and establish generalization guarantees for
FIDEM, respectively.
Rademacher complexity has been used in FL to obtain theoretical guarantees on the centralized model (Mohri et al., 2019) and personalized model (Mansour et al., 2020).
Mohri et al.
(2019) considered a scenario where a single target distribution is modeled as an unknown mixture of multiple domain distributions and obtained a global modal by minimizing the worst-case loss.
This is different from our setting where we consider multiple test distributions for clients and focus on the overall test error.
Mansour et al.
(2020) studied personalization under the same training/test data distribution assumption over each client, which is different from our setting.
Bias-variance decomposition provides a relatively refined characterization of generalization error (or excess risk), where a large bias indicates that a model is not flexible enough to learn from the data and a high variance indicates that the model performs unstably.
Bias-variance decomposition is typically studied in two settings, i.e., the fixed and random design setting, which is categorized by whether the (training) data are fixed or random.
This technique has been extensively applied in least squares (Hsu et al., 2012; Dieuleveut et al., 2017), analysis of SGD (Jain et al., 2018;
Zou et al., 2021), and double descent (Adlam & Pennington, 2020).
Information-theoretic bounds on the generalization error and privacy leakage in federated settings were established in (Yagli et al., 2020).
Under partial participation of clients, Yuan et al.
(2021) proposed a framework, which distinguishes performance gaps due to unseen client data from performance gap due to unseen client distributions.
Still, these work study FL under the same training/test data distribution assumption over each client.
C FIDEM WITH A FOCUS ON MINMIZING R
Without loss of generality and for simplicity of notation, in this section, we set l = 1 .
We consider four typical scenarios under various distribution shifts and formulate their FIDEM with a focus on minmizing R 1 .
The details of these scenarios are summarized in Table 4.
Remark 4 .
Covariance shift (as well as its assumption) is the most commonly used and studied in theory and practice in distribution shifts (Sugiyama et al., 2007; Kanamori et al., 2009; Kato & Teshima, 2021; Uehara et al., 2020; Tripuraneni et al., 2021; Zhou & Levine, 2021).
Handling covariate shift is a challenging issue, especially in federated settings (Kairouz et al., 2021).
No intra-client covariate shift: (No-CS)
For description simplicity, we assume that there are only 2 clients but our results can be directly extended to multiple clients.
This scenario assumes p tr k ( x ) =
p te k
( x ) for k
= 1 , 2 .
Client 1 aims to learn h w assuming p tr
1 ( x ) p tr
( x ) is given.
We consider the following FIDEM that is proved to be consistent in terms of minimizing minimizing R 1 :
[Equation]
C
[Equation]
Covariate shift only for client 1: (CS on one)
We now consider covariate shift only for client 1, i.e., p tr1 ( x ) ̸
= p te1 ( x ) and p tr2 ( x ) =
p te2 ( x ) .
We consider the following FIDEM
[Equation]
Covariate shift for both clients: (CS on both)
We assume p tr1 ( x ) ̸
= p te1 ( x ) and p tr2 ( x ) ̸
= p te2 ( x ) , i.e., covariate shift for both clients.
The corresponding FIDEM is the same as Eq.
(C.2).
Multiple clients: (CS on multi.)
Finally, we consider a general scenario with K clients.
We assume both intra-client and inter-client covariate shifts by the following FIDEM:
[Equation]
where (cid:80)
Kk =1 λ k = 1 and λ k ≥ 0 .
Proposition 2.
Let l ∈
[ K ] .
In above settings, FIDEM defined in Eqs.
(C.1) , (C.2) , and (C.3) is consistent.
i.e., the learned function converges in probability to the optimal function in terms of minimizing R 1 .
Proposition 2 implies that, under various settings, FIDEM outputs an unbiased estimate of a minimizer of the true risk .
Proof.
For the scenario without intra-client covariate shift, FIDEM in Eq.
(C.1) can be expressed as
[Equation]
For the scenario with covariate shift only for client 1 or for both clients
[Equation]
FIDEM in Eq.
(C.2) admits
[Equation]
We note that p te1 ( x ) p tr2 ( x ) =
p te1 ( x ) p tr1 ( x ) p tr1 ( x ) p tr2 ( x ) , which is the product of ratios due to intra-client covariate shift on client 1 and inter-client covariate shift.
For multiple clients, let k ∈
[ K ] .
Similarly, we have
[Equation]
Then we have
[Equation]
The consistency of FIDEM, i.e., convergence in probability, is immediately followed the standard arguments in e.g., (Shimodaira, 2000)[Section 3] and (Sugiyama et al., 2007)[Section 2.2] using the law of large numbers.
■
Note that to solve Eq.
(C.3), client 1 needs to estimate p te1 ( x )
p tr k ( x ) for all clients k with λ k > 0 in (C.3).
Remark 5 .
Scaling (cid:80)
Kk
=
λ k does not affect the optimal parameters in Eq.
(C.3).
For rotational simplicity, we set λ k = 1 for k ∈ [ K ] .
C. N O INTRA - CLIENT SHIFT
In this section, we consider the important and special case of the setting described in Section 2.
under no intra-client covariate shifts but inter-client covariate shifts.
For simplicity, we consider a two clients with train/test distributions P and Q whose train/test densities are denoted by p and q , respectively.
We also suppose that we have a sample z ∼ P and z ′ ∼ Q to learn with the goal is to find an unbiased estimate of the overall risk with the smallest variance.
In this setting, the classical ERM (FedAvg) objective ℓ ( z, θ ) + ℓ ( z ′ , θ ) is an unbiased estimate for the overall risk
L ( θ ) =
E P [ ℓ ( z, θ )]
+ E Q [ ℓ ( z ′ , θ )] 5 .
In this setting, the objective of FIDEM, i.e., 12 (ˆ L P ( θ ) + ˆ L Q ( θ )) with ˆ L P ( θ ) =
(cid:0) 1 + q ( z ) p ( z ) (cid:1) ℓ ( z, θ ) and ˆ L Q ( θ )
= (cid:0)
+ p ( z ′ )
q
( z ′ ) (cid:1) ℓ ( z ′ , θ ) is an unbiased estimate for the overall risk L ( θ ) .
We now show that the our method (FIDEM) has a smaller variance than FedAvg under certain conditions.
Let E P [( ℓ ( z, θ ) − E P [ ℓ ( z, θ )]) 2 ]
= σ 2 P and E Q [( ℓ ( z ′ , θ ) − E Q [ ℓ ( z ′ , θ )]) 2 ] =
σ 2 Q .
For FedAvg, the variance is given by
[Equation]
For FIDEM, the variance is given by
[Equation]
where V P = 14 E P [(ˆ L P ( θ ) − L ( θ )) 2 ] and V Q = 14 E Q [(ˆ L Q ( θ ) − L ( θ )) 2 ] .
We now expand each term V P and V Q .
We can show that
[Equation]
where ˜ σ
[Equation]
=
[Equation]
) p ( z ) ℓ ( z, θ ) −
E Q [ ℓ ( z ′ , θ )] (cid:17)(cid:105) .
Similarly, we have
[Equation]
where ˜ σ 2 Q = E Q (cid:104)(cid:16) p ( z ′ )
q ( z ′ ) ℓ ( z ′ , θ ) − E P [ ℓ ( z, θ )] (cid:17) 2 (cid:105) + 2 E Q (cid:104)(cid:16) ℓ ( z ′ , θ ) − E Q [ ℓ ( z ′ , θ )] (cid:17)(cid:16) p ( z ′ )
q ( z ′ ) ℓ ( z ′ , θ ) −
E P [ ℓ ( z, θ )] (cid:17)(cid:105) .
We note if ˜ σ 2 P + ˜ σ 2 Q ≤ 3( σ 2 P + σ 2 Q ) then, FIDEM will have smaller variance than FedAvg, i.e.,
V P + V Q ≤ σ 2 P + σ 2 Q .
The exact condition depends on the loss and densities.
To show a concrete example, for the more general and practical case with both intra/inter-client , in Section 4.2, we show that FIDEM results in smaller excess risk compared to FedAvg through a refined bias-variance decomposition.
Given two distributions, considering the case of no intra-client shift is a special case, where it is true that FedAvg is an unbiased estimate of the overall risk.
However, this unbiasedness breaks as soon as there is only one client whose test and train distributions are different, which is very common in theory and practice.
Please note that FIDEM is an unbiased estimate of the overall risk in a general FL setting without requiring any prior knowledge/assumptions on the potential covariate shifts.
D R ATIO E STIMATION D. NN BD DRM FOR A SINGLE CLIENT
For simplicity, we firstly focus on the problem of estimating r ( x ) =
p te ( x ) p tr ( x ) and then extend our consideration to the estimation of r k ( x ) in Eq.
Let r ∗ denote the true density ratio.
Our goal is to estimate r ∗ by optimizing our ratio model r .
The discrepancy between r and r ∗ is measured by E p tr [BD f ( r ∗ ( x ) ∥ r ( x ))] .
We note that E p tr [BD f ( r ∗ ( x ) ∥ r ( x ))]
=
E f ( r ) + E p tr
[ f ( r ∗ ( x ))]
where
E f ( r ) =
E p tr [ ∇ f ( r ( x ))
r ( x ) − f ( r ( x ))] − E p te [ ∇ f
( r ( x ))] .
Note that E p tr [ f ( r ∗ ( x ))] is constant w.r.t.
r .
Let { x tr i } n tr i =1 and { x te j } n te j =1 denote unlabelled samples drawn i.i.d from distributions p tr and p te , respectively.
Let H r ⊂ { r : X → B f } denote a hypothesis class for our model r .
Using an empirical approximation of E f ( r ∗ ( x )
∥ r ( x )) , Sugiyama et al.
(2012) formulated BD-based DRM problem as min r ∈H r ˆ E f ( r ) where
[Equation]
Sugiyama et al.
(2012) showed that BD-based DRM unifies well-known density ratio estimation methods by substituting an appropriate f in (D.1).
However, it is shown that solving BD-based DRM with highly flexible models such as neural networks typically leads to an over-fitting issue (Kato &
Teshima, 2021; Kiryo et al., 2017).
In particular, Kato & Teshima (2021) called such issue “train-loss hacking” where − 1 n te (cid:80) n te j =1 ∇ f
( r ( x te j )) in (D.1) diverges if there is no lower bound on this term.
Even when there exists a lower bound, the model r tends to increase to the largest possible values of its output range at points { x te j } n te j =1 .
To resolve such issue, Kato & Teshima (2021) proposed to use non-negative BD (nnBD) DRM, i.e., min r ∈H r ˆ E + f ( r ) where
[Equation]
ReLU(
[Equation]
( z ) = ∇ f ( z )
z − f ( z ) , and ℓ 2 ( z ) =
C ( ∇ f ( z )
z − f ( z ))
− ∇ f ( z ) .
Substituting f ( z ) =
( z − 1) 2into (D.2), the least-squares importance fitting (LSIF) variant of nnBD is given by
[Equation]
In Appendix F, we show explicit expressions for unnormalized Kullback–Leibler (UKL), logistic regression (LR), and positive and unlabeled learning (PU) variants of nnBD.
Estimating r = sup x ∈X tr r ∗ ( x ) is a key step for density ratio estimation.
It is shown that underestimating C leads to significant performance degradation (Kato & Teshima, 2021, Section 5).
Kato & Teshima (2021) considered C as a hyper-parameter, which can be tuned.
However, obtaining an efficient estimate of r is desirable, in particular when training a deep model.
Let B ⊂
X
tr .
Assume p tr and p te are continuous.
Since B is connected and Lebesgue-measurable with finite measure, by applying intermediate value theorem (Russ, 1980), there exist ˜ x
tr and ˆ x te such that Pr { X tr ∈ B} = p tr (˜ x tr )Vol( B ) and Pr { X te ∈ B} = p te (ˆ x te )
Vol( B ) where Vol( B ) =
(cid:82) x ∈B d x .
We note that sup x ∈B r ∗ ( x ) ≤ sup x ∈B p te ( x ) inf x ∈B p tr ( x ) and p te (ˆ x te ) p tr (˜ x tr ) ≤ sup x ∈B p te ( x ) inf x ∈B p tr ( x ) .
We partition
X tr into M bins where for each bin B m , if there exists some x tr i ∈ B m , then we define ˜ r m :=
Pr { X te ∈B m } Pr { X tr ∈B m } ≃ 1 n te (cid:80) n te j =1 1 ( x te j ∈B m )n tr (cid:80) n tr i =1 1 ( x tr i ∈B m ) for m ∈ [ M ] .
Otherwise, ˜ r m = 0 .
Finally, we propose to use
C ≤ 1˜ r where ˜ r = max { ˜ r 1 , · · · , ˜ r M } .
Convergence of ˜ r to r is established in Appendix G.
Now, suppose there are K clients where each client provides n te unlabelled test samples to the pool of samples.
Our goal is to estimate r k in Eq.
(3.1) for k = 1 , · · · , K .
The BD-based DRM for client k is given by min r k ∈H r ˆ E f
( r k ) where ˆ E f ( r k )
= 1 n tr k (cid:80) n tr k i =1 (cid:16) ∇ f
( r k ( x tr k,i ))
r k ( x tr k,i ) − f
( r k ( x tr k,i )) (cid:17) − 1 n te (cid:80) n te j =1 (cid:80) Kl =
1 ∇ f
( r k ( x te l,j )) .
The nnBD DRM problem for client k is min r k ∈H r ˆ E + f
( r k ) where ˆ E + f ( r k ) =
ReLU( ˆ S 1 ,ℓ 1 ) + 1 n te n te (cid:88)
j =
K (cid:88) l =
[Equation]
, and r k = sup x ∈X tr r ∗ k ( x ) .
Substituting f ( z ) =
( z − 1) 2into (D.3), the LSIF variant of nnBD for client k is given by min r k ∈H r ˆ E +LSIF
( r k ) where
[Equation]
and ˆ S LSIF = 12 n tr k
[Equation]
n tr k
[Equation]
=
[Equation]
r
[Equation]
x tr
[Equation]
) −
[Equation]
(cid:80)
[Equation]
Kl
[Equation]
r
[Equation]
k ( x te l,j ) .
We provide explicit expressions for
UKL, LR, and PU variants of nnBD for client k in Appendix H.
Our goal is to estimate r k = sup
[Equation]
For HDRM method, we first partition X tr into M bins where for each bin B m , if there exists some x tr k,i ∈ B m , then we define ˜ r k,m :=
[Equation]
for m ∈
[ M ] .
Otherwise, ˜ r k,m = 0 .
Finally, we propose to use C k
= 1˜
r k where ˜ r k = max { ˜ r k, 1 , · · · , ˜ r k,M } .
D. BD- BASED DRM FOR FL
Our goal is to estimate r k by minimizing the discrepancy E p tr k [BD f ( r ∗ k
( x ) ∥ r k ( x ))] , which is equivalent to min r k ∈H r E f
( r k ) where
[Equation]
since E p tr k [BD f ( r ∗ k
( x ) ∥ r k ( x ))]
=
E f ( r k ) + E p tr k [ f ( r ∗ k ( x ))] and E p tr k [ f ( r ∗ k ( x ))] is constant w.r.t.
r k .
Let { x tr k,i } n tr k i =1 and { x te l,j } n te j =1 denote unlabelled samples drawn i.i.d from distributions p tr k and p te l , respectively, for l ∈ [ K ] .
A natural way to solve min r k ∈H r E f ( r k ) is to substitute empirical averages in Eq.
(D.5)
(Sugiyama et al., 2012), leading to BD-based DRM for FL: min r k ∈H r ˆ E f
( r k ) where
[Equation]
j =
K (cid:88) l = ∇ f ( r k ( x te l,j )) .
E C OMMUNICATION COSTS AND FIIDEM
To estimate density ratios for FIDEM, clients require to send a few unlabelled test samples only once .
The server shuffles those samples and broadcasts the shuffled version to clients only once .
The communication overhead to estimate ratios is negligible compared to the communication costs for sharing high-dimensional stochastic gradients over the course of training.
Consider the example of CIFAR10 consisting of 32 by 32 images with 3 channels represented by 8 bits.
If one shares 1000 unlabelled images 6 , the communication amounts to sharing roughly 3 × 10 6 values each with 8 bits, i.e., 25 × 10 6 total communication bits or 3 . 1 MB.
In contrast, during training, the network size alone easily surpasses this size (e.g. the common ResNet-18 has 11 million parameters, each represented by a 32-bit floating point).
Standard training of ResNet-18 requires 8 × 10 4 iterations and aggregations, which amounts to 2 .
816 × 10 13 total communicated bits per client, i.e., 3 . 5 TB during training.
In other words, the number of communication bits needed during training in standard federated learning is usually many orders of magnitudes larger than the size of samples shared for estimating the ratios.
To further reduce communication costs of density ratio estimation and gradient aggregation, compression methods such as quantization, sparsification, and local updating rules, can be used along with FIDEM on the fly (Alistarh et al., 2017).
Alternatively, to eliminate any privacy risks, clients may minimize the following surrogate objective, which we name FIIDEM:
[Equation]
where ˜ F
[Equation]
We note that privacy risks are eliminated by solving E.1.
However, to exploit the entire data distributed among all clients and achieve the optimal global model in terms of overall test error, clients need to compromise some level of privacy and share unlabelled samples generated from their test distribution with the server.
Hence, in this paper, we focus on the original objective F ( w ) in FIDEM, which is different from ˜ F ( w ) .
F V ARIANTS OF NN BD.
In this section
[Equation]
we show explicit expressions for unnormalized Kullback–Leibler (UKL), logistic regression (LR), and positive and unlabeled learning (PU) variants of nnBD.
Substituting f ( z )
=
z log( z ) − z into Eq.
(D.2)
, we have ℓ 1 ( z ) =
z and ℓ 2 ( z ) = zC − log
( z ) , and the UKL variant of nnBD is given by
[Equation] [Equation]
Substituting f ( z )
=
z log( z ) − ( z + 1) log( z + 1) into Eq.
(D.2)
, we have ℓ 1 ( z )
= log( z + 1) and ℓ 2 ( z ) =
C log( z + 1) + log (cid:0) z +1 z (cid:1) , and the LR (BKL) variant of nnBD is given by
[Equation]
Substituting f ( z ) =
C log(1 − z )
+ Cz (log( z ) − log(1 − z )) into Eq.
(D.2)
, we have ℓ 1 ( z ) =
− C log(1 − z ) and ℓ 2 ( z ) = − C log( z )
+ ( C − C 2 ) log(1 − z ) , and the PU variant of nnBD is given by
[Equation]
G C ONVERGENCE OF ˜ r AND k - MEANS CLUSTERING .
Lemma 2.
If n tr k , n te , and M go to infinity with sup m Vol( B m ) → 0 , then ˜ r k → r k .
Proof.
Let x ∈ X tr .
Note that when n tr k , n te , and M go to infinity, the numerator and denominator of ˜ r k become (cid:80) Kl =1 p te l ( x )Vol( B m ) and p tr k ( x )Vol( B m ) , respectively, where x ∈ B m .
■
Please note that our density ratio in Eq.
(3.1) is in the form of a sum of test densities over own train density.
So even if one or a few number of ratios are poorly estimated, it will not impact the entire ratio in Eq.
(3.1) as nested estimation errors.
The error does not propagate in a multiplicative manner but in an additive way.
k -means clustering for HDRM.
We note that partitioning the space and counting the number of samples in each bin is not necessarily an easy task when data is high dimensional.
In practice, one simple method is to cluster train and test samples using an efficient implementation of k -means clustering with M clusters and count the number of train and test samples in each cluster (Lloyd, 1982).
To estimate the ratios, we need a batch of samples from the test distribution of each client in addition to a batch of samples from the train distribution for each estimating client.
The running time of Lloyd’s algorithm with M clusters is O ( nd x M ) where n is the total number of samples with dimension d x .
H UKL, LR, AND PU VARIANTS OF NN BD FOR MULTIPLE CLIENTS .
In this section, we provide explicit expressions for UKL, LR, and PU variants of nnBD for client k .
The UKL variant of nnBD for client k is given by min r k ∈H r ˆ E + UKL ( r k )
where
[Equation]
The LR variant of nnBD for client k is given by min r k ∈H r ˆ E +LR
( r k ) where
[Equation]
K
[Equation]
log
[Equation]
The PU variant of nnBD for client k is given by min r k ∈H r ˆ E +PU ( r k ) where
[Equation]
I P ROOF OF T HEOREM
In this section, we prove Theorem 1, which establishes an upper bound on the generalization error of nnBD DRM (HDRM method with an arbitrary f ) for client k in terms of BD risk, which holds with high probability along the lines of (Kiryo et al., 2017; Lu et al., 2020; Kato & Teshima, 2021).
We remind that client k ’s goal is to estimate this ratio:
[Equation]
For client k , the BD risk given by
[Equation]
where ˜ E k :
[Equation]
ℓ 1 ( z ) =
∇ f ( z ) z − f ( z ) , and ℓ 2 ( z )
=
C ( ∇ f ( z )
z − f ( z ))
− ∇ f ( z ) .
We note that the definition of C k implies ˜ p
k = p tr k − C k (cid:80) Kl =1 p te l > 0 .
We remind that f : B f
→ R is a strictly convex function with bounded gradient ∇ f where B f ⊂ [0 , ∞ ) , and H r ⊂ { r : X → B f } denotes a hypothesis class for our model r .
The nnBD DRM problem for client k is min r k ∈H r ˆ E + f
( r k ) where
[Equation]
with ˆ E p tr k is the sample average over { x tr k,i } n tr k i =1 , and ˆ E p te l is the sample average over { x te l,j } n te j =1 .
In the following, we denote ˆ E k :
=
ˆ E p tr k − C k (cid:80) K l =1 ˆ E p te l for notational simplicity.
Let ˆ r k :=
arg min r k ∈H r ˆ E
+ f ( r k ) and r ∗ k :=
arg min r k ∈H r E f ( r k ) .
We first decompose the generalization error into maximal deviation and bias terms:
[Equation]
where the second inequality holds since ˆ r k :
= arg min r k ∈H r ˆ E
+ f ( r k ) .
The first term in the RHS of (I.4) is the maximal derivation and the second term is the bias.
In the following two lemmas, we find an upper bound on the maximal deviation sup r k ∈H r | ˆ E + f ( r k ) −
E [ ˆ E + f ( r k )] | and bias sup r k ∈H r | E [ ˆ E + f ( r k )] − E f ( r k ) | , respectively.
Lemma 3 (Maximal deviation bound) .
Denote ∆ ℓ := sup z ∈B f max i ∈{ 1 , 2 } | ℓ i ( z ) | , then for any 0 < δ < 1 , the maximal deviation term is upper bounded with probability at least 1 − δ
[Equation]
Proof.
Denote
[Equation]
with
S
[Equation]
Let S ( i ) k be obtained by replacing element i of set S k by an independent data point taking values from the set X tr .
We now measure the absolute value of the difference caused by changing one data point in the maximal deviation term (I.5), i.e., | Φ( S k ) − Φ( S ( i ) k ) | .
If the changed point is sampled from p tr k , then the absolute value of the difference caused in the maximal deviation term is upper bounded by 2∆ ℓ n tr k .
If the changed point is sampled from p te l , the the absolute value of the difference caused in the maximal deviation term is upper bounded by 2∆ ℓ ( C k +1) n te for l = 1 , · · · , K .
Applying McDiarmid’s inequality (McDiarmid et al., 1989), with probability at least 1 − δ , we have
[Equation]
In the following, we establish an upper bound on the expected maximal deviation
E [sup r k ∈H r | ˆ E + f ( r k ) − E
[ ˆ E + f ( r k )] | ] by generalization the symmetrization argument in (Kiryo et al., 2017; Lu et al., 2020) followed by applying Talagrand’s contraction lemma for two-sided Rademacher complexity.
Let m ∈
[ M ] and N m ∈ Z + for M ∈ Z + .
Let g m : R → R be a L g m -Lipschitz function.
Let p m,p denote a probability distribution over X tr .
Suppose that { x i } n m,p i =1 are drawn i.i.d.
from p m,p for p ∈ [ N m ] and m ∈ [ M ] .
Let ℓ m,p : B f
→ R + be a L m,p -Lipschitz function and ˜ C m,p be a constant ∀ m, p .
Consider the following stochastic process:
[Equation]
where ˆ E m,p denotes sample average over { x i } n m,p i =1 .
In the rest of the proof, we show that
[Equation]
To prove (I.6), we consider a continuous extension of ℓ ( m,p ) defined on the origin.
We note that such extension does not change ˆ R k ( r k ) since ℓ ( m,p ) takes values only in B f .
If B f = { ( z 1 , z 2 ) } for some 0 ≤ z 1 < z 2 , then for any z ∈ [0 , z 1 ] , we define ℓ ( m,p ) ( z ) =
lim z ↓ z 1 ℓ ( m,p ) ( z ) where lim z ↓ z 1 ℓ ( m,p ) ( z ) exists since ℓ ( m,p ) is uniformly continuous due to Lipschitz continuity.
Then ℓ ( m,p ) will be L m,p -Lipschitz on z ∈ [0 , z 2 ] .
Let { ˜ x i } n m,p i =1 be an independent copy of { x i } n m,p i =1 .
Let denote δ
ˆ R :=
E [sup r k ∈H r | ˆ R k
( r k ) − E
[ ˆ R k ( r k )] | ] .
Following a symmetrization argument (Vapnik, 1999), an upper bound on the symmetrized process can be established by Rademacher complexity:
[Table]
[Equation]
I
[Equation]
where σ m,p are Rademacher variables uniformly chosen from {− 1 , 1 } , ˜ E and ˆ˜ E m,p denote the expectation and sample average over data distribution p m,p and the independent copy { ˜ x i } n m,p i =1 , respectively, the third inequality holds by the Lipschitz continuous property of g m , and the last inequality is obtained by applying Talagrand’s contraction lemma for two-sided Rademacher complexity (Ledoux & Talagrand, 1991; Bartlett & Mendelson, 2002).
Applying (I.6), we can show that
[Equation]
which completes the proof.
■
Next we find an upper bound on the bias sup r k ∈H r | E [ ˆ E + f
( r k )]
− E f ( r k ) | .
Lemma (Bias bound) .
Denote
[Equation]
:=
[Equation]
| .
Assume inf r ∈H r E
[ˆ E k [ ℓ 1 ( r k ( x ))]] > 0 for k ∈
[ K ] .
Then, an upper bound on the bias term is given by
[Equation]
for some constant η k > 0 .
Proof.
Let ˆ˜ E k :
=
ˆ E p tr k − C k (cid:80) Kl
=
ˆ E p te l .
We first note that | E [ ˆ E + f ( r k )] − E f ( r k ) | = | E [ ˆ E + f ( r k ) − ˆ E f ( r k )] | =
[Equation]
ReLU
[Equation]
ℓ
[Equation]
r
[Equation]
x
[Equation]
−
[Equation]
k [ ℓ
[Equation]
≤
[Equation]
−
[Equation]
where the third inequality holds due to Jensen’s inequality.
We note that ˆ˜ E k [ ℓ 1 ( r k ( x ))] ≤
[Equation]
[Equation]
KC k )∆ ℓ implies sup
[Equation]
Due to the assumption inf r ∈H r E [ˆ E k [ ℓ 1 ( r k ( x ))]] > 0 , there exists an η k > 0 such that
E [ˆ E k [ ℓ 1 ( r k ( x ))]] ≥ η k for all r k ∈ H r .
Then we have
[Equation]
where (cid:94) ReLU( z ) =
ReLU( z ) − z .
Denote ˜Φ( S k ) :=
ˆ˜ E k [ ℓ 1 ( r k ( x ))] where S k
= { x tr k, 1 , · · · , x tr n tr k , 1 , x te1 , 1 , · · · , x te K,n te } .
Let S ( i ) k be obtained by replacing element i of set S k by an independent data point taking values from the set
X tr .
We now measure the absolute value of the difference caused by changing one data point in | ˜Φ( S k ) − ˜Φ( S ( i ) k ) | .
If the changed point is sampled from p tr k , the the absolute value of the difference caused in the maximal deviation term is upper bounded by ∆ ℓ n tr k .
If the changed point is sampled from p te l , the the absolute value of the difference caused in the maximal deviation term is upper bounded by ∆ ℓ C k n te for l = 1 , · · · , K .
Finally, McDiarmid’s inequality (McDiarmid et al., 1989) implies:
[Equation]
which completes the proof.
■
Substituting the upper bounds in (I.5) and (I.8) into (I.4), with probability at least 1 − δ , we have
E
[Equation]
(I.9) where Ψ = ∆ ℓ (cid:113) 8( 1 n tr k + K (
[Equation]
+ C k ) 2 n te ) log
[Equation]
for some constant η k > 0 .
This completes the proof.
J G ENERALIZATION BOUND FOR MULTI - LAYER PERCEPTRON AND MULTIPLE CLIENTS
Our generalization error bound for client k depends on the Rademacher complexity of the hypothesis class for our density ratio model H r ⊂ { r : X → B f } w.r.t.
client k train distribution p tr k and all client’s test distributions p te l for l ∈ [ K ] .
By restricting a function class for density ratios and substituting an upper bounds on its Rademacher complexity, we can obtain explicit generalization error bounds in terms of n tr k , n te in a special case.
As an example, the following corollary establishes a generalization bound for multi-layer perceptron density ratio models in terms of the Frobenius norms of weight matrices.
Example J.1 (Complexity for multi-layer perceptron class (Golowich et al., 2018)) .
Assume that distribution p has a bounded support S p :=
sup x ∈ supp( p ) ∥ x ∥ < ∞ .
Let H be the class of real-valued neural networks with depth L over the domain X tr , W i be the network weight matrix i .
Suppose that each weight matrix has a bounded Frobenius norm ∥ W i ∥ F ≤ ∆ W i for i
∈
[ L ] and the activation ϕ is 1-Lipschitz, and positive-homogeneous function, i.e., ϕ ( αz ) =
αϕ ( z ) , which is applied element-wise.
Then we have
[Equation]
Remark 6 .
To control the upper bound ∆ W i for i ∈
[ L ] , it is natural to employ the sparsity of the weights, e.g., (Golowich et al., 2018, Section 4) and (Hanin & Rolnick, 2019).
We consider a special network architecture where diag( W i ) ’s are close to 1-sparse unit vectors for i ∈
[ L ] , which implies that the matrices W i ’s will be almost rank- 1 .
Then ∥ W i ∥ F is upper bounded by 1 for i ∈
[ L ] .
Corollary 1 (Generalization error bound under Example J.1) .
For Example J.1 and loss functions described in Theorem 1, with probability at least 1 − δ , we have
[Equation]
where K tr k = O ( L 1 S p tr k √ L (cid:81) Li =1 ∆ W i ) , K te l
=
O (max { L 1 , L 2 } S p te l √ L (cid:81) Li
=1 ∆ W i ) , and Ψ = ∆
[Equation]
(cid:113)
[Equation]
for some constant η
[Equation]
> 0 .
Finally, we apply union bound and obtain a global generalization error bound that holds for all clients:
Corollary 2 (Generalization error bound for multiple clients) .
Let 0 < δ k < 1 for k ∈ [ K ] .
Let
K tr = max k ∈
[ K ] K tr k .
For Example J.1 and loss functions described in Theorem 1, with probability at least 1 − (cid:80)
Kk =1 δ k , we have
[Equation]
where Ψ = ∆
[Equation]
max
[Equation]
n
[Equation]
min k ∈ [ K ] n tr
[Equation]
min k ∈ [ K ] δ k
[Equation]
and η
[Equation]
min k ∈ [ K ] η k
[Equation]
The rates match the optimal minimax rates for example for a density estimation problem when the density belongs to the Hölder function class (Tsybakov, 2008)[Section 2] with a sufficiently large β based on Definition 1.2 of Tsybakov (2008).
The Ω(1 / √ n ) lower bounds are obtained for important problems including nonparametric regression, estimation of functionals, nonparametric testing, and finding a linear combination of M functions to be as close as the target data generating function (Nemirovski, 1998)[Section 5.3].
K A DDITIONAL E RROR DUE TO E STIMATION OF r k
In this section, we consider a practical scenario where we have access to only in imperfect estimate of r k = sup x ∈X tr r ∗ k
( x ) to find C k in Eq.
In particular, we find additional error when using ˜ C k
= 1˜ r k where ˜ r k is obtained by HDRM in Section 3.
The nnBD DRM problem for client k using ˜
C k is min r k ∈H r ˆ E + f
( r k ) where
[Equation]
Along the lines of the proof of Lemma
[Equation]
we can show that the maximal deviation term using
˜ C k is upper bounded with probability at least 1 − δ :
[Equation]
Under perfect estimate of r k = sup x ∈X tr r ∗ k ( x ) with C k = 1 r k , the nnBD DRM problem for client k is min r
[Equation]
ˆ E
[Equation]
+ f
[Equation]
r k ) where
[Equation]
Applying triangle inequality, we first decompose the bias term
[Equation]
An upper bound on sup r k ∈H r | E [ ˆ E ++ f ( r k )] − E f ( r k ) | is established similar to the proof of Lemma 4:
[Equation]
Substituting Eq.
(K.1) and Eq.
(K.3) into | E [ ˆ E + f ( r k ) − ˆ E ++ f ( r k )] | , we have
[Equation]
which together with ReLU( a ) − ReLU( b ) ≤ | a − b | is used to establish the following upper bound:
[Equation]
Let m ∗ = arg max m ∈
[ M ] ˜ r k,m .
We note that by the construction of HDRM, there is a constant lower bound on the numerator of ˜ r k , i.e., 1 n te (cid:80) n te j =1 (cid:80) Kl =1 1 ( x te l,j ∈ B m ∗ ) ≥ c , that is achieved when { x te l,j } n te j =1 are distributed uniformly across M bins.
Let x ∈ X and let ˆ p tr k ( x ; M ) denote a histogram-based density estimate of p tr k ( x ) with M bins.
The maximum value of ˜ C k is attained when 1 n te (cid:80) n te j =1 (cid:80) Kl =1 1 ( x te l,j ∈ B m ∗ ) meets its lower bound, which leads to the maximum deviation from
C k ≤ ˜ C k .
Assuming p tr k ( x ) is L k -Lipschitz with sup x ∈X p tr k ( x ) < ∞ , the mean squared error of a histogram-based density estimate with M bins is upper bounded by (Wasserman, 2006)[Section 6]: E | ˆ p tr k ( x ; M ) − p tr k ( x ) |
= O
( L 2 k /M
+ M/n tr k ) .
Putting together with a constant lower bound on the numerator of ˜ r k and applying Jensen’s inequality, we have:
[Equation]
L P ROOF OF L EMMA
We first note that
[Equation]
We first find the expression for ˆ θ considering the ridge regression problem assuming p te ( x ) p tr ( x ) is given.
FIDEM problem with Tikhonov regularization is given by
[Equation]
where w i = p te ( x i ) p tr ( x i ) and λ is the regularization parameter.
This is a reweighted least squares problem.
The objective function above is strongly convex and differentiable.
Applying the fist-order condition, the unique minimum is as follows:
[Equation]
where W = diag( w 1 , · · · , w n ) .
Substituting y
=
X θ ∗ + ϵ into (L.1), we note that
[Equation]
and
[Equation]
We now characterize the bias B (ˆ θ ) and variance V (ˆ θ ) terms when the model estimate is given by (L.1).
Let ∥ x ∥ 2 A :=
x ⊤ Ax .
Substituting the expression for ˆ θ into R (ˆ θ ) , the excess risk can be decomposed into a bias and a variance term as follows:
[Equation]
where the bias is given by
[Equation]
with
[Equation]
, and the variance is given by
[Equation]
=
[Equation] [Equation]
where Φ V = (cid:0) X ⊤ WX
+ λ I d (cid:1) − 1 X ⊤ W 2 X (cid:0)
X ⊤ WX
+ λ I d (cid:1) − 1 Σ te .
M P ROOF OF T HEOREM
In the one-hot case, it is clear that X ⊤ X = (cid:80) ni
=1 x i x ⊤ i and X ⊤ WX =
(cid:80) ni
=
w i x i x ⊤ i are diagonal matrices.
For bias in the one-hot setting, we have
B (ˆ θ ) =
λ 2 (cid:20) θ ⊤ ∗ (cid:16) X ⊤ WX
+ λ I (cid:17) − 1 Σ te (cid:16)
X ⊤ WX
+ λ I (cid:17) − 1 θ ∗ (cid:21) =
λ 2 d (cid:88) i =
[( θ ∗ ) i ] 2 λ
′ i
( λ i ( X ⊤ WX ) + λ )=
λ 2 d (cid:88) i =
[( θ ∗ ) i ] 2 λ
′ i [ µ i w i
+ λ ] where the equation holds by the fact that, all matrices are diagonal including X ⊤ X , X ⊤ WX , and Σ te .
Accordingly, we have λ
i ( X ⊤ WX )
=
λ i ( X ⊤ X )
λ i ( W ) with i ∈ [ d ] .
For the classical ERM, the bias is
[Equation]
where λ i is the eigenvalue of Σ tr .
To achieve B (ˆ θ ) ⩽ B ( θ v ) , we have to make some assumptions on the relationship between λ i , λ ′ i and w i .
Our analysis of error bound requires
[Equation]
which implies
[Equation]
such that Eq.
(M.1) holds where we use the inequality a + c b + c ⩽ ab + 1 for any a, b, c > 0 .
For the vanilla ERM, the variance is
[Equation]
For FIDEM, the variance is
[Equation]
We note that V (ˆ θ ) ≤
[Equation]
( θ v ) can be achieved by
[Equation]
This can be obtained by
[Equation]
which implies w i ⩽ ξ i (cid:113)
λ i λ
i .
Combining Eqs.
(M.2) and (M.3), the proof is complete.
N W HEN FIDEM CANNOT OUTPERFORM ERM
In this section, we provide a counterexample to show that, under which certain case, FIDEM cannot provably outperform ERM.
Proposition 3.
Under the same setting of Theorem 2, i.e., the fixed-design setting and label noise assumption, under the following condition
[Equation]
If the ratio satisfies
[Equation]
then we have
[Equation]
Proof.
According to Eq.
(M.1), B ( θ v )
⩽ B (ˆ θ ) holds by
[Equation]
which is equivalent to
[Equation]
According to Eq.
(M.3), V ( θ v ) ⩽
V (ˆ θ ) holds by
[Equation]
which is equivalent to
[Equation]
Combining Eqs.
(N.2) and (N.3), the proof is complete.
To validate the condition in Eq.
(N.1), we require each term in the RHS to be nonnegative.
That implies (cid:115)
[Equation]
which is our condition in Proposition 3.
By checking Eqs.
(N.2) and (N.3), in both cases (cid:113) λ
′ i
λ
i ≥ 1 and (cid:113) λ ′
i
λ i ≤ 1 , we have w i ⩽ 1 .
■
O E XPERIMENTAL DETAILS AND ADDITIONAL EXPERIMENTS
Datasets : We make use of three datasets in the experiments: MNIST (LeCun et al., 1998), Fashion
MNIST 7 (Xiao et al., 2017), and CIFAR10 (Krizhevsky et al., 2009).
MNIST consists of images depicting handwritten digits from 0 to 9 .
The resolution of each image is 28 × 28 .
The dataset includes 60 , 000 images for training.
Similarly Fashion MNIST includes grayscale images of clothing of resolution 28 × 28 .
The training set consists of 60 , 000 examples, and the test set of 10 ,examples.
CIFAR10 consists of colored images with a resolution of 32 × 32 .
The training set contains 50 , 000 examples while the test set contains 10 , 000 examples.
Experimental setup : For all experiments we use the cross entropy loss.
The stochastic gradient for each of the clients are computed with a batch size of 64 and aggregated on the server, which uses the
Adam optimizer.
Experiments on MNIST and Fashion MNIST uses a LeNet (LeCun et al., 1998), a learning rate of 0 .
001 , no weight decay, and runs for 5 , 000 iterations.
For CIFAR10 experiments we use the larger ResNet-
(He et al., 2016).
Batch normalization in ResNet-18 is treated by averaging the statistics on the server and subsequently broadcasting to the workers.
A learning rate of 0 .and
weight decay of 0 . 0001 are used.
We report the best iterate in terms of average test accuracy after 20 , 000 iterations.
All reported mean and standard deviations are computed over 5 independent runs except for CIFAR10 which uses 3 independent runs.
For target shift the randomisation is also over the realization of the class distributions to ensure that the conclusions are not due to the particularities of the sub-sampled images.
All experiments are carried out on an internal cluster using one GPU.
[Table] [Table]
[Table]
[Table]
[Table]
Test
[Table]
Client
[Table]
Train
[Table]
Test
[Table]
Train
[Table]
Test
[Table]
[Table]
O. T ARGET SHIFT
For the target shift experiments on Fashion MNIST in Table 1, we summarize the different number of data points for each dataset split in Table 9.
A similar distribution across clients is used for the additional experiments for FIDEM and FedAvg on CIFAR10 (Table 8).
CIFAR10 differs from Fashion
MNIST in the number of examples due to the training set being smaller.
The results for CIFAR10 in
Table 7 shows that FIDEM uniformly improves the accuracy over FedAvg on this difficult target shift instance.
We additionally include a two-client setting in Table 10 with the associated distribution described in Table 11.
To model a scenario closer to real-world FL, we consider a setting with 100 clients on CIFAR10 under challenging distribution shifts and partial participation of clients, which is a requirement for crossdevice FL (Kairouz et al., 2021;
Wang et al., 2021).
We sub-sample 5 clients uniformly at random at every round for 200 , 000 iterations.
The target distribution is described in Table 5 and experimental results can be found in Table 6.
We observe that FIDEM uniformly improves the test accuracy when compared with FedAvg and that the gap is especially large between the worst-performing clients.
To compute the exact ratio r ( x ) we will assume that the distributions are separable.
Definition 3 (Separability) .
A distribution over X × Y is separable if there exists a partition ( X i ) mi = of X such that p ( y i |X i )
= 1 for some y i ∈ Y and all i ∈ [ m ] .
We denote the associated deterministic label assignment as g : X → Y .
[Table] [Table]
Proposition 4.
Assume that the distributions p te ( x , y ) and p tr ( x , y ) are both separable.
Then the ratio can be computed based on the associated label y :
= g ( x ) as follows,
[Equation]
Proof.
Due to separability, p te ( y | x ) = p tr ( y | x ) .
So
[Equation]
It follows that,
[Equation]
Using the definition of the target shift assumption, p te ( x | y )
= p tr ( x | y ) , the conditional distributions cancel and we obtain the claim.
■
Proposition 4 provides a way to compute the ratio r ( x ) when the labels are available and the shift is known.
O. C OVARIATE SHIFT
The color flipping probability used to generate each of the colored MNIST datasets for the covariate shift experiment can be found in Table 12.
We consider an asymmetric client setup where client 1 in addition has 40 times less training examples than client 2.
O. V ERIFYING ASSUMPTIONS
Consider the two datasets used for the main experiments in Table 1 and Table 2.
We verify in
Figure 2 that the eigenvalues of the training distribution and test distribution for each client satisfy (cid:113)
[Equation]
(cid:17) in Theorem 2.
[Table]
[Table]
P C OMPUTATIONAL COMPLEXITY OF A LGORITHM
We note that clients compute the ratios in parallel where each client needs to estimate one ratio.
To estimate density ratios for FIDEM, clients require to send a few unlabelled test samples only once .
The server shuffles those samples and broadcasts the shuffled version to clients only once .
Compared to FedAvg, the additional computational cost per client is O ( TN k ) where T is the number of iterations for Algorithm 1 to converge and N k is the number of batches for ratio estimation.
Compared to baseline FedAvg, the additional computation of FIDEM is negligible but leads to substantial improvements of the overall generalization in settings under challenging distribution shifts.
Q L IMITATIONS
In this paper, we focus on settings where ratio estimation is required once prior to model training.
Handling distribution shifts in complex non-stationary settings where ratio estimation is an ongoing process is an interesting problem for future work.
In addition, various personalization methods have been proposed to improve fairness in terms of uniformity of model performance across clients (Li et al., 2021a;b).
To meet specific requirements of each client, our global model can be combined with a personalized model on each client.
Developing new variants of FIDEM with a focus on fairness is an interesting problem for future work.
[Table]
p tr1 ( x ) p te1 ( x ) p tr2 ( x ) p te2 ( x )
Probability of flipping color 0.
[Table]
To estimate { r k ( x ) } Kk =1 , clients need to send unlabelled samples x te l,j for l ∈ [ K ] and j ∈ [ n te ] from their test distributions.
We note that instead of their true samples, clients can alternatively send samples generated from a generative model (Goodfellow et al., 2020).
Note that training GANs may be computationally extensive due to required computational resources and availability of representative samples.
However, we propose to use GANs as an alternative method with clear caveats, only when 1) clients have sufficient computational resources and 2) they are unwilling to share unlabelled data with the server.
As a partial mitigation of privacy risks, we introduced FIIDEM.
FIIDEM does not require any data sharing among clients and does not require any GAN training.
In this paper, we focus on FIDEM since it outputs an unbiased estimate of a minimizer of the overall true risk , and enables us to theoretically show the benefit of importance weighting in generalization.
One particular challenge in real-world cross-device FL is to estimate ratios on real-world datasets such as WILDS (Koh et al., 2021) and LEAF (Caldas et al., 2019).
WILDS has been mostly used for domain generalization, where the setting is not similar to ours.
We still have to decide on an arbitrary test/train split.
LEAF mainly captures inter-client distribution shifts and settings where different clients have different numbers of examples over thousands of clients.
This work is not about scalability to thousands of clients experimentally using our single GPU simulated setup.
While we anticipate efficient ratio estimation will improve over time, our FIDEM and FIIDEM formulations along with improved ratio estimates will provide reasonable solutions to learn an effective global model in real-world cross-device FL under covariate shifts.