R OBUST AND P ERSONALIZED F EDERATED L EARNING WITH S PURIOUS F EATURES : AN A DVERSARIAL A PPROACH
Anonymous authors
Paper under double-blind review
A BSTRACT
A common approach for personalized federated learning is ﬁne-tuning the global machine learning model to each local client.
While this addresses some issues of statistical heterogeneity, we ﬁnd that such personalization methods are often vulnerable to spurious features , leading to bias and diminished generalization performance.
However, debiasing the personalized models under spurious features is difﬁcult.
To this end, we propose a strategy to mitigate the effect of spurious features based on our observation that the global model in the federated learning step has a low accuracy disparity due to statistical heterogeneity.
Then, we estimate and mitigate the accuracy disparity of personalized models using the global model and adversarial transferability in the personalization step.
We theoretically establish the connection between the adversarial transferability and the accuracy disparity between the global and personalized models. Empirical results on MNIST, CelebA, and Coil20 datasets show that our method reduces the accuracy disparity of the personalized model on the bias-conﬂicting data samples from 15.12% to 2.15%, compared to existing personalization approaches, while preserving the beneﬁt of enhanced average accuracy from ﬁne-tuning.
I NTRODUCTION
Federated learning (FL) is a leading framework for clients to collaboratively train a shared global machine learning (ML) model without releasing their local private datasets (McMahan et al., 2017;
Kairouz et al., 2019).
The jointly trained global model could be further ﬁne-tuned on each client’s local dataset to produce personalized models (Fallah et al., 2020; T. Dinh et al., 2020; Li et al., 2021).
While existing theoretical and empirical results highlight how personalized models improve accuracy on local data, few works consider what features the personalized models learn from the local dataset.
Our motivating hypothesis is that not all local features are beneﬁcial .
For example, consider a gender prediction task using face images, where the ML model learns to predict gender based on hair color because females are more likely to have blond hair (Sagawa et al., 2020).
In this case, the hair color is called a spurious feature because it only statistically correlates with the gender on biased-aligned samples but not necessarily on the overall population.
Thus, the accuracy of a model that relies on spurious features such as hair color is likely to drop signiﬁcantly on bias-conﬂicting samples where the spurious correlation does not hold, e.g., for blond male images (Sagawa et al., 2020). This paper calls the accuracy difference of an ML model on the dataset with spurious features and the dataset without spurious features accuracy disparity .
More broadly, the accuracy disparity caused by spurious features leads to issues in both fairness (McNamara et al., 2019; Zhao & Gordon, 2019; Agarwal et al., 2019; Chi et al., 2021), i.e., racial bias (Khani & Liang, 2021) and robustness, i.e., accuracy decrease under distribution shift (Zhao et al., 2019; Koh et al., 2021).
Compared to the global model, because of the local ﬁne-tuning, the personalized models are more vulnerable to spurious features and have a larger accuracy disparity.
Empirically, we observe that the typical non-i.i.d.
data distributions in FL settings reduce the accuracy disparity of the global model.
One potential explanation is that the statistical heterogeneity (Wang [Figure] et al., 2021) under spurious features across users is larger than that of non-spurious features.
As a result, the aggregation operation in the central server averages out the diverse shifts under the spurious features (Figure 1) so that the global model becomes more robust against spurious features on the benchmark datasets.
On the other hand, the local ﬁne-tuning step will result in a personalized model that entangles spurious features and becomes biased.
Hence, it remains a key challenge how to debias the personalized models.
Various methods (Wang et al., 2019; Sagawa et al., 2020; Liu et al., 2021) have been developed to disentangle spurious features from ML models, but few can be easily applied to personalized models under the federated learning setting.
The main reason lies in their reliance on bias-conﬂicting samples.
The bias-conﬂicting samples can be rare.
For example, in the CelebA dataset, only around 1.7k samples are bias-conﬂicting out of more than170k samples.
If we distribute the CelebA dataset across users, nearly half do not have any bias-conﬂicting samples.
Prior work (Li & Wang, 2019) has tried using a global proxy dataset for training FL models.
However, a global proxy dataset may not characterize the local samples well.
Furthermore, estimating the accuracy disparity of the personalized model becomes difﬁcult without access to bias-conﬂicting samples.
To approach this problem, we propose a novel method to reduce the accuracy disparity of personalized models.
Our proposed method does not rely on the bias-conﬂicting samples, e.g., blond male images in the aforementioned task of gender prediction, which are often rare and may not be available for every user.
Instead, inspired by prior works (Tram`er et al., 2017; Liang et al., 2021) on the transferability of adversarial examples, we use the global model as a reference and the adversarial transferability between the global model and the personalized models as a proxy to estimate the accuracy disparity of the personalized models.
The intuition is that if two ML models use disjoint subsets of features, the adversarial examples that one ML model generates do not transfer to the other
ML model.
Based on this intuition, we propose the following hypothesis:
If the personalized models entangle spurious features (thus increasing the accuracy disparity), the adversarial examples generated by the global model (that uses non-spurious features) do not transfer to the personalized models.
Empirically, we show that the adversarial transferability between the global and personalized models strongly correlates with the accuracy disparity between the global and personalized models (Section 4), validating our hypothesis.
We further theoretically connect the adversarial transferability to the accuracy disparity (Section 5). Based on the empirical observations and theoretical results, we develop a method that enforces adversarial transferability between the global and the personalized models to reduce the accuracy disparity of personalized models (Section 6).
Our contributions are summarized as follows
: • We empirically evaluate the accuracy disparity of the global and personalized models in a federated learning setting with spurious features, highlighting a risk of existing personalization methods.
• We design a method to estimate the accuracy disparity of the personalized models, based on the low accuracy disparity global model and the adversarial transferability between the global and personalized models.
• We theoretically connect the adversarial transferability and the accuracy disparity of the global and personalized models. • We develop a methods to reduce the accuracy disparity of personalized models by enforcing the adversarial transferability between the global and personalized models.
Empirically, we conduct extensive experiments to validate the effectiveness of the proposed methods in mitigating accuracy disparity under the FL setting.
Our experiments on MNIST (Deng, 2012),
CelebA (Liu et al., 2015), and Coil20 (Nene et al., 1996)
datasets show that the proposed approach reduces the accuracy disparity of personalized models from 15 .
12% to 2 .
15% , which is closer to that of the global model ( − 0 .
Our method also preserves the beneﬁt of the enhanced average accuracy from ﬁne-tuning, resulting in 3 .
43% accuracy improvement on the biased test set and 0 .
85% accuracy improvement on the biased-conﬂicting test set.
R ELATED W ORK
Personalized Federated Learning Fine-tuning is typical for personalization methods.
The metalearning-based method ﬁrst trains a global model and ﬁne-tunes the global model locally (Fallah et al., 2020).
Other methods using multi-task learning (Li et al., 2021) or Moreau envelopes (T. Dinh et al., 2020) have an interpretation as ﬁne-tuning the local model along with training the global model.
Fine-tuning is also compatible with clustering-based method (Ghosh et al., 2020).
Debiasing ML Models A few prior works (Li & Vasconcelos, 2019; Sagawa et al., 2020) utilize group labels, which might require human annotation, to debias ML models.
For example, the group distributional robust optimization (DRO) method (Sagawa et al., 2020) aims to optimize the worst-case error rate of ML models across different (often manually annotated) groups.
Some groups contain bias-conﬂicting samples while others do not.
Residual learning-based methods (He et al., 2019; Nam et al., 2020; Liu et al., 2021) train a biased ML model and up-weight the residual, which mainly contains bias-conﬂicting samples that the biased ML model mis-predicts.
Chi et al.
(2021) aims to mitigate the accuracy disparity in regression problems via learning the appropriate representations.
However, all these methods rely on the explicit access to the bias-conﬂicting samples, making them difﬁcult to apply on personalized federated learning, where bias-conﬂicting samples may not be accessible for every client.
P RELIMINARIES
Deﬁnitions and Notation A data sample is a vector x =
[ x r , x s ] , where x r corresponds to the robust and non-spurious features and x s are the spurious features. d s is the dimension of spurious features.
Let y be a label, and deﬁne (cid:96) :
Y × Y −→ R to be a λ -smooth, twice differentiable loss function and L ( f, D ) = E ( x ,y ) ∼D
[ (cid:96) ( f ( x ) , y )] to be the empirical risk. w g and w p are the weights for the global model f g : X −→ Y and personalized model f p : X −→ Y , respectively.
γ is the ratio between the gradient norms of the global and personalized models, (cid:107)∇ x (cid:96) ( f g ( x ) ,y ) (cid:107) (cid:107)∇ x (cid:96) ( f p ( x ) ,y ) (cid:107) .
Let supp( D ) be the support of distribution D .
We deﬁne a global data distribution D g , a biased local data distribution D b , and assume a bias-conﬂicting local data distribution D bc .
We deﬁne the “pseudo-gradient” as the difference between the updated local model and the global model from the previous round (sometimes we will use the term “gradient” when it is clear from context).
(cid:104)· , ·(cid:105) denotes an inner product of two vectors and · (cid:95) · denotes a concatenation of two vectors.
θ is the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x (cid:96) ( f p ( x ) , y ) .
θ g is the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x r (cid:96) ( f g ( x ) , y ) (cid:95) 0 , which measures the entanglement of the global model to spurious features.
Training and Personalization Methods
We train the global model using the federated averaging algorithm
(McMahan et al., 2017), which learns a model f : X −→ Y that minimizes: L ( f, D g ) =
(cid:80)
Ni =1 |D b i | / |D g | · E x ,y ∼D bi (cid:96) ( f ( x ) , y ) , where N is the number of clients,
D b i is the biased local dataset for client i , and D g = ∪ N i =1 D b i is the global dataset.
When the global model f g converges, f g will be sent to local clients for further ﬁne-tuning by minimizing E x ,y ∼D bi (cid:96) ( f ( x ) , y ) .
Adversarial Examples and Transferability We can generate an adversarial example x adv given data sample x with label y by solving:
[Equation]
[Figure] [Figure]
where f is the victim ML model, and (cid:15) is the attack budget.
In this example, we consider the Lattacks, but extensions to general L p attacks are straightforward.
We say an adversarial example to be transferable if it also fools another ML model (e.g., a personalized model) other than the original victim model f
(e.g., the global model).
A N E MPIRICAL S TUDY WITH S PURIOUS F EATURES
To gain some insights into the problem, we ﬁrst perform an empirical study on the accuracy disparity of the global and personalized models in an FL setting. In this study, the personalization method is ﬁne-tuning.
Our results highlight the risk of existing ﬁne-tuning-based personalization methods and the difﬁculty of mitigating the risk.
We also highlight the correlation between the adversarial transferability and the accuracy disparity between the global and personalized models. We provide additional theoretical analysis in Section 5 to support the observed correlation.
The spurious features in the empirical study are as follows.
Spurious Features We consider color as the spurious feature for the MNIST, CelebA, and Coil datasets.
In the MNIST and Coil20 datasets, we manually color the objects according to their labels to create spurious correlations, as Figure 2 shows.
The spurious correlations vary across clients for the MNIST and Coil20 data (e.g., the red color correlates with label zero on the ﬁrst client and with label one on the second client) to create additional statistical heterogeneity.
In the CelebA dataset, the hair color attribute correlates with the gender label.
We assign disjoint subsets of celebrities to different users, which naturally increases statistical heterogeneity for the spurious correlation.
4. STATISTICAL HETEROGENEITY R EDUCES A CCURACY D ISPARITY
Figure 3 shows the accuracy disparity of ML models on biased and bias-conﬂicting test sets.
Compared to the models trained in the centralized setting, where the spurious correlations are ﬁxed, the accuracy disparity of models trained in the federated setting decreased signiﬁcantly.
These empirical results suggest that the global model in FL is more robust to spurious features if the spurious features are non-i.i.d.
across clients.
To explain this observation, consider the relationship between the gradient directions and the learned features.
For the spurious features, its correlation with the label may change across clients.
As [Figure] [Figure] an example, in some users’ local datasets, the blond hair does not correlate with the gender label because the dataset does not contain any blond female image or the dataset has blond male images, as visualized in Figure 1a.
Therefore, the gradient directions for the spurious features are diverse across clients, as shown in Figure 1b.
The divergence between the gradient directions, as a consequence, makes learning spurious features difﬁcult.
In contrast, the non-spurious features, e.g., shape features, are more consistent across clients, leading to a more consistent gradient direction.
4. P ERSONALIZATION M AY E XACERBATE A CCURACY D ISPARITY
Although the global model in the federated setting has a lower accuracy disparity than in the centralized setting, the advantage could vanish during the personalization step.
The Personalized Model Entangles Spurious Features As can be observed in Figure 4, the accuracy ﬁrst increases and decreases on the MNIST bias-conﬂicting test set and slowly decreases on
Coil20.
These two observations indicate that the personalized model entangles spurious features and exacerbates the accuracy disparity in a few batches.
Although in principle, one may resort to early stopping, this is not feasible when the bias-conﬂicting dataset is unavailable or scarce.
4. A DVERSARIAL T RANSFERABILITY I NDICATES A CCURACY D ISPARITY
Because the bias-conﬂicting test set is often unavailable, it is infeasible to directly measure the accuracy disparity across personalized models.
To this end, in this section, we focus on methods that implicitly measure the accuracy disparity and determine whether the personalized models entangle spurious features.
Following our hypothesis in Section 1, we consider using the adversarial transferability between the global and personalized models as a proxy for the accuracy disparity measurement.
Figure 5 plots the accuracy disparity and the adversarial transferability during ﬁnetuning.
As the accuracy disparity of the personalized models increases and drifts away from that of the global model, the adversarial transferability between the global and personalized models decreases.
This result empirically validates our hypothesis.  T HEORETICAL I NSIGHTS
This section presents our theoretical result that supports our hypothesis in Section 1 and the experimental results in Section 4.3. Our theoretical result applies the loss, which has similar behavior to the accuracy as is shown in the empirical results in Section 7. Before we proceed, some additional deﬁnitions and notations are needed for the presentation, and we provide a table summarizing all the notations used in Appendix A to ease the reading. Then, we connect both the loss disparity and the adversarial transferability to the angle between the gradients of the global and personalized models.
In what follows, we shall show an upper bound of the loss disparity of a personalized model, which consists of the adversarial transferability between the global and personalized models and an indicator of the entanglement of the global model to spurious features. 5. M ORE D EFINITIONS AND N OTATION We deﬁne natural perturbation ∆ to model the distribution shift between the bias-conﬂicting D bc and biased D b . ∆ could change a bias-aligned sample to a corresponding bias-conﬂicting sample.
The distribution D ∆ | x of the natural perturbation ∆ conditions on the data sample x . Formally, for any x ∈ R d , we have: [Equation] Running Example For a non-blond male image x , we could draw a natural perturbation ∆ from
D ∆ | x that change the hair color in x to blond. That is saying, x + ∆ is a blond male image. Iteratively drawing data samples from the biased dataset and applying the sampled natural perturbations to the data samples result in a dataset with bias-conﬂicting samples. Another perturbation to consider is the adversarial perturbation δ f,(cid:15) =
x adv − x that is generated using f with budget (cid:15) . Plugging the deﬁnition of δ f,(cid:15) into Eq. (1), we have δ f,(cid:15) = arg max (cid:107) δ (cid:107)≤ (cid:15) (cid:96) ( f ( x + δ ) , y ) .
Since the budget (cid:15) is small, we could approximate the loss function (cid:96) using the ﬁrst-order gradient: δ f,(cid:15) =
[Equation]
( x ) ,y ) (cid:107) (Miyato et al., 2018; Liang et al., 2021).
With the adversarial perturbation, we deﬁne the adversarial transferability loss:
[Equation]
which indicates the effectiveness of the adversarial perturbation generated using the global model applied to the personalized models.
5. L OSS D ISPARITY AND A DVERSARIAL T RANSFERABILITY
With the deﬁnitions of natural and adversarial perturbations, we have the following lemmas, showing that both the loss disparity and the adversarial transferability connect to an angle θ .
Lemma 1.
Let ∆ be the natural perturbation, θ be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x (cid:96) ( f p ( x ) , y ) , θ g be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x r (cid:96) ( f g ( x ) , y ) (cid:95) 0 , and γ be
[Equation]
x (cid:96)
[Equation] f g ( x ) ,y ) (cid:107) [Equation] x (cid:96) ( f p ( x ) ,y ) (cid:107) , we have:
[Equation]
Lemma 1 connects the loss disparity to θ . The θ g , differing from θ , is an indicator of the entanglement of the global model to spurious features and is a constant during the personalization step. Lemma 2. Let (cid:15) be the attack budget, θ be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x (cid:96) ( f p ( x ) , y ) , γ be (cid:107)∇ x (cid:96) ( f g ( x ) ,y ) (cid:107) (cid:107)∇ x (cid:96) ( f p ( x ) ,y ) (cid:107) , and the loss function (cid:96) :
Y × Y −→ R be λ -smooth, twice differentiable, we have
[Equation] [Equation] y [Equation]
Lemma 2 connects the adversarial transferability loss to θ .
In the following analysis, we connect the loss disparity to adversarial transferability via θ .
5. A G ENERALIZATION U PPER B OUND
We now present an upper bound of the disparity L ( f p , D bc ) − L ( f p , D b ) .
The main idea is to derive an upper bound of (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107) · sin θ in Eq.
(2) from Eq.
Next we outline the assumption.
Assumption 1.
The distribution shift does not exacerbate the entanglement of a model to spurious features:
[Equation]
With Assumption
[Equation]
the following theorem holds:
Theorem 3. Let γ min be the minimum of γ , under Assumptions 1 and Lemmas 1-2, we have: [Equation] Theorem 3 suggests (1) debiasing the global model f g , whose entanglement to spurious features is measured by θ g , and (2) enforcing the adversarial transferability between f g and f p help reducing the loss disparity of personalized models.
For the constants γ min and λ , we further explore their impacts in the following sections.
With the empirical and theoretical results in Sections 4.3 and 5, respectively, it is natural to ask if enforcing the adversarial transferability in the personalization step reduces the accuracy disparity.
In this section, we ﬁrst introduce adversarial examples to the personalization step as a regularization term added to the original loss function, aiming to enforce the adversarial transferability .
However, the accuracy disparity still increases, albeit much slower, even if the adversarial transferability remains high.
One possible reason is that the personalized model increases its gradient norm, which helps preserve the adversarial transferability but does not prevent the personalized model from entangling spurious features.
To this end, we add an L 2 regularization term to the loss function, aligning the gradient norms of the global and personalized models .
Combing these two methods addresses the accuracy disparity.
Both methods are relatively light-weight from a computational perspective.
6. E NFORCING A DVERSARIAL T RANSFERABILITY
We enforce that global and personalized models make consistent predictions on adversarial examples, such that adversarial examples transfer from one to the other.
Adversarial Examples
The projected gradient descent (PGD) attack (Madry et al., 2018) is an effective attack method that uses the neural network’s ﬁrst-order gradient, and is easy to compute.
Additionally, . The attack solves x
[Equation]
=
[Equation]
( f ( x
[Equation]
) , y ) iteratively.
At iteration t + 1 , the adversarial example is:
[Equation]
x tadv ) , y ))) , where Proj is a projection operator.
Enforcing Consistent Predictions Both the global model f g and the personalized model f p take the adversarial example x adv as input and output z g and z p from their last layers, respectively.
We enforce the adversarial transferability by adding the following regularization term, which maximizes the cross-entropy between z g and z p .
Since the global model f g is ﬁxed as a reference in the personalization step and its low accuracy disparity is desirable, we use z g as the ground-truth:
[Equation]
where K is the number of classes.
The local model has access to the global model, so there is no additional communication overhead for implementing this regularization. The adversarial examples are computed using the global model once for all.
The computation only needs a few back-propagations, much less than training the global model.
6. A LIGNING G RADIENT N ORMS
In Eq.
(3), we have seen that the adversarial transferability loss depends not only on the angle θ , which connects the transferability to the disparity but also on γ := (cid:107)∇ x (cid:96) ( f g ( x ) ,y ) (cid:107) (cid:107)∇ x (cid:96) ( f p ( x ) ,y ) (cid:107) .
A small γ indicates that the personalized model increases its gradient norm.
With a larger gradient norm, the personalized model could entangle the spurious features and increase θ without impacting the adversarial transferability.
To prevent γ from decreasing, we employ a simple and effective strategy by adding an L 2 regularization term R L 2 = (cid:107) w g − w p (cid:107) 2 to the loss function.
The motivation behind the L 2 term is straightforward: if two models have similar weights, they have similar gradients.
Empirical results in
Appendix D.2 show the effectiveness of the L 2 term in controlling γ .
Although prior works (Li et al., 2020; T. Dinh et al., 2020; Li et al., 2021) have explored similar regularization methods, we develop the regularization term from a different perspective.
E XPERIMENTS
This section presents our experimental results, demonstrating that our method reduces the accuracy disparity.
We also show that the beneﬁt of enhanced average accuracy from ﬁne-tuning is preserved.
7. S ETTING
Data Partition We distribute the MNIST and Coil20 dataset across 50 clients where each client have 5 different classes.
The local dataset on each client is further partitioned to train/validation/test set with a ratio of 72:8:20, following prior work (Li et al., 2021).
We make two data partitions for
CelebA: CelebA R using a real partition and CelebA S using a synthetic partition, both haveclients.
In both CelebA partitions, each client represent a disjoint set of celebrity Li et al.
Due to the limited space, we further detail the data partition in Appendix C.1, report the hyperparameters in Appendix C.2, and list the neural network architecture in Appendix C.3.
[Figure]
7. T HE E FFECTIVENESS OF P ROPOSED M ETHODS
We conduct an ablation study on the MNIST dataset.
Figures 6b and 6f demonstrate the effectiveness of enforcing adversarial transferability.
However, the accuracy and loss disparity still increase during personalization, which is potentially caused by the gradient norm issue (Section 6.2).
Aligning the gradient norms by applying the L 2 regularization term while enforcing adversarial transferability address the accuracy and loss disparity as Figures 6d and 6h show, respectively.
Figure 6c and 6g further show that applying the L 2 regularization term alone does not address the accuracy or loss disparity. Compared to naive ﬁne-tuning, which is reported in Figures 6a and 6e, our method mitigates the accuracy and loss disparities by ∼ 50% [Table] [Table]
7. A NALYSIS
We compare our method to no personalization (Global), naive ﬁne-tuning (FT), Ditto (Li et al., 2021), up-weighting (UW) (Sagawa et al., 2020), and just train twice (JTT)
(Liu et al., 2021).
The up-weighting method is implemented via sampling bias-aligned and bias-conﬂicting samples with equal probability (Sagawa et al., 2020).
Up-weighting and JTT are not applicable to Coil20 due to the lack of bias-conﬂicting samples.
We use the local ﬁnetuning version of the Ditto solver because the local ﬁnetuning solver performs fewer local updates than that of the joint optimization solver and therefore entangles spurious features less.
Each experiment is repeated 9 times with 3 random seeds for the federated learning step and 3 for the personalization step.
We select models using the validation accuracy minus the decrease of adversarial transferability and using the validation accuracy for other baseline and competitor methods.
Tables 1 shows the main result.
Our method reduces the accuracy disparity of personalized models from 15 .
12% to 2 .
15% , compared to other personalization methods.
Our method also preserves the enhanced average accuracy from ﬁne-tuning, resulting in 3 .
43% accuracy improvement on the biased test set and 0 .
85% improvement on the biased-conﬂicting test set compared to the global model.
In contrast, the naive ﬁne-tuning method sacriﬁces the accuracy on the bias-conﬂicting test set by up to 14 .
3% and increase the accuracy disparity by 15 .
We also ﬁnd that our methods outperform the supervised up-weighting method and the unsupervised JTT method, which increase the average accuracy disparity to 7 .
56% and 17 .
76% , respectively.
One possible reason is that the diversity of the up-weighted bias-conﬂicting samples are small.
Therefore, the neural network could memorize them instead of discarding spurious features.
Appendix D.3 further shows empirical results that support our analysis.
C ONCLUSION
In this work, we explor the risk of federated learning personalization methods with spurious features, which lead to high accuracy disparity between the global and local models.
Through empirical and theoretical explorations, we identity a strong positive correlation between accuracy disparity and the adversarial transferability between the global and local models.
Inspired by this observation, we develop a strategy by enforcing the adversarial transferability between the global and personalized models to mitigate this risk and hence reduce the accuracy disparity.
Our empirical results show that enforcing transferability reduces the accuracy disparity of personalized models.
E THICS S TATEMENT
Our method mitigates the issue of spurious features, which lead to bias towards minority groups, in personalized federated learning.
However, completely disentangling spurious features remains challenging and is an issue for many federated learning methods.
R EPRODUCIBILITY S TATEMENT
Our implementation, including data partition scripts, is based on the FedML library (He et al., 2020), which is open-sourced. The proofs of the theoretical results are in Appendix B.
The datasets in our experiments are publicly available.
We detail the data partition in Appendix C.1, report the hyper-parameter tuning in Appendix C. and list the neural network architecture in Appendix C
Appendix A N OTATION
[Table] robust feature and spurious features in x = [Table]
[Table]
respectively
[Table]
dimension of
[Table]
respectively
[Table]
An adversarial purtubation generated by the global model f g with attack budget (cid:15) ∆
An natural perturbation, which could ﬂip the spurious attribute
D g
The global distribution, which is the union of local distributions
D b
A biased local distribution
[Table]
A bias-conﬂicting local distribution
[Table]
The distribution of
[Table]
perturbation
[Table]
The support of distribution D
[Table]
An inner product of two vectors ·
[Table]
A concatenation of two vectors
B P ROOFS B. P ROOF OF L EMMA
Lemma 1.
Let ∆ be the natural perturbation, θ be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x (cid:96) ( f p ( x ) , y ) , θ g be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x r (cid:96) ( f g ( x ) , y ) (cid:95) 0 , and γ be (cid:107)∇ x
[Equation]
g
[Equation]
x ) ,y )
[Equation]
x (cid:96) ( f p ( x ) ,y ) (cid:107) , we have:
[Equation]
Proof.
Rewriting L ( f p , D bc ) and introducing ∆ :
[Equation]
Moving L ( f p , D b ) to the left-hand-side (LHS), we have:
[Equation]
According to Assumption 1, we further have:
[Equation]
Next, we connect (cid:104)∇ x s (cid:96) ( f p ( x + α · ∆ ) , y ) , 1 (cid:105) to (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107)· sin θ .
The ﬁrst step is connecting (cid:104)∇ x s
[Equation]
[Equation]
p (
[Equation]
[Equation]
[Equation]
[Equation]
to (cid:107)∇ x s (cid:96) ( f p ( x ) , y ) (cid:107) using Cauchy-Schwarz inequality:
[Equation]
[Equation]
Then, we connect (cid:107)∇ x s (cid:96) ( f p ( x ) , y ) (cid:107) to (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107) .
Assuming the global model f g entangles spurious features and the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x r (cid:96) ( f g ( x ) , y ) (cid:95) 0 is θ g , we have:
[Equation]
Since it is easy to see that θ ∈ [0 , π 4 ] and the gradient of sin θ is monotonically decreasing in [0 , π 4 ] , we have:
[Equation]
Combining Eq.
(7) and Eq.
(8), we have:
[Equation]
Recalling the deﬁtion of γ :
= (cid:107)∇ x (cid:96) ( f g ( x ) ,y ) (cid:107) (cid:107)∇ x (cid:96) ( f p ( x ) ,y ) (cid:107) and combining Eq.
(4), Eq.
(5), Eq.
(6), Eq.
(9) complete the proof.
B. P ROOF OF L EMMA
Lemma 2.
Let (cid:15) be the attack budget, θ be the angle between ∇ x (cid:96) ( f g ( x ) , y ) and ∇ x (cid:96) ( f p ( x ) , y ) , γ be (cid:107)∇ x (cid:96) ( f g ( x ) ,y ) (cid:107) (cid:107)∇ x (cid:96) ( f p ( x ) ,y ) (cid:107) , and the loss function (cid:96) :
Y × Y −→ R be λ -smooth, twice differentiable, we have
[Equation]
trans (
[Equation]
[Equation]
[Equation]
, y )
[Equation]
Proof.
Under the deﬁnition of the adversarial perturbation, it is easy to see that δ f,(cid:15) =
(cid:15)
[Equation]
x )
[Equation]
and δ f,(cid:15) increases the loss by:
[Equation]
where ˜ x g is a linear interpolation between x and x + δ f g ,(cid:15) , by the Lagrange’s mean-value theorem.
Similarly, for a transferable adversarial example from f g applies to f p , δ f g ,(cid:15) could increase the loss of f p by:
[Equation]
where cos θ =
[Equation]
Plugging the approximations above to the adversarial transferability loss
[Equation]
we have:
[Equation]
Under the λ -smooth assumption on the loss function
[Equation]
the spectral norms of the Hessian metrics are bounded.
Therefore, we could bound the norm of the deviate between the quadratic terms (Nesterov, 2003, Proof of Theorem 2.1.5) in the adversarial transferability loss:
[Equation]
Since the quadratic terms in Eq.
(10) are scalars
[Equation]
we have:
[Equation]
Plugging Eq.
(11) and the deﬁnition of γ to (cid:96) trans ( f g , f p , x , y ) completes the proof
[Equation]
B. P ROOF OF T HEOREM
Theorem 3.
Let γ min be the minimum of γ , under Assumptions 1 and
Lemmas 1-
[Equation]
we have:
[Equation]
Proof.
According to Lemma 2, we know:
[Equation]
where cos θ =
[Equation]
x (cid:96) (
[Equation]
p
[Equation]
x )
[Equation]
y ) (cid:107) .
Then, we derive an upper bound of (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107) · sin θ from (cid:96) trans ( f g , f p , x , y ) .
It is easy to see that θ ∈ [0 , π 4 ] .
Therefore, we have: [Table] [Equation] [Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
According to Lemma
[Equation]
we know:
[Equation]
[Equation]
Combining Eq.
(13) and Eq.
(12), and taking expectation of x , y over D b :
[Equation]
Plugging Eq.
(14) back to Eq.
(13) completes the proof.
C M ORE E XPERIMENTAL S ETTING C. D ATA P ARTITION
We distribute the MNIST and Coil20 dataset across 50 clients.
Each client has data from 5 different classes.
The local dataset on each client is further partitioned to train/validation/test set with a ratio of 72:8:20, following prior work (Li et al., 2021).
The test set here is biased.
We alternate the spurious features in biased test sets by recoloring the data to create a bias-conﬂicting test set.
For the
CelebA dataset, we consider two partitions.
In the ﬁrst partition (CelebA R), each client represents 20 celebrities.
One celebrity only appears on one client.
The blond male images in the biased test sets are copied to bias-conﬂicting test sets.
We use all clients for training the global model.
In the personalization step, we select the clients who have more than 5 blond female training samples and more than 5 blond male test samples.
We select these clients because they provide enough samples to create spurious correlations and bias-conﬂicting test sets.
Although the ﬁrst partition on CelebA is real, the number (161) of blond male images is small.
To make the result clearer, we create another synthetic CelebA partition.
In the second partition (CelebA S), there are 650 blond male images, which achieve a similar bias-conﬂicting test set size as prior works Sagawa et al.
Liu et al.
The 650 images are distributed to 3 clients with 2350 other images.
The rest of the images are distributed in the same way as the ﬁrst partition.
Tables 3, 4, and 5 provide more details about the 3 clients.
[Table]
[Table]
[Table]
C. H YPER - PARAMETERS
We use Adam optimizer (Kingma & Ba, 2015) through our experiments with learning rate 1e-4.
Although stochastic gradient descent (SGD) optimizer is more common in vision-related tasks, we ﬁnd that the Adam optimizer always leads to lower accuracy disparity.
We train the global model for 500 rounds.
5 clients are selected per round, and each performs 5 epochs of local updates.
We tune the coefﬁcients of the adversarial transferability and L 2 regularization terms from { 0 . 01 , 0 . 1 , 1 . 0 , 10 . 0 } and select the largest value that does not decrease the validation accuracy during penalization.
We start the attack budget at 0 .
031 and gradually decrease it such that 30% − 50% of the attack succeeds.
A large budget will make the attack too strong and push the adversarial examples far across the decision boundary, making the regularization method less effective.
We conﬁgure (cid:15) to 0.031/0.01/0.015 for
MNIST/CelebA/Coil20, respectively.
We ﬁne-tune the global model for 5 epochs on MNIST and
10 epochs on CelebA/Coil20, which are sufﬁcient for the personalized models to converge.
The clients with the most data samples ﬁne-tune the penalized models for a total of 80/40/30 batches on
MNIST/CelebA/Coil20 datasets.
Note that we may not select the personalized model with the most ﬁne-tuning batches for reporting.
In the just train twice (JTT) method, we up-sample the residuel by a factor of 50.
In Ditto, we tune its λ from { 0 . 1 , 1 . 0 } .
C. N EURAL N ETWORK A RCHITECTURE
We use CNN 28x28 for MNIST dataset and CNN 64x64 for CelebA and Coil20 datasets.
[Table]
D M ORE E XPERIMENTAL R ESULTS D. F IRST - ORDER A PPROXIMATION OF A DVERSARIAL T RANSFERABILITY L OSS
To explore the impact of the λ term in Lemma 2 and Theorem 3, we measure the relative difference between (cid:96) trans ( f g , f p , x , y ) and (cid:15) · (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107) · (1 − 1 γ · cos θ ) .
In other words, we measured the accuracy of a ﬁrst-order approximation of (cid:96) trans ( f g , f p , x , y ) .
If the approximation is accuracy is high, it implies that the impact of λ · (cid:15) 2 is low. Speciﬁcally, we compute an approximation error: E ( x ,y [Table]
On MNIST, CelebA and Coil20 datasets, we ﬁnd that the approximation error is 0.019, 0.058, 0.084, respectively.
These results suggests that using (cid:15) · (cid:107)∇ x (cid:96) ( f g ( x ) , y ) (cid:107) · (1 − 1 γ · cos θ ) to approximate (cid:96) trans ( f g , f p , x , y ) results in a decent accuracy and the impact of λ · (cid:15) 2 is low.
The possible reason is that the attack budget (cid:15) is usually small (e.g., 0.031), such that the gradient of a function changes little in a small neighborhood deﬁned by (cid:15) .
D. E FFECTIVENESS OF L 2 R EGULARIZATION T ERM
Figure 7 shows the distribution of γ before and after applying the L 2 regularization term on the
CelebA dataset.
Here, the γ is computed once per data sample.
We keep the global model ﬁxed and ﬁne-tune the personalized model for 1 epoch.
D. D IVERSITY OF B IAS - CONFLICTING S AMPLES I MPACTS D EBIASING
To explore the impact of the diversity of bias-conﬂicting samples on debiasing, we vary the diversity of bias-conﬂicting samples and adjust the up-weighting factors accordingly.
Speciﬁcally, we sample a factor of 0.02, 0.025, 0.033, 0.05, and 0.1 biased data samples from the MNIST dataset and re-color them to become bias-conﬂicting.
The factor in the sampling step is called sampling factor.
Then, we up-weight the bias-conﬂicting samples by a factor of 50, 40, 30, 20, 10, respectively, keeping the total number of bias-conﬂicting samples consistent.
Here, the bias-conﬂicting samples have less diversity if generated by a small number of biased data samples with a large up-weighting factor.
Experimental results in Figure 8 show that, as the diversity reduces, the accuracy disparity of personalized model on the biased dataset and bias-conﬂicting dataset increases, supporting our analysis.
Therefore, our method is applicable in the scarcity of bias-conﬂicting samples while the up-weighting method fails.