Robust and Personalized Federated Learning with Spurious Features: an Adversarial Approach
Authors: Xiaoyang Wang
Han Zhao
Klara Nahrstedt
Oluwasanmi O Koyejo
Venue: ICLR 2022 Submitted (2022)
Keywords: federated learning
personalization
spurious features

Abstract: A common approach for personalized federated learning is fine-tuning the global machine learning model to each local client. While this addresses some issues of statistical heterogeneity, we find that such personalization methods are often vulnerable to spurious features, leading to bias and diminished generalization performance. However, debiasing the personalized models under spurious features is difficult. To this end, we propose a strategy to mitigate the effect of spurious features based on our observation that the global model in the federated learning step has a low accuracy disparity due to statistical heterogeneity. Then, we estimate and mitigate the accuracy disparity of personalized models using the global model and adversarial transferability in the personalization step. Empirical results on MNIST, CelebA, and Coil20 datasets show that our method reduces the accuracy disparity of the personalized model on the bias-conflicting data samples from 15.12% to 2.15%, compared to existing personalization approaches, while preserving the benefit of enhanced average accuracy from fine-tuning.

Under review as a conference paper at ICLR 2022
ROBUST AND PERSONALIZED FEDERATED LEARNING WITH SPURIOUS FEATURES: AN ADVERSARIAL AP- PROACH
Anonymous authors Paper under double-blind review
ABSTRACT
A common approach for personalized federated learning is fine-tuning the global machine learning model to each local client. While this addresses some issues find that such personalization methods are often vulnerable to spurious features, leading to bias and diminished generalization performance. However, debiasing the personalized models under spurious features is difficult. To this end, we propose a strategy to mitigate the effect of spurious fea- tures based on our observation that the global model in the federated learning step has low accuracy disparity due to statistical heterogeneity. Then, we estimate and mitigate the accuracy disparity of personalized models using the global model and adversarial transferability in the personalization step. We theoretically establish the connection between the adversarial transferability and the accuracy disparity between the global and personalized models. Empirical results on MNIST, CelebA, and Coil20 datasets show that our method reduces the accuracy disparity the personalized model on the bias- chas-connii data samples from 15.12% to 2. 251,, compared to existing personalization approaches, while preserving the benefit enhanced average accuracy from fine-tuning.
of statistical heterogeneity,
INTRODUCTION
Federated learning (FL. is leading framework for clients to collaboratively train shared global
machine learning (ML) model without releasing their local private datasets (McMahan et al. 2017
Kairouz et al., 2019). The jointly trained global model could be further fine-tuned each client's
local dataset to produce personalized models (Fallah et al. 2020 T. Dinh et al. 2020 Li et al. 2021).
While existing theoretical and empirical results highlight how personalized models improve accuracy
on local data, few works consider what features the personalized models learn from the local dataset. Our motivating hypothesis that not all local features beneficial.
For example, consider a gender prediction task using face images, where the ML model learns to predict gender based on hair color because females are more likely have blond hair (Sagawa et al.) 2020). In this case, the hair color is called spurious feature because it only statistically correlates with the gender on biased-aligned samples but not necessarily accuracy of a model that relies spurious features such as hair color is likely to drop significantly on bias-conflicting samples where the spurious correlation does not hold, e.g., for blond male images (Sagawa et al.) 2020). This paper calls the accuracy difference of an MI model on the dataset with spurious features and the dataset without spurious features accuracy disparity. More broadly, the accuracy disparity caused by spurious features leads to issues in both fairness (McNamara et al. Zhao & Gordon, 2019 Agarwal et al. 2019 Chi et al. 2022I) ). i.c., racial bias (Khani Liang and robustness, i.e., accuracy decrease under distribution shift (Zhao al. 2019 Koh et al. Compared to the global model, because of the local fine-tuning, the personalized models are more vulnerable spurious features and have larger accuracy disparity.
the overall population. Thus, the
2019
2021
2021
Empirically, we observe that the typical non-i.i.d. data distributions in FL settings reduce the accuracy disparity of the global model. One potential explanation is that the statistical heterogeneity (Wang

---

Under review as a conference paper at ICLR 2022
Per User Gradient
Aggregated Gradient
HAT soler and gesder
Brows her Perede
Blad har- Female
Honthea Pensle
Towerds Spurious Features
Towards Features
Global Mocel
Personal Model
(b) Gradient Divergence between Users
(a) Spurious Correlation Varies across Users
Figure 1: The statistical heterogeneity of spurious features leads to diverse gradients. The statistical heterogeneity leads to global ML model with low accuracy disparity federated learning setting because the aggregation step in the central server averages out the gradients resulted from local spurious features.
jet al. 2021) under spurious features across users is larger than that of non-spurious features. As result, the aggregation operation in the central server averages out the diverse shifts under the spurious features (Figure T so that the global model becomes more robust against spurious features on the benchmark datasets. On the other hand, the local fine-tuning step will result in a personalized model that entangles spurious features and becomes biased. Hence, it remains a key challenge how to debias the personalized models.
Various methods (Wang et al. 2019 Sagawa et al. 2020 Liu et al. 2021) have been developed to disentangle spurious features from ML models, but few be easily applied personalized models under the federated learning setting. The main reason lies in their reliance on bias-conflicting samples. The bias-confficting samples
be rare. For example, in the CelebA dataset, only around 1.7k samples are bias- coaasscnnii out of more than samples. If we distribute the CelebA dataset across users, nearly half do not have any bias-confiicting samples. Prior work (Li Wang 2019) has tried using a global proxy dataset for training FL models. However, global proxy dataset may not characterize the local samples well. Furthermore, estimating the accuracy disparity of the personalized model becomes difficult without access to bias-conflicting samples.
To approach this problem, we propose novel method to reduce the accuracy disparity of personalized models. Our proposed method does not rely on the bias-conficting samples, e.g., blond male images in the aforementioned task of gender prediction, which are often rare and may not be available for every user. Instead, inspired by prior works (Tramèr et al. 2017: Liang et al. 202 l) the transfcrability of adversarial examples, we use the global model as a reference and the adversarial transferability between the global model and the personalized models accuracy disparity of the personalized models. The intuition is that if two ML models use disjoint subsets of features, the adversarial examples that one ML model generates do not transfer to the other ML model. Based on this intuition, we propose the following hypothesis:
proxy to estimate the
If the personalized models entangle spurious features (thus increasing the accuracy disparity), the adversarial examples generated by the global model (that uses non-spurious features) do not transfer to the personalized models.
Empirically, we show that the adversarial transferability between the global and personalized models strongly correlates with the accuracy disparity between the global and personalized models (Section 4.. validating our hypothesis. We further thcoretically connect the adversarial transferability to the accuracy disparity (Section 5). Based on the empirical observations and theoretical results, develop a method that enforces adversarial transferability between the global and the personalized models to reduce the accuracy disparity of personalized models (Section C). Our contributions summarized as follows:
We empirically evaluate the accuracy disparity of the global and personalized models in fed- erated learning setting with spurious features, highlighting a risk of existing personalization methods.
We design method to estimate the accuracy disparity of the personalized models, based on the low accuracy disparity global model and the adversarial transferability between the global and personalized models.
2

---

Under review as a conference paper at ICLR 2022
We theoretically connect the adversarial transferability and the accuracy disparity of the global and personalized models.
We develop methods to reduce the accuracy disparity of personalized models by enforcing the adversarial transferability between the global and personalized models.
Empirically, we conduct extensive experiments to validate the effectiveness of the proposed methods in mitigating accuracy disparity under the FL. setting. Our experiments MNIST (Deng) 2012),
CelebA (Liu et al. 2013), and Coi120 (Nene et al. 1996) datasets show that the proposed approach reduces the accuracy disparity of personalized models from 15. 15.12% 2.15%, which is closer to that of the global model (-0.63%). Our method also preserves the benefit of the enhanced average accuracy from fine-tuning, resulting in 3.43% accuracy improvement on the biased test set and 0.85% accuracy improvement on the biased-confe biased-confficting test set.
2
RELATED WORK
Personalized Federated Learning Fine-tuning is typical for personalization methods. The meta- Seurning-based method first trains global model and fine-tunes the global model locally (Fallah
et al. 2020). Other methods using multi-task learning (Li et al. (02T) or Moreau envelopes (T. Dinh et al. 2020 have an interpretation as fine-tuning the local model along with training the global model.
Fine-tuning is also compatible with clustering- based method (Ghosh et al. 2020).
Debiasing ML Models A few prior works (Li & Vasconcelos 2019 Sagawa et al., 2020) utilize group labels, which might require human annotation, to debias ML models. For example, the group distributional robust optimization (DRO) method (Sagawa al. 2020) aims optimize the worst-case error rate of ML models across different (often manually annotated) groups. Some groups contain caas-confii samples while others do not. Residual learning- based methods (He) et al. 2019: Nam et al. 2020 Liu et al.. (021) train a biased ML model and up-weight the residual, which mainly contains bias-conflicting samples that the biased ML model mis-predicts. Chi et al. (202T) aims to mitigate the accuracy disparity in regression problems via learning the appropriate representations. However, all these methods rely on the explicit access to the bias-confticting samples, making them difficult to apply on personalized federated learning, where bias-confliting samples may not be accessible for every client.
PRELIMINARIES
Definitions and Notation A data sample is vector æ [ar,, 2s]. where H, corresponds to the robust and non-spurious features and x. are the spurious features. d, is the dimension of spurious features. Let y be label, and define f y x y R to be X-smooth, twice differentiable loss function and L(f, D) = ((((()) v)] to be the empirical risk. Wy and wP are the weights for the global model fa X - y and personalized model fp : X' + V1 respectively. 7 the ratio between the gradient norms of the global and personalized models, [V [[ Let supp(D) be the support of distribution D. We define global data distribution Dg biased local data distribution as the
D,,, and assume a bias- coafffinfi local data distribution Pbc- We define the difference between the updated local model and the global model from the previous round (sometimes we will use the term "gradient" when it is clear from context). (:-r) denotes an inner product of two vectors and
denotes a concatenation of two vectors. is the angle between Vz (x()), y) and Vx ( y). ) Og is the angle between V=l(fg()) y) and V ((f(()) v) ~ O, which measures the entanglement of the global model spurious features.
Training and Personalization Methods We train the global model using the federated averaging algorithm (McMahan et al. 2017). which learns a model f X + y that minimizes: L(f, D2) EN I/1pyl Dx (1f( v). where N is the number of clients, Do, is the biased local dataset for client i, and D, , is the global dataset. When the global model f, converges, fg will be sent to local clients for further fine-tuning by minimizing Ezy.D,, e( {((x))))) v))
A Adversarial Examples and Transferability We can generate an adversarial example Eade given data sample with label y by solving:
T addo
arg max ((fx'),)) 2
(1)
Se
3

---

Under review as a conference paper at ICLR 2022
0
/
2
5
6 7 8
9
(c) Coil20
(a) MNIST
(b) CelebA
Figure 2: Datasets with spurious features. The object color spuriously correlates with the label in MNIST (a) and Coil20 (c) datasets. The hair color spuriously correlates with gender in the CelebA
dataset (b).
1.0
3.988
1.0
0.975
0.930
D
0.904
0.9
0.643
0LEA3
0.9
Accuracy
0.0
0.8
0.7
CLEAT
0.7
0.6
Accuraccy Blased Calaset
0.6
Accuracy on Bissed Datasar: Accuracy Bies Conllecing Dulasel
Accuracy on bes-Conflictng
Accuracy on Dies Conllicing Detasel
0.5
Cortralized
Federated
Centralized rederzted
0.5
Centralied
Federated
Training Setting
Training Setting
Training Setting
(a) MNIST
(b) CelebA
(c) Coil20
The accuracy of ML models Biased Dataset and Bias-Confficting Dataset under centralized and federated training settings. In the centralized setting, an ML model is trained by single dataset that contains all the samples with fixed spurious correlation. The global models the federated setting achieve smaller accuracy disparities between biased and bias-conflicting datasets.
Figure 3:
where f is the victim ML model, and is the attack budget. In this example, we consider the L2 attacks, but extensions to general Lp attacks straightforward. We say adversarial example to be transferable it also fools another ML model (e.g., a personalized model) other than the original victim model f (e.g., the global model).
AN EMPIRICAL STUDY WITH SPURIOUS FEATURES
To gain some insights into the problem, we first perform an empirical study on the accuracy disparity of the global and personalized models in an FL setting. In this study, the personalization method is fine-tuning. Our results highlight the risk of existing fine-tuning-baser personalization methods and the difficulty of mitigating the risk. We also highlight the correlation between the adversarial transferability and the accuracy disparity between the global and personalized models. We provide additional theoretical analysis in Section 5 support the observed correlation. The spurious features in the empirical study are follows.
Spurious Features We consider color as the spurious feature for the MNIST, CelebA, and Coil20 datasets. In the MNIST and Coil20 datasets, we manually color the objects according to their labels to create spurious correlations, Figure 2 shows. The spurious correlations vary across clients for the MNIST and Coil20 data (e.g.. the red color correlates with label zero on the first client and with label one the second client) to create additional statistical heterogeneity. In the CelebA dataset, the hair color attribute correlates with the gender label. We assign disjoint subsets celebrities to different users, which naturally increases statistical heterogeneity for the spurious correlation.
4.1 STATISTICAL HETEROGENEITY REDUCES ACCURACY DISPARITY
Figure 3 shows the accuracy disparity of ML models biased and bias-conflicting test sets. Com- pared to the models trained in the centralized setting, where the spurious correlations are fixed, the accuracy disparity of models trained in the federated setting decreased significantly. These empirical results suggest that the global model FL is more robust to spurious features the spurious features non-i.i.d. across clients.
To explain this observation, consider the relationship between the gradient directions and the learned features. For the spurious features, its correlation with the label may change across clients. As
4

---

Under review as a conference paper at ICLR 2022
1.0
1.00
Accuracy
0.9
Act B
Accuracy
0.9
0.95
0.8
Acc
0.8
Acc BC
0.85
0.90
Acc
Acc_p BC
8
12 16
Batch
Number of Batch
Number of Batch
(a) MNIST
(b) CelebA
(c) Coil20
Figure 4: The accuracy of personalized model on Biased Dataset (Acc_B) and Bias-Conflic Conflictinn Dataset AAc_BCr with increasing fine-tuning batches. The personalized models entangle spurious
features and increase accuracy disparities between biased and conffcctfi datasets.
1.00
Accuracy Difference
Accuracy Difference
Accuracy Difference
0,050
0.2
0.75
0.025
0.50
Acc_ B- Acc_ BC
0.000
Acc Adv
0.1
0.25
Acc B Acc BC
-0.025
Acc, B Acc BC
Acc_Adv
-0.050
Acc Adv
0.00
0.0
12
0
2
0
4
Number of Batch
Number of Batch
Number of Batch
(a) MNIST
(b) CelebA
(c) Coil20
Figure 5: The accuracy disparity of personalized models biased dataset and bias-conflicting dataset (Ace_B Acc_BC) and their accuracy on adversarial examples (Acc_Adv), As the personalized models entangle spurious features and increase the accuracy disparity, the accuracy of the personalized models on adversarial examples increases, which indicates the adversarial transferability between the global and personalized models decreases.
an example, in some users' local datasets, the blond hair does not correlate with the gender label because the dataset does not contain any blond female image or the dataset has blond male images, as visualized in Figure la Therefore, the gradient directions for the spurious features are diverse across clients, as shown in Figure [b] The divergence between the gradient directions, as consequence, makes learning spurious features difficult. In contrast, the non-spurious features, e.g., shape features, are more consistent across clients, leading more consistent gradient direction.
4.2 PERSONALIZATION MAY EXACERBATE ACCURACY DISPARITY
Although the global model in the federated setting has lower accuracy disparity than in the centralized setting, the advantage could vanish during the personalization step.
The Personalized Model Entangles Spurious Features As can be observed in Figure 4 the accuracy first increases and decreases on the MNIST bias-conflictnng test set and slowly decreases on Coil20. These two observations indicate that the personalized model entangles spurious features and exacerbates the accuracy disparity in a few batches. Although in principle, one may resort to early stopping, this is not feasible when the bias-conflicting dataset is unavailable scarce.
4.3 ADVERSARIAL TRANSFERABILITY INDICATES ACCURACY DISPARITY
Because the bias-conflicting test set is often unavailable, it is infeasible to directly measure the accuracy disparity across personalized models. To this end, in this section, we focus on methods that implicitly measure the accuracy disparity and determine whether the personalized models entangle spurious features. Following our hypothesis in Section []] we consider using the adversarial transferability between the global and personalized models as proxy for the accuracy disparity measurement. Figure FigureB]poos the accuracy disparity and the adversarial transferability during fine- tuning. As the accuracy disparity of the personalized models increases and drifts away from that of the global model, the adversarial transferability between the global and personalized models decreases. This result empirically validates hypothesis.
5

---

Under review as a conference paper at ICLR 2022
5 THEORETICAL INSIGHTS
This section presents our theoretical result that supports our hypothesis in Section [ and the experi- mental results in Section 4.3 Our theoretical result applies the loss, which has similar behavior to the accuracy as is shown in the empirical results in Section 7 Before we proceed, some additional definitions and notations are needed for the presentation, and we provide table summarizing all the notations used in Appendix to ease the reading. Then, we connect both the loss disparity and the adversarial transferability the angle between the gradients of the global and personalized models. In what follows, we shall show an upper bound of the loss disparity of personalized model, which consists the adversarial transferability between the global and personalized models and an indicator of the entanglement of the global model to spurious features.
5.1 MORE DEFIITTION AND NOTATION
We define natural perturbation A to model the distribution shift between the bias-conflicting Die and biased D.. A could change a bias- alained sample a corresponding blas-conflicting sample. The distribution D Daj= of the natural perturbation A conditions on the data sample Formally, for any
Rd have: Pra-Dsa
(x)
=E ER' AER 1 (m=='+a PI (x') PrA-DAlar (A).
could draw natural perturbation A from Dair that change the hair color in æ to blond. That is saying, æ A is a blond male image. Iteratively drawing data samples from the biased dataset and applying the sampled natural perturbations the data samples result in a dataset with bias-conflicting samples.
Running Example For non-blond male image
Another perturbation consider is the adversarial perturbation oj. Badv that is generated using ff with budget e. Plugging the definition of ,,, into Eq. (I), we have o = arg max d|| {(f(x y). Since the budget is small, we could approximate the loss function & using the first-order gradient: o, arg sa v= y) T
(()()( (Miyato et al. 2018
Liang
al. 202T With the adversarial perturbation, we define the adversarial transterability loss:
ttaan (Sg. fp. x, = (e(f(( fg,e). - {(fg(z).))) (eUp((x v) ((fp(())))))))))))())
which indicates the effectiveness of the adversarial perturbation generated using the global model applied to the personalized models.
5.2 LOSs DISPARITY AND ADVERSARIAL TRANSFERABILITY
With the definitions of natural and adversarial perturbations, this section shows that both the loss disparity and the adversarial transferability connect an angle 0. Next, we outline the assumption: Assumption 1. The distribution shift does not exacerbate the entanglement of model to spurious features Fs which is measured by
V [.f((()) y):
E (. If (Va, Vxl((f( A),9) 1)da] (Ve v), a=0
Under Assumption 1, the following Lemmas hold.
Lemma 1. Under Assumption 1, let A be the natural perturbation, 0 be the angle between [ (m)) y) and V=l(fp())) u). 0g be the angle between (=((f(())) y) and Va,, (Jg())))))) y)
(F we have:
O, and be [V (x))))))
C(fp, Dbc) L(fp Db) B(a.))D..A-aa.
a.. (Va, E(fp((((x a A). 1)da]
aa=
Vd
B(a.g)-D,
(Ve((f(()) w)1l (sind, + sinO]]
Lemma CaanmaToooa connects the loss disparity to 0. The 0,. differing from 8, is an indicator of the entanglement of the global model to spurious features and is constant during the personalization step.

---

Under review as a conference paper at ICLR 2022
Lemma 2. Let be the attack budget, 0 be the angle between Val(fg(æ), y) and V Va((fp(z),)) y) be (., and the loss function & y y + R be X-smooth, twice differentiable, we have I1 (m).)il
1 cos0) x x S brrons (Sar S 2p, 22
II (I))) -11-
(3)
c (a((ff( w)ll cost) + x
Lemma connects the adversarial transferability loss 0. In the following analysis, we connect the loss disparity to adversarial transferability via 0.
5.3 A GENERALIZATION UPPER BOUND
We now present an upper bound of the disparity L(fp Dbc) f.. Do). The main idea is derive an upper bound of [V (x)), ())l,))] sinO in Eq. ② from Eq. (3)
Theorem 3. Let Timi be the minimum of 7, with Lemmas 1-2, we have:
[(Sp: [(pp: Do) < Vd,, ((Sinl, - V2
1) ( ( v)ll
Yinin
1 + E(..y) (trans (so fp. x y)] - x-c)
Theorem suggests (1) debiasing the global model fg. whose entanglement to spurious features is measured by gg. and (2) enforcing the adversarial transferability between fg and f, help reducing the loss disparity of personalized models. For the constants "/in and we further explore their impacts in the following section and Appendix D.I. respectively.
6 METHODS
With the empirical and theoretical results in Sections 4.3 and 5 respectively, it is natural to ask if enforcing the adversarial transferability in the personalization step reduces the accuracy disparity. In this section, we first introduce adversarial examples to the personalization step a regularization term added to the original loss function, aiming to enforce the adversarial transferability. However, the accuracy disparity still increases, albeit much slower, even if the adversarial transferability remains high. One possible reason that the personalized model increases its gradient norm, which helps preserve the adversarial transferability but does not prevent the personalized model from entangling spurious features. To this end, we add an L2 regularization term to the loss function, aligning the gradient norms of the global and personalized models. Combing these two methods addresses the accuracy disparity. Both methods are relatively light-weight from a computational perspective.
6.1 ENFORCING ADVERSARIAL TRANSFERABILITY
We enforce that global and personalized models make consistent predictions on adversarial examples, such that adversarial examples transfer from one to the other.
Generating Adversarial Examples The projected gradient descent (PGD) attack (Madry et al. 2018) is an effective attack method that uses the neural network's first-order gradient, and easy to compute. Additionally, The attack solves Fade arg max -== -1 ((f((() (==),, y) iteratively. At itera-
tion / + 1, the adversarial example is: - = Proji =lIse (x + a sign( (V sta (
where Proj is a projection operator.
Enforcing Consistent Predictions
Both the global model fa and the personalized model fp take the adversarial example Lade as input and output 2, and z, from their last layers, respectively. We enforce the adversarial transferability by adding the following regularization term, which maximizes the cross-entropy between Zg and Zp- Since the global model fg is fixed as a reference in the personalization step and its low accuracy disparity is desirable, we use Zy as the ground-truth:
K
Radi (Zg. xp) = Dizgi log(2p;) + (1 - 2g,) log(1 -2pp]l
7

---

Under review as a conference paper at ICLR 2022
where K is the number of classes. The local model has access to the global model, so there is no additional communication overhead for implementing this regularization. The adversarial examples are computed using the global model once for all. The computation only needs a few back-propagations, much less than training the global model.
6.2 ALIGNING GRADIENT NORMS
In Eq. (3). we have seen that the adversarial transferability loss depends not only on the angle 8, which connects the transferability to the disparity but also on 7 = [V,,([[(()),. [V=()>(0).)]] A small
indicates that the personalized model increases its gradient norm. Then, the personalized model could entangle the spurious features and increase without decreasing the adversarial transfcrability.
To prevent 7 from decreasing, we employ simple and effective strategy by adding an Lg regu- larization term Rt,, = ||w, - wpll² to the loss function. The motivation behind the L2 term is straightforward: if two models have similar weights, they have similar gradients. Empirical results in Appendix [D.2 show the effectiveness of the L2 term in controlling y. Although prior works (Li et al. 2020: T. Dinh et al. 2020 Li et al. 202I) have explored similar regularization methods, we develop the regularization term from a different perspective.
7
EXPERIMENTS
This section presents our experimental results, demonstrating that our method reduces the accuracy disparity. We also show that the benefit of enhanced average accuracy from fine-tuning is preserved.
7.1 SETTING
Data Partition We distribute the MNIST and Coil20 dataset across 50 clients where each client have 5 different classes. The local dataset on each client is further partitioned to vrain/vaiiaaiiontesst set with ratio of 72:8:20, following prior work (Li et al. 2021). We make two data partitions for CelebA: CelebA.R using a real partition and CelebA_S using a synthetic partition, both have 508 clients. In both CelebA partitions, each client represent a disjoint set of celebrity (Li ct al. 202T).
Due to the limited space, we further detail the data partition in Appendix C.T report the hyper- parameters in Appendix 2 and list the neural network architecture in Appendix C.3
7.2 THE EFFECTIVENESS OF PROPOSED METHODS
We conduct an ablation study on the MNIST dataset. Figures 6b and 6f demonstrate the effectiveness of enforcing adversarial transferability. However, the accuracy and loss disparity still increase during personalization, which is potentially caused by the gradient norm issue (Section 6.2). Aligning the gradient norms by applying the L2 regularization term while enforeing adversarial transferability address the accuracy and loss disparity as Figures 6d and 6h show, respectively. Figure (6c) and 6g further show that applying the L2 regularization term alone does not address the accuracy or loss disparity. Compared to naive fine-tuning, which is reported in Figures 6a and 6e method mitigates the accuracy and loss disparities by ~ 50%.
7.3 ANALYSIS
We compare our method to no personalization (Global), naive fine-tuning (FT), Ditto (Li al. 2021). up-weighting (UW) (Sagawa et al.) (020), and just train twice (JTT) (Liu et al. 2021). The up-weighting method is implemented via sampling bias-aligned and bias-conflicting samples with equal probability (Sagawa et al. 2020). Up- neightihgin and JTT are not applicable to Coil20 due to the lack of bias-conflicting samples. We use the local finetuning version of the Ditto solver because the local finetuning solver performs fewer local updates than that of the joint optimization solver and therefore entangles spurious features less. Each experiment is repeated 9 times with 3 random seeds for the federated learning step and 3 for the personalization step. We select models using the validation accuracy minus the decrease of adversarial transferability and using the validation accuracy for other baseline and competitor methods.
Tables T shows the main result. Our method reduces the accuracy disparity of personalized models from 15.1 15.12% 2. 2.15, compared other personalization methods. Our method also preserves the
8

---

by up Under review as a conference paper at ICLR 2022
Arcurnyy bifference
1.0
1.0
Acc_
Act_B B
00.5
0.8
Acc, Adv
0.5
0.0
Acc Adv
0.6
0.6
0.6
Acc_B B Acc BC Acc Adv
Accuray
0.6
0.4
0.4
0.7
0.2
0.2
02
0.0
0.0
0.0
0.0
12 Number Batch
16
12
16
Number of Batch
Number Batch
Number of Batch
(a) Naive Fine-tuning
(b) Enforcing Adversarial Transferability
(c) Aligning Norms
Gradient (d) Applying Both Meth- ods
Loss Differenca
1.5
Loss Locs
MAX Loss, No Loz
1.5
L00s BC Lasz
MAX Loss Ads Loss Adv
MAX Loss, NV Loss Mdx
Locs E
2.0
1.
1.0
0.3
Loss, SC Loss, B
0.5
0.5
MAX Lass, Adv Loss Adv
0.0
0.0
0.0
12 Number Batch
16
12
Number of Batch
Number Batcn
Number of Batch
(e) Naive Fine-tuning
(1) Enforcing Adversarial (g) Aligning Transferability
Gradient (h) Applying Both Meth- ods
Norms
Figure 6: The accuracy disparity (Ace B Acc BC) and adversarial transferability accuracy (Ace_Ady) and the loss disparity (Loss_BC Loss_B) and adversarial transferability loss (MAX-Loss Adv Ldv)_Adv) with different methods. Combining the proposed methods ad- dresses the accuracy and loss disparities. Acc BC/Loss BC and Acc B/Los. B are the accuracy/losses on the bias-conflicting and biased test sets, respectively. Acc_Ao//Loss .Avv the accuracy/loss on adversarial examples. Adv and MAX Loss_Ads Loss Ldss which measures the decrease of Loss Adv, indicate the decrease of adversarial transferability.
Table Accuracy of personalized Models on Biased Test Set (Acc.B) and Bias- Confficnni Test Set (Ace B). Our proposed method achieves the lowest accuracy disparity (2. 15%) compared to other personalization methods (15.1 155.1.5%/15538 538%) and 3.43% accuracy improvement on the biased test set and 0.85% improvement on the biased-conflicting test set compared to the global model.
MNIST
CelebA S
CelebA R Acc BC
Coil20
Method
Acc
Ace BC
Acc B
Acc BC
Ace
Acc B
Ace BC
Global
.852 a 2e4 .847 :: 6e-4
.930 :: ses .910 3e4
.909 i 6e .929 = 5e5
.882 = .6 .903 :: 7e4
FT
.989 = 6e-7 .704
+ 3o4
.952 = .786
= le-3 .963 .849
.931 = .891 = 2--4
Ditto
.982 = 36- .724
= te-3
.948 s-s .715 + se4
.966 + les
.884 2o-4
.939 = 4-5 .897 + se4
UW
-968 = 2 .823
= 7e-4
.930 = 7e6 .889 1:54 Se-
= 4e-4 .956 2-3 .836
.936 le5 .895
N/A
N/A
JTT
985 2e- .707
6e-5
.952 2-6 887 + 3e-4
le-3 .925 26-5 .927
N/A
N/A
Ours
.951 = 2655 .870
= Be-4
.932 = 3e-5 .910
= 86-5
.901 3o-4
.916 = 5c-8
enhanced average accuracy from fine-tuning, resulting in 3.43% accuracy improvement the biased test set and 0.85% improvement on the biased-conflicting test set compared to the global model. In contrast, the naive fine-tuning method sacrifices the accuracy on the bias-conflicting to 14.3% and increase the accuracy disparity by 15. 12%. We also find that our methods outperform the supervised up-weighting method and the unsupervised JTT method. which increase the average accuracy disparity to 7.56% and 17.76% respectively. One possible reason is that the diversity of the up- up-weighted bias-conflicing samples are small. Therefore, the neural network could memorize them instead of discarding spurious features. Appendix D.3 further shows empirical results that support our analysis.
8 CONCLUSION
In this work, we show the risk of prior federated learning personalization methods with spurious features, which lead to high accuracy disparity between the global and local models. Then, we develop strategy by enforcing the adversarial transferability between the global and personalized models to reduce the accuracy disparity. Both empirical and theoretical results show that our strategy is effective.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
Our method mitigates the issue of spurious features, which lead to bias towards minority groups, in personalized federated learning. However, completely disentangling spurious features remains challenging and is an issue for many federated learning methods.
10 REPRODUCIBILITY STATEMENT
Our implementation, including data partition scripts, is based on the FedML library (e et al. 2020).
which is open-sourced. The proofs of the theoretical results are in Appendix B The datasets in
experiments are publicly available. We detail the data partition in Appendix C.I report the
hyper-parameter tuning in Appendix and list the neural network architecture in Appendix
REFERENCES
Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. Fair regression: Quantitative definitions and reduction-based algorithms. International Conference on Machine Learning, pp. 120 129. PMLR, 2019.
Jianfeng Chi, Yuan Tian, Geoffrey Gordon, and Han Zhao. Understanding and mitigating accuracy disparity in regression. In International Conference on Machine Learning (ICML), 2021.
Li Deng. The mnist datahase of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6): 141-142, 2012.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33, 2020.
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered federated learning. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 19586-19597. Cur- Associates, Inc., 2020. URL F2le//222c0000001111666660222282
neurips cttssccc/ppe ccpppper/222///
Chaoyang He, Songze Li, Jinhyun So, Mi Zhang, Hongyi Wang, Xiaoyang Wang. Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Li Shen, Peilin Zhao, Yan Kang, Yang Liu, Ramesh Raskar, Qiang Yang. Murali Annavaram, and Salman Avestimehr. Fedml: A research library and benchmark for federated machine learning. arXiv preprint vrXiv:2007.13516 2020.
He He, Sheng Zha, and Haohan Wang. Unleam dataset bias in natural language inference by fitting the residual. arXiv preprint arXiv: 1908. 10763, 2019.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 2019.
Fereshte Khani and Percy Liang. Removing spurious features can hurt accuracy and affect groups disproportionatelyy In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 196-205, 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URLhttp //axxiv. www. abs / 1412 22..21222 6980
Pang Wei Koh, Shiori Sagawa, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, et al. Wilds: A benchmark of in-the-wild distribution shifts. International Conference on Machine Learning, pp. 5637-5664. PMLR, 2021.
10

---

Under review as a conference paper at ICLR 2022
Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv: 2019.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. iopoulos. and V. Sze (eds.), Proceedings of Machine Learning and Systems, volume 2. pp. 429 450, 2020. URL 7642e http:///....gggg//////////////// 3 F10
Dhillon, D. Papail-
.pdf
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, pp. 6357-6368. PMLR, 2021.
Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pullern Recognition, pp. 9572-9581 2019.
Kaizhao Liang, Jacky Y Zhang, Boxin Wang, Zhuolin Yang, Sanmi Koyejo, and Bo Li. Uncovering the connections between adversarial transferability and knowledge transferability. In International Conference on Machine Learning, pp. 6577-6587. PMLR, 2021.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. In International Conference on Learn- ing Representations, 2018. URL /ttpp:://oeerevview wttee//oooonnii=111122e.
Brendan McMMaan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication-effiieeeen learning of deep networks from decentralized data. Artificial Intelli- gence and Statistics, pp. 1273-1282. PMLR, 2017.
Daniel McNamara, Cheng Soon Ong, and Robert C Williamson. Costs and benefits of fair representa- tion learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 263-270, 2019.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 1118)117991199)) 2018.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
Sameer A. Nene, Shree K. Nayar, and Hiroshi Murase. Columbia object image library (coil-20. Technical report, 1996.
Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2003.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https
//openreview.oo net/ forrmm
Canh T. Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. In H. Larochelle. M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21394 24005, Curran Associates, Inc., 2020.
11

---

Under review as a conference paper at ICLR 2022
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. The space of transferable adversarial examples. arXiv preprint arXiv: 3rXiv:/704.03453, 2017.
Haohan Wang, Zexue He, Zachary L. Lipton, and Eric P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
net/ www.ooommnnnner....
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H Brendan McMahan, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, al. field guide federated optimization. arXiv preprint
2021.
Han Zhao and Geoff Gordon. Inherent tradeoffs in learning fair representations. Advances in neural information processing systems, 32: 12215565-15685, 2019.
Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In International Conference on Machine Learning, pp. 7523-7532. PMLR, 2019.
12

---

Under review as a conference paper at ICLR 2022
Appendix
A NOTATION
Table Table of Notation
Symbol
Description
A pair of data sample and label The robust feature and spurious features 2,), respectively
Ty Le
d, dy, ds
The dimension of ær 2.. respectively
Sg
The global model
fp
The personlized local model
/
An adversarial purtubation generated by the global model fy with attack budget e An natural perturbation, which could flip the spurious attribute
A
DD
The global distribution, which the union of local distributions A biased local distribution
Db
Dho
A bias-conflicting local distribution
DaIzy Supp(D) (1)))
The distribution of natural perturbation
The support of distribution 'D
An inner product of two vectors A concatenation of two vectors
B PROOFS
B. PROOF OF LEMMA
Lemma 1. Let A be the natural perturbation, 0 be the angle between Val(f( y) and
(L)(F y), Og be the angle between (=l(f((z),,)) u) and Va, (lg(())))) ~ 0, and y be I. (w) we have: (=)()()))))
jazo C(fp, Doe) C(fp.Do) Do /' (V (xfp(1 a A). 1)da]
Vde
E( [V w)ll -sinn,, sinO)]
Proof. Rewriting C(fp, Doe) and introducing A:
C(fp, Doe) (n.g)~D (()()))))))))))))))())(()))))( v)]
E(E,))DD (2(sp(x + A),
E( Pai
+ ((fp(1
- -
- ((fp(t))
+ (( [2(f((x + A),y) - -((f((x).v)
=((aN)~D, (1) yi]
+ -E [ + A) )), {(f((z).v))
=L(fp, Do) Emy
(Vellfp(z + +a A). 1)da]
a==
C[(pp Db) +
Ve,l(fp(a) a A)))), 1)da
a=0
13

---

Under review as a conference paper at ICLR 2022
Moving L(fp. Db) to the left-hand-side (LHS), we have:
C(fp,Dbe) C(fp: Do) B(a.g)- aa
a A).1),1)da) (4)
According to Assumption further have:
E (e..)) A
(V2,(f A),8). 1 & VV, (5)
a=
|/ {(),(()) f) sinO. The first step is connecting Next, connect (V Nn, -))
<V (x + ,),1),1) to V=, ((f((x),)))
using
inequality:
(Va, (a- A).)
- NVV,, + a A),J), Va, ((p(x + a. A),1)) (1,1)
(6)
Vd. VV (ff(
Then, we connect V. ((p())) 1) to [v((f(())) y) Assuming the global model fg entangles
spurious features and the angle between V (z) y) and V (x) y) ~ 0 is 0g, we have:
VV (f)))))))) - V (z), g)ll sin(0, + 0).
Since is easy to see that E [0, 71 and the gradient sinf is monotonically decreasing in [0, ] have:
+8
s0 + =
Vsinddo
+0
cosOd0
0,
cos0dd + f e
cosod#
sinO, + sinO
Combining Eq. 7 and Eq. 8 we have:
IV ((f((()))))lll IV (ff(((()) (sin0, sinf)
Recalling the defition of = [V)(f(())))) and combining Eq. (4 Eq. (=)))(()).)]]
55
Eq.
(6)
Eq.
complete the proof.
B.2 PROOF OF LEMMA
Lemma 2. Let e be the attack budget, 0 be the angle between V [al(g(()) y) and V Val(s(a))
be [V,(f, (w)).))l and the loss function 8 y y + R X-smooth, twice differentiable, we have
IV (w),))
e
v)
cost) - X x. 2 S lerana Sps F, x.
1
Sc. V (( v0ll
cosf) + x 222
14

---

Under review as a conference paper at ICLR 2022
Val(s(().) Proof. Under the definition of the adversarial perturbation, it easy that ,,,- [[Va)((()))) and ,,, increases the loss by:
( + +z,,,)))))
= =,, =e. =c. =E.11V=((,,(00,) 2
8 S,c V (Z,))
1 /.. V2, ( (1)) y)
where d a linear interpolation between a and x: + 5 f/.ee by the Lagrange's mean-value theorem. Similarly, for transferable adversarial example from fg applies fp. òf could increase the loss of fo by:
{(fp(æ + )fz,e),)) - -S,, V
-
Sanc V (()(t)).1)001.
1 = =c. [ b)|l cost + J0 V2, (z)()))
2
where cos@ V [= Vad
[ (z).)) [[ Plugging the approximations above to the adversarial
transferability loss, we have:
(rr Jp. v. (eUg(z
- - = (2ll(VV((g(()) - ( 9) coso
fe.e (
5 V2 (ff(()))))))
Under the A-smooth assumption on the loss function, the spectral norms of the Hessian metrics are bounded. Therefore, we could bound the norm of the deviate between the quadratic terms (Nesterov 2003 Proof of Theorem 2.1.5) in the adversarial transferability loss:
||6 j, V2 (Zg)))))))))
v)0 f. V2 (fp((p)) s 2A 57 1/,,0/p. = -2 (10)
Since the quadratic terms in Eq.
110 are scalars, we have:
2A 2 Ju.
By (Zg())))))) T% V2 ())))) S 22x
(11)
Plugging Eq.
11 and the definition of 7 lernns (Syz Sps nr v) completes the proof.
B.3 PROOF THEOREM 3
Theorem 3. Let Tmin be the minimum of under Assumptions I and Lemmas 1-2, have:
C(fp, - L(fp Dz) < Vd, ( ((in)), - V2
1) (B.. (, 2 v)ll
Vini
1 E(r.y) lerans (szr Sps 20, w)] xe)
Proof. According to Lemma
we know:
lians (fan x, 2e (V (1 cos@) - A
where cosf = Velllclow///
[V V V.1111111 Then, we derive an upper bound of || (f)
sinO from Lirans (Sgr fp. x, y). is easy see that 0 E [0, z.. Therefore, we have:
15

---

Under review as a conference paper at ICLR 2022
trons 2z.
1 7 2 Val(Sg(() w)1l (1 -cosf) x.
= - [Vll(f(()) f)1l (1- (1-cos8 + - X.
ç [V (1-cos/)
+ 6.(7-11 11
f)ll - x
7
\Vel(f(()) v) sin"
2
0
e. 6-(7-1)
(V) vl) -
2
7
E [V M)1 (sin0 + 2 sin 20
-sin0)
2
(1 [[ 22
[Vel([ sin0 [VV[(((())(()))
w)ll (
sin? sin0)
2
c. 1) ( w)1 x
( sin0 + E , ( V2)
1- Val(f( 1)1l c
7
c. v2)
= [Vel((( U)
sin0
([l 2
?
Moving || V (x)) w| sin0 to the left hand side (LHS):
(( (x)) w)ll sinO
(12)
Lranno (fg. fp. x. (V2-)) +7.
According to Lemma know:
D(Sp, Doc) L(Sp, Dz) B(aww) Do ANDAIE
(V., (fp(ar + (1,8)1)dd
a=
(13)
Vd.
E, Dn [V v) (sinO, sinf)]
Combining Eq.
13 and Eq. 12 and taking expectation of x, y over Dp:
EanDs Vd,
(=((f((z)..)) v)ll (sino, + sinf)]
ViTe
< (law) y) sind,y + B( lerans Sos xr, v)
+ +E(ay) (V2-))
x-
(14)
5 Vd. (1 (sin02 + /2
1) (x,))) 1V x)
7min
+ -e Ltrans x, 8) + +
Plugging Eq. 14) back to Eq.
13 completes the proof.
16

---

Under review as a conference paper at ICLR 2022
C MORE EXPERIMENTAL SETTING
C.I DATA PARTITION
We distribute the MNIST and Coil20 dataset across 50 clients. Each client has data from 5 different classes. The local dataset each client is further partitioned to irain/validation/lest set with a ratio of 72:8:20, following prior work (Li et al. 2021). The test set here biased. We alternate the spurious features in biased test sets by recoloring the data to create bias-conflicting test set. For the CelebA dataset, we consider two partitions. In the first partition (CelebA R), each client represents 20 celebrities. One celebrity only appears sets are copied to bias-conflicting test sets. We use all clients for training the global model. In the personalization step, we select the clients who have more than blond female training samples and more than 5 blond male test samples. We select these clients because they provide enough samples to create spurious correlations and bias-conflicting test sets. Although the first partition on CelebA is real, the number (161) of blond male images is small. To make the result clearer, we create another synthetic CelebA partition. the second partition (CelebA S), there are 650 blond male images, which achieve similar bias-conflicting test set size prior works (Sagaaa et al. (2020); Liu et al. (202T). The 650 images are distributed to 3 clients with 2350 other images. The rest of the images distributed in the same way as the first partition. Tables and provide more details about the
client. The blond male images in the biased test
3 clients.
Table 3: Number of Train and Validation Samples in CelebA S
Client ID
Non-blond Female
Non-Blond Male
Blond Female
Blond Male
0
55
31
12
2
30
68
0
2
2
59
28
11
2
Table 4: Number of Biased Test Samples in CelebA
Client ID
Non-blond Female
Non-Blond Male
Blond Female
Blond Male
0
115
60
45
2
60
75
79
0
2
86
111
14
0
Table 5: Number of Biased- Slased-Confficting Test Samples in CelebA.S
Client ID
Non-blond Female
Non-Blond Male
Blond Female
Blond Male
200
1
0
203
2
204
C.2 HYPER-PARE PARAMETERS
We use Adam optimizer (Kingma & Ba 2015) throughout our experiments with learning rate le-4. Although stochastic gradient descent (SGD) optimizer is more common in vision-related tasks, we find that the Adam optimizer always leads to lower accuracy disparity. We train the global model for 500 rounds. 5 clients are selected per round, and each performs 5 epochs of local updates. We tune the coefficients of the adversarial transferability and L2 regularization terms from (0.01, 0.1, 1.0, 10.03 and select the largest value that does not decrease the validation accuracy during penalization. We start the attack budget at 0.031 and gradually decrease it such that 30% 50% of the attack succeeds. A large budget will make the attack too strong and push the adversarial examples far across the decision boundary, making the regularization method less effective. We configure e to 0.03 1/0.01/00.01 for MNIST/CellcA/CCiil) respectively. We fine-tune the global model for 5 epochs on MNIST and
17

---

Under review as a conference paper at ICLR 2022
10 epochs CelebA/Coil200, which sufficient for the personalized models to converge. The clients with the most data samples fine-tune the penalized models for a total of 80/40/30 batches on MNISTTCelebA/Coii20 datasets. Note that may not select the personalized model with the most fine-tuning batches for reporting. In the just train twice (JTT) method, we up-sample the residuel by factor of 50. In Ditto, tune its À from (0.1,1.0).
C.3 NEURAL NETWORK ARCHITECTURE
We use CNN 28x28 for MNIST dataset and CNN 64x64 for CelebA and Coil20 datasets.
Table 6: Neural Network Architecture
CNN 28x28
CNN 64x64
Input: 32--28-28
Input: R 3--6--44
4.4 conv, 64 BN LReLU, stride 2 4-4 cony, 128 BN IReLU, stride FC 4096 ReLU
4.4 conv, 64 BN LReLU, stride 2 4-4 conv, 64 BN IReLU, stride 2 FC 4096 ReL.U
FC 10
FC 10
D MORE EXPERIMENTAL RESULTS
D.1 FIRST-ORDER APPROXIMATION OF ADVERSARIAL. TRANSFERABILLTY LOss
To explore the impact of the À term in Lemma 2 and Theorem
measure the relative difference cost). In other words, we measured
between frans (f . fpr x, y) and e | [ (σ), w)ll the accuracy of first-order approximation of trans (fg. fp. æ, v). If the approximation is accuracy high, implies that the impact of X is low. Specifically, we compute an approximation error:
E( Lerana(Ja Spr æ. v) - VV
(1 1 cos0)
Etrane fps x; y)
On MNIST, CelebA and Coil20 datasets, we find that the approximation error is 0.019, 0.058, 0.084, respectively. These results suggests that using e & (a)), w) (1 - coso) to approximate lerans (fg. fp. æ, y) results in decent accuracy and the impact of À c² is low. The possible reason that the attack budget is usually small (e.g., 0.031), such that the gradient of a function changes little in small neighborhood defined by
D.2 EFFECTIVENESS OF La REGULARIZATION TERM
Figure 7 shows the distribution of before and after applying the L2 regularization term on the CelebA dataset. Here, the 7 is computed once per data sample. We keep the global model fixed and fine-tune the personalized model for epoch.
2.0
Density
1.
Method L
wa L2
0.5
10a
10a
gamma
Figure 7: With the L2 regularization term, the distribution of y is centered around 1, with small variance. The minimum of closer to
18

---

Under review as a conference paper at ICLR 2022
D.3 DIVERSITY OF BIAS-CONFLICTING SAMPLES IMPACTS DEBIASING
To explore the impact of the diversity of bias-conflicting samples on debiasing, we vary the diversity of bias-conflicting samples and adjust the up-weighting factors accordingly. Specifically, sample a factor of 0.02, 0.025, 0.033, 0.05, and 0.1 biased data samples from the MNIST dataset and re-color them to become binscconf,. The factor the sampling step called sampling factor. Then, we up-weight the bias-con colfficnni samples by factor of 50, 40, 30, 20, 10, respectively, keeping the total number bias-conflicting samples consistent. Here, the bias-conflicting samples have less diversity if generated by small number of biased data samples with large up-weighting factor. Experimental results in Figure 8 show that, as the diversity reduces, the accuracy disparity of personalized model on the biased dataset and bias-conflictnng dataset increases, supporting analysis. Therefore, our method is applicable in the scarcity of bias-conflicting samples while the up- up-weighting method fails.
0.95
Acc
0.90
0.85
Acc BC
0.025
0.075 Sampling Factor
0.050
0.100
Figure 8: The up-wen spiihtitt method is less effective, resulting in large accuracy disparity of personal- ized model on biased dataset and bias-conflicting dataset, if the bias-com chassconfictiin samples is generated by a small number of biased data samples using a small sampling factor and a large up-weighting factor. The up-weighting factor is set to be the reciprocal of the sampling factor.
19


Reviews:
[{"review_id": "teFxg2An8Iz", "reviewer": "Reviewer_CHgD", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n- The idea of using adversarial examples and minimizing entropy loss of global and local models' outputs is normal. The intuitions are straightforward to understand.\n- Authors have spent a decent amount of effort explaining the relationship between the spurious features and adversarial transferability. It is helpful for audiences.\n- The results do show certain improvements over the other baselines.\n\nWeakness:\n- The presentation has space for improvement, please explicitly explain all the critical terms (accuracy disparity, bias-conflicting, etc) at the first occurrence in this paper.\n- Many subjective descriptions exist in the paper, e.g., if you claim the distribution shift of spurious features is a major effect of accuracy disparity, it would be really important that you give theoretical proof and/or empirical support to verify your claim.\n- It seems like the spurious features are handcrafted, and we don't have a clear solution of how to automatically choose the spurious features in real applications. \n- The adversarial examples are generated using a global model, however, the way of generating adversarial examples in FL worth a lot of analysis and description of the details. There exist some papers that discuss the best way of improving adversarial robustness with adversarial training. Here, the same strategy should be applied to compare.\n- The experiments are a bit disappointing. Without a comprehensive comparison, a replication of the results is almost impossible. So many factors in FL can dramatically change the results. The authors didn't provide a fair and reproducible setting for the results. The experiment is incomplete and the results are not convincing. \n\n", "summary_of_the_paper": "The authors have proposed a new FL training strategy to reduce the performance discrepancy between the central model and the client models. The PGD generated adversarial examples are fed into both central model and client models and their outputs are used to minimize the entropy loss. The computation is further simplified using Taylor expansion.", "summary_of_the_review": "The idea is good and novel, however, the presentation is disappointing and the experiments are weak. I recommend rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "qP-fv1VOQn3", "reviewer": "Reviewer_eYjN", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n- The work exposes a possible generalisation issues in personalised federated learning and proposes a novel approach to tackle it\n- The idea is well motivated, paper is generally well written and experiments are provided to substantiate the claims\n\nWeakness\n- Use of some non-standard hyperparams like 0.031 eps budget for MNIST and the batches of 96, 40, 30. Similarly the 5 epochs of local training seem larger than the conventional 1 or 2. Could the authors provide an explanation?\n- Doesn’t include exploration for other modalities like text or large scale setups like FEMNIST \n\nFew open questions\n- Any thoughts on how the method could compare to say the adversarial training objective in combination with personalisation?\n", "summary_of_the_paper": "This work explores the possibility of personalisation methods entangling spurious features that can undermine their generalization in case of federated learning. It proposes to use a combination of a consistency term for adversarial transferability and an L2 regularisation term to help reduce this disparity.  The approach is evaluated on artificial settings with spurious features.  ", "summary_of_the_review": "I think this work tackles an interesting hypothesis that can limit generalization in case of personalised FL. The proposed solution is principled and well motivated with appropriate ablation study. The only drawback would be lack of experimentation on large scale problems which would certainly make this a valuable piece of work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hJpD_niDyxu", "reviewer": "Reviewer_jRKj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n- `The paper identifies a relevant problem.` The paper empirically shows that prior work on personalized federated learning induces high accuracy disparity, which is a relevant problem to the fairness community.\n- `The method is somewhat novel and technically sound.` The paper demonstrates a link between adversarial transferability and accuracy disparity of personalized models. Thus, the paper employs adversarial transferability and weight regularization (which has already been proposed by prior work, as the paper acknowledges) to learn personalized federated models that achieve high accuracy and lower accuracy disparity than prior work. \n\n**Weaknesses**\n\n- `Certain presentational aspects could be improved.` For example, \"bias-conflicting examples\" are mentioned in the introduction but only defined in section 4 (leaving the reader to guess what these \"bias-conflicting examples\" are). Furthermore, the experimental setup for figures 4 and 5 is not provided (e.g., which personalization method was used?). Moreover, in section 6.1 two sentences are repeated (“there are 650 blond…” and “there are 650 blond...”). Finally, there are a few typos (Section 4.1 “models on bias**ed** and bias-conflicting”, section 6.1 “through**out** our experiments”, section 6.1 “residu**a**l”).\n- `The method lacks key motivation.` Adversarial transferability is introduced to reduce the accuracy disparity of the personalized models by forcing them to be vulnerable to the same adversarial examples as the global model. However, the adversarial transferability between the global and local models is just a proxy for the similarity of the two models, similar to, e.g., KL divergence.\n- `Baselines are misrepresented.` In figure 3, the centralized model is trained on a dataset where the spurious correlations are fixed, whereas the federated model is trained on multiple client datasets, each with different spurious correlations. Accordingly, it is unsurprising that the centralized model has a significantly lower accuracy on the bias-conflicting dataset, as it was not exposed to the same data distribution shifts during training as the federated model. I would expect the centralized model to perform on par with the federated model when trained on the same data, invalidating the paper’s claim.\n- `Results are misrepresented.` In section 5, the paper states that low adversarial transferability indicates high accuracy disparity for the personalized models. However, the paper merely shows a correlation between the two metrics (as they both estimate the similarity between the global and personalized models). I would be very surprised if one could not train global and personalized models that achieve “high transferability and high disparity” or “low transferability but high disparity” (as I believe that transferability and accuracy disparity are only weakly correlated). Unfortunately, the paper does not provide empirical evidence to substantiate these claims. (In fact, it provides evidence for the opposite as “accuracy disparity still increases, even if the adversarial transferability remains high”).\n- `The paper makes unsubstantiated claims.` In section 5, the paper claims that “Both methods are relatively light-weight from a computational perspective” but does not provide any further evidence for or analysis of this statement. \n- `Results are unclear.` Figure 7 compares the losses on the biased and bias-conflicting datasets. However, the loss is only an approximation of the accuracy, which is the quantity that we are ultimately interested in. Therefore, it would be more meaningful to compare the accuracies on the different datasets.\n- `Results are insignificant.` Comparing the results for the global model and the proposed method in table 1, the method achieves roughly the same accuracy on the bias-conflicting dataset and only a minor increase in accuracy on the biased dataset (except for the MNIST dataset). Moreover, the prior personalization approaches achieve significantly higher accuracies on the biased datasets. Thus, the method just provides a “little less” personalization, yielding personalized models that are more similar to the global model (with corresponding performance).", "summary_of_the_paper": "The paper considers the problem that personalization methods in federated learning may cause the personalized models to overfit on spurious features, thereby increasing the accuracy disparity compared to the global model. To mitigate this accuracy disparity, the paper investigates adversarial transferability, which is shown to correlate with disparity. Thus, the paper proposes a federated personalization approach based on adversarial transferability and catastrophic forgetting that reduces accuracy disparity to the level of the global model while maintaining the higher accuracy of prior personalization methods. The paper evaluates the approach on three real-world datasets.", "summary_of_the_review": "The paper identifies an interesting shortcoming of prior personalization approaches. However, given the various weaknesses outlined above (e.g., insignificant results, misrepresentation of results) and the limited novelty, I do not believe that the paper meets the bar for publication in its current form.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper talks about a novel setting in Federated Learning and argues that personalization methods may cause the personalized models to overfit on spurious features, thereby increasing the accuracy disparity compared to the global model. To this end the authors propose a debiasing strategy using a global model and adversarial tranferability. 

 There were some positive opinion about the problem being interesting .However reviewers had several concerns about the validity of assumption and hand wavy arguments used in the solutions for existence adversarial tranferability. Overall, the settings and the need for removing personalization bias needs to be validated more convincingly and rigorously, with concrete real scenarios and experiments.
Decision: Reject