Personalized Federated Learning with Spurious Features: An Adversarial Approach
Authors: Xiaoyang Wang
Han Zhao
Klara Nahrstedt
Sanmi Koyejo
Venue: Accepted by TMLR ()

Abstract: One of the common approaches for personalizing federated learning is fine-tuning the global model for each local client. While this addresses some issues of statistical heterogeneity, we find that such personalization methods are vulnerable to spurious features at local agents, leading to reduced generalization performance. This work considers a setup where spurious features correlate with the label in each client's training environment, and the mixture of multiple training environments (i.e., the global environment) diminishes the spurious correlations. In other words, while the global federated learning model trained over the global environment suffers less from spurious features, the local fine-tuning step may lead to personalized models vulnerable to spurious correlations. In light of this practical and pressing challenge, we propose a novel strategy to mitigate the effect of spurious features during personalization by maintaining the adversarial transferability between the global and personalized models. Empirical results on object and action recognition tasks show that our proposed approach bounds personalized models from further exploiting spurious features while preserving the benefit of enhanced accuracy from fine-tuning.

Published in Transactions on Machine Learning Research (03/2024)
Personalized Federated Learning with Spurious Features: An Adversarial Approach
Xiaoyang Wang
ru28@rlknois.edu
Deportment of Computer Science University of Mlinois Urbana- Champaign
Han Zhao
hanzhao@hllinois.ccomm edu
Department Computer Science
University of Blinois at Urbona- Chaap-nn
Klara Nahrstedt
klaru@illioois. edu
Department of Computer Science University of Blinois at Urbona- Champann
Sanmi Koyejo
edu
Deportment of Computer Science Stanford University
Reviewed on OpenReview: https: // openreview. net/ forum? id= 92wHHUVHHH
Abstract
One of the common approaches for personalizing federated learning is finc-tuning the global model for each local client. While this addresses some issues statistical helerogeneity, we find that such personalization methods are vulnerable to spurious features at local agents, leading to reduced generalization performance. This work considers a setup where spurious features correlate with the label in each client's training enviromment, and the mixture of multiple training environments (i.e., the global environment) diminishes the spurious correlations. In other words, while the global federated learning model trained over the global environment suffers less from spurious features, the local fine-tuning step may lead to personalized models vulnerable to spurious correlations. In light of this practical and pressing challenge, we propose a novel strategy to mitigate the effect of spurious features during personalization by maintaining the adversarial transferability between the global and personalized models. Empirical results on object and action recognition tasks show that our proposed approach bounds personalized models from further exploiting spurious features while preserving the benefit of enhanced accuracy from fine-tuning.
1 Introduction
Federated learning (FI.) is a leading framework for clients collaboratively train a shared global machine learning (ML) model without releasing their local private datascts (McMahan et al. 2017 Kairouz al. 2019). The jointly trained global model can be further fine-Luned
each client's 's local dataset to produce personalized (local) models (Fallah et al. 2020 T. Dinh et al. 2020 Li et al. 2021 While existing theoretical and empirical results highlight how personalized models improve accuracy on local data, few works consider what features the personalized models learn from the local dataset. Our motivating hypothesis is that, not all local features are beneftcial.
Spurious features, which correlate with labols some environments but do not generalize across all environ- ments, are ubiquitous (Geirhos al.] 2020 Singla & Feizil 2022p. For example, for activity recognition, a rocky cliff can correlate with climbing activity (Nm et al. 2020 for classification, flowers can be spurious

---

Published in Transactions on Machine Learning Research (03/2024)
Table I: Requirement comparison. Our approach limits personalization from increasing the entanglement level to spurious features without requiring access to multiple environments or their indexes. Environment indexes are part of the label of data samples, indicating which environment the data sample comes from.
Method
Environments Environment. Indexes
Invariant risk minimization (Arjovsky et al. 2019
Must-have
Must-have
Distributionally robust optimization (Sagawa et al. 2020
Must-have
Must-have
Just-train-twice (JTT)
(Liu et al. 2021
Must-have
Need-nol
Ours
Need-not
Need-not
feature in butterfly images [Singla & Feizi 2022;; ; for recognition, hair color may confound the gender of face images (Liu et al., 2021p. These examples are illustrated with images in Appendix A In these examples, the predictive performance of model that entangles spurious features can degrade in environments where the spurious correlation no longer holds (e.g., for the activity detection example, consider climbing on icy walls). Such non-uniform performance across environments due to the entanglement of spurious features has led to
issues in robustness and fairness, among others (He et al., 2019 Nam et al. 2020 Liu et al. 2021
This paper considers a sctup where each user holds training samples from their own environment but may move other environments with a deployed model. Such a setup applies federated learning systems with diverse and heterogeneous clients, who tend collect training samples around their current environment and may later travel with their mobile devices. Examples include mobile augmented reality applications (Cao et al, 2022,, where client-side distributions change spatially. In such cases, although spurious features correlate with the label on each client's 's Iraining enviroument, aggregating the clients over different environments (i.e., the global environment) contributes to mitigating the spurious correlations.
In FL, even if the spurious correlations in the global environment can be diminished, naive local fine-tuning surely increases the entanglement level of personalized models to spurious features of their local training environments. Thus, lcarning a personalized model for each client without risking entanglement to spurious features is non-trivial. Common methods that aim to disentangle spurious features from models (Wang et al. 2019 Sagawa et al.] 2020: Liu et al., 2021 Wang et al., 2022b) require full access to multiple environments and are, therefore, not applicable to our federated setting. On the other hand, although using the off-the-shelf global model in local agents without fino-tuning avoids further spurions correlations, the performance on each local client's dataset can be sub-optimal.
To this end, we propose novel method to limit personalization from increasing the entanglement level to spurious features without requiring access to multiple environments (Table [ and subsequently improve the generalization performance of personalized models measured by the accuracy disparily (Zhao & Gordon 2022p the difference between model's accuracies across environments. We call the difference between the entanglement level of the global and personalized models entanglement deviation. Then, our mothod uses the adversarial transferubility between the global and personalized models as a proxy to bound the entanglement deviation. The adversarial transferability is measured by the percentage of adversarial examples generated by the global model that also "fool" personalized models. The intuition is that if two models entangle features in the same way, the adversarial examples are more likely to transfer from one to another and vice versa. Based on this intuition, we propose the following hypothesis:
If personalized models increase their entanglement to spurious features, fewer adversorial ezamples generated by the global model transfer to personalized models.
Note that our approach does not require entangling spurious features to be an exclusive cause of decreasing the adversarial transfcrability and remains effective as long as such connection exists. Empirically, we observe that the adversarial transferability between the global and personalized models often decreases when personalized models entangle spurious features. In such cases, the entanglement deviation will increase (Section 3 validating our hypothesis. However, naively including adversarial examples with flipped labels (i.c., mispredictions) to fine-tune and maintain the adversarial transfcrability is sub-optimal in bounding the entanglement deviation (Section 3.3 Therefore, we theoretically analyze the connection between the
2

---

Published in Transactions on Machine Learning Research (03/2024)
adversarial transferability and the entanglement deviation of the global and personalized models. In addition, we show that the disparity can be bounded by the entanglement and present conditions under which the disparity upper bound of personalized models can be close to that of the global model (Section 4 Based on the theoretical results, we develop an improved method to bound the entanglement deviation and the accuracy disparity of personalized models (Section 5) thus improving the robustness of personalized models to local spurious features. Our main contributions arc:
We empirically evaluate the vuincrability of personalized models to spurious features in foderated learning setting, highlighting critical risk of existing personalization methods.
We theoretically counect the adversarial transferability and the entanglement levels of the global and personalized models to spurious features.
We develop method to bound the increased entanglement of personalized models to spurions features by maintaining the adversarial transferability between the global and personalized models.
We conduct extensive experiments to validate the effectiveness of the proposed methods under FL settings. Our experiments on MNIST (Deng 2012), Coil20 (Neue al. 1996), CelebA (Liu et al.) 2015 Caldas el al.) 2018], and biased action recognition (BAR) (Nam et al.) 2020) datasets show that the proposed approach reduces the accuracy disparity of personalized models from 18.38% to 3.42%. Our method also preserves the benefit of the enhanced average accuracy from fine-tuning, resulting in 4.48% accuracy improvement in the global environment.
2 Related Work
Personalized Federated Learning. Fine-tuning is typical for personalizing FL. The meta-learning-bart based method first trains a global model and fine-tunes the global model locally (Fallah et al. 2020). Other methods using multi-task learning (Li et al. 2021 or Moreau envelopes (T. Dinh et al. 2020) have an interpretation as fine-tuning the local model along with training the global model. Fine-tuning is also compatible with clustering- based methods (Ghosh et al. 2020p. FedPAC (Xu et al., 2023) and FEDORA (Wu et al. 2023 further integrate the idea of clustering and fine-tuning: they enable knowledge transfer between personalized models clients that are similar to each other. There are also approaches investigating fine-tuning using sub-networks (Shamsian et al. 2021) or using k-ncarest-ncighhor (kNN) classifier as augmentation to a local model (Marfoq et al. 2022). Our work focuses on limiting the entanglement of personalized models during fine-tuning via efficient regularization. Therefore, we will compare our approach with other fine-tuning approaches that focus on the loss function design (Li et al. 2021 Xu et al., 2023 Wu et al. 2023). The sub-network and kNN approaches are not direct competitors to our approach, and their combination with our work can be interesting for future work.
Debiasing Machine Learning Models. Debiasing is way to disentangle spurious features from the model. Chi et al. (2021) aims to mitigate the accuracy disparity in regression problems via learning the appropriate representations. A few prior works (Li & Vasconcelos] 2019 Sagawa et al. 2020 Wang et al.) 2022b) utilize group labels, which might require human annotation, to debias ML models. Residual learning- based methods (He et al.) 2019 Nam et al. 2020 Liu et al., 2021 train a biased ML model and up-weight the residual, which mainly contains samples from under-represented environments that the biased ML model mis-predicts. Inspired by distributional robust optimization (DRO) (Duchi al. 2016: 2023), the group DRO approach (Sagawa et al., 2020) minimizes the worst-case training loss over mixture of pre-defined groups via selectively assigning higher weights to underfitted groups. However, disadvantage of the residual lcarning-basee based and the DRO-based approaches is their requirement for procise group annotations (Table I. A follow-up work, called just-train-twice (JTT) (Liu el al. 2021), further removed this requirement, which is comparable to our work and is included in the experiments. Note that our approach does not require group annotations either. In addition, we find an issuc with the approach of "assigning higher weights reereseeddedented environments during training" in the federated learning personalization step- when each client has very few samples. The samples from under-representted environments can be too few for the model
3

---

Published in Transactions on Machine Learning Research (03/2024)
1.0
1.00
1.00
Disparity
Acc Disparity
Accuracy
0.75
0.75
0.8
Acc Acc' AdvTrans
Acc Acc AdvTrans
0.50
0.50
0.6
0.25
0.25
Acc
0.00
0.00
0
12 Number of Batch
16
4
8
12
16
0
12 Number Batch
4
8
16
Number of Batch
(a) Accuracy disparity.
(b) Accuracy disparity correlates with (c) Naively maintaining high adversar- ial transferability does not help.
adversarial transferability.
Figure An empirical study to evaluate the transferability hypotheses with spurious features. (a) The accuracy of personalized models the training environments (Ace) and the other environments that are not accessíble during training (Acc') with increasing fine-tuning batches. The personalized models entangle the spurious feature and increase accuracy disparities between environments. (b) As the personalized models entangle the spurious feature and increase their accuracy disparity (Acc Acc'), adversarial transferability (AdvTrans) decreases. (c) Naively maintaining high adversarial transferability (Adv' aadvTrans) helps mitigate the accuracy disparity (Acc - Acc'), but the accuracy disparity still increases from 0.06 to 0.12 along with the increasing adversarial transferability.
reach a desirable performance even if we assign them higher weights (see experiments in Section 6.2.2). In contrast, our approach does not require access samples from minority groups during personalization and find that our approach does not suffer from such limitation.
Adversarial Transferability. Prior works [Tramer et al. 2017 Charles et al. 2019) attempt to understand adversarial transfcrability. A recent work connects adversarial transferability to knowledge transferability in a transfer learning setting (Liang et al. 2021). Our paper builds on this work to show how to solve a new problem bounding the personalized models' entanglement to spurious features- by maintaining adversarial transferability between the global and personalized models.
3 An Empirical Study with Spurious Features
We first perform an empirical study on the entanglement deviation between the global and personalized models in an FL setting to gain some insights into the problem. This section focuses on the MNIST dataset due to the limited space, and Appendix further provides results on the other datasets. In this study, the personalization method is fine-tuning. Our results highlight the risk of existing fine-tuning-baee based personalization methods and the difficulty of mitigating the risk. We also highlight the correlation between adversarial transferabilit, and entanglement deviation. We provide additional theoretical analysis in Section for more insights into the observed correlation. The client setup and evaluation protocol this empirical study are as follows.
Client Setup. There are two enviromments whose combination the global enviromment. Each client gets different set of grayscale MNIST digits, then creates a spurious correlation by coloring the grayscale objects according to their labels differently for each environment (e.g., the red color correlates with digit 0 in the first environment and with digit in the second environment). Then, each client picks random environment for training, and all clients collaboratively train global model. After the global model converges, clients further fine-tune the global model in their training environments.
Evaluation Protocol. We evaluate the accuracy of personalized models in their training environments (Acc) and the other environments that are not accessible during training (Ace') with an increasing number of fino-tuning steps (i.c., batches). The accuracy disparity indicates entanglement to spurious fcatures, defined as the accuracy difference between the training environment and the environment that not accessible during training (i.e., Acc- Acc').
4

---

Published in Transactions on Machine Learning Research (03/2024)
3.1 Personalization May Exacerbate Entanglement
As can be observed in Figure 1a the accuracy disparily of personalized models increases within a few fine-tuning batches, deviating from that of the global model. This observation indicates that personalized models gradually increase the entanglement to the spurious feature. Although, in principle, one may resort to early stopping, this is not feasible using a single training environment alone.
3.2 Transferability Correlates with Entanglement
Because only the training enviromment accessible during personalization, it infeasible to directly measure personalized models' entanglement or accuracy disparity. To this end, in this section, we focus on methods that implicitly measure and determine to which level personalized models entangle the spurious feature. Following our hypothesis in Section 1 we consider using the adversarial transferability between the global and personalized models as a proxy for measuring their entanglement deviation.
An adversarial attack aims to add a calibrated adversarial perturbation to the input to mislead a model's 's prediction. A perturbed input is an adversarial example. A common way to calibrate the adversarial perturbation is leveraging the model's first-order gradient w.r.t. the input (Madry al.) 2018 Miyato et al. 2018 Liang et al., 2021). Such a first-order gradient suggests direction along which the prediction changes significantly. In our setting, the first-order gradient shall direct more perturbations to robust features to mislead the global model, which entangles spurious features less. In contrast, if personalized models entangle the spurious features more, they are more likely to resist the attack that targets the global model by using the spurious features with fewer perturbations. Following this intuition, we employ the projected gradient. descent (PGD) attack (Madry et al.] 2018) that iteratively calibrates the adversarial perturbation using the first-order gradient. Specifically, at a point æ with label y, for the global model fg with loss function E, iteration & ++1 1, the adversarial example is:
Adversarial Attack
x tt Proji|aan seles (aady + a signnVai
v))),,
where Proj is projection operator, =e x, is the attack budget, and a is the attack step size.
Adversarial Transferability We generate adversarial examples using the global model and collect the ones that cause the global model to mispredict. Then, personalizod models make predictions on the collected adversarial examples. The adversarial transferability is defined as the percentage of the collected adversarial examples that also cause personalized models to misprodict. Recalling that the collected adversarial examples embed more perturbation on robust features, personalized models may resist the adversarial examples if they use spurious features to predict.
Figure 1b plots the accuracy disparity and the adversarial transferability during fine-tuning. As personalized models increase their entanglement to spurious features, the accuracy disparity of the personalized model increases and deviates from that of the global model. Then, the adversarial transferability between the global and personalized models decreases. This result empirically validates our hypothesis.
3.3 Can Maintaining Transferability Mitigate Entanglement Deviation?
Following the empirical observation of the adversarial transferability and the entanglement deviation, we add the collected adversarial examples with flipped labels (i.e., mispredictions) to the training set during personalization, aiming to maintain the adversarial transferability. However, Figure [[c shows that the accuracy disparity still doubles (0.06 to 0.12), indicating that the entanglement deviation increases, even if the adversarial transfcrability increases. The following section presents thcoretical analysis, outlining conditions under which maintaining the adversarial transferability helps bound the entanglement deviation.
4 Theoretical Insights
In this section, we present a theoretical analysis that supports our hypothesis in Section 1 the experimental results in Section 3.2 and analyze the failure in Section 3.3 Before we proceed to the detailed analysis, some
5

---

Published in Transactions on Machine Learning Research (03/2024)
additional definitions and notations are needed for the presentation (we provide table summarizing all the notations used in Appendix B to ease the reading).
4.1 Setting
Data Model. A data sample æ Cat(ær, xt)) is concatenation of robust features Ør (i.e., dim((z)) 2 2) and a spurious feature I.. The label is binary, E {0,1}-
Environment. We consider a setting with two environments On the 7th client, D; denotes the joint distribution of æ and y in the training environment and Dj denotes the future environment of the i-th client that is not accessible during training. In addition, we use Drr to denote the joint distribution of æ, and y and use a to denote the joint distribution of æs and y. Our analysis applies each client's D and D', and shall omit the subscripts when it is clear from the context.
Distribution Shift. Our analysis focuses conditional distribution shift caused by spurious correlations, which imply PD(x,, v) * Poi v) but Po(æ, y) = Pa-(a, y) and Po(y) = (v).
Data Generating Process. We consider a data- tata-generating process that produces a pair samples with four steps: (1) draw a label y according to Pp(y), (2) draw a robust feature æ, under Po(ax, y), (3) draw a spurious feature according to Pp(x,, y) and concatenate features æ Cat(æ, æx), and (4) draw spurious feature with Po (ar, y) and concatenate features x' Cat w?)) By the linearity of expectation, the four-step data-generating process does not affect the loss disparity:
Epl((f(z). E (((f(()) will = (B, (((f(x). ((f(('),
(1)
Hypothesis Class. We consider logistic regression model f(x) = (w x)) with a Sigmoid activation function 0, and a logistic loss l. Similar to the notations in the data model, we let w = Cat(w,, w.), where w, includes the weights on robust features D, and w, denotes the weight of spurious feature æs Sg denotes the global model with weight wy and fi denotes the personalized model with weight wo Wg.s denotes the global model's weight of the spurious feature.
4.2 Preliminaries
Entanglement. For a given model f with weight w, its entanglement to the spurious feature ||>,|| is defined as w,. The entanglement ||w,]| can be further decomposed into norm term and an angle term: ||w=_ll || cos 0, where 0 is the angle between and Cat w..). Such definition of f implies that 0 E [0, } and coso E [0, 1]. Compared to existing definitions such w, 0 from the literature discntangling spurious features [Rosenfeld el al. 2021 Wang el al. 2022a), our definition allows quantilative measurement of the entanglement/ wtwpg//eeeentttittaannnnnnee /diennangglemen to spurious features and includes the standard objective as special case (i.c., ||/||| = 0 or = 0).
Angles Let 0' arccos model's weight wi, 0g be the angle between Wy and Cal(O, wg.s), and Or be the angle between and Cat(O, w)
we || be the angle between a global model's weight. wy and a personalized
w. w
Loss Disparity The loss disparity of a model f is defined as: En.(((((<)) v) - Ex([(f(x). y)l, quantifying the generalization performance of a model across environments.
Adversarial Perturbation. The adversarial perturbation 85 Badv x generated using model f with budget e. A common way to generate adversarial examples is solving maximization problem: Badv arg maxj= - s {( (f())))))) which returns data sample that maximizes the model's loss and likely
'Our analysis could generalized more than two environments straightforward way.

---

Published in Transactions on Machine Learning Research (03/2024)
causes misprediction. Plugging the definition of ,,,e into Tadv arg m ((f((*))))) have ff.- = arg j {[f(z + y). With small budget we can approximate the loss function E using the first-order gradient (Miyato et al. 2018 Liang et al. 2021]:
f.c arg max V=((f(x).)) c vel(f()),)) [ve(f(=)))))]
(2)
Se
Adversarial Transferability Measure. With the adversarial perturbation, we define the adversarial transferability measure that positively correlated with the adversarial transferability:
Eg fi, y) (eff((z - o5.,)),) -
(
(eU, ds -((((())))))
(3)
measuring how effective the adversarial perturbation ,/, generated by the global model Sg is after applying to the personalized model Ji- Specifically, if the adversarial perturbation ofg is less effective on the personalized model ft due to decreased adversarial transferability, we have (f fi, fi.x,)) < 0. Otherwise, fg-1(sgr Jts x. y) 2 O. For a logistic regression model, its first-order gradient at a is (a(w" T x)) w. Here, we can see that the gradient direction depends on the weight w at given data sample. This allows us connect the adversarial transferability measure to entanglement using the angles between weights.
4.3 An Upper Bound of the Loss Disparity
Disparity Upper Bound. To show the necessity of reducing the entanglement the spurious feature, we connect the entanglement to the spurious feature and the loss disparity, empirical metric that quantifies model's generalization performance across environments. The following theorem suggests that the loss disparity of a model is upper bounded by its entanglement to spurious features, quantified by the angle 0 between w and Cat w.).
Theorem 1. Under the setting in Section 4.1 for a model f paremeterized by w, assume the composition f a of the activation function a and the loss function is p-Lipschitz, let be the angle between w and Cot ws); we have:
[ED((((Z))v) ED (( 112 WD,,,, = p|/ cos 0. WD, D)).
(4)
Note that cos 0 E [0, 1] by its definition. Theorem implies that the loss disparity upper bounded by the entanglement measure ||l]|a cos@ (Section 1.2p. the Lipschitz constant P. and the Wasserstein-1 distance Wi(D,, D.). Such result is intuitive: for given distribution shift on spurious features æ, that can be quantified by the Wasserstein-1 distance Wi (D. D2), its impact on the loss disparity can be further amplified by up to the product of the entanglement measure ||w,|| = || cos 1 and the Lipschitz constant of foo.
Disparity Deviation Upper Bound Following the result in Theorem ! we further present an upper bound on the disparity deviation between the global and local models. Our upper bound includes the three angles that are defined Section 4.2 (1) 0g: (2) 0,, and (3) 6', where the first two angles measure how
much the global and personalized models entangle the spurious feature and the angle O' quantifies the angle difference between wy and wi.
Corollary 1. Under the setting in Section and the definitions in Section with Theorem have:
[E)((((())) v)] Ro (2]) w)] [F (B) Ep
(((((x))))
(5)
pp Sp-Wi(D.y (D. D') (Iw cos 0, + ||l/||| cos 0/
In addition, 0' < 0g we have
|E (x)) y)1 ED (e [E](((((x))
v) - ED (e(f((z))
(6)
p Wi (D.,,, D') (110,,1 || cos 0, + || cos(O, - -0')
7

---

Published in Transactions on Machine Learning Research (03/2024)
Otherwise, 8' > 0g. have:
[DDe(f v\l ED ((]f)())))))))) w/1l [EDl((f(Z)) ED (=((((((()))))))
(7)
Sp-Wi D2) (1w0 cos 0 + ||wil])
Corollary [ suggests that the disparity deviation between pair of models is bounded by the difference between their entanglement measures, where each is composed of an angle term and norm term. The intuition behind Equations equation 6 and equation is straightforward: in a higher dimensional setting (e.g., dim(x,) 2 2 in Section [. the angle 8: that quantifies the entanglement of the local model fi to the spurious feature æs falls within an interval [0g - 0' 0' + 0 > Og, the angle O is possibly 0 all the time, entangling the local model fi to the spurious feature. Furthermore, we present a directed analysis of the loss disparity of local model.
Corollary 2. Under the setting in Section 4.1 and the definitions in Section 4.2 with Theorem if 8' S 8g. ue have
[p(((f((()) 8)] ED [((f(=)) v)ll < p|| W1 WD D')
= p||well cos 0: (D, D') S p|| cos - 0') W (D,,,,, D').
(8)
Otherwise, if 0' > gg. we have:
[D((([((z)))) Ed ((([(())))))) S e| (D,,,, D().
(9)
Corollary shows that the loss disparity upper bound of local model fi can be upper bounded by its weight norm wy and the angle difference 0' between the global and personalized models. Next, we further investigate the relationship between the adversarial transferability between the global and personalized models and the angle o' between wy and wi.
Adversarial Transferability. Now, we investigate the connection between adversarial transferability and the loss disparity as well as its deviation. The following theorem bridges the connection between the adversarial transferability measure Epl.y Eplly-i(fg- fi, v)] and the angle difference f' between the global and personalized models in Corollaries and We show that high adversarial transferability measure indicates small 6' that further implics a low disparity deviation (Corollary D
Theorem 2. Under the setting in Section 4.1 let be the attack budget, and 0' be the angle between wa and we have:
0' arrcos (-- (
[c- +++ Eollg-i fo.
(10)
I)], = ( v)Il,, and o = E - (2,2(xaav) is the expected difference of second-order Taylor remainders between fy and fi.
where O =
Theorem connects the adversarial transferability measure EDlgg-st(f J1. y)] and the angle 0' which is a key factor in the loss disparity of a personalized model (Corollaries and 2 Such a connection gives rise an opportunity of minimizing the adversarial transfcrability mcasure for reducing the loss disparity of personalized model. However, in addition to the adversarial transferability measure, Theorem 2 also includes gradient norm term O from the personalized model, a constant D, and second-order Taylor remainder term o, which may need additional treatments.
4.4 Necessary Condition
Since the connection between the adversarial transferability measure and the entanglement deviation is angle-based (i.e., 0'), the norm terms (e.g., O EB[l (((((()) I)ll may corrupt such connection and

---

Published in Transactions on Machine Learning Research (03/2024)
cause the sub-optimal result in Section 3.3 For example, increasing the gradient norm term O of the personalized model ft may increase 0' with fixed adversarial transferability. A Also, directly increasing the weight norm ||w/] || can increase the entanglement deviation even if 0' = 0. Therefore, we need to stabilize the gradient norm term O and the weight norm w throughout the personalization step to make the angle-basod connection helpful. The following section lists two practices for maintaining adversarial transferability while stabilizing the aforementioned norm terms.
5 Methods
In Section Theorem 1 suggests that the accuracy disparity is upper bounded by the Lipschitz constant, our entanglement measure, and the distribution shift measured in Wasserstain-1 distance. Then, in Corollary [] we show that the disparity deviation between a pair of models is bounded by the difference between their entanglement measures. Further, Theorem 2 shows that maintaining the adversarial transferability can help reduce the difference between the angle terms in the entanglement measures thus illustrating how we can limit the disparity deviation by maintaining adversarial transferability.
However, such connection may break if the weight norm of local models significantly increases or decreases during personalization (Section 4.4). To this end, present an improved approach to maintaining the adversarial transferability while stabilizing the norm terms. Concretely, we first introduce regularizer to encourage the global and personalized models to make consistent, predictions on adversarial examples, maintaining the adversarial transferability. In addition, we add another Le regularization term to align the weight of the global and personalized models because similar weights load to similar norm terms. Appending the two regularizers to the loss function during personalization miligates the increasing accuracy disparity.
5.1 Maintaining Adversarial Transferability
Generating Adversarial Examples. We employ the same projected gradient descent (PGD) attack (Madry et al.] 2018)) as Section 3.2 which iteratively perturbs the input using the model's first-order gradient. At iteration ++ 1, the adversarial example under budget is: 2 Proji ( ( asign( V Val aaa (())))) where Proj is a projection operator and a is the attack step size.
Enforcing Consistent Predictions. Both the global model fg and the personalized model fi take the adversarial example Fady as input and output 2g and ž from their last layers, respectively. We increase the adversarial transferability by adding the following regularization term, which minimizes the cross-entropy between 2g and 2 after softmax normalization. Since the global model fy is fixed as reference in the personalization step, and its low accuracy disparity is desirable, we zy as the ground truth. With K classes, we have:
K
Radv (Zg: 21)
Σ a
F
log
K
k Di=]
The local model has access to the global model, so there is no additional communication overhead for implementing this regularization. The adversarial examples are computed using the global model once and for all. The computation only needs a few back propaarron,,,ion, much less than training the global model. Suppose there are N data samples, and the personalization needs E epochs, our approach only generates N adversarial examples once using the global model. In contrast, the standard adversarial training (Madry et al. 2018 generales N x E adversarial examples.
5.2 Aligning Weights and Norms
In addition, we employ a simple and effective strategy by adding an L2 regularization term to the loss function, as a means to minimize the interference from norm terms as is discussed in Scction 4.4
RIg (Wg: ur) 1000 w/ll-
(11)
9

---

Published in Transactions on Machine Learning Research (03/2024)
Table 2: Worst-case accuracy AcCworst and accuracy disparity Accdiap across environments of personalized models. Our proposed method improves the worst-environment accuracy and mitigates the accuracy disparity during personalization.
MNIST
Coil20
CelebA
BAR
Method
Accworst
AcCdisp
Aorwerst
ACCdisp
AcCworst
ACCdisp
AcCworst
AcCdisp
Global
847
.005
.888
.001
.906
.002 +
.720
.002
FT
.704 ae-4
285
.804
.156
4c-3
.849
1e-3
.114
1e-3
.620
.180 20-4
Ditto
.724 1e-3
.258
+ 1e-3
.952 1 2e-4
.031
2e-4
.884
2e-1
.082
1 2e-4
.644 1 20-6
.151 20-8
UW
.823 Te-4
.145
+ 2c-5
.894 + 4c-4
.021
40-4
.895
Be-4
.041
+ 4o-4
N/A A
N/A
JTT
.707 = 6e05
.278
+ 0e-5
.836 + 1e-3
.060
= 1e-3
.836
1 2e-4
.120
1 26-4
N/A
N/A
FedPAC
.755 Ge-4
.236
+ 1e-8
.859 + 1e-5
.102
2e-3
.873
1o-3
.082
+ 2c-3
.654 + Se-B
.122 1e-3
FEDORA
.721 2e-3
.269
2e-3
.847
1e-3
.115
1e-3
.851
2e-3
.109
4 3e-3
.643
4 1n-3
.146 10-3
Ours
.870
.081
20-5
.963
5e-5
.023
1e-5
.925
.002
Be-5
.730
+ 20-4 .031 + 22-4
Each experiment is repeated 9 times with 3 random seeds for the federated learning step and 3 random seeds for the personalization step.
The motivation behind the L2 term is straightforward: two models have similar weights, they have similar weight norms and gradient norms, as is shown in the following proposition.
Proposition 1. Under the setlings in Section 4-7 assume the composition lo of the activation function and the loss function & is p- Lipschitz, have:
[|wlll|
| |II S
2w wgll
(12)
and
|EDll V vo. ((f(()) V
vIll < pEp ||||||||| wgll
(13)
We empirically verify that our Lg regularizer reduces the weight norm deviation in Appendix E prior works (Li et al. 2020 T. Dinh et al. 2020 Li et al. 2021 have explored similar regularization methods, we develop the regularization term from a different perspective.
Although
6 Experiments
This soction presents our exporimental results under a federated learning setup with four datasets: MNIST Deng 2012), Coil20 (Nene et al.) 1996), CelebA (Liu et al. 2015 Caldas al. 2018 and BAR (Nam et al. 2020 demonstrating that our method mitigates the accuracy disparity. We also show that the benefit of enhanced average accuracy from fino-tuning is preserved. Appendix E further presents simulation with Gaussian data, experimental results with imperfect global models, and an evaluation of our approach against adversarial attacks.
6.1 Setup
Spurious Features The MNIST and Coil20 datasets employ synthetic spurious feature where the object color correlates with the label. Such a synthetic spurious feature allows precise intervention on the spurious attribute of each data sample without interfering with the robust fcatures, matching the setting in Section
4.1 The spurious feature in the BAR dataset is the action background (e.g., rocky cliff). The hair color is spurious feature that correlates with the gender label in the CelebA dataset.
We consider a two-environment setup. Each user gets a different set of samples (Caldas et al. 2018), allocates the samples to the two environments, and randomly picks one environment as the training environment. Combining the training environments eliminates the synthetic spurious correlation in MNIST and Coil20 datasets. The realistic spurious correlation in the BAR and CelebA datasets can diminish but does not necessarily disappear in the global environment.
Environments
10

---

Published in Transactions on Machine Learning Research (03/2024)
Metrics We consider three metrics in evaluating our approaches: (1) the accuracy disparity Accdisp across environments, (2) the worst-case accuracy AcCworst among all environments, and (3) the average accuracy in the mixture of all environments (i.e., the global environment). Metrics (1) and (2) measure the generalization performance of personalized models under environment shifts after deployment. Metric (3) quantifics the overall utility of personalized models.
We provide detailed experimental setup in Appendix D
6.2 Result and Comparison
There are two naive personalization methods: using the global model (Global) and vanilla fine-tuning (FT). In addition, we employ three recent. works that are applicable to the federated learning setting (Section 3. Up-weighting (UW) (Sagawa et al.) (020) method is implemented via sampling biased samples and counterexamples with equal probability. Just-train-twice (JTT) (Sagawa et al. 2020) leverages models that have already entangled spurious features (e.g., personalized models with naive fine-tuning) to identify and up-weight counter-cxamples without explicit annotations. Ditto (Li et al.) (021) adds a regularizer A||ww - wy during personalization to limit the deviation of personalized models from the global model. We select models using the validation accuracy minus the adversarial transferability degradation for ours and using the validation accuracy for others.
6.2.1 Summary of Results
Tables 2 andd 3 show the main results with the two metries on generalization performance in Section 6.1 Our method reduces the accuracy disparity of personalized models from 18.38% to 3.42% compared to vanilla fine-tuning, outperforming other competitors (Table On average, our method also preserves the enhanced average accuracy from fine-tuning, resulting in 4.48% accuracy improvement in global environments compared to the global model (Table 3
6.2.2 Analysis of Results
One limitation of up-weighting-basse methods (i.c., UW and JTT) relates to the diversity of counter-cxamples, which do not agree with spurious correlations. Counter-examples are few in datasets, leading to low diversity.
To explore the impact of counter-exampler diversity on disentangling spurious features from a model, we vary the diversity of counter-examples and adjust the up-weighting factors accordingly. Specifically, we sample a factor of 0.02, 0.025, 0.083, 0.05, and 0.1 biased samples from the MNIST dataset and re-color them to become counter-examples. The factor the sampling step is called the "sampling factor". Then, we up-weight the counter-examples by factor of 50, 40, 30, 20, and 10, respectively, keeping the total number of counter-example consistent. Here, the counter-examples have less diversity if generated by small number of biased data samples with large up- up-weighting factor. Experimental results in Figure 2 show that, as the diversity reduces, the accuracy disparity of personalized models increases, supporting our hesothesis on the sample diversity. Therefore, our method is applicable in the scarcity of counter-examples while up-ueipe weighting based methods fail.
The issuc with Ditto is that groedily maximizing the accuracy training environments often leads to small X, giving personalized models spaces to entangle spurious features. FedPAC (Xu et al. 2023) and FEDORA (Wu et al. 2023). which promote knowledge transfer between personalized models clients that similar to each other, can exacerbate the entanglement to spurious features because they encourage the knowledge transfer between clients that share similar spurious features while isolating clients from different environments.
6.3 Ablation Study
We conduct an ablation study on the MNIST dataset and plot the accuracy and loss curves, which are shown to have similar behaviors. Figures 3b and 3i demonstrate that only enforcing consistent prodictions on adversarial examples is insufficient. Figure 3c and 3g further show that applying the L2 regularization term alone does not address the accuracy loss disparity. Aligning weights of the global and personalized models
11

---

Published in Transactions on Machine Learning Research (03/2024)
0.95
Acc
Metric
Acc
0.85
Acc'
0.80
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.10
Sampling Factor
The up-weighting method is less effective with few counter-cxamples, resulting in a large accuracy disparity of personalized model in the training environment (Acc) and the global environment (Ace'), ) if the counter-examples are generated by a small number of biased data samples using a small sampling factor and large up-weighting factor. The up-weighting factor is set to be the reciprocal of the sampling factor.
Figure 2:
Table 3:
Average accuracy personalized models over different environments. Our proposed method preserves
the enhanced accuracy of personalization.
Method
MNIST
Coil20
CelebA
BAR
Global
849 +
.888 10-3
.919
.721
FT
.846 i:
.882
4m-3
-906
::2 1e-3
.710 +:
Ditto
.833
.968
2o-4
.925
.719
FedPAC
.873
.910
20-3
.914
.715
FEDORA
.855 1 Re-3
.904
1e-3
.905 3e-3
.716
+ 1e-3
Ours
.911 20-5
.974
1e-5
.926
.745
2e-4
by applying the L2 regularization term while maintaining the adversarial transfcrability address the accuracy and loss disparity as Figures 3d and 3h show, respectively. Both components our methods are helpful, and a combination of them achieves the best results.
We also explored enforcing the global and personalized models to make consistent predictions benign samples. However, the accuracy disparily of personalized models increases by 9.66% (19.25% relative increase) with fine-tuning 16 batches. We hypothesize that personalized models could use combination of robust and spurious features to make the same prediction as the global model on benign samples, which only uses robust features and is disentangled from spurions features.
6.4 Visualizing Gradient Magnitudes
We plot the distribution of the ratio (()((()))))
[V across data samples with 16 batches of fine-tuning the MNIST dataset, using the global model reference. Figure suggests enforcing consistent predictions using logits (our approach) achieves lower gradient magnitude deviation than maintaining the adversarial transferability using flipped labels (naive approach). The Le regularizer also stabilizes gradient magnitudes.
7 Conclusion and Future Work
In this work, we show a risk of prior federated learning personalization methods with spurious features, which lead to high accuracy disparity between environments. Then, we develop a strategy to reduce the accuracy disparity by maintaining the adversarial transfcrability between the global and personalized models. Both empirical and theoretical results show that our strategy is effective. Extensions of this work include incorporating text data, which often requires different methods for generating adversarial examples.
12

---

Published in Transactions on Machine Learning Research (03/2024)
1.00
Ace Disparity
Disparity
0.8
Acc Disparity
0.8
Acc Disparity
0.8
0.75
Acc Ace
0.6
Acc Acc'
0.6
Acc Acc'
0.6
Acc Acc' Advirans
0.50
Advirans
0.4
AdvTrans
0.4
Advlrans
0.4
0.25
0.00
0.0
0.0
0.0
8
12
15
12
16
0
8
12
16
12
16
Number of Batch
Number of Batch
Number of Batch
Number of Batch
(a) Naive Fine-tuning
(b) Consistent Prediction
(c) Aligning Weights
(d) Ours
Lass Disparity
1.5
Loss Dispanty
1.5
Loss Disparity
1.5
Advirans + 1.7
Loss Disparity
1.5
1.0
1.0
1.0
1.0
Irans
0.5
0.0
16
12
16
Number Batch
Number Batch
Number of Batch
Number Batch
(e) Naive Fine-tuning
(I) Consistent Prediction
(g) Aligning Weights
(h) Ours
Figure 3: The accuracy disparity (Ace Acc') and the loss disparity (Loss' Loss) between the training and the global environments. These two disparities correlate with the adversarial transferability measure (AdvTrans) differently with different methods. We add 1.7 to the adversarial transfcrability mcasure over the loss ease reading. Accuracy and loss disparities have similar behaviors. Combining the two practices in our approach addresses the accuracy and loss disparities.
1.00
0.75
Method
0.50
Logit w
Label L2
0.25
Logit w/o L2
10f
10a
Ratio
Figure 4: Distribution of the ratio [| (()))))))) Using approach and enforcing consistent predictions with logits achieves lower gradient magnitude deviation than adding adversarial examples with flipped labels personalization. The L2 regularizer also stabilizes gradient magnitudes.
[V
8 Acknowledgement
XW and KN are partially supported by a research grant from the C3.ai Digital Transformation Institute (DTI). This work of SK is partially supported by NSF III 2046795, IIS 1909577, CCF 1934986, NIH 1R01MH1 HETHH1622666 6226-01 A, NIFA award 2020-67021-32799 the Alfred P. Sloan Foundation, and Google Inc. The work of HZ is partially supported by the Defense Advanced Research Projects Agency (DARPA) under Cooperative Agreement Number: HR00112320012 and a research grant from the Amazon-lllinoi Center on AI for Interactive Conversational Experiences (AICE).
References
Martin Arjovsky, Téon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint Xrc::19977002899 2019.
Scbastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMa- han, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXis:1812.00009 2018.
13

---

Published in Transactions on Machine Learning Research (03/2024)
Jacky Cao, Kit-Yung Yung Lam, Lik-Hang Lee, Xiaoli Liu, Pan Hui, and Xiang Su. Mobile augmented reality:
User interfaces, frameworks, and intelligence. ACM Compul. Surv., aug 2022. TSSN 0360-0300. doi:
10.1145/355799. URL httpss///oi. http:///ooo.og//1....o 111///..11///357999999
Zachary B. Charles, Harrison Rosenberg, and Dimitris Papailiopoulos. geometric perspective on the transferability of adversarial directions. In AISTATS, 2019.
Jianseng Chi, Yuan Tian, Geoffrey J Gordon, and Han Zhao. Understanding and mitigating accuracy disparity in regression. In International Conference on Machine Leorning (ICML), 2021.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141 142, 2012.
John Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses for latent covariate mixtures. Oper. Res., 71(2):649-66) 664, mar 2023. ISSN 0030-364X. doi: 10. 1112877ppre.2022.00 URL https://aoi. 10. 1211 htt./////pp...0/// 2022.2363
John C. Duchi, Peter W. Glymn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. Math. Oper. Res., 46:946-969, 2016. URL https://api semanticscholar
org/Corrppao hog//orrpssDD:7726000
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar, Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances Neural Information Processing Systems, 33, 2020.
Robert Geirhos, Jörn- HenriH Jacobsen, Claudio Michaclis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11): 665 673, 2020.
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An efficient framework for clustered fed- erated learning. In H. Larochelle, M. Ranzato, Hadsell, M. F. Balcan, and H. Lin (eds.), Advances Neu- rol Information Processing Systems, volume 33, pp. 19586 19597. Curran Associates, Inc., 2020. URL (ttpss http://...........coooooo//////a
He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fitting the residual. Xiv preprint arXiv:1908 1908. 10763, 2019.
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual lcarning for image recognition. 2016 IEFE Conference on Computer Vision and Paltern Recognilion (CVPR), pp. 770 778, 2016.
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning, Xix preprint or orXiv:1912.0499 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, Son Diego, CA, USA, May 7-9, 2015, Conference Truck Proceedings, 2015. URL
6980
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In I. Dhillon, D. Papailiopoulos, and V. Sze (eds.), Proceedings of Machine Learning and Systems, volume 2, pp. 429 450, 2020. URL htttps:/ .wtssssmsss......m .tw.syssog////.. Paper. .ddf
Tian Li, Shengynan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust foderated learning through personalization. In International Conference on Machine Learning, pp. 6357 6368. PMLR, 2021.
Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pallern Recognition, pp. 9572-9581, 2019.
14

---

Published in Transactions on Machine Learning Research (03/2024)
Kaizhao Liang, Jacky Y Zhang, Boxin Wang, Zhuolin Yang, Sanmi Koyejo, and Bo Li. Uncovering the connections between ad adversarial transferability and knowledge transferability. In International Conference on Machine Learning, pp. 1577-6587) PMLR, 2021.
Evan % Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. Tn International Conference on Machine Learning, pp. 6781-6792 1691-679.222 PMLR, 2021.
Ziwei Tiu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. URL http:p///ppppuaaaaaaaaeeeeeerraa net:forummi.ddxxx.BBB22
Othmane Marfoq, Giovanni Neglia, Richard Vidal, and Laetitia Kameni. Personalized federated learning through local memorization. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 15090.11092. PMLR, 17-23 Jul 2022. URL https: patpss///ccceedngs..... mlr. prres//162//arfoqq2 html
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pp- 1273-1282. PMLR, 2017.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii Virtual adversarial training: a regular- ization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 1999919999,, 2018.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Newral Information Processing Systems, 2020.
Sameer A. Nene, Shree K. Nayar, and Hiroshi Murase. Columbia object image library (coil-20. Technical report, 1996.
Flan Rosenfeld, Pradeep Kumar Ravikumar, and Andrej Risteski. The risks of invariant risk minimization. In International Conference on Learning Representations, 2021. URL http::///pparreveer..eetttttttt
id=BbMIbVPJ-42
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://openreviier
htt./forummidrryyGuuutt
Aviv Shamsian, Aviy Navon, Ethan Fetaya, and Gal Chechik. Personalized foderated learning using hyper- networks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. Jul 2021. URL
PMLR, 18-24
Sahil Singla and Sohcil Feizi. Salient imagenct: How to discover spurious fcatures in deep learning? In International Conference on Learning Represendations, 2022. URL https://openreviee: http://ppaarrreeeeeeeeeeerrrriia fet/ XVPqLyNxSyh
Canh T. Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21394-2144 21405. Curran Associates, Inc., 2020.
Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick Mcdaniel. The space of transferable adversarial examples. ArXio, abs/ 1704.0345, 2017.
15

---

Published in Transactions on Machine Learning Research (03/2024)
Haohan Wang, Zexue He, Zachary L. Lipton, and Eric P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019. URL
https://openrevier
Haoxiang Wang; Haozhe Si, Bo Li, and Han Zhao. Provable domain generalization via invariant-feature subspace recovery. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabalo (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. https://proceediin mlr. hwtp:ssww......seeeeeememmme wwwppp/eweeeeaagg//p/p hhmmll
PMLR, 17 23 Jul 2022a. URL
Haoxiang Wang; Haozhe Si, Bo Li, and Han Zhao. Provable domain generalization via invariant-foature subspace recovery, In International Conference on Machine Learning. PMLR, 2022b.
Jun Wu, Wenxuan Bao, Elizabeth Ainsworth, and Jingrui Hc. Personalized federatod lcarning with parameter propagation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, KDD '23, pp. 2594 2605, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701030, doi: 10.1145 10.1145/3580305.33 /35/333500033994644 URL
http://60...0rr/10.114////3/3333 .5999444
Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. In The Eleventh International Conference Learning Representations, 2023. URL
Han Zhao and Geoffrey J Gordon. Inherent tradeoffs in learning fair representations. Journal of Machine Learning Research, 23:1 23:1-26, 2022.
16

---

Published in Transactions on Machine Learning Research (03/2024)
A
Spurious Feature Examples
0
/ 2 4
5
6 7 8 9
(a) MNIST
(b) CelebA
(c) BAR
Figure 5: Datasets with spurious features. The object color spuriously correlates with the digit in MNIST (a).
The hair color spuriously correlates with the gender in the CelebA dataset (b). The background spuriously correlates with the action in the BAR dataset (c).
B
Notation
Table 4: Table of Notation
Symbol
Description
x, y
A pair of data sample and label
(:)(;;)
A concatenation of two vectors
Br Ds
The robust features and spurious features in æ Cat( æt), respectively The global model
fg
fi
The personalized local model
An adversarial perturbation generated by the global model fg with attack budget
D
A distribution in the training environment
A distribution in the other environment that is not accessible during training Wasscrstain-1 distance between two distributions
W1(+))
C Proofs
Theorem 1. Under the selling in Section 4.1 for model f parameterized by w, assume the composition l of the activation function and the loss function e is p-Lipschitz, let 0 be the angle between w and COt(O, w.), we have:
[D[((f(z)) 2)1 ED [[[f(x)) v)]l < wll WI,,,, D') 2| W1 WD,,,, D.).
(14)
Proof. With the data generating process in Section 4.1 we have:
Ep((((()) v)1 -E (()((())) w)ll [ED,)) ((F()))) [
(15)
Since the loss function is p-pisschitz we further have:
Ep. [((f((). v) - {(f(2') < (f(x)
(16)
Under the given data generating process in Section 4.2, we know that æ x' Cat x, xx),, yielding: [p,lff() f(x')] < |,, ED, Dy jx. æ'Jl ||l||2 cos OIE D 1x, x']1l
(17)
By the definition of Wasserstein-11 distance, we have:
ED,.Dy xx]] S E, - =x,1] S (D. DD..
(18)
17

---

Published in Transactions on Machine Learning Research (03/2024)
Theorem 2. Under the setting in Section 4.1 let e be the attack budget, and 0' be the angle between wy and ue have:
0' = arccos c.
[s. [+
St, 21, w,
(19)
where O ED V I)l. == cBDlIIV, (() y)ll,, and o E ( the expecled difference of second-order Taylor remainders between fg and fi-
Proof. Expanding ( J1, y) (xr v) and recalling the definition of adversarial perturbation ö5, V.llf(a)))) in Equation (2), we have:
e (V)
lo-i x,
(eVi(x + 8f.r). - ((Ur( + 8),,))))- ((G)(())))))))
(20)
val(fi(()).) v) 0 (1 vol(s(()). 3) 18 (2
( v (x),, w)ll + R1,2(Badv) (
Val(Si(a).)) [V v)]
where Tady x + of and Rr,2(aaat (g.2( is the difference of second-order Taylor remainders between fg and fi.
For logistic regression models fg and fi, we have:
v=((f(()).)) = (o( - u) and V=((f(()).)) = (a(w, x) y)ur.
(21)
Since the label y {0, 1) of each data sample æ is fixed, and the logits a(w, T x)) and ow x) are in (0, 1), the product of scalars aw, J x)) y and aw x) y is always positive. This further implies that the angle between gradients V=((fg()),) and V=l(f(()), y) equals the angle 8' between weight vectors wy and wi- In addition, the angle 0' bet weight vectors is consistent across data samples. Therefore, we further have:
v=l(f((z)))) ee. vo((I(().v) = ((l((()))) coll'"..
(22)
(f) w)
Plugging Equation 22 into Equation 20 and taking the expectation over D, we have:
Epllg-1(f fi, x)
(23)
= VV v) v) costf (EDDll/a v)ll + Ep [Rz R.
Rearranging the torms in Equation 23
we have:
cEp V (f + Ep Rg (xndv) 21,2( Epleg-r1 fi, xx, v)1
coso'
elE V ae (f((2))))))
(24)
+ Epl.g (f fl. vi]
Then, taking the arccos operator completes the proof.
Proposition 1. Under the settings in Section 4.1 assume the composition f of the activation function o and the loss function & is p- Lippshiizz, we have:
|||
1 |ll S ||w; wyll
(25)
and
Epll V., (f((x). v)ll Epl[ w, (xf(()) |I|| S |Ep||æ=1||b - w,y.
(26)
18

---

Published in Transactions on Machine Learning Research (03/2024)
Proof.
With the triangle inequality, we have:
| = | + W. wgl
||0g || ||:0 wgl
(27)
and
[0 g
||ug + wt w/l
|| | + 1|wg - will
(28)
Since 1|2, - will = ww wgll moving || and ||ai||| to the hand-hand- in Equations respectively, we have ||| ||
27 and
28
wgg S || 2ggl.
Similarly, we have:
VI ((i(())))
= V
- ( V {ffg( v)ll
I V w
(29)
x Vw, [1) +
wgl
and
w, (fg(()) (z), f)لl - (v,, v)1l +
w, wi ||-
(30)
Recalling that ||.g will = || 8. combining Equations 29 and 30 results in:
[V ((g(()) vl V , (fi v)1l Vw, (xfg(a)) v)
( م |||ج||||
will-
(31)
Taking the expectation and absolute value over both sides of Equation 31 completes the proof.
D Detailed Experimental Setting
D.1 Data Partition
We distribute the MNIST, Coil20, and BAR datasets across 50 clients. Each client has data samples from 5 different classes. These data samples are further grouped into two environments according to their spurious correlations. Each client randomly picks one environment for training. Local datasels are further partitioned to train/velr hatie//aaiiaiion/teee /est set with a ratio of 72:8:20, following prior work (Li et al. 2021 Combining test sets from the two environments gets test sets in global environments.
For the CelebA dataset, let each client represent 20 celebrities. One celebrity only appears one client. The blond hair correlates with the female in the training environment and correlates with the male in the other environment that is not used for training. Note that non-blond hair colors do not correlate with any gender. In the personalization step, we select the clients with more than 5 blond female data samples in the training environment and more than 5 blond male data samples in the other environment that not used for training. We select these clients because they provide enough samples to create the spurious correlation and evaluate entanglements to the spurious correlation.
D.2 Hyper- saper-parameters
We use Adam optimizer [Kingma & Ba 2015) throughout our experiments with a learning rate of 1e-4 for MNIST, CelebA, and BAR and 2e-4 for Coil20. Although stochastic gradient descent (SGD) optimizer more common in vision-related tasks, the Adam optimizer always leads lower accuracy disparity. We train the global model for 500 rounds. 5 clients selected per round, each performing 5 epochs of local updates. We tune the coefficients of the adversarial transferability and L2 regularization terms from {0.01, 0.1, 1.0, 10.0) and select the largest value that does not decrease the validation accuracy during penalization. We start the attack budget at 0.031 (i.e., 255) and gradually decrease it such that 30%-500 50% of the al lack succeeds. A large budget will make the attack too strong and push the adversarial examples far across the decision boundary, making the regularization method less effective. We configure c 0.031/0.20 00.01/0.0031 /0.031 for MNIST CCt//C/ll//oll//// respectively. We fine-tune the global model for epochs on LNNIT/BBAK and 10 epochs on Coil20/ /CelebA, which are sufficient. for the personalized models to converge. The clients with the most data samples fine-tune
8
19

---

Published in Transactions on Machine Learning Research (03/2024)
the penalized models for total of 80/40/30 batches MNIST MINIST/CeleAA/Col22 datasets. Note that may not select the personalized model with the most fine-tuning batches for reporting. In the up-weighting (UW) and just-train-twier (JTT) method, we up-sample the residual by a factor of 50. In Ditto, we tume its X from (0.1,1.0}
D.3 Neural Network Architecture
We use CNN 28x28 for the MNIST dataset and CNN 64x64 for the CelebA and the Coil20 datasets, as are listed in Table 5 We use a pre-trained ResNet-18 (He 2016) for the BAR dataset.
Table 5: Neural Network Architecture
CNN 28 x 28
CNN 64 x 64
Iuput: 23x28x28
Input: 33x64664
4x4 cony, 64 LReLU, stride 2
4x4 conv, BN LReLU, stride 4x4 conv, 64 BN LReLU, stride 2 FC 4096 ReLU
4x4 conv, 128 BN LReLU, stride
FC 4096 ReLU
FC 10
FC 10
E
More Experimental Results
E.1
Transferability Correlates with Disparity
The additional empirical studies on Coil20 and CelebA datasets are shown in Figure 6
1.00
1.0
1.0
Acc Disparity
Disparity
Disparity
0.75
Acc Acc'
Acc' AdvTrans
Acc Acc' AdvTrans
0.50
0.5
0.5
AdvTrans
0.25
0.00
0.0
12 16
2
4
2
4
Number of Batch
Number of Batch
Number of Batch
(a) MNIST
(b) Coil20
(c) CelebA
Figure 6: As the personalized models entangle the spurious feature and increase their accuracy disparity (Acc - Acc'), the adversarial transferability (AdvTrans) decreases. Here, the accuracy disparity deviation also increases as the accuracy disparity of personalized models deviates from that the global model, which is fixed as a reference.
E.2 Simulations
We construct synthetic dataset where y [0, 1], = [x,, and x, (2(2y-1,, 1) and x, (c(2y-11)) where the is if the client index mods 2 is 0. Otherwise, the e -1 (Figures 7a and 7ch. In the experiments shown in Figures 7b and 7d. the is 1 if the client index mods 2 is O; otherwise, the e is 0. We employ a linear model w æ where w Cat (w,, w.).
Our simulation starts with the global model. We use the federated averaging algorithm with 50 clients, each with 128 data samples, and select clients per round. Figure shows that the gradient Iw. of uz noisier and closer than the gradient Iw, of w, As result, w, increases during the training while w, does not increase increases much slower, is shown in Figure These results suggest that when the spurious
20

---

Published in Transactions on Machine Learning Research (03/2024)
correlation varies across clients, the model does not entangle the spurious feature or only slightly entangle the spurious feature.
0.02
2.0
2.0
0.01
1.5
1.5
W65
0.00
0.00
1.0
-0.01
W.T.
W r
-0.02
Q_W_5
0.0
50
100
50 Round Index
100
0
50
75
100
25
50
75
100
Round Index
Round Index
Round index
Spurious Correlation
(b) 50% Users Have Sta-(c) Spurious Correlation
(d) 50% Users Have Sta-
Shifts Across Users
tionary Spurious Correlation Shifts Across Users
tionary Spurious Correlation
Figure 7: The gradients and weights of a linear model during training. The gradient Iv, of w. is more noisy and closer to 0 than the gradient 9w, of w,. Therefore, w, stays around 0.
Then, we personalize the global model whose w [2.0, 0.0) on clients with shifting spurious correlations. Results Figure suggest that our approach limits wy around with minimal fluctuation.
2.0
1.5
1.5
1.5
1.0
: 1.0
W.F
M.
0.5
0.5
0.5
0.0
0.0
0.0
10
10 Number of Batch
5
15
10 Number of Batch
Number of Batch
Number of Batch
Naive Fine-tuning
(b) Consistent Prediction
(c) Aligning Weights
(d) Ours
Figure 8: Weights of linear model during fine-tuning. Our approach reduces the co-efficiency w, of the spurious feature F, most effectively. Aligning weights also limits w, but has a more significant. fluctuation than ours.
E.3 Personalization with Imperfect Global Model
Our approach still bounds the disparity deviation of personalized models when the global model already entangles spurious features. Specifically, for global models with accuracy disparities of 0.5% and 2.1% MNIST, the accuracy disparity of personalized models is 8.1% and 12.4%, respectively. In contrast, fine-tuning increases both disparities to 28.5%.
E.4 Robustness Adversarial Attack
Although we add adversarial examples with flipped labels during personalization, the robustness of personalized models from approach does not degrade significantly. We evaluate our approach against transferable adversarial examples from the global model and direct adversarial attacks on personalized models with attack configurations in Appendix D Our approach only increases the success rate of transferable attacks from 56.1% to 68.2% and that direct attacks from 47.8% to 50.1% on MNIST. Both transferable attacks use the global model to generate adversarial examples. The success rate of transferable attacks is not 100% because personalized models are trained with different benign and adversarial samples locally. Transferable attacks can have higher success rates than direct attacks since only the adversarial examples that "fool" global models are transferred.
21


Reviews:
[{"review_id": "0Nk9VcrZl0", "reviewer": "Reviewer_x621", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- This paper is tackling a novel problem.\n- The proposed method seem to achieve stronger empirical performance compared to prior baselines.\n\nWeakness:\n- The paper is quite hard to read. There are a lot of names and concepts in Section 1 and 3. For example, what does it mean to say 'two environments whose combination is the global environment'? What is the formal definition of entanglement and environment? A lot of the definitions are in Section 4, which causes a lot of troubles for readers to understand the paper. Some other examples include not defining $W_1$ until its last appearance in the appendix. From my point of view, the presentation needs a lot of improvement.\n- The motivation of problem setup is a bit unclear to me. It seems that the authors want to build some connection between personalization and distribution shift. However, the authors seem to implicitly assume that although the distribution shift for each client individually, the global distribution remains the same. This seems to be an assumption that is too strong (e.g. I don't think that's typically true for temporal distribution shift). Could the authors provide a few examples / justifications on why such scenario is practical?\n- Fig 1a seems to be a natural outcome of ERM: the model wouldn't generalize to out-of-distribution samples under ERM objective. Personalization just seems to be a special case of this. As a result, a large body of discussion with prior efforts on distributional robust optimization is missing. I'm also curious about how applying vanilla DRO training while finetuning could affect the behavior.\n- Theoretical results are hard to parse. What do we learn from Theorem 1 and 2 and how do they help in designing the method in Section 5. These details seems to be missing.\n- The algorithm is extremely costly as it requires to compute the adversarial example at every iteration. This is equivalent to the time of federated adversarial training. The training time could be multiple times more than that of Ditto, which similarly uses the L2 regularization term but not the adversarial example.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors propose studying a new problem in federated learning, namely how personalization relates to distributional robustness in federated learning. The authors empirically show how personalization can overfit to locally spurious features and how this relate to adversarial transferability. Based on these observations, the authors propose new personalization method to overcome such difficulty.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Improve the presentation to clearly write out the setup, motivation of setup, and definitions.\n- Discuss how this work related to DRO research and DRO research's application to personalized FL.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QH5jSkeIvG", "reviewer": "Reviewer_VDA3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The relation between adversarial transferability and vulnerability to spurious features in FL is interesting and less studied in the literature,\n2. The paper makes an effort to provide theoretical insights of the proposed solution. \n\nWeaknesses & questions:\n\n1. Mathematically, the entanglement of a model $w$ to the spurious feature is defined as $w_s$ with $||w_s|| = ||w||cos(\\theta)$. I am not sure such quantification is reasonable. More justification is helpful. \n2. All theoretical results are limited to the logistic regression model, I wonder whether it can be extended to more general settings such as generalized linear models or non-linear models. If not, what are the challenges? \n3. Theorem 2 gives an upper bound of $\\theta’$ using arccos function. It is not clear to me if the upper bound is valid. Because the domain of arccos is [-1,1], is the sum of terms inside the parenthesis within [-1,1]? \n4. It is not clear to me how the theoretical upper bounds are leveraged for algorithmic design to help generalization. Because the bounds are functions of Lipschitz constant $\\rho$, distribution shift, the norm of model parameter, angles between parameters, all are difficult to control. How are these bounds used in the paper?\n5. The paper only mentions the related work very briefly in section 2, the comparison is not discussed sufficiently. Moreover, the method is only compared with a few algorithms in the literature, including UW and JTT proposed in 2020, Ditto proposed in 2021. I believe many more personalized methods have been proposed in 2022 and 2023. It is important to compare the proposed method with the state-of-the-art methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the connections between personalization and vulnerability to spurious features in FL. It shows that the personalization strategy such as fine-tuning may worsen the local models by making them more vulnerable to spurious features, which further hurt the generalizability. By exploring the relation between adversarial transferability and the entanglement to spurious features, the paper proposes a method based on adversarial samples that mitigate the effect of spurious features during personalization.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. add more literature reviews and experiments to compare with SOTA methods\n2. add more interpretation for the theoretical modeling and the bounds, especially how the bounds are related to the algorithmic design\n3. see Weakness section", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "132VySJi92", "reviewer": "Reviewer_rNGy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1.\tThe motivation is clear and the route is sensible.\n2.\tThe completeness of the paper is good, not only has the experiment but also provides the corresponding theoretical proof.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work addresses the challenge of personalizing federated learning models, where fine-tuning global models for local clients can lead to reduced generalization performance due to spurious features at local clients. In this setup, the spurious features correlate with the label in each client's training environment, but the global environment diminishes these correlations. While global federated learning models suffer less from spurious features, local fine-tuning may reintroduce vulnerability to them. To mitigate this, the authors propose a novel strategy that maintains adversarial transferability between global and personalized models. Empirical results on object and action recognition tasks demonstrate that this approach prevents personalized models from exploiting spurious features while retaining the benefits of accuracy enhancement through fine-tuning.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tThe comparative experiments in this paper are not sufficient.", "broader_impact_concerns": "No ethical concerns have identified for this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In my own reading of the paper, I also found the paper hard to follow. For example, concepts like "adversarial transferability loss" and "adversarial transferability accuracy" are not intuitive, easy to mix and confusing. I found the "adversarial transferability loss" especially confusing, as I would have expected a high value of "adversarial transferability" to indicate high level of transferability, and the word "loss" is simply confusing in this context.

Based on this, I am recommending acceptance with minor revision subject to clarifying the presentation. This would include at least:
1. Make sure that all terminology is clearly explained. If possible, avoid confusing terminology like "adversarial transferability loss/accuracy".
2. Make sure that responses to all relevant questions from the reviewers are reflected in the paper contents. Reviewer x621 in particular raised concern that the relation to DRO, which the reviewer considered crucial for the paper, is not discussed in the text.

When revising the paper, I would like to remind the authors that TMLR **does not have a strict page limit**, so space limitation is not an excuse for omitting some important material or presenting things unclearly.
Decision: Accept (poster)