A ct P er FL : Active Personalized Federated Learning (2022)

Abstract: In the context of personalized federated learning (FL), the critical challenge is to balance local model improvement and global model tuning when the personal and global objectives may not be exactly aligned. Inspired by Bayesian hierarchical models, we develop ActPerFL, a self-aware personalized FL method where each client can automatically balance the training of its local personal model and the global model that implicitly contributes to other clients’ training. Such a balance is derived from the inter-client and intra-client uncertainty quantification. Consequently, ActPerFL can adapt to the underlying clients’ heterogeneity with uncertainty-driven local training and model aggregation. With experimental studies on Sent140 and Amazon Alexa audio data, we show that ActPerFL can achieve superior personalization performance compared with the existing counterparts.

## introduction federated learning (fl) @xcite is transforming machine learning (ml) ecosystems from "centralized in-thecloud" to "distributed across-clients," to potentially leverage the computation and data resources of billions of edge devices @xcite , without raw data leaving the devices. as a distributed ml framework, fl aims to train a global model that aggregates gradients or model updates from the participating edge devices. recent research in fl has significantly extended its original scope to address the emerging concern of personalization, a broad term that often refers to an fl system that accommodates client-specific data distributions of interest @xcite . in particular, each client in a personalized fl system holds data that can be potentially non-iid. for example, smart edge devices at different houses may collect audio data of heterogeneous nature @xcite @xcite due to, e.g., accents, background noises, and house structures. each device hopes to improve its on-device model through personalized fl without transmitting sensitive data. while the practical benefits of personalization have been widely acknowledged, its theoretical understanding remains unclear. existing works on personalized fl often derive algorithms based on a pre-specified optimization formulation or model aggregation rule. in this work, we start with a toy example and develop insights into the nature of personalization from a statistical uncertainty perspective. in particular, we aim to answer the following critical questions regarding personalized fl. both questions are challenging. the question (q1) demands a systematic way to characterize the client-specific and globally-shared information. to this end, we draw insights from a simplified and analytically tractable setting: two-level bayesian hierarchical models, where the two levels respectively describe inter-client and intra-client uncertainty. we make the following technical contributions: to our best knowledge, actperfl is the first work that utilizes uncertainty quantification to drive fl personalization. from the server's perspective, it is postulated that data z 1 , . . . , z m are generated from the following two-layer bayesian hierarchical model: for all clients with @xmath0, . . . , m . here, σ 2 0 is a constant, and θ 0 ∼ π 0 (•) is a hyperparameter with a non-informative flat prior. the above model represents both the connections and heterogeneity across clients. in particular, each client's data are distributed according to a client-specific parameter (θ m ), which follows a distribution decided by a parent parameter (θ 0 ). the parent parameter is interpreted as the root of shared information. without loss of generality, we study client 1's local model as parameterized by θ 1 . under the above model assumption, the parent parameter θ 0 that represents the global model has a posterior distribution from the perspective of client m, we suppose that the postulated model is the same as above for @xmath1, . . . , m , and θ 1 = θ 0 . it can be verified that the posterior distributions of θ 1 without and with global bayesian learning are p( the first distribution above describes the learned result of client 1 from its local data, while the second one represents the knowledge from all the clients' data in hindsight. using the mean square error as risk, the bayes estimate of θ 1 or θ 0 is the mean of the posterior distribution, namely θ (l) 1 and θ (fl) ## proposed solution for personalized fl our proposed actperfl framework has three key components as detailed in this section: (i) proper initialization for local clients at each round, (ii) automatic determination of the local training steps, (iii) discrepancy-aware aggregation rule for the global model. these components are interconnected and contribute together to actperfl's effectiveness. note that points (i) and (iii) direct actperfl to the regions that benefit personalization in the optimization space during local training, which is not considered in prior works such as ditto @xcite and pfedme @xcite . therefore, actperfl is more than imposing implicit regularization via early stopping. in this section, we show how the posterior quantities of interest in section 2 can be connected with fl. recall that each client m can obtain the floptimal solution θ (fl) m with the initial value θ init m in eqn. (6) and tuning parameters η, ℓ in eqn. (7). also, it can be shown that θ init m is connected with the global-optimal θ (g) in eqn. (1) through in general, the above σ 2 0 , σ 2 m represent "inter-client uncertainty" and "intra-client uncertainty," respectively. when σ 2 0 and σ 2 m 's are unknown, they can be approximated asymptotically or using practical finite-sample approximations. sgd-based practical algorithm for dl. for the above training method, the quantities σ 2 0 and σ 2 m are crucial as they affect the choice of learning rate η m and the early-stop rule. however, these two values are unknown in complex learning models. to approximate the uncertainty quantities, we generally treat σ 2 m as "uncertainty of the local optimal solution θ (l) m of client m", and σ 2 0 as "uncertainty of clients' underlying parameters." assume that for each client m, we had u independent samples of its data and the corresponding local optimal parameter θ m,1 , . . . , θ m,u . we could then estimate σ 2 m by their sample variance. in particular, at round t, we approximate σ 2 m with: likewise, at round t, we estimate σ 2 0 by: for multi-dimensional parameters, we introduce the following counterpart uncertainty measures. for vectors x 1 , . . . , x m , their empirical variance is defined as the trace of @xmath2, which is the sum of entry-wise empirical variances. ## experimental studies experimental setup. we evaluate actperfl's performance on two nlp datasets: sentiment140 @xcite and private amazon alexa audio data. sent140 is a text sentiment analysis dataset with two output classes and 772 clients. we generate non-i.i.d. data following fedprox @xcite . the audio dataset is collected for wake-word detection task (i.e., binary classification). this dataset contains 39 thousand hours of training data and 14 thousand hours of test data. we use a two-layer algorithm 1 active personal fl (actperfl) input: a server and m clients. communication rounds t , client activity rate c, client m's local data d m and learning rate η m . for each communication round @xmath3, . . . t do sample clients: m t ← max(⌊c • m ⌋, 1) for each client @xmath4 lstm model and an 11-layer cnn model for these two datasets, respectively. for comparison, we also evaluate the personalization performance of fedavg @xcite , ditto @xcite , perfedavg @xcite , and pfedme @xcite . ## concluding remarks we proposed actperfl to address the challenge of balancing local model training and global model aggregation in personalized fl. our solution adaptively adjusts local training with automated hyper-parameter selection and performs uncertainty-weighted global aggregation. empirical studies show that actperfl can achieve promising performance on nlp applications.