PeFLL: Personalized Federated Learning by Learning to Learn
Authors: Jonathan Scott
Hossein Zakerinia
Christoph H Lampert
Venue: ICLR 2024 poster (2024)
Keywords: Personalized Federated Learning
Learning-to-Learn
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: PeFLL is a hypernetwork-based approach for personalized federated learning. Based on a learning-to-learn approach it efficiently generates accurate individual models for current as well as future clients.

Abstract: We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the low-data regime, and not only for clients present during its training phase, but also for any that may emerge in the future; 2) it reduces the amount of on-client computation and client-server communication by providing future clients with ready-to-use personalized models that require no additional finetuning or optimization; 3) it comes with theoretical guarantees that establish generalization from the observed clients to future ones. 
At the core of PeFLL lies a learning-to-learn approach that jointly trains an embedding network and a hypernetwork. The embedding network is used to represent clients in a latent descriptor space in a way that reflects their similarity to each other. The hypernetwork takes as input such descriptors and outputs the parameters of fully personalized client models. In combination, both networks constitute a learning algorithm that achieves state-of-the-art performance in several personalized federated learning benchmarks.

Published as a conference paper at ICLR 2024
PEFLL: PERSONALIZED FEDERATED LEARNING BY LEARNING TO LEARN
Jonathan Scott
Hossein Zakerinia
Institute of Science and Technology Austria (ISTA) Jonathan. scott @ st.tss..
Institute of Science and Technology Austria (ISTA) hossein, n. zakeri n iiaaiaa atist.....tttae...tt...iiiassase ac.at
Christoph H. Lampert
Institute of Science and Technology Austria (ISTA) ch1@ist.c ac.at
ABSTRACT
We present PeFLL,, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the low-data regime, and not only for clients present during its training phase, but also for any that may emerge in the future; 2) it reduces the amount of on-client computation and client-server communication by providing future clients with ready-to-use personalized models that require no additional finetuning or optimization; 3) comes with theoretical guarantees that establish generalization from the observed clients to future ones.
At the core of PeFLL lies a learning-to-lee to-learn approach that jointly trains an embedding network and hypernetwork. The embedding network is used to represent clients in a latent descriptor space in a way that reflects their similarity each other. The hypernetorkk takes input such descriptors and outputs the parameters of fully personalized client models. In combination, both networks constitute learning algorithm that achieves state-of-the-as performance in several personalized federated learning benchmarks.
INTRODUCTION
Federated learning (FL) (McMahan et al., 2017) has emerged as a de-facto standard for privacy- preserving machine learning in a distributed setting, where multitude of clients, e.g., user devices, collaboratively learn prediction models without sharing their data directly with any other party. In practice, the clients' data distributions may differ significantly from each other, which makes learning a single global model sub-optimal (Li et al.) 2020; Kairouz et al. 202I). Personalized federated learning (pFL.) (Smith al. 2017)
of dealing with such statistical heterogeneity, by allowing clients to learn individual models while still benefiting from each other.
A key challenge of personalized federated learning lies in balancing the benefit of increased data that is available for joint training with the need to remain faithful to each client's 's own distribution. Most methods achieve this through a combination of global model training and some form of finetuning or training of a client-specific model. However, such an approach has number of shortcomings. Consider, for instance, a federated learning system over millions of mobile devices, such as Gboard (Hard et al.) 2018). Only fraction of all possible devices are likely to be seen during training, devices can drop out or at any time, and it should be possible to produce models also for devices that enter the system at a later time. Having to train or finetune new model for each such new device incurs computational costs. This is particularly unfortunate when the computation happens the client devices, which are typically of low computational power. Furthermore, high communication costs emerge when model parameters or updates have to be communicated repeatedly between the server and the client. The summary effect is a high latency before the personalized model ready for use. Additionally, the quality of the personalized model unreliable, as it depends on the amount of data available the client, which in personalized federated learning tends to be small.

---

Published as a conference paper at ICLR 2024
In this work, we address all of the above challenges by introducing a new personalized federated learning algorithm. In PeFLL (for Personalized Federated Learning by Learning to Learn), it is the server that produces ready-to-use personalized models for any client by means of a single forward pass through a hypernetwork. The hypernework''s s input is a descriptor vector, which any client can produce by forward passing some of its data through an embedding network. The output of the hypernetwork is the parameters of a personalized model, which the client can use directly without the need for further training or finetuning. Thereby PeFLL overcomes the problems of high latency and client-side computation cost in model personalization from which previous approaches suffered when they required client-side optimization and/or multi-round client-server communication.
The combined effect of evaluating first the embedding network and then the hypernetwork constitutes a learning method: it produces model parameters from input data. The process is parametrized by the networks' weights, which PeFLL trains end-to-end across training set of clients. This learning-to- viewpoint allows for a number of advantageous design choices. For example, PeFLL benefits when many clients are available for training, where other approaches might suffer in prediction quality. The reason is that the client models are created by end-to-end trained networks, and the more clients participate in the training and the more diverse these are, the better the networks' generalization abilities to future clients. Also, PeFLL is able to handle clients with small datasets better than previous approaches. Because the descriptor evaluation is a ferd-forward process, it is not susceptible to overfitting, as any form of on-client training or finetuning would be. At the same time, already during training the client descriptors are computed from mini-batches. thereby anticipating the setting of clients with little data. Once the training phase is concluded, PeFLL can even be used to create personalized models for clients with only unlabeled data.
PeFLL's s strong empirical results, on which we report in Section 4 are not a coincidence, but they can be explained by its roots in theory. Its training objective derived from a new generalization bound, which we prove in Section 3.2 and which yields tighter bounds in the federated setting than previous ones. PeFILL's training procedure distributes the necessary computation between the server and the clients in a way that ensures that the computationally most expensive task happen on the server yet stays truthful to the federated paradigm. The optimization is stateless, meaning that clients can drop out of the training set or enter it at any time. Nevertheless, as we show in Section 3.1 it converges at a rate comparable to standard federated averaging.
RELATED WORK
Personalized federated learning (pFL) is currently one of the promising techniques for learning
models that are adapted to individual clients' data distributions without jeopardizing their privacy.
Existing approaches typically follow one of multiple blueprints: multi-task methods (Smith et al.)
/202e/aazzely Hanzely al.] 20200] Hanzely and Richtárik| 2020: Ye et al. [023] Zhang et al. 2023 Qin et al. (023) learn individual
2017| et al. [202T] Dinh et al., 2022 Li et al.; 202T Dinh et al.] [2020]
per-client models, while sharing information between clients, e.g. through regularization towards
central model. Meta-learning methods learn shared model, which the clients can quickly adapt or finetune to their individual data distributions, e.g. by a small number of gradient updates (Fallah
let al. 2020; Tiang et al., 2019 Wu et al. 2023a). Decomposition-buuse methods (Arivazhagan et al. 2019 Collins et al. 2021] Bui et al.] L et al.] [2020] Chen et al., (202)] Wu et al.) (002b))
split the learnable parameters into two groups: those that are meant to shared between clients (e.g.
a feature representation) and those that are learned on a per-client basis (c.g. classification heads).
Clustering- based methods (Ghosh et al., 2020. Mansour et al. (020) divide the clients into fixed number of subgroups and learn individual models for each cluster.
Hypernetworks (Ha et al.] (017)) have only recently been employed in the context of pFL (Mal et al. 2022: Yi et | 2023). Typically, the hypernetworks reside on the clients, which limits the methods efficiency. Closest to our work are [Shamsian et al. (202T) and [L et al.; (202)), which also generate each clients" personalized model using server-side hypernetworks. These works adopt a non-parametric approach in which the server learns and stores individual descriptor vectors per client. However, such an approach has number of downsides. First, it entails a stateful optimization problem, which is undesirable because it means the server has to know at any time in advance which client is participating in training to retrieve their descriptors. Second, the number of parameters stored at the server grows with the number of clients, which can cause scalability issues in large-scale applications. Finally, in such setting the hypernework can only be evaluated for clients with which
2

---

Published as a conference paper at ICLR 2024
the server has interactive before. For new clients, descriptors must first to be inferred, and this requires an optimization procedure with multiple client-server communication rounds. In contrast, in PeFLL the server learns just the embedding network, and the client descriptors are computed on-the-fly using just forward-evaluations on the clients. As a consequence, clients can drop in or out at any time, any number of clients can be handled with fixed memory budget, and even for previously unseen clients, no iterative optimization is required.
Learning-to-Leam. Outside of federated learning, the idea that learning from several tasks should make it easier to learn future tasks has appeared under different names, such as continual learn-
ing (Ring, 1994, Mitchell al., 2015). lifelong learning (Thrun and Mitchell [995] Chen and Liu 2018; Parisi et al., 2019), learning to learn (Thrun and Pratt] [998] Andrychowicz et al. (0166).
inductive bias learning (Baxter 2000), meta-learning (Vilalta and Drissi 2002] Finn et al. 2017
(202T. In this work, we use the term Learning-ton -toiinggtt---rrn, because it best describes the aspect that the central object that PeFLL learns a mechanism for predicting model parameters from training data, i.e. essentially it learns learning algorithm.
Hospedales et al.
The central question of interest is the learned learning algorithm generalizes, i.e., if it produces good models not only for the datasets used during its training phase, but also for future datasets. Corresponding results are typically generalization bounds in a PAC- -ay-Baaesi framework (Pentina
and Lampert 2014 Amit and Meir 2018; Rothfuss et al., 202T Guan and Lu, 2022 Rezazadel 2022). as we also provide in this work. Closest to our work in this regard is Rezazadeh (2022), which addresses the task of hyperparameter learning. However, their bound is not well adapted to the federated learning setting in which the number of clients is large and the amount of data per client
might be small. For more detailed comparison, see Section 3.2 and Appendix B
3
METHOD
We now formally introduce the problem we address and introduce the proposed PeFLL algorithm. We assume a standard supervised federated learning setting with a (possibly very large) number, n, of clients. Each of these has data set, S, {(x1 vi...... mm V Vnm.) }} C X D for i E {1..... n). sampled from a client-dependent data distribution D. Throughout the paper, we adopt non-i.i.d setting, i.e., the data distributions can differ between clients, D; * Dj for i + j. For any model, 0 E TR.d (we identify models with their parameter vectors), the client can compute its training loss, C(0; -/ Σø ((x), vý. f), where é X x y x Rd + R+ is a loss function. The goal is to learn client-specific models, 0,, in a way that exploits the benefit of sharing information between clients, while adhering to the principles of federated learning.
PeFLL adopts hypernetwork approach: for any client, a personalized model, 0 E Rd, is produced as the output of shared deep network, h IR'' + Rd, that takes as input a client descriptor e E R' To compute client descriptors, we use an embedding network, Φ x x y - TRd which takes individual data points as input and averages the resulting embeddings! (S)) = [TT E(ayles (xx, y). We denote the hypernetwork's parameters as Th and the embedding network's ' parameters as Te- As shorthand, we write 1) (m))).))
Evaluating the embedding network followed by the hypernetwork can be seen as a learning algorithm: it transforms set of training data points into model parameters. Consequently, learning ") can be seen as a form of learning-to-lean. Specifically, we propose the PeFLI (Personalized Federated Learning by Learning to Learn) algorithm, which consists of solving the following optimization problem, where || denotes the L2"norm,
" An Anl1mall1 Am + Σ [(M(o(San):))) mo); S:) + No mo
min Tho
mm):
(I)
PeFLL performs this optimization in a way that is compatible with the constraints imposed by the federated learning setup. Structurally, it splits the objective (T) into several parts: a server objective, which consist of the first two (regularization) terms, f(p, Th) = > An.l7 2| >-mllm and multiple per-client objectives, each of which consists of the terms inside the summation f0, St) = [(O,; S,) + \o| |2.. The terms are coupled through the identity 0, = h(v(S, 7x); m.). This split
This construction is motivated by the fact that e should be permutation invariant (set) function Zaheer 2017), but for the sake of conciseness, one can also take it simply as efficient design choice.
3

---

Published as a conference paper at ICLR 2024
Server
Algorithm PeeLLi-predict
input target client with private dataset S
network Siv
embedding
diient descriptor
model a
personal
1: Server sends embedding network y/ client
2: Client selects a data batch -
3: Client computes v(B;
4: Client sends descriptor to server
Client
Server computes h(v; m))
6: Server sends personalized model 0 to client
Communication protocol of PeFLL- predict for generating personalized models.
Figure 1:
Algorithm PeFLL-trail
Model (personalized, client)
input number of training steps, T
input number clients to select per step, c
Ad,
I: for t 1...., . do
2: P + Server randomly samples clients
3: Server broadcasts embedding network T)
Hypernetwork 1ps (shared, on server)
for client E P parallel do Client selects a data batch B S.
6: U v(B; no), client computes descriptor
7:
Client sends U2 to server
8:
0, h(w;; Mb. server computes personalized model
9:
Server sends
client
10:
8:em + client runs &s steps of local SGD on f((O.)
Embedding network (shared, on client)
11:
Client sends 20, := 6:TH 0g to server
12:
(Av). (1) Av.) <- server backprop with error vector A0,
13:
Server sends Av, to client
Data flow for PeFLL model generation (forward pass, left) and training (backward pass, right). The client descriptor, Vis and the client model 0; are small. Transmitting them and their up- date vectors efficient. The hy- pernetwork, Th can be large, but remains on the server.
Figure 2:
AnE +- client runs backprop with error vector Av;
15:
Client sends Ano server
16: end for
17: Te < 28 mm + P Eier
18:
Ne <- (1 23A,) Mo + IPT EiEr Arp
19:
end for
output network parameters (1p4, 1))
allows PeFLL to distribute the necessary computation efficiently, preserving the privacy of the client data, and minimizing the necessary communication overhead. Pseudocode of the specific steps is provided in Algorithms/ Tand]
We start by describing the PeFLL PeFLL-predice routine (Algorithm [T) which can predict a personalized model for any target client. First, the server sends the current embedding network, Ne. to the client (line which evaluates it on all or subset of its data compute the client descriptor (line B}. Next, the client sends its descriptor to the server (line 4 who evaluates the hypernetwork on it (line 5). client (6n where is ready for use. Overall, The resulting personalized model sent back communication steps are required before the client only two server-to-client and one client-tosserve has obtained a functioning personalized model (see Figure I).
The reFLL-tral raiin routine (Algorithm 2) mostly adopts a standard stochastic optimization pattern federated setting, with the exception that certain parts of the forward (and backward) pass take place on the clients and others take place on the server. Thus PeFLL follows the Split Learning (Gupaa and Raskar 2018) paradigm in Federated Learning (Thapa et al. 2022). In each iteration the server selects batch of available clients (line 2 and broadcasts the embedding model, 7/o, to all of them (line 33). Then, each client in parallel computes its descriptor, v, (6). sends to the server (line 7. and receives personalized model from the server return (line 8 9.. At this point the forward pass is over and backpropagation starts. To this end, each client performs local SGD for steps on its personalized model and personal data (line It sends the resulting update, A0,, the According the chain rule,
server (line 11 where it acts as a proxy for 0f. 36, 0f: 0/: 00. The server evaluate both expressions using backpropagation (line
0h.
0f: 00: and
9mh
89; 8nj.
12 because all
Bu,
88, de:
4

---

Published as a conference paper at ICLR 2024
required expressions are available to it now. It obtains updates Am/y ())
and Ae;, the latter of which it
sends to the client (line 13) as a proxy for 04, おहe" Again based on the chain rule Cmp= De. Dn. ) the
0f. del)
client computes an update vector for the embedding network, Anj (line 14), and sends it back to the
(t)
server (line [5). Finally, the server updates all network parameters from the average of per-client
contributions as well as the contributions from the server objective (lines 17 18
The above analysis shows that PeFLL has a number of desirable properties: 1) It distributes the necessary computation in a client-friendly way. The hypernetwork is stored on the server and evaluated only there. It is also never sent via the communication channel. This important, because hypernetwrrks can be quite large, as already their output layer must have as many neurons as the total number of parameters of the client model. Therefore, they tend to have high memory and computational requirements, which client devices might not be able to provide.
2) It has low latency and communication cost. Generating a model for any client requires com- municating only few and small quantities: a) the parameters of the embedding network, which are typically small, b) the client descriptors, which are low dimensional vectors, and c) the personalized models, which typically also can be kept small, because they only have to solve client-specific rather than general- cenrrol-purpose purpose tasks. For the backward pass in the training phase, three additional quantities need to be communicated: a) the clients' suggested parameter updates, which are of equal size the model parameters, b) gradients with respect to the client descriptors, which are of equal size as the descriptors, and c) the embedding network's updates, which are of the same size as the embedding network. All of these quantities are rather small compared to, e.g., the size of the hypernetwork, which PeFLL avoids sending.
3) It respects the federated paradigm. Clients are not required to share their training data but only their descriptors and-during training- the gradient of the loss with respect to the model parameters. Both of these are computed locally on the client devices. Note that we do not aim for formal privacy guarantees in this work. These could, in principle, be provided by incorporating mulli-parly computation (Evans et al. (018)) and/or differential privacy (Dwork and Roth, 20014). Since a descriptor is an average a batch of embeddings, this step can be made differentially private with some added noise using standard DP techniques such as Gaussian or Laplace mechanisms. This, however, could add the need for additional trade-offs, so we leave the analysis to future work.
3.1 CONVERGENCE GUARANTEES
We now establish the convergence of PeFLL's training procedure. Specifically, we give guarantees in the form of bounding the expected average gradient norm, as is common for deep stochastic optimization algorithms. The proof and full formulation can be found in Appendix C
Theorem 3.1. Under standard smoothness and boundedness assumptions (see appendix), PeFLL's optimization after T steps fulfills
1 T Σ BIIFF(N) (F.) + I(607+4h78) + 224cI36703 VCT
8bR03 14T70303 + b +
(2)
kveT
T
b
where F is the PeFl. objjective ① with lower bound F.. To are the parameter values at initialization, ",....,Tr TIr are the intermediate parameter values. L., In are smoothness parameters of F and the local models. by, b2 are bounds on the norms of the gradients of the local model and the hypernetwork, respectively, ai is bound on the variance of stochastic gradients of local models, and a2, are bounds on the variance due to the clients generating models with data batches of size b instead of their whole training set. "G is a bound on the dissimilarity of clients, is the number of clients participating at each round, and k the number of local SGD steps performed by the clients.
The proof resembles convergence proofs for FedAvg with non-i.i.d clients, but differs in three aspects that we believe makes our result of independent interest: 1) due to the nin-linyarity of the hypernetwork, gradients computed from batches might not be unbiased; 2) the updates the network parameters are not simple averages, but consist of multiple gradient steps on the clients, which the server processes further using the chain rule; 3) the objective includes regularization terms that only the server can compute.
Note that all of these operations are standard first-order derivatives and matrix multiplications. In contrast to some meta-learning algorithms, second-order derivatives, such gradients of gradients, are required.
5

---

Published as a conference paper at ICLR 2024
Discussion Theorem 3.1 characterizes the convergence rate of PeFLL's optimization step in terms of the number of iterations, T, and some wroblem-speciffe constants. For illustration, we first discuss the case where the client descriptors are computed from the complete client dataset (B S; in Algorithm|] [ line 5). In that case, da and d3 vanish, such that only three terms remain in (2) The first two are of order vT. while the third one is of order T For sufficiently large T, the first two terms dominate, resulting in the same order of convergence as FedAvg (Karimireddy et al. 2020). clients compute their descriptors from batches (B Ç S,), two additional variance terms emerge in the bound, which depend on the size of the batches used by the clients to compute their descriptors. It is always possible to control these terms, though: for large S,, one choose B sufficiently large to make the additional terms as small as desired, and for small S,, setting B = S; is practical, which will make the additional terms disappear completely, see above.
3.2 GENERALIZATION GUARANTEES
In this section, we prove a generalization bound that justifies PeFLL's learning objective. Following prior work in learning-to-lcs -e-ar theory adopt PAC-Bayesian framework. For this, we assume that the embedding network, the hypernetwork and the personalized models are all Iearned stochastically. Specifically, the result of training the hypernetwork and embedding network distributions over their parameters, Qn N (m) Id), and Q. = N ((),; Id, for some fixed Orp G2 > 0. The mean vectors 77,, and th are the learnable parameters, and Id is the identity matrix of the respective dimensions. The (stochastic) prediction model for a client with dataset S is obtained by sampling from the Gaussian distribution Q = (0; agld) for 0- = h(v(S; m)); 7m)) and some ax > 0.
Gaussian probability
We formalize clients tuples (D,, where D, is their data distribution, and S, i.ddd. D, is training data. For simplicity, we assume that all data sets are of identical size, m. We assume that both observed clients and new clients are sampled from a client environment, T (Baxter (2000). that is, probability distribution over clients. Then, we can prove the following guarantee of generalization from observed to future clients. For the proof and more general form of the result, see Appendix B
Theorem 3.2. For all ó > the following statement holds with probability at least ô over the randomness of the clients. For all parameter vectors, π (Mi No):
n S
E
(x, y,
E m A.
(Z,
(D,Sy~ (a,9)~ D AI On Bar
(a.y))
17
2 log(
1
2ag 1= [1((v(S no)s 12 log(
8mn
)+1
+1
20h
20
E
2n
1A 2
2mn
Th ~
Discussion Theorem 3.2 states that the expected loss of the learned models on future clients (which is the actual quantity of interest) can be controlled by the empirical loss on the observed clients' data (which we can compute) plus two terms that resemble regularizers. The first term penalizes extreme values in the parameters of the hypernetwork and the embedding network. Thereby, it prevents overfitting for the part of the learning process that accumulates information across clients. The second term penalizes extreme values in the output of the hypernetwork, which are the parameters of the per-client models. By this, it prevents overfitting on each client. Because the guarantee holds uniformly over all parameter vectors, we can optimize the right hand side with respect 1 and the guarantee will still be fulfilled for the minimizer. The PeFLL objective is modeled after the right hand side of (3) to mirror this optimization in simplified form: we drop constant terms and use just the mean vectors of the network parameters instead of sampling them stochastically. Also, we drop the square roots from the regularization terms to make them numerically better behaved.
Relation to previous work A generalization bound of similar form as the one underlying Theo- rem 3.2 appeared (Rezazadeh) 2022, Theorem 5.2). That bound, however, is not well suited the federated setting. First, it contains term of order (x+mm) log m/nn which is not necessarily small for large n (number of clients) but small m (samples per client). In contrast, the corresponding terms in our bound, log and log are both small in this regime. Second, when instantiating the bound from Rezazadeh] (2022) an additional term of order J |lal| would appear, which can be large in the case where the dimensionality of the network parameters large but m small.
nm nm

---

Published as a conference paper at ICLR 2024
4 EXPERIMENTS
In this section we report on our experimental evaluation The values reported in every table and plot given the mean together with the standard deviation across three random seeds.
For our experiments, we use three datasets that are standard benchmarks for FL: CI- FARTO/CIFAR (Krizhevsky) 2009) and FEMNIST (Caldas et al.] 2018). Following prior pFI. works, for CIFARIO and CIFAR1OG we form statistically heterogeneous clients by randomly assign- ing a fixed fraction of the total number of C' classes to each of clients, for n E {100, 500, 1000). The clients then receive test and train samples from only these classes For CIFAR 10 each client has 2 of the C= 10 classes and for CIFARI00 each client has 10 of the C 100 classes. FEMNIST is a federated dataset for handwritten character recognition, with C = 62 classes (digits and lower/upper case letters) and 817,851 samples. We keep its predefined partition inoo 3597 clients based on writer identity. We randomly partition the clients into 90% seen and 10% unseen. Seen clients are used for training while unseen clients do not contribute to the initial training and are only used to later assess each method's performance new clients described below. Additional experiments on the Shakespeare dataset (Caldas et al. 2018) are provided in Appendix
Datasets
Baselines We evaluate and report results for the following pFL methods, for which we build on the FL-Bench repository Per-FedAvg (Fallah et al. 2020), which optimizes the MAML (Finn et al. 2017) objective in federated setting; FedRep (Collims et al. (022I) which trains global feature extractor and per-client classifier heads; pFedMe (Dinh et al. 2020). which trains a personal model per client using regularization term penalize differences from global model; kNN-Per (Marfog eet al. 2022), which trains a single global model which each client uses individually to extract features of their data for use in
wewtrrettnnighbb--abeseeeee classifier; pFedHN (Shamsian et al. 202 which jointly trains a hypernetwork and per client embedding vectors to output a personalized model for each client. For reference, we also include results of (non-ped coon-prsonnllzed) FedAvg (McMahan et al., 2017) and of training a local model separately on each client.
Constructing models for unseen clients We are interested in the performance of PeFLL not just on the training clients but also on clients not seen at training time. As described in Algorithm [ inference on new clients is simple and efficient for unseen clients as it does not require any model training. by either the client the server. With the exception of KNN-Per all other methods require some form of finetuning in order to obtain personalized models for new clients. Per-FedAvg and PFedMe obtain personal models by finetuning the global model locally at each client for some small number of gradient steps. FedRep freezes the global feature extractor and optimizes a randomly initialized head locally at each new client. pFedHN freezes the trained hypernetwork and optimizes a new embedding vector for each new client, which requires not just local training but also several communication rounds with the server. The most efficient baseline for inference on new client is kNN-Per, which requires only a single forward pass through the trained global model and the evaluation of a menearrst-nnighoor
predictor.
Models Following prior works in pFL the personalized model used by each client is a LeNet-style model (eecun et al., 1998) with two convolutional layers and three fully connected layers. For fair comparison we use this model for PeFLL as well as all reported baselines. PeFLL uses an embedding network and hypernetwork to generate this personalized client model. For our experiments the hypernetwork is three layer fully connected network which takes as input client descriptor vector, V E TRL, and outputs the parameters of the client model, E TRdd. For further details, see Appendix A Note that the final layer of the client model predicts across all classes in the dataset. For the embedding network we tested two options, a single linear projection which takes as input a one-hot encoded label vector, and LeNet-type ConvNet with the same architecture the client models except that its input is extended by C extra channels that encode the label in one-hot form. The choice of such small models for our embedding network is consistent with the fact that the embedding network must be transmitted to the client. We find that for CIFARIO and FEMNIST the ConvNet embedding network produces the best results while for CIFARI00 the linear embedding is best, and these are the results we report.
Training Details We train all methods, except Local, for 5000 rounds with partial client partic- ipation. For CIFARIO and CIFAR 100 client participation is set to 5% per round. For FEMNIST
'We provide the code as supplemental material. We will publish it when the anonymity requirement is lifted. hhttps://gt thub. com/ / Ka rhou'/am/ I.
7

---

Published as a conference paper at ICLR 2024
CIFARIO
CIFAR100
FEMNIST
#tm.clients
n
450
900
450
900
3237
Local
82.2 = 0.6
70.9 + 0.5
65.5 0.7
39.4 0.2
19.7 +
11.3
62.2 = 0.1
FedAvg
47.5 0.5
50.4 0.6
51.9 0.7
16.4 0.4
20.2 0.1
20.2
82.1
Per-FedAvg
79.1 = 2.1
76.9 0.9
7..61...3
40.0
31.1 0.2
2.1
22.700.1
FedRep
84.6 = 0.8
79.2
0.3
42.7 0.8
35.5
30.8
83.6 - 0.8
pFedMe
83,9 = 1.2
7.0
$.9
39.6
0.5
40.5
0.4
34.6
85,9 - 0.8
kNN-Per
84.0 = 0.3
82.0 0.5
42.2
+ 0.9
38.1 + 0.4
34.5
0.5
85.2 - 0.3
pFedHN
87.8 0.5
77.4 1.3
66.0
1.1
53.6
+ 0.2
25.5
+ 0.3
20.0 - 1.2
83.8 - 0.3
PeFLL
89.0 1 0.8
88.9 1 0.4
87.8 1 0.4
56.0
0.3
43.2
1 0.8
40.9 = 0.6
90.1 + 0.1
(a) Accuracy clients observed at training time (best values per setup in bold)
CIFARIO
CIFARTO0
FEMNIST
#tm.clients
n 90
450
900
90
450
900
3237
Local
81.6 = 0.7
70.6 = 1.8
65.5 = 0.5
38.7 + 0.9
19.9 = 1.1
11.0
= 0.5
62.1 = 0.2
FedAvg
48,6 0.3
49.6 + 1.0
51.7 1 0.7
17. 1 0.9
19.3 1 2.0
20.5 1: 1.0
81.9 + 0.4
Per-FedAvg
77.6 1.9
69.0 + 2.5
67.3=00
35.6 + 1.5
30.6 + 0.4
20.5 + 0.5
81.1 + 1.5
FedRep
81.8 1.0
77.4 1.7
76.0
38.6 + 1.1
35.9 + 1.5
31.8 1.6
82.8 = 0.7
PFedMe
78.4 2.2
74.3 = 1.4
72.3
31.1 + 1.4
32.3
1 0.5
28.9 - 0.2
86,1 = 0.4
kNN-Per
82.4 0.7
80.8 = 1.5
78.4 1.1
41.8 1.2
37.2
33.7 = 0.7
84.6 = 0.6
PFedHN
63.3 3.7
60.9 = 2.4
57.8 2.0
24.1 +
1.2
21.5 1.4
20.8
1.2
82.5 = 0.1
PeFLL
89.3 1 2.2
0.6
88.5
1 0.3
35.2 1:
1.4
38.1 0.4
38.4 0.9
90.7 1 0.2
(b) Accuracy on clients not observed training time (best values per setup bold)
Table 1: Experimental results on standard pFL benchmarks. In all settings, except CIFAR100 with 100 clients, PeFLL achieves higher accuracy than previous methods, and the accuracy difference between training clients (top table) and unseen clients (bottom table) is small, if present at all.
we fix the number of clients participating per round to 5. The Local baseline trains on each client independently for 200 epochs. The hyperparameters for all methods are tuned using validation data that was held out from the training set (10,000 samples for CIFAR1O and CIFAR 100, spread across the clients, and 10% of each client's data for FEMNIST). The optimizer used for training at the client is SGD with a batch size of 32, a learning rate chosen via grid search and momentum set to 0.9. The batch size used for computing the descriptor is also 32. More details of the hyperparameter selection for each method as well as ablation studies are provided in Appendix A
4.1 RESULTS
T shows the results for PeFLL and the baseline methods. In all cases, we report the test set accuracy on the clients that were used for training (Table Tab and on new clients that were not part of the training process (Table PeFLL achieves the best results in almost all cases, and often by large margin. The improvements over previous methods are most prominent for previously unseen clients, for which PeFLL produces model of almost identical accuracy as for the clients used for training. This is especially remarkable in light of the fact that several of the other methods have computationally more expensive procedures for generating models in this setting than PeFLLI, in particular requiring on-client or even federated training to produce the personalized models. We see this result as a strong evidence that PeFLL successful generalizes, as predicted by Theorem 3.2
Comparing PeFLL's results to the most similar baseline, pFedHN, one observes that the latter's performance decreases noticeably when the number of clients increases and the number of samples per client decrease accordingly. We attribute this to the fact that pFedHN learns independent client descriptors for each client, which can become unreliable if only few training examples are available for each client. Similarly, for Per- Per-FedAvg. PFedMe and FedRep, which construct personalized models by local finetuning, the model accuracy drops when the amount of data per client decreases, especially in the more challenging CIFAR100 setup. kNN-Per maintains good generalization from train to unseen clients, but its performance worsens when the number of samples per client drops and the kNN predictors have less client data available. In contrast, PeFLL's performance remains stable, which we attribute to the fact that it learns the embedding and hypernetwork jointly from all available data. The new client does not have to its data for training, but rather just to generate client descriptor. As a pure feed- forward process, this does not suffer from overfitting in the low-data regime.
8

---

Published as a conference paper at ICLR 2024
CIFARIO
CIFARTOO
FEMNIST
#trn.clients
11 = 90
450
900
90
450
900
3237
FedAvg
18.6 + 0.3
49.6 +: 1.0
51.7 + 0.7
17.1
+ 0.9
19.3 + 2.0
20.5 4 1.0
81.9 + 0.4
PeFLL
84.5 1 0.6
83.6
1.0
81.5 1 0.7
26.3
0.4 28.2
0.6 30.3 1 0.6
91.4 + 0.1
Table 2: Test accuracy on CIFARIO and CIFARIOO for (test) clients that have only unlabeled data.
Personalization for clients with only unlabeled data A scenario of particular interest is when some clients have only unlabeled data available. PeFLL can still provide personalized models for them in the following way. To train PeFLL, as before one uses only clients which do have labeled data. However, the embedding network is constructed to take only the unlabeled parts of the client data as input (i.e. it ignores the labels). Consequently, even clients that only have unlabeled data can evaluate the embedding network and obtain descriptors, so the hypernetwork can assign personalized models to them during the test phase. We report results for this in Table 2 Note that none of the personalized bascline methods from Section are able to handle this setting, because they train or finetune on the test clients. Therefore, the only alternative would be to train a ponppesson model, c.g. from FedAvg, and use this for clients with only labeled data. One can see that PeFLL improves the accuracy over this baseline in all cases.
Analysis of Client Descriptors Our work relies on the hypothesis that clients with similar data dis-
Rank Correlation
tributions should obtain similar descriptors, such
0.8
that the hypernetwork then produces similar models for them. To study this hypothesis empirically, we create clients of different similarities to each other
0.6
in the following way.
1000
2000 3000 Round
4000
5000
Let C denote the number of classes and n the num- ber of clients. Then, for each client i we sample a vector of class proportions, π; E A from a Dirichlet distribution Dir(a:) with parameter vector (0.1,.... 0.1). where AC is the unitary sim- plex of dimension C. Each client then receives a dataset, S. of samples from each of the classes according to its class proportion vector R, We randomly split the clients into train and unseen clients, and we run PeFLL on the train clients. Throughout training we periodically use the embedding network to compute all client descriptors in order to track how they evolve.
Figure 3: Correlation between client descrip- tor similarity obtained from the embedding network and ground truth similarity over the course of training.
For any pair of such clients, we compute two similarity measures: dij = || TT which provides a ground truth notion of similarity between client distributions, and di =
which measures the similarity between client descriptors. To measure the extent to which d" reflects dt we use their average rank correlation in the following way. For any unseen client i we a form vector of similarities d. (di ... di E R", and compute its Spearman's rank correlation coefficient (Spearman 1904) to the corresponding ground truth vector dī = (dft..., df,, E IR"..
-
Figure 3 shows the average of this value across the unseen clients after different numbers of training steps (CIFAR 10 dataset, 1000 clients in total). One can see that the rank correlation increases over the course of training, reaching very high correlation values of approximately 0.93. Results for other numbers of clients are comparable, see Appendix [A] This indicates that the embedding network indeed learns to organize the descriptor space according to client distributional similarity, with similar clients being mapped to similar descriptors. Moreover, because these results are obtained from unseen clients, it is clear that the effect does not reflect potential overfitting of the embedding network to the training clients, but that the learned similarity indeed generalizes well.
5 CONCLUSION
In this work, we presented PeFLL, a new algorithm for personalized federated learning based on learning-to-learn. By means of an embedding network that creates inputs to hypernetwork it efficiently generates personalized models, both for clients present during training and new clients that appear later. PeFLL has several desirable properties for real-world usability. solid theoretical foundation in terms of convergence and generalization. It is trained by stateless optimization and does not require additional training steps to obtain models for new clients. Overall, it has low latency and puts little computational burden on the clients, making it practical to use in large scale applications with many clients each possessing little data.
stands on

---

Published as a conference paper at ICLR 2024
Acknowledgements. This research supported by the Scientific Service Units (SSU) of ISTA through resources provided by Scientific Computing (SciComp).
REFERENCES
R. Amit and R. Meir. Meta-learning by adjusting priors based on extended PAC-Bayes theory. In International Conference on Machine Learning (ICML), 2018
M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, B. Shillingfor, and N. De Freitas. Learning to learn by gradient descent by gradient descent. In Conference on Neural Information Processing Systems (NeurIPS), 2016.
M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, Federated learning with personal- ization layers. CoRR, abs/1912.0081s 2019. URL/ttp::/arxxi..oo www.s///w...../a..aaao.o..asssst 1912. 00818
J. Baxter. A model of inductive bias learning. Journal of Artificial Intelligence Research (JAIR), 12: 149-198, 2000.
D. Bui, K. Malik, J. Goetz, H. Liu, S. Moon, Kumar, and K. G. Shin. Federated user representation learning. CoRR, abs//909..m
2019. WRRLhttp:/// http://aaaa..oo///////1000.0.../
Caldas, P. Wu, T. Li, Konečný, H. B. McMahan. V. Smith, and A. Talwalkar. LEAF: benchmark for federated settings. CoRR, abs/ 0tss/18122 812.01097, 2018. URL pattp::/aarse http:///w...a/////////////////// 01197
D. Chen, L. Yao, D. Gao, B. Ding, and Y. Li. Efficient personalized federated learning via sparse model-adaptation. In International Conference Machine Learning (ICML), 2023.
Z. Chen and B. Liu. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(3), 2018.
L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai. Exploiting shared representations for personalized federated learning. In International Conference on Machine Learning (ICML), 2021.
C. Dinh, N. H. Tran, and T. D. Nguyen. Personalized federated learning with moreau envelopes. In Conference Neural Information Processing Systems (NeurIPS), 2020.
C. T. Dinh, T. T. Vu, N. H. Tran, M. N. Dao, and H. Zhang. A new look and convergence rate of federated multi-tast learning with Laplacian regularization. IEEE Transactions Neural Networks (TNN), 2022.
C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Machine Learning, 9(3 2014.
D. Evans, V. Kolesnikov, and M. Rosulek. A pragmatic introduction to secure marti computation. Foundations and Trends in Privacy and Security, 2(2-3), 2018. ISSN 2474-1558.
A. Fallah, A. Mokhtari, and A. E. Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (ICML), Proceedings of Machine Learning Research, 2017.
A. Ghosh, J. Chung, D. Yin, and K. Ramchandran. An efficient framework for clustered federated learning. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
J. Guan and Z. Lu. Fast-rate PAC-Bayesian generalization bounds for meta-learning. In International Conference on Machine Learning (ICML), 2022.
O. Gupta and R. Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116, 2018
D. Ha, A. M. Dai, and Q. V. Le. Hypernetworks. In International Conference Learning Represen- fations (ICLR), 2017.
F. Hanzely and P. Richtárík. Federated learning of mixture of global and local models. CoRR, abs/2002.05516. 2020.
05516
10

---

Published as a conference paper at ICLR 2024
F. Hanzely, S. Hanzely, S. Horváth, and P. Richtárik. Lower bounds and optimal algorithms for per- sonalized federated 2020.
Tedrnng learning. In Conference on Neural Information Processing Systems (NeurIPS),
A. Hard, K. Rao, R. Mathews, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage. Federated learning for mobile keyboard prediction. CoRR, 1////8811.03606 2018. URL wwaxxiv.org/ab....
03604
Hospedales, A. Antoniou, P. Micaelli, and A. Storkey. Meta-leaming in neural networks: A survey, IEFE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(9):5149-33 2021.
Y. Jiang, J. Konečný, K. Rush, and S. Kannan. Improving federated learning personalization via model agnostic meta learning, CoRR, pes//999.124880 2019. URL htttp://arx.v.ogg///aa 1909. 12488)
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji. K. Bonawitz, Z. Charles, G. Cormode, R. Cummings. R. G. L. D' D'Oliveira, H. Eichner, S. E. Rouayheb, D. Evans, J. Gardner, Z. Garrett, A. Gascón, Ghazi, P. Gibbons. M. Gruteser, Z. Harchaoui, C. He, L. He, Z. Huo,
B. Hutchinson, J. Hsu. M. Jaggi, T. Javidi, G. Joshi, M. Khodak, J. Konecný, A. Korolova,
F. Koushanfar. S. Koyejo, T. Lepoint, Y. Liu, P. Mittal, M. Mohri, R. Nock, A. Özgür, R. Pagh, H. Qi. D. Ramage, R. Raskar, M. Raykova, D. Song, W. Song, S. U. Stich, Z. Sun, A. T. Suresh, F. Tramèr, Vepakomma, J. Wang, I.. Xiong. 7.. Xu, Q. Yang, F. X. Yu, H. Yu, and S. 7hao. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1-2), 2021.
S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich,, and A. T. Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference Machine Learning (ICML), 2020.
A. Krizhevsky. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
H. Li, Z. Cai, J. Wang, J. Tang, W. Ding, C.-T. Lin, and Y. Shi. FedTP: Federated learning by transformer personalization. IEEE Transactions on Neural Newworks and Learning Systems, 2023.
Li, A. K. Sahu, A. Talwalkar, and V. Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50-60, 2020.
Li, S. Hu, A. Beirami, and V. Smith. Ditto: Fair and robust federated learning through personaliza- tion. In International Conference on Machine Learning (ICML), 2021.
P. P. Liang, T. Liu, 7. Liu, R. Salakhutdinov, and L.. Morency. Think locally, act globally: Federated learning with local and global representations. CoRR, abs/2001. .01523, 2020. URL http
//arxiv.orge hts//bbb/20000 22001. 01523
X. Ma, Zhang, S. Guo, and W. Xu. Layer-wised model aggregation for personalized federated learning. In JEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Y. Mansour, M. Mohri, J. Ro, and A. T. Suresh. Three approaches for personalization with applications to federated learning. CoRR, abs/2002.10619, 2020. URLL//ttpss://xxx..cco////a 10619
O. Marfoq, G. Neglia, A. Bellet, L. Kameni, and R. Vidal. Federated multi-task learning under a mixture of distributions. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
O. Marfoq, G. Neglia, R. Vidal, and L. Kameni. Personalized federated learning through local memorization. In International Conference on Machine Learning (ICML), 2022.
A. Maurer. A note on the PAC Bayesian theorem. CoRR, cs.LG/0411 1099, 2004. URL http:
7/arxiv. arg/aba// /s LG/0411000
D. A. McAllester. Some PAC-Bayesian theorems. In Conference on Computational Learning Theory (COLT), 1998.
11

---

Published as a conference paper at ICLR 2024
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Areas. Communication-effen learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
T. Mitchell, W. Cohen, E. Hruschka, P. Talukdar, J. Betteridge, A. Carlson, B. Dalvi, M. Gardner, B. Kisiel, J. Krishnamurthy, N. Lao, K. Mazaitis, T. Mohamed, N. Nakashole, E. Platanios, A. Ritter, M. Samadi, B. Settles, R. Wang. D. Wijaya, A. Gupta, X. Chen, A. Saparov, M. Greaves, and J. Welling. Never-ending learning. In Conference Artificial Intelligence (AAAI), 2015.
G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter. Continual lifelong learning with neural networks: review. Neural networks, 11::54-71, 2019.
A. Pentina and C. H. Lampert. PAC-Bayesian bound for lifelong learning. International Conference on Machine Learning (ICML), 2014.
A. Pentina and C. H. Lampert. Lifelong learning with non-iid tasks. In Conference on Neural Information Processing Systems (NeurIPS), 2015.
M. Pérez-Ortin Ortiz, O. Rivasplata, J. Shawe-Taylor, and C. Szepesvári. Tighter risk certificates for neural networks. The Journal of Machine Learning Research. 22(1):13326-10355 2021.
Z. Qin, I.. Yang, Q. Wang, Y. Han, and Q. Hu. Reliable and interpretable personalized federated learning. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023.
A. Rezazadeh. A unified view on PAC-Bayes bounds for meta-learning. In International Conference on Machine Learning (ICMI), 2022.
M. B. Ring, Continual Learning in Reinforcement Environments. PhD thesis, University of Texas at Austin, Computer Science Department, 1994.
Rothfuss, V. Fortuin, M. Josifoski, and A. Krause. PACOH: Bayes-optimal meta-learning with PAC-guarantees. In International Conference on Machine Learning (ICML), 2021.
Y. Seldin. F. Laviolette. N. Cesa-Bianchi, J. Shawe-Taylor and P. Auer. PAC-Bayesian incqualities for martingales. IEEE Transactions on Information Theory, 58(12):7086 2012.
A. Shamsian, A. Navon, F. Fetaya, and G. Chechik. Personalized federated learning using hypernet- works. In International Conference on Machine Learning (ICML), 2021.
V. Smith, C. Chiang, M. Sanjabi, and A. Talwalkar. Federated multi-task learning. In Conference on Neural Information Processing Systems (NeurIPS), 2017.
C. Spearman, The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72-101, 1904.
C. Thapa, M. A. P. Chamikara, S. Camtepe, and L. Sun. SplitFed: When federated learning meets split learning. In Conference on Artificial Intelligence (AAAI), 2022.
S. Thrun and T. Mitchell. Lifelong robot learning. Robotics and Autonomous Systems, 15(1):25 46, 1995.
S. Thrun and L. Pratt, editors. Learning to Learn. Kluwer Academic Press, 1998.
R. Vilalta and Y. Drissi. A perspective view and survey of meta-learning Artifial Intelligence Reviews, 18(2):77-95, 2002.
J. Wu, W. Bao, E. Ainsworth, and J. He. Personalized federated learning with parameter propagation. In International Conference on Knowledge Discovery and Data Mining (KDD), 2023a.
Y. Wu, S. Zhang, W. Yu, Y. Liu, Q. Gu, D. Zhou, H. Chen, and W. Cheng. Personalized federated learning under mixture of distributions. In International Conference on Machine Learning (ICML), 2023b.
R. Ye, Z. Ni, F. Wu, S. Chen, and Y. Wang. Personalized federated learning with inferred collaboration graphs. In International Conference on Machine Learning (ICMI.), 2023.
L. Yi, X. Shi, N. Wang, Z. Xu, G. Wang, and X. Liu. PFedLHNs: Personalized federated learning via local hypernetworks. In International Conference on Artificial Neural Networks (ICANN), 2023.
M. Zaheer, S. Kottur, S. Ravanbakhsh, Poczos, R. R. Salakhutdinov, and A. Smola. Deep sets.
In Conference on Neural Information Processing Systems (NeurIPS), 2017.
H. Zhang, C. Li, W. Dai, J. Zou, and H. Xiong. FedCR: Personalized federated learning based on across-client common representation with conditional mutual information regularization. In International Conference Machine Learning (ICMI.), 2023.
12

---

Published as a conference paper at ICLR 2024
Layer
Shape
Nonlinearity ReLU
Layer
Shape
Nonlinearity
Conv1
3 x 5 x 5 x 16
Conv1
3 x 5 x 5 x 16
ReLU
MaxPool
2 x 2
MaxPool
2 x
Conv2
16 x 5 32
ReLU
Conv2
16 x 5 x 32
ReLU
MaxPool
2 x 2
flatten
MaxPool
2 x 2
flatten
FC1
800 x 120
ReLU
FC1
800 x 120
ReLU
FC2
120 x 84
ReLU
FC2
120 84
ReLU
FC3
84 x C
softmax
FC3
84 x
none
Table 3: LeNet-based ConvNet of personalized client models. Convolutions are without padding. C' is the number of classes.
Table 4: LeNet-based embedding network. Con- volutions are without padding. I the dimen- sionality of the client descriptor.
Layer
Shape
Nonlinearity
FCI
x 100
ReLU
FC2
100 x 100
ReLU
FC3
100 x 100
ReLU
FC4
100 x 100
ReLU
FC5
100 D
none
Table 5: Fully connected hypernetwork. D is the dimensionality of the model to be created. Note that because FC5 alone the hypernetwork is at least 100 bigger than the client models.
A APPENDIX EXPERIMENTS
A.1 MODEL ARCHITECTURES
We used LeNet-based architecture for the personalized model on each client. The exact architecture is shown in Table 3 After each convolutional and fully connected layer we apply a ReL.U non- linearity except for the final layer which uses softmax for input into the cross-entropy loss. The embedding network uses the same architecture except that the final layer has dimension 84 x and we do not apply a non-linearin (Table(4) 4 For the hypernetwork we use a fully connected network. The exact architecture is shown in Table 5 Again we apply a ReLU after each layer except the last one, which does not have non-linearity and simply has output of size D, where D denotes the number of parameters in the client personalized model.
A.2 HYPERPARAMETER SETTINGS
For PeFLL., each client uses a single batch data (batch size 32) as input to the embedding network. For the hypernetwork parameter settings we use the recomended values given in (Shamsian et al. 2021) we found these to work well in setting well. Specifically, the dumension of the embedding vectors
2 and the number of client SGD steps parameters for the embedding network and hypernetwork are set d = A = 10-3, while the output regularization is \ =
50. The regularization
For Per-FedAvg we used the first order (FO) approximation detailed in Fallah et al. 2020), and following the recomendations we set the number of client local steps is set to be single epoch. For the number of client updates to the head and body to 5 and I epochs respectively as recommended in (Collins et al. 202T). h. To infer on new client we we randomly initialized head and trained only the head locally for 20 epochs. For PFedMe we follow the recomendations
FedRep
in (Dinh et al. 2020) and set the regularization parameter to À = 15, the computation parameter to K = 3 and B I. To infer for new client we first initialize the clients personalized model to the global model and finetuned this for 20 epochs using the pFedMe loss function. For kNN-Per for the scale parameter in the KNN classifier we searched over the values recommended in (Marfoq et al.) 2022), namely 10. 1, 10. 100), we found that setting the scale parameter to 100 to worked best. The k parameter for the kNN search and the weight parameter for how much weight the client local vs global model were chosen individually on each client by cross-valion coss-validation searching over (5, 10} and (0.0, 0.1, 0.2, 0.2...., 1.0) respectively. For pFedHN we set the hyperparameters following the recommendations in (Shamsian al. 2021) For inferring new client followed the
13

---

Published as a conference paper at ICLR 2024
procedure given in (Shamsian al. 2021 and randomly initialized new embedding vector and optimized only this using 20 rounds of client server communication.
A.3 ADDITIONAL EXPERIMENTS
Comparison of communication and computation cost with FedAvg We compare the perfor- mance of PeFL.L. to FedAvg with respect to communication costs of training. each global round, the communication cost of PeFLL between the server and e clients is 2c([ + +ll--l + ||vl]) while for FedAvg 2c||@]|. In our setting, the emhedding network and client models are of comparable size and the size of client descriptors (v) is negligibly small compared to them. Therefore, the per round communication cost of PeFL. is about twice that of FedAvg. In Figures [ we plot the accuracies during training for given communication cost (measured in GB of information transferred between server and clients) as well computational cost for the clients (measured in number of global rounds). As shown in the figures, for any given communication or computational budget PeFLL outperforms FedAvg on these datasets.
Shakespeare dataset In order to test the effectiveness of PeFILL in non-image tasks, we also provide results for the Shakespeare dataset (Caldas et al. 2018), which is next character prediction task. We use the LSTM model from Caldas et al. (2018) as the client model, and as the embedding network. The hypernetwork is the same as in all other experiments. We compare the performance in comparison to a number of baselines. Figure 5 shows the comparison of the accuracies for train and test clients during training with respect to the number of global rounds. PeFLL outperforms all baselines when comparing performance with respect to the number of global rounds. We additionally compare the performance of PeFLL with FedAvg with respect accuracy for given communication We observe similar performances for a fixed communication budget.
cost, see Figure 6
ResNet Experiments In order to test the effectiveness of PeFLL for more modern architectures, we also performed experiments using a ResNet20 mode f for CIFARIO dataset. We compared the performance of PeFLL with FedAvg, and the results are shown in Figure 7 for 100, 500, and 1000 clients, respectively. These figures show PeFLL outperforms FedAvg also for this model, and support the feasibility of generating modern models as part of our pipeline.
For all experiments clients were related in the sense that they come from the same client environment. In this section, we examine how well PcFLL is able to generalize beyond this, by studying its extrapolation performance, where the new clients come from a different client environment than the train clients. We follow the same procedure as in the previous section to simulate heterogeneous clients and use sampled Dirichlet class proportions to assign classes. For the train clients we again use Ormmin (0.1, to generate each clients class proportion vector. However, for the new clients we use different Dirichlet parameter Onnw = (a..... ar). For each a {0.1, 0.2, 0.3,.... 1.0) we generate a group of new clients using this parameter. We run PeFLL on the train clients then use the trained embedding and hypernetworks to generate model for each of the new clients.
Analysis of Client Extrapolation
far, clients at training time and new
0.1)
new client accuracy
PeFLL
kNN-Per
0.8
PFedMe
Local
0.6
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 alpha
Figure 4: Accuracy in client extrapolation. Larger values of a indicate new clients that are more dissimilar to the train clients.
implementation from pytorch-ci for except without the batch normalization layers.
'ee
use
the
14

---

Published as a conference paper at ICLR 2024
FedAvg
Accuracy
0.4
PeFLL
FedRep
pFedMe
0.2
Per-FedAvg(FO)
0
500
1000
1500
2000
2500
Global Round
FedAvg
Accuracy
0.4
PeFLL
FedRep
pFedMe
0.2
Per-FedAvg(FO)
0
500
1500 Global Round
1000
2000
2500
Figure 5: Test Accuracies for train clients (top row) and test clients (bottom row) during different steps of the training for PeFI.I and baselines, for the Shakespeare dataset.
0.4 0.4
Accuracy
Accuracy
0.4
0.2
FedAvg
0.2
FedAvg
PeFLL
PeFLL
20
40
60
80
0
20
40
60
80
Communication Cost (GB)
Communication Cost (GB)
Figure 6: Test Accuracies for train clients (left) and test clients (right) over the course of training measured with respect to total communication cost for PeFLL and FedAvg, for the Shakespeare dataset.
Figure 4
(show shows the resulting accuracy values for PeFLL and the best performing baselines of Table as well The baseline purely local per-client training. Note that as a increases so does the difficulty of the client's 's problem, as illustrated by the fact that the accuracy of purely local training decreases. Despite this increased task difficulty and distributional difference PeFLL still obtains strong results. Even
1, PeFLL produces models that are more accurate than those learned by the other methods for smaller values of a, and far superior purely local training on the new clients.
Integrating known client label sets In the vanilla setting of Section 4 only the client training data is used to create model. By design, each model was multi-class classifier across all possible classes, even if only subset of those are relevant for the client. However, it is also reasonable to assume that clients might have prior knowledge which of the output classes they actually want to predict. This information can included by label masking (LM). in which the client overwrites the output probabilities of classes that cannot occur for by 0, such that they will never predicted. report results for this in Table 6 One can see that this has minor effects for CIFAR but the results for CIFAR 100 are clear improved. We attribute this to the fact that the larger label set and the smaller number of examples per class increase the chances of spurious classes being predicted.
15

---

Published as a conference paper at ICLR 2024
0.8
0.8
Accuracy
00.6
0.6
Accuracy
0.6
w/ M/
0.4
0.4
FedAvg PeFLL
FedAvg
0.2
0.2
PeFLL
1000
2000
3000
4000
0
1000
2000
3000
4000
Global Round
Global Round
Accuracy
0.6
M/ MW w\ Ma pa
Accuracy
0.6
M Mhn Maa WV My
0.4
0.4
FedAvg PeFLL
FedAvg
0.2
0.2
PeFLL
1000
2000
3000
4000
1000
2000
3000
4000
Global Round
Global Round
0.6
mmM
0.6
Accuracy
Accuracy
0.4
0.4
FedAvg PeFLL
FedAvg
0.2
0.2
PeFLL
1000
2000
3000
4000
0
1000
2000
3000
4000
Global Round
Global Round
Figure 7: Test Accuracies for train clients (left) and test clients (right) during different steps of the training for PeFLL and FedAvg for the CIFAR10 dataset with 100 clients (top row), 500 clients (middle row) and 1000 clients (bottom row). PeFLL consistently outperforms FedAvg.
0.75
0.75
Accuracy
Accuracy
0.50
0.50
FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000
2000
3000
4000
5000
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.75
0.75
Accuracy
Accuracy
0.50
0.50
Mer FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000
2000
3000
4000
5000
5
10
25
Global Round
Total Communication Cost (GB)
Figure 8: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFI.L and FedAvg, for the CIFAR dataset, 100 clients. PeFI.L's accuracy is higher than FedAvg's 's with respect to the number of rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
16

---

Published as a conference paper at ICLR 2024
0.75
0.75
Accuracy
Accuracy
0.50
0.50
FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000
2000
3000
4000
5000
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.75
0.75
Accuracy
Accuracy
0.50
0.50
FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000
2000 3000
4000
5000
5
10
25
Global Round
Total Communication Cost (GB)
Figure 9: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLL and FedAvg, for the CIFAR1O dataset, 500 clients. PeFLL's accuracy is higher than FedAvg's ss with respect to the number of rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
0.75
Accuracy
Accuracy
0.50
0.50
FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000
2000
3000
4000
5000
0
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.75
0.75
Accuracy
Accuracy
0.50
0.50
FedAvg
FedAvg
0.25
PeFLL
0.25
PeFLL
1000 2000 3000
4000
5000
5
10
15
20
25
Global Round
Total Communication Cost (GB)
Figure 10: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLL and FedAvg, for the CIFARIO 10 dataset, 1000 clients. PeFLL's accuracy is higher than FedAvg's 's with respect to the number of rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
A.4 ADDITIONAL FIGURES
We include additional figures for the correlation between the distances between client descriptors and the true client distributional distances described in Section 4 In Figures 15 [7 we show the results of the experiment for different numbers of clients, 100, 500 and 1000. As we can see in all cases high correlation achieved. With more clients we obtain higher levels of correlation and less variability in the results.
A.5 ABLATION STUDIES
We include several ablation studies to explore how different design choices impact PeFLL's perfor- mance.
17

---

Published as a conference paper at ICLR 2024
0.6
Accuracy
0.4
Accuracy
0.4
FedAvg PeFLL
FedAvg
PeFLL
0.2
0.2
0.0
0.0
1000
2000
3000
4000
5000
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.3
0.3
Accuracy
Accuracy
0.2
0.2
0.1
FedAvg
0.1
FedAvg
PeFLL
PeFLL
0.0
0.0
1000
4000 Global Round
2000 3000
5000
20 25
Total Communication Cost (GB)
Figure 11: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLL and FedAvg, for the CIFAR 100 dataset, 100 clients. PeFLL's 's accuracy is higher than FedAvg's 's with respect to the number of rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
0.4
0.4
Accuracy
Accuracy
0.2
0.2
FedAvg PeFLL
FedAvg
PeFLL
0.0
0.0
1000
2000
3000
4000
5000
0
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.3
Accuracy
Accuracy
0.2
0.2
FedAvg PeFLL
0.1
FedAvg
PeFLL
0.0
0.0
1000 2000 3000
4000
5000
0
5
10
15
20
25
Global Round
Total Communication Cost (GB)
Figure 12: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLI. and FedAvg, for the CIFAR 100 dataset, 500 clients. PeFLL''s accuracy is higher than FedAvg's 's with respect to the number of rounds (which roughly reflect the computational cost for the clients) well as the communication cost.
Hypernetwork Size. We investigate how the size of the hypernetwork impacts performance. We keep the same hypernetwork architecture as described in Table vary the number of hidden layers. We have small network (S) with only single hidden layer, medium network (M) with 3 hidden layers and a large network (L.) with 5 hidden layers. We evaluate on CIFARIO and CIFAR 100 using the same experimental setup as for previous experiments. with 100, 500 and 1000 clients. The results are shown in Table ? As we can see in the results, for CIFARIO all 3 networks are reasonably similar with slight drop in performance observed for the S network. For CIFAR 100 it is more pronounced, with modest increase in performance for the L network and noticeable drop for the S network.
Regularization Parameters. We examine the impact of varying the regularization parameters of the objective (I) on the performance PeFLL. We consider different values for the regularization pa- rameters and study the values in (0, 5 10-5 5,55 5 10-3 3 5 10-1) for client regularization parameter,
18

---

Published as a conference paper at ICLR 2024
0.4
0.4
Accuracy
Accuracy
0.2
0.2
FedAvg
FedAvg
PeFLL
PeFLL
0.0
0.0
1000
2000
3000
4000
5000
0
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.4
0.3
Accuracy
Accuracy
0.2
0.2
FedAvg PeFLL
0.1
FedAvg
PeFLL
0.0
0.0
1000
2000 3000
4000
5000
10 15
Global Round
Total Communication Cost (GB)
Figure 13: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLL and FedAvg, for the CIFAR 100 dataset, 1000 clients. PeFLL's accuracy is higher than FedAvg's 's with respect to the number of rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
0.75
0.75
Accuracy
Accuracy
0.50
0.50
0.25
FedAvg
0.25
FedAvg PeFLL
PeFLL
0.00
0.00
1000
2000
3000
4000
5000
5
10
15
20
25
Global Round
Total Communication Cost (GB)
0.75
0.75
Accuracy
Accuracy
0.50
0.50
0.25
FedAvg
0.25
FedAvg
PeFLL
PeFLL
0.00
0.00
1000
2000 3000
4000
5000
5
10
15
20 25
Global Round
Total Communication Cost (GB)
Figure 14: Test Accuracies for train clients (top) and test clients (bottom) during different steps of the training for PeFLL and FedAvg, for the FEMNIST dataset. PeFLL's accuracy is higher than FedAvg's 's with respect to the number rounds (which roughly reflect the computational cost for the clients) as well as the communication cost.
and values in (0, 10-5, 5.5 x 10-3) for embedding and hypernetwork regularization. The results are shown Table 8 As one can see, PeFLL's performance is rather robust over the regularization strengthh. However, consistent with the theory, modest amounts of hypernetwork regularization parameter help preventing overfitting on training clients and improve client-level generalization. Similarly, modest amounts of client regularization improve sample-level generalization. As can be expected, very large values this parameter leads underfitting.
Embedding network architecture. We also study the effect of different architecture choices for the embedding network. We compare the performance of a single-layer MLP network layer and a LeNet-style ConvNet for CIFAR10 and CIFAR100. As we can see in Table outperforms the MLP in all experiments.
9 the CNN embedding
19

---

Published as a conference paper at ICLR 2024
CIFARIO
CIFARI00
#trn.clients
T 90
450
900
450
900
PeFLL
89.0 0.8
88.9 = 0.4
8..
56.0 + 0.3
43.2 + 0.8
PeFLL + LM
88.9 + 0.7
88.9 = 0.3
8.44
59.5 1 0.7
53.2 + 0.6
51.4
CIFARTO
CIFARTIN 100
#trn.clients
n=
450
900
90
450
900
PeFLL
89.3 + 2.2
88.9 + 0.6
88.5
0.3
35.2 + 1.4
38.1 + 0.4
38.4 1 0.9
PeFLL + LM
88.9 + 2.4
88.2 + 0.9
87.4 + 1.3
55.6
+ 1.6
53.4 + 0.1
51.1 + 0.2
Table 6: Test accuracy on CIFARIO and CIFARI00 for training clients (top) and test clients (bottom) without and with label masking (LM).
Rank Correlation
0.8
0.6
0
1000
2000
3000
4000
5000
Round
Figure 15: Correlation between client descriptor similarity obtained from the embedding network and ground truth similarity over the course of training. CIFARIO, 100 Clients.
Rank Correlation
0.8
0.6
0
1000
2000
3000
4000
5000
Round
Figure 16: Correlation between client descriptor similarity obtained from the embedding network and ground truth similarity over the course of training. CIFARIO, 500 clients.
Rank Correlation
0.8
0.6
0
1000
2000
3000
4000
5000
Round
Figure 17: Correlation between client descriptor similarity obtained from the embedding network and ground truth similarity over the course of training. CIFARIO, 1000 clients.
20

---

Published as a conference paper at ICLR 2024
CIFARIO
CIFAR100
#Clients
100
500
1000
100
500
1000
S
88.7 + 0.3
88.2 0.4
87.7 0.4
57.9 + 0.3
51.9 + 0.6
4.44
88.9
0.7
0.3
88.1 0.3
59.4 + 0.2
53.1 0.4
50.8 0.5
L
88.7
+ 0.8
88.8 0.3
88.2 1
59.8 + 0.1
53.8 0.2
52.6 + 1.0
Table 7: Test accuracy on CIFAR10 and CIFAR100 using different sizes of hypemetworks, small (S), medium (M) and large (L).
dn = A
A
0
10-0
10
0
87.4 + 1.1
87.4 1.0
0.6
5
10
88.7 + 0.7
88.7 0.6
88.8
10-3
87.8
+ 0.8
87.6 0.8
87.5
0.5
10
82.5
+ 1.5
83.8
1.1
66.2
Table 8: Test accuracy on CIFARIO with 100 clients for different settings of the regularization param- eters. On the left is the regularization of the client network, and along the top is the regularization of the hypernetwork and embedding network.
CIFARTO
CIFAR 100
#Clients
100
500
1000
100
500
1000
MLP
88.5 1.2
87.8 1.1
2.1
59.1 + 0.2
53.4 + 0.4
51.2 0.5
CNN
88.9 - 0.7
88.9
= 0.3
0.4
59.4 0.2
54.4 0.3
5 0.3
Table 9: Test accuracy on CIFARIO and CIFAR100 using different embedding network architectures.
MLP is single hidden layer fully connected network and CNN is a LeNet-style ConvNet.
21

---

Published as a conference paper at ICLR 2024
B APPENDIX GENERALIZATION
In this section we prove a new generalization bound for the generic learning-to- setting that, in particular, will imply Theorem 3.2 as a special case. Before formulating and proving our main results, we remind the reader of the obtaining guarantees for learning-to-lears
learning framework (McAllester 1998) and its use for
In standard PAC-Bayesian leamnnn, we are given set of possible models, H. and a prior distribution over these P M(H), where MM-) denotes the set of probability measures over a base set. Given a dataset, S, learning model means constructing new (posterior) distribution, Q E M(H), which is meant to give high probability to models with small loss. The posterior distribution, Q. induces stochastic predictor: for any input, one samples a specific model, I ~ Q, and outputs the result of this model applied to the input. Note that Q can in principle be a Dirac delta distribution at a single model, resulting in a deterministic predictor. However, for large (uncountably infinite) model such a choice typically does not lead to strong generalization guarantees. For conciseness of notation, in the following we do not distinguish between the distributions over models and their stochastic predictors.
PAC-Bayesian learning and learning-to-lcar
The quality of a stochastic predictor, Q. on a data point (x,y) is quantified by its expected loss, ((x,)) (Q) = BBnn ((x, y, s)- From this, define the empirical error dataset, S. as [8] E(z.))Es (Q) and its expected loss with respect to a data distribution, D, as ((a.))) (a.,)) (Q). Ordinary PAC-Bayesian generalization bounds provide -rgo-brobability up- per bounds to the expected loss of stochastic predictors by the corresponding empirical loss as well as some complexity terms, which typically include the Kullback- divergence between the chosen posterior distribution and the original (training-dats independent) prior, DKL (QI()))) (McAllester 1998).
Typically, the posterior distribution is not chosen arbitrarily, but it the result of a learning algorithm, A: (X's x y)m MMH), which takes as input the training data and, potentially, the prior distribution. The idea of learning-to-leet to-learn (also called meta-learning lifelong learning in the literature) is to learn the learning algorithm (Baxter 2000, Pentina and Lampert 2015).
To study this theoretically, we adopt the setting where n learning tasks available, which we write as tuples, (S,, D:), for i E (1...., n), each with data set S; C X' x y that is sampled from corresponding data distribution D, E M(X x ))) For simplicity, we assume that all datasets are of the same size, m. We assume tasks are sampled i..d. from a task environment, T, which is simply data distribution over such tuples.
Again adopting the PAC-Bayesian framework, we assume that data-independent (meta-) distribution over learning algorithms is available, P E MMA)), where A is the set of possible algorithms, and the goal is use the observed task data to construct a (meta- ppetaa)posterior distribution, Q E (A). As before, the resulting procedure stochastic: at every invocation, the system samples an algorithm A ~ Q. It applies this to the training data, obtaining a posterior distribution A(S), and it makes predictions by sampling models accordingly, A(S).
Analogously to above situation, we define two measures of quality for such a stochastic algorithms. Its empirical loss on the data of the observed clients:
()))
= 71
1 Σ
E
Σ
(l uat),))
(4)
A
m
1=1
1=1
and its expected loss on future clients,
er( =
E
E E
l( ((r,,,(())
(5)
(D.S)N ANQ (aW)D
The following theorem provides connection between both quantities.
Theorem B.1. Let P E M(A) and (MH) be a meta-prior and prior distribution, respectively,
which are chosen independently of the observed training data, Si...., Sm.- Assume that the loss
function is bounded in [0, M). Then, for all 5 2 0 it holds with probability least 8 over the
22

---

Published as a conference paper at ICLR 2024
sampling of the datasets, that for all distributions Q M((A over algorithms,
( 1y 0
E (AL(()())
Smn
o )
+
er(Q) - M
+ME 1~
2n
2mn
(6)
where DEL denotes the Kullback-Leebber divergence.
Proof. The proof resembles previous proofs for PAC-Bayesian learning-to-lean to-lerr learning, but differs in some relevant steps. We discuss this a Discussion section after the proof itself.
First, we observe that (6) depends linearly in M. Therefore, if suffices to prove the case M = 1, and the general case follows by applying the theorem to E/M with subsequent rescaling. Next, define the expected loss for the n training clients, er Q), as an intermediate object:
1 Σ ni ANQ
(r(2)
E
E (m,y)-,
(, y, A(S.))
To bound @r(2) (r) we divide it in two parts: en( ēń( er(2) and éi(Q) ér()). We then bound each part separately and combine the results.
Part I For the former, for any h H let (.) 1
m ブラー ( yj h) (x A(z, y, h).
and (Q) = TREq (h),, such that (r()) - ēr(() = EA~o Σ (A))())) Then, for any 0,
eπ(@) - ( Ane
(A((()(S)) x.. A(S.)) X x P)
(8)
1
(.())) 1 Dx. ... x...xxxxxxx....... X x
< sup
(9)
Q On
e (h.))
S
(10)
h KPP
where the second inequality is due to the change of measure inequality (eelin al. 2012)
Because for each ....., n, independent of Si, Sn....n Sn, we have
E
e
E E (A)))) S1 ha
EEE (ha))
E
(11)
hi
S, R.MP
Each (,) is a bounded random variable with support in an interval of size 1. By Hoeffding's lemma we have
E E e
< mm
(12)
S.
h
Therefore,
E
II
n
2 (hi)
<
22 @aam
(13)
S1 hy
R
and by Markov's inequality, for any e R.
A (h())) 2 26 ) 22 e -e
Ps..... Sn h
e
(14)
Hence
Psi....S. (3Q Qn n
1 12 (()).) 1 xx..lQix....Qullll. xP) 2 1)
1
)t)
<
12
(15)
23

---

Published as a conference paper at ICLR 2024
equivalently,
"
Sp...... Sn (VQI..... On (()):)
(16)
7
1
E
DKL (QilP)))P) + log(
1
2
+
A
2
1
8
(17)
0
8nmm
2
By applying union bound for all the values of À E A with A = = . 4mn) we get that
Ss....... (VQI.....
1 Qn. AA
(Q))))
(18)
n
1 (KI(P))P)) log(
8mn
5
2 1
(19)
8nm
2
Note that [A] < A and
1 S x Therefore, for all values of A E (1, (1,4)) have [
Ps Ss Sn (VQ....., Qni AA E (1, 4mm) 1 n
Σ A())))
(20)
8mn
A Smn
8
DKL DKL (QillP) + log(
2
1
(21)
2
=:(())
For any choice of QI...., On, let X 4mn, that implies (Dku holds trivially. Otherwise, X E (0, 4mm), so Inequality
V /mm Σ - DKL (Q:lIP) + log( 8mm +1. If + log( Smn
&
1) 2mn and r(>*) so Inequality
21
(1 also holds. Therefore, we
P Ps......s (4Q (VQI.... Qn: Σ ((()))
En (Kl(Q.))P) log( 8mmn 5
+
8
2 1
2mn
2
1=1
(22)
In combination with (10). we obtain:
D DKL. (A(S)))))) log(
8mn 8
+
6
Ps..... Ps - S
E A~Q
2 1
2mn
2
(23)
Part For upper bounding er( ēr Q) we have by a standard PAC-Bayesian bound Maurer
2004 Pérez-Ortiz et al. 2021
P Ps....... S. (VQ: er(2) - ( <
DKL DKL log
Ayn 5
8
2 1
(24)
2n
2
Combination We combine
23 and 244 by a union bound to obtain
DKL(Q + log(
4
P Ps Sn (vQ:er(Q) - E(( S
(25)
2n
Σn
(KL(A(S))IPP + log( 8mn
+
E
21-8.
(26)
2mn
This concludes the proof of the theorem.
Discussion The difference in our proof and previous bounds in the similar settings (Pentina and
Lampert 2015 Rezazadeh 2022) in the upper bound for er Q) ér(() which is the average of
24

---

Published as a conference paper at ICLR 2024
the generalization gaps for the training tasks that we observe. In our we prove a generalization bound for arbitrary posteriors to be used by the tasks, using the change-of-measurg inequality (CMI) for the transition from Q1 X x Qn to P x ... x P. Because our bound holds uniformly in Qi,... On. it also holds for the posteriors produced by a deterministic learning algorithm (a fixed A) or any distributions of algorithms (a hyperposterior Q over algorithms), This way we do not have to include the transition from Q to P in the CMI, all prior work did. Consequently, in part of the proof we avoid getting (L()))) term which otherwise would be divided only by factor, which is undesirable in the federated setting. Additionally compared to (Rezazadeh) 2022) we have
better logarithmic dependency. They have a term of order logmy/T which is not negligible in the case that we have large number of tasks and small number of samples per task, which is usual in the federated setting. In contrast, our logarithmic terms are divided by n.
We now instantiate the situation of Theorem 3.2 by specific choice of prior and posterior distributions. Let the learning algorithm be parameterized by the hypernetwork weights, Mh, and the embedding networks weights, 7p- As meta-posterior we use a Gaussian distribution, Q = Qn x 2. for 2h = N (T) I,ld), and Q. = N (m)o a,ld), where ?i and 8/o are learnable and on and on, are fixed. For any (îj,, f7))
Proof of Theorem 3.2
and training set S. the learning algorithm produces posterior distribution Q = (0; agld), where 0 h(v; mh) with
[$] (x.))) Φ(2,, y; mu). As
prior, we use (O0 agld). With these choices, we have (2r, P) = a (rxl11al1 I1 + anlnvl?? and
(u, P) = |=| Inserting these into Theorem B.1 we recover Theorem 3.2
C APPENDIX - OPTIMIZATION
In this section, we formulate the technical assumptions for Theorem 3.1 and prove it. For the convenience of reading we repeat some notation from Section For each client 1,....1 n and parameter 0,, we note the client objective as f(())) and F((T) = (h((S, n)), where i(S, 7) = h(v(S, m))) is shorthand notation for the combination of hypernetwork and embedding network. The server objective we denote by f(), which in our setting consists only of regularization terms. The overall objective is F(n) = Σx= E1) + f(7). For the rest of this section, by 77, we mean the parameters at iteration I, and Dt,i.j the stochastic gradient for client at global step local step 1.
C.I ANALYTICAL ASSUMPTIONS
We make the following assumptions, which are common in the literature (Karimireddy et al. 2020 Fallah et al. 2020). Since our algorithm consists of multiple parts, we have to make the assumptions for difterent parts separately.
Bounded Gradients: there exist constants, b1, b2, such that
[Vo](00)] s bi, I[V,,(S, n)il s b2
(27)
which, in particular, implies
[M(S, 1((S,v))ll s b2117 7/1l Smoothness: there exist constants, L1, L2, La. such that
(28)
(V) Vefi(0'))l S 1 0'1l V.,((. m) - V,,h(S, r/)l Lalln 7'
(29)
(30)
(V) v,f(r/11 < salln-n'll n''l
(31)
Bounded Dissimilarity: there exists a constant 70 with, such that with F(n)
E F((r):
1
B(IVF -
(32)
25

---

Published as a conference paper at ICLR 2024
Bounded Variance: there exists constants 01, 02. 03, such that, for batches B, C S of size b |
E ((Vo(f((0) vo(f (TVo((())-<<(((())3>=>><>> oी
(33)
E [Vh(B, Vh(S,))) 1)) - 0f
(34)
E [|( 1) M(S, n)1 of
(35)
b
C.2 CONVERGENCE
Lemma C.1. There exists a constant which F is L-smooth.
Proof. For each F; we have
(V,) - - ".A
(36)
IV Vof(b(S..))VV,MSS...)
(37)
+ Vef.(M(S,
< VeffAHSS.., M
(38)
(39)
+ (IVof(((SS.:)),,((5..1)) Vof(4MS,,,1))V,,,55....7
(40)
$ 4/ - v/1l + b1bb2ll11 - 7/ = (2 = 7/1
(41)
Therefore for F we have
& 1 Σ V
IV F(/'/
V F1(')) v,,('ii
(42)
1 V,FF((n)
V, VV v,,(i) (L' L3))1n 1n - 1/'l
L| 7/
n
(43)
Lemma C.2. For independent random vectors 01 ... we we have
E
a < Σ Bllla - E[a:]1]]
(44)
SEC
Also E[ S a for each i, we have
E
a:
Billa: E|a:]]]9 caa
(45)
IC
Proof. By expanding the power 2 get
[||
Σ
a. -[aa]]]]]]]
EE
|a,
B[a:]11 E. Σ (a: B|a:), aj E[ali)]
(46)
IEC
sec
izjEC
(1-2)
Billa - E[a+]|]3+ +
E (a; - Ela.]. ag - E(as])]
(47)
2 n-2
E Billa; E|a.1119
c-2
Bllaa: E[a,]11
(48)
E E[la, [[[a]]19]
(49)
26

---

Published as a conference paper at ICLR 2024
For the second part we have
ElI
Σ
a.
1
||
Σ
a
B[aj] + [|a]]]
(50)
IEC
IEC
BlIl
Σ -
Blai11 + III
B || + 2
- Elas], Σ
B[a:)]
(51)
iEC
iEC
REC
B(|] Σ Fla.|]1
[
R Bla.]| [1 v Σ
Bllla; B[ai]]1 + e²a²
(52)
iEC
EEC
Lemma C.3. For any step and client the following inequality holds:
1 E (a.
([) Vof((((B..)) m.)
(53)
5=0
$ 227176339009 of) + 3b100 66
(54)
Proof. By adding and subtracting some immediate terms get
E BIVA(( VoJi(h(B,.n)) E It.. 9 (MBB
(55)
= =V] - Voli(h(s,,))
3
Σ
(56)
1=0
1-1
+ (1) - 3
Σ
vl(B,,))))
(57)
1=0
1-1
(Vof( m²)
Σ
Ie.s.
(h(S,, (h
Ve.(((BB.)))
E
g1, (h(B, C
1=
3=0
(58)
And by Cauchy-Schward inequality get
27

---

Published as a conference paper at ICLR 2024
B[[FF((H) VoJi(h(B,,n))
Σ gt.s (LL
(59)
NO - $ Σ 91, (.
(60)
+3E Vol n)) - Σ (MA - Vh(S,, T()
(61)
+3E (Vol( - B Σ
11
Me.s (h(,,,((,,,)))) V
Σ
1=0
1=0
(62)
(3 Voli(h(S,,,)) m)) - gti (MM5
(63)
j=0
+3B(()Vof((((S) nd 3
Vh(S:
(64)
1-1
1-1
+ VofMM md
Σ
It. -
Vofi(((Ss - (Ms)) m))
1=0
5=0
(65)
31,6388 E
Σ
gr.i.j (k),
+b6 VM
(66)
1=U
Σ
+ mz) (M5
E
Ih (N(S:;,)))
gr.i. (h( 11
j=0
j=u
(67)
<
188-88088---0---033--088-----883 +
(68)
(69)
which used the assumptions on the bounded gradients and bounded variance and Lemma C.2
Lemma C.4. For any step the following inequality holds:
-38K IVF
C((F(()) Mh+1 m)) <
+
+
3340 + +63kL3030
4
(70)
28

---

Published as a conference paper at ICLR 2024
Proof.
8 ΣΣ
B(VF(m)) 71+1 mb)l = BKV (IIVF(()))))) (m))
It. (h m)) - BbVf(m)))
SEC 2m1
(71)
= = 3
VoSi( -
Br.s (B
(72)
1=
+ +E((F(()) BK f(m)))
(73)
3
(MTF(,)VV.. Vo) - B gead (..
(74)
+
(VEF())
(75)
By definition F(n) = Ei= Fi(n) + f(n), and by Young's inequality and lemma C.3 we have
(VF(()) M++1 -1)i
(76)
3k
8 12
H gt... (M(B,,
$
E VF(m)) Vofi(h(B,,n) nd) -) -
Σ
4
5=0
(77)
Bkk VF()))1
(78)
-38k
8
[F(()]]1
(27L36887066 + + 6L 6L,333
(79)
4
i=
-38k
[VF(())]]
271,16550 +0 33 663T2
(80)
4
Lemma C.5. For any step t the following inequality holds:
4 4.
- -
01
2/
432
(81)
108L7633*<<003 (6} + 12b 32
24
(82)
Proof. The change of 1 in step of optimization is the average of the change caused by selected clients plus function (π] at the server.
3
Ellmn - m 1 = Ell
Ir.i (BB -Bm (83)
IEC 1=1
29

---

Published as a conference paper at ICLR 2024
Which write
82
k
Bl|l++1
T/8 1
It. (1a) ckV fm=))l
(84)
IEC
1=1
82
k
- E
gr.dd (h(B;, Wh) TV)
(4 nt)
ge.s (b(B,
IEC 1=1
5=0
(85)
V Vofi(h(B..nh)) -
VF((n))
(86)
IEC 1=1
j=0
k
(VF(()) + Vf(m) VF(N)) +
(87)
IEC 1=1
SEC 1=1
And by Sauchy-Schwarz inequality we get
E Bi17 m/p 1
(88)
4,32
$
E[l
Σ
Ie.s
Vofi(h(Br, me) - g (LL ())
19
iEC
(89)
4,2 0²
E[1 Voli(h(B,,.)) - Σ (Ma,(((B m. (.;(n)))
+
(90)
iEC 1=1
482
4,2
+ ²²
ElI
Σ
vR()) + VS())) - VF(m))] +
El|
V VF()))1
(91)
IEC 1=1
iEC 1=1
Note that VF - V/(m) = - VE()). By lemma
C.2 C.3
the bounded dissimilarity
assumption and simplifying terms we get
E Elllnt+1 1
(92)
k
4.
cn Σ B[I - - Σ
0f +
(93)
+ 10817 3 (12 + of) + 120 2112
(94)
4
2.
01
26 14 11| V (1)
(95)
+ 32
(b} + of) 12b 24L
(96)
We now ready prove Theorem 3.1 which restate for the convenience of the reader.
Theorem 3.1
Under the assumptions
C.1 PeFLL's 's optimization after steps fulfills
(F(no) F.)
22
Σ EIIF(())
+ 8b302 14
(97)
VeT
T
42 + k72)
(98)
kveT
Proof. By smoothness of F we have
7 E 1|
E[F(n+1) F F(n)] < E(V((n). M441 mm)
mm12
(99)
2
30

---

Published as a conference paper at ICLR 2024
And by combining it with Lemma C.4 and C.5
and simplifying terms get
L T. |1/ - 17el2
E
F((+1))))
F(n)) - (.V)(m)) 178) +
(100)
2
33k
E
(m))l
+ + +f + 333kb003 + 6,3kL,6,03 (101)
4
kL
o 2L
+
10 2
(102)
+
(103)
-33k
+ +2 +50
(104)
+ 3,362002
(105)
LB2K
k2 10
(106)
E ([ +22 43kb 7
(107)
+ k of +
kL
G
(108)
Therefore
3k
[TF((++) F(m)) -
E
+ +436b/02 +
+7BkL,6303
(109)
2
3L k 0 +
kL
NG
(110)
Hence
2 ETF(n) ((n-1)) +663366662233 + 88302 + 14L76303
VF()))1 -
(111)
3k
6LB
4L33
of +
16
(112)
And
1 T
((F( - FF.)
+ 56133663332 + 86f03 + 14L36303
(113)
T
BKT
6TB
4L33
of
(114)
by setting 8
2 K/T
we get
T E VF(2)) < (F(1)) - F.).
T
bb
Σ
+ 88 + 141,
(115)
VCT
T
1=1
I(Go? + 4k-7)
(116)
kveT
31


Reviews:
[{"review_id": "zjBDwmhekR", "reviewer": "Reviewer_Couf", "summary": "This paper proposes a learning-to-learn approach to personalized federated learning. A hypernetwork residing on the server outputs the parameters of the personalized models given “descriptors” of the clients. The descriptor is generated by an embedding model that is shared by the server to the clients so that they can use it on their private data. This method avoids some issues in personalized federated learning by construction, such as the need to fine-tune, how to generalize to unseen clients, clients having little data, system/data/model heterogeneity, etc. Convergence proofs and generalization bounds are given for their method PeFLL. Experiments show that this model is indeed effective by a large margin, especially at generalizing to unseen clients.", "questions": "- Can you share some details about training time, memory consumption, and computational efficiency of PeFLL vs. regular models?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and the approach is interesting.\n- Heterogeneity issues can be alleviated with a learning-to-learn approach.\n- Generalizing to unseen clients makes sense with the embedding model. We intuitively assume that client similarity is dictated by the similarity of their data. Thus, clients with similar embeddings should have similar personalized models.\n- Algorithms are well detailed and the code is shared in the supplementary materials for reproducibility.\n- Experiments show significant improvements on FEMNIST and CIFAR-10/100, especially on clients unseen during training.\n- The benefits of client embedding network is justified experimentally.", "weaknesses": "- I think that the experiments are not sufficient to conclude that PeFLL is superior. You have to demonstrate its improvement on architectures more modern than LeNet as well. It would also be great to report performance on federated datasets other than FEMNIST (e.g. other LEAF datasets).\n- The computational efficiency of the model is questionable given that the hypernetwork is at least a 100 times bigger than the client's model, and this is only the case for these particular experiments. It might require a wider net to generate larger personalized models. In other words, LeNet on MNIST and CIFAR-10/100 might only need 100 \"ranks\" to perform well, whereas on more complicated models and datasets, a wider net would be needed for good performance, which makes scalability an issue. That is why I think experiments covering these settings are needed to conclude the practical benefits of PeFLL.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZWzNgJ6OPx", "reviewer": "Reviewer_Fw5z", "summary": "The authors present a federated training algorithm which allows clients to receive a personalized model at test-time, based on their individual local dataset. This is achieved by predicting personalized network parameters based on an individual client's descriptor. This descriptor (a vector embedding), is generated by passing the local dataset through a set network. The embedding computation is done locally, while the (expensive) hypernetwork is run on the server. \nThe training algorithm involves reversing the forward-pass and computing partial gradients at the server and the individual clients.\n\n-- post rebuttal --\nThe authors have addressed my concerns and I do recommend acceptance.\nWhile some question with respect to the method's generalizability to new tasks & models remain, I believe it to be a worthwhile method to experiment with for a new personalization use-case.", "questions": "- The authors only consider the label-skew setting of object-classification computer vision problems C10/C100 and Femnist. Specifically for C10/C100, the authors assign two labels per client. The provided code (mask_absent is True by default) suggest that their method consequently reduces to a binary classification problem at test time. Can you confirm this choice of evaluation is consistent with all baselines? Evaluation in line 374 & 375 of train.py concerns the test-clients. Following into line 20, evaluation is done. The default args embed_split=True suggest that you are using the training-set to compute embeddings, as well as (lines 58-76) using it to test for presence of classes on that client. It seems that for all (train & test) clients, you assume the presence of a labelled training set, which allows for masking your logits. Lines 24-38 suggest you're treating training & test clients identically. I would expect a \"test-client\" to not have any labelled data available, meaning that the logit-masking should not be allowed. Can you clarify your evaluation procedure for me? Do \"test-clients\" use a different training-set for embedding computation? I would expect those to be identical. \nFor the training-clients, I would be curious to understand if you use the labelled training data for embedding computation, or throw those away for the validation-set in Table 1a. You claim that the client descriptor network is not susceptible to overfitting. I would ask you to show evidence for that claim. \n- How does PeFLL fair in non-label skew settings? E.g. I would recommend the Shakespeare dataset (https://arxiv.org/abs/1812.01097) \n- In order to fairly compare PeFFL against baselines, I believe it is necessary to introduce the communication overhead that is introduced by the different methods. You argue that \"It has low latency and communication costs\", which should be clarified to be true only at test-time (and even then is higher than alternatives that function offline, such as FedAvg). During training, PeFFL is constrained to \"FedSGD\", i.e. a client cannot amortize communication with additional computation. Instead, forward-backward pass requires communication of various elements: Embedding, client-specific model and the gradients corresponding to both. I would like to see two analyses:\n1.) GB of communication on the x-axis and test-client accuracy at the y-axis. This allows to draw iso-lines and understand the impact of the high-frequency communication of PeFFL.\n2.) Introduction of another baseline, in which a larger, more flexible model is trained through FedAdam (https://arxiv.org/abs/2003.00295) with properly tuned hyper-parameters. The hypothesis I would like to see tested is if a more flexible model is able to generalize across all train & test clients without requiring personalization. For a given communication budget, should I choose PeFFL or can I fall back to a larger FedAvg-trained model?\n- Another interesting experiment (though not required), would be to see a combination of classical FedAvg and PeFFL. Consider applying PeFFL only to the last layer(s) of a model, while training the rest through standard FedAvg. For object classification, it is usually the last layers that require personalization, while the initial layers can share jointly relevant features. Such a setup would allow the embedding network to be a separate head of the FedAvg-trained network. \n\nShould the authors address my questions, I will consider raising my score", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This paper addresses a highly relevant problem in an innovative way. The proposed test-time generation of personalized models through a server-side hypernetwork is original and well-described. The paper reads very well, I enjoyed following the authors' red line through the paper.", "weaknesses": "The weaknesses of this paper revolve around the experimental evaluation, specifically the base-lines that the authors consider given their choice of the experimental setting. The evaluation pipeline raises som questions, detailed below. \nThe comparison to baselines is somewhat unfair, due to a lack of measuring communication overhead.\nFinally, i believe the discussion of related works should include the topic of Split-Learning, a version of FL where parts of the forward-pass are performed across different devices (e.g. client and server). PeFFL is an example of split-learning under that definition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zKZoDlblG6", "reviewer": "Reviewer_tVNw", "summary": "This paper proposes a learning-to-learn framework for personalized FL, where the goal is to learn a global hypernetwork that outputs each clients' personalized model. A shared embedding network is also trained, which produces each client's descriptor based on its local dataset. During inference, any client (even clients that did not participate during training) can send its descriptor to the server and receive the personalized model. This enables each client to obtain the model without further finetuning at test-time. Convergence analysis as well as generalization bound analysis are provided. The effectiveness of proposed idea is also confirmed via experiments.", "questions": "See weakness above.\n\n\n\n\n------------------------------------------------\n\nFinal Update: I appreciate the authors for providing additional experiments. I think this paper is above the acceptance threshold, and would like to keep my original score 6.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of taking advantage of learning-to-learn and hypernetworks in personalized FL is interesting.\n\n2. The proposed idea has several advantages. Specifically, any client can obtain its personalized model by conducting only forward propagations.\n\n3. Experimental results show significant performance advantage of the proposed method.\n\n4. Theoretical results are also provided, further strengthening the paper.", "weaknesses": "1. All experiments are conducted using LeNet-style models, which are relatively outdated. I would like to see the performance on ResNet-style models.\n\n2. In Table 2, the performance of \"Local\" is missing. Moreover, during experiments, what is the number of local updates for the baselines like Per-FedAvg and pFedMe. How do their performance improve as the number of local updates increases, and can they achieve similar performance compared to the proposed PeFLL?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)