Federated User Representation Learning

1 INTRODUCTION . Collaborative personalization , like learning user embeddings jointly with the task , is a powerful way to improve accuracy of neural-network-based models by adapting the model to each user ’ s behavior ( Grbovic & Cheng , 2018 ; Ni et al. , 2018 ; Lee et al. , 2017 ; Jaech & Ostendorf , 2018 ; McGraw et al. , 2016 ; Vosecky et al. , 2014 ) . However , model personalization usually assumes the availability of user data on a centralized server . To protect user privacy , it is desirable to train personalized models in a privacy-preserving way , for example , using Federated Learning ( McMahan et al. , 2016 ; Konen et al. , 2016b ) . Personalization in FL poses many challenges due to its distributed nature , high communication costs , and privacy constraints ( Li et al. , 2019a ; Bonawitz et al. , 2019 ; Caldas et al. , 2018 ; Li et al. , 2019b ; 2018 ; Liu et al. , 2018 ; Yang et al. , 2019 ; Konen et al. , 2016a ) . To overcome these difficulties , we propose a simple , communication-efficient , scalable , privacypreserving scheme , called FURL , to extend existing neural-network personalization to FL . FURL can personalize models in FL by learning task-specific user representations ( i.e. , embeddings ) ( Lerer et al. , 2019 ; Grbovic & Cheng , 2018 ; Ni et al. , 2018 ; Lee et al. , 2017 ; Jaech & Ostendorf , 2018 ) or by personalizing model weights ( Tang & Wang , 2018 ) . Research on collaborative personalization in FL ( Smith et al. , 2017 ; Sebastian Caldas , 2019 ; Chen et al. , 2018 ; Yao et al. , 2019 ) has generally focused on the development of new techniques tailored to the FL setting . We show that most existing neural-network personalization techniques , which satisfy the split-personalization constraint ( 1,2,3 ) , can be used directly in FL , with only a small change to Federated Averaging ( McMahan et al. , 2016 ) , the most common FL training algorithm . Existing techniques do not efficiently train user embeddings in FL since the standard Federated Averaging algorithm ( McMahan et al. , 2016 ) transfers and averages all parameters on a central server . Conventional training assumes that all user embeddings are part of the same model . Transferring all user embeddings to devices during FL training is prohibitively resource-expensive ( in terms of communication and storage on user devices ) and does not preserve user privacy . FURL defines the concepts of federated and private parameters : the latter remain on the user device instead of being transferred to the server . Specifically , we use a private user embedding vector on each device and train it jointly with the global model . These embeddings are never transferred back to the server . We show theoretically and empirically that splitting model parameters as in FURL affects neither model performance nor the inherent structure in learned user embeddings . While global model aggregation time in FURL increases linearly in the number of users , this is a significant reduction compared with other approaches ( Smith et al. , 2017 ; Sebastian Caldas , 2019 ) whose global aggregation time increases quadratically in the number of users . FURL has advantages over conventional on-server training since it exploits the fact that models are already distributed across users . There is little resource overhead in distributing the embedding table across users as well . Using a distributed embeddings table improves the memory locality of both training embeddings and using them for inference , compared to on-server training with a centralized and potentially very large user embedding table . Our evaluation of document classification tasks on two real-world datasets shows that FURL has similar performance to the server-only approach while preserving user privacy . Learning user embeddings improves the performance significantly in both server training and FL . Moreover , user representations learned in FL have a similar structure to those learned in a central server , indicating that embeddings are learned independently yet collaboratively in FL . In this paper , we make the following contributions : • We propose FURL , a simple , scalable , resource-efficient , and privacy preserving method that enables existing collaborative personalization techniques to work in the FL setting with only minimal changes by splitting the model into federated and private parameters . • We provide formal constraints under which the parameter splitting does not affect model performance . Most model personalization approaches satisfy these constraints when trained using Federated Averaging ( McMahan et al. , 2016 ) , the most popular FL algorithm . • We show empirically that FURL significantly improves the performance of models in the FL setting . The improvements are 8 % and 51 % on two real-world datasets . We also show that performance in the FL setting closely matches the centralized training with small reductions of only 0 % and 4 % on the datasets . • Finally , we analyze user embeddings learned in FL and compare with the user representations learned in centralized training , showing that both user representations have similar structures . 2 RELATED WORK . Most existing work on collaborative personalization in the FL setting has focused on FL-specific implementations of personalization . Multi-task formulations of Federated Learning ( MTL-FL ) ( Smith et al. , 2017 ; Sebastian Caldas , 2019 ) present a general way to leverage the relationship among users to learn personalized weights in FL . However , this approach is not scalable since the number of parameters increases quadratically with the number of users . We leverage existing , successful techniques for on-server personalization of neural networks that are more scalable but less general , i.e. , they satisfy the split-personalization constraint ( 1,2,3 ) . Transfer learning has also been proposed for personalization in FL ( Hartmann , 2018 ) , but it requires alternative freezing of local and global models , thus complicating the FL training process . Moreover , some versions ( Zhao et al. , 2018 ) need access to global proxy data . Chen et al . ( 2018 ) uses a twolevel meta-training procedure with a separate query set to personalize models in FL . FURL is a scalable approach to collaborative personalization that does not require complex multiphase training , works empirically on non-convex objectives , and leverages existing techniques used to personalize neural networks in the centralized setting . We show empirically that user representations learned by FURL are similar to the centralized setting . Collaborative filtering ( Ammad-ud-din et al. , 2019 ) can be seen as a specific instance of the generalized approach in FURL . Finally , while fine-tuning individual user models after FL training ( Popov & Kudinov , 2018 ) can be effective , we focuses on more powerful collaborative personalization that leverages common behavior among users . 3 LEARNING PRIVATE USER REPRESENTATIONS . The main constraint in preserving privacy while learning user embeddings is that embeddings should not be transferred back to the server nor distributed to other users . While typical model parameters are trained on data from all users , user embeddings are very privacy-sensitive ( Resheff et al. , 2018 ) because a user ’ s embedding is trained only on that user ’ s data . FURL proposes splitting model parameters into federated and private parts . In this section , we show that this parameter-splitting has no effect on the FL training algorithm , as long as the FL training algorithm satisfies the split-personalization constraint . Models using common personalization techniques like collaborative filtering , personalization via embeddings or user-specific weights satisfy the split-personalization constraint when trained using Federated Averaging . 3.1 SPLIT PERSONALIZATION CONSTRAINT . FL algorithms typically have two steps : 1 . Local Training : Each user initializes their local model parameters to be the same as the latest global parameters stored on the server . Local model parameters are then updated by individual users by training on their own data . This produces different models for each user.1 2 . Global Aggregation : Locally-trained models are ” aggregated ” together to produce an improved global model on the server . Many different aggregation schemes have been proposed , from a simple weighted average of parameters ( McMahan et al. , 2016 ) , to a quadratic optimization ( Smith et al. , 2017 ) . To protect user privacy and reduce network communication , user embeddings are treated as private parameters and not sent to the server in the aggregation step . Formal conditions under which this splitting does not affect the model quality are described as follows . Suppose we train a model on data from n users , and the k-th training example for user i has features xik , and label y i k. The predicted label is ŷ i k = f ( x i k ; wf , w 1 p , . . . , w n p ) , where the model has federated parameters wf ∈ Rf and private parameters wip ∈ Rp ∀i ∈ { 1 , . . . , n } . In order to guarantee no model quality loss from splitting of parameters , FURL requires the splitpersonalization constraint to be satisfied , i.e. , any iteration of training produces the same results irrespective of whether private parameters are kept locally , or shared with the server in the aggregation step . The two following constraints are sufficient ( but not necessary ) to satisfy the splitpersonalization constraint : local training must satisfy the independent-local-training constraint ( 1 ) , and global aggregation must satisfy the independent-aggregation constraint ( 2,3 ) . 3.2 INDEPENDENT LOCAL TRAINING CONSTRAINT . The independent-local-training constraint requires that the loss function used in local training on user i is independent of private parameters for other users wjp , ∀j 6= i . A corollary of this constraint is that for training example k on user i , the gradient of the local loss function with respect to other users ’ private parameters is zero : ∂L ( ŷik , y i k ) ∂wjp = ∂L ( f ( xik ; wf , w 1 p , . . . , w n p ) , y i k ) ∂wjp = 0 , ∀j 6= i ( 1 ) Equation 1 is satisfied by most implementations of personalization techniques like collaborative filtering , personalization via user embeddings or user-specific model weights , and MTL-FL ( Smith et al. , 2017 ; Sebastian Caldas , 2019 ) . Note that ( 1 ) is not satisfied if the loss function includes a norm of the global user representation matrix for regularization . In the FL setting , global regularization of the user representation matrix is impractical from a bandwidth and privacy perspective . Even in centralized training , regularization of the global representation matrix slows down training a lot , and hence is rarely used in practice ( Ni et al. , 2018 ) . Dropout regularization does not violate ( 1 ) . Neither does regularization of the norm of each user representation separately . 1Although not required , for expositional clarity we assume local training uses gradient descent , or a stochastic variant of gradient descent . 3.3 INDEPENDENT AGGREGATION CONSTRAINT . The independent-aggregation constraint requires , informally , that the global update step for federated parameters is independent of private parameters . In particular , the global update for federated parameters depends only on locally trained values of federated parameters , and optionally , on some summary statistics of training data . Furthermore , the global update step for private parameters for user i is required to be independent of private parameters of other users , and independent of the federated parameters . The global update for private parameters for user i depends only on locally trained values of private parameters for user i , and optionally , on some summary statistics . The independent-aggregation constraint implies that the aggregation step has no interaction terms between private parameters of different users . Since interaction terms increase quadratically in the number of users , scalable FL approaches , like Federated Averaging and its derivatives ( McMahan et al. , 2016 ; Leroy et al. , 2018 ) satisfy the independent-aggregation assumption . However , MTL-FL formulations ( Smith et al. , 2017 ; Sebastian Caldas , 2019 ) do not . More formally , at the beginning of training iteration t , let wtf ∈ Rf denote federated parameters and wi , tp ∈ Rp denote private parameters for user i . These are produced by the global aggregation step at the end of the training iteration t− 1 . Local Training At the start of local training iteration t , model of user i initializes its local federated parameters as utf , i : = w t f , and its local private parameters as u i , t p , i : = w i , t p , where u represents a local parameter that will change during local training . ui , tp , i denotes private parameters of user i stored locally on user i ’ s device . Local training typically involves running a few iterations of gradient descent on the model of user i , which updates its local parameters utf , i and u i , t p , i . Global Aggregation At the end of local training , these locally updated parameters u are sent to the server for global aggregation . Equation 2 for federated parameters and Equation 3 for private parameters must hold to satisfy the independent-aggregation constraint . In particular , the global update rule for federated parameters wf ∈ Rf must be of the form : wt+1f : = af ( w t f , u t f,1 , . . . , u t f , n , s1 , . . . , sn ) ( 2 ) where utf , i is the local update of wf from user i in iteration t , si ∈ Rs is summary information about training data of user i ( e.g. , number of training examples ) , and af is a function from Rf+nf+ns 7→ Rf . Also , the global update rule for private parameters of user i , wip ∈ Rp , must be of the form : wi , t+1p : = ap ( w i , t p , u i , t p , i , s1 , . . . , sn ) ( 3 ) where ui , tp , i is the local update of w i p from user i in iteration t , si ∈ Rs is summary information about training data of user i , and ap is a function from R2p+ns 7→ Rp . In our empirical evaluation of FURL , we use Federated Averaging as the function af , while the function ap is the identity function wi , t+1p : = u i , t p , i ( more details in Section 3.4 ) . However , FURL ’ s approach of splitting parameters is valid for any FL algorithm that satisfies ( 2 ) and ( 3 ) .

Summary: Authors proposed a formal training scheme (FURL) for personalized federated models. The claimed benefits of such models are 1) preservation of user privacy by keeping the personalized parameters locally on each user's device, and 2) reduced data exchange to make the training complexity grow linearly with the number of users. Authors approached the problem with defining the constraint of split personalization, and argued that common FL setting such as Federated Averaging could satisfy this constraint.