Federated User Representation Learning
Authors: Duc Bui
Kshitiz Malik
Jack Goetz
Seungwhan Moon
Honglei Liu
Anuj Kumar
Kang G. Shin
Venue:  (2020)
Keywords: Machine Learning
Federated Learning
Personalization
User Representation

Abstract: Collaborative personalization, such as through learned user representations (embeddings), can improve the prediction accuracy of neural-network-based models significantly. We propose Federated User Representation Learning (FURL), a simple, scalable, privacy-preserving and resource-efficient way to utilize existing neural personalization techniques in the Federated Learning (FL) setting. FURL divides model parameters into federated and private parameters. Private parameters, such as private user embeddings, are trained locally, but unlike federated parameters, they are not transferred to or averaged on the server. We show theoretically that this parameter split does not affect training for most model personalization approaches. Storing user embeddings locally not only preserves user privacy, but also improves memory locality of personalization compared to on-server training. We evaluate FURL on two datasets, demonstrating a significant improvement in model quality with 8% and 51% performance increases, and approximately the same level of performance as centralized training with only 0% and 4% reductions. Furthermore, we show that user embeddings learned in FL and the centralized setting have a very similar structure, indicating that FURL can learn collaboratively through the shared parameters while preserving user privacy.

Under review as a conference paper at ICLR 2020
FEDERATED USER REPRESENTATION LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Collaborative personalization, such as through learned user representations (em- beddings), can improve the prediction accuracy of neural-nettork--baee models significantly. We propose Federated User Representation Learning (FURL), a simple, scalable, privacy-preservee and resource-efficient way utilize existing neural personalization techniques in the Federated Learning (FL) setting. FURL divides model parameters into federated and private parameters. Private param- eters, such as private user embeddings, are trained locally, but unlike federated parameters, they are not transferred to averaged on the server. We show the- oretically that this parameter split does not affect training for most model per- sonalization approaches. Storing user embeddings locally not only preserves user privacy, but also improves memory locality of personalization compared to on- server training. We evaluate FURL on two datasets, demonstrating a significant improvement in model quality with 8% and 51% performance increases, and ap- proximately the same level of performance as centralized training with only 0% and 4% reductions. Furthermore, we show that user embeddings learned in FL and the centralized setting have a very similar structure, indicating that FURL can learn collaboratively through the shared parameters while preserving user privacy.
INTRODUCTION
Collaborative personalization, like learning user embeddings jointly with the task, is a powerful way to improve accuracy of neuuallnetwork-baaee
models by adapting the model to each user's behavior (Grbovic & Cheng. 20188: Ni et al. 2018 Lee et al.) 2017 Jaech & Ostendorf] 2018 McGraw et al.) 2016) Vosecky et al. 2014). However, model personalization
usually assumes the centralized server. To protect user privacy, it is desirable to train personalized models a privacy-preserving way, for example, using Federated Learning (McMahan 2016b). Personalization in FL poses many challenges due to its distributed 2019
availability of user data
al. 2016 Konen al.
nature, high communication costs, and privacy constraints (Li et al.) 2019a Bonawitz et al.)
Caldas et al. 2018 Li et al. 2019b 2018
Liu et al. 2018 Yang
2019 Konen al. 201
To overcome these difficulties, we propose a simple, communication--ffiiii scalable, privacy- preserving scheme, called FURL, extend existing neural-network personalization FI.. FURL can personalize models in FL by learning task-specific user representations (i.c.. embeddings) (eerer al. 2019 Grbovic & Cheng 2018 Ni al. 2018 Lee al., 2017 Jacch & Ostendori 2018) or by personalizing model weights (Tang & Wang 8). Research on collaborative personalization
in FL (Smith et al. 2017 Sebastian Caldas] 2019 Chen al. 2018 Yao et al. 2019) has generally focused on the development of new techniques tailored to the FL setting. We show that most existing
neural-network personalization techniques, which satisfy the split-personalliatiin constraint 1 3)
can be used directly in FL, with only a small change to Federated Averaging (McMahan et al. 2016
the most common FL training algorithm.
Existing techniques do not efficiently train embeddings in FL since the standard Federated Av- craging algorithm (McMahan et al. 2016) transfers and averages all parameters on a central server. Conventional training assumes that all user embeddings all user embeddings to devices during FL training is prohibitively resource-expensive (in terms of communication and storage user devices) and does not preserve user privacy.
part of the same model. Transferring
FURL defines the concepts of federated and private parameters: the latter remain on the user device instead of being transferred the server. Specifically, we use a private embedding vector on

---

Under review as a conference paper at ICLR 2020
each device and train it jointly with the global model. These embeddings never transferred back to the server.
We show theoretically and empirically that splitting model parameters in FURL affects neither model performance nor the inherent structure in learned user embeddings. While global model ag- gregation time in FURL increases linearly in the number of users, this is significant reduction compared with other approaches (Smith et al. 2017 /Sebastian Caldas 2019) whose global aggre- gation time increases quadratically in the number of users.
FURL has advantages over conventional on-server training since it exploits the fact that models are already distributed across users. There little resource overhead in distributing the embedding table across users as well. Using distributed embeddings table improves the memory locality of both training embeddings and using them for inference, compared on-server training with a centralized and potentially very large user embedding table.
Our evaluation of document classification tasks on two real-world datasets shows that FURL has similar performance to the server-only approach while preserving user privacy. Learning user em- beddings improves the performance significantly in both server training and FL. Moreover, user representations learned in FL. have similar structure to those learned in a central server, indicating that embeddings are learned independently yet collaboratively FI..
In this paper, we make the following contributions:
We propose FURL, a simple, scalable, aasource-efficient and privacy preserving method that enables existing collaborative personalization techniques work in the FL setting with only min- imal changes by splitting the model into federated and private parameters.
We provide formal constraints under which the parameter splitting does not affect model per- formance. Most model personalization approaches satisfy these constraints when trained using Federated Averaging (McMahan al. 2016), the most popular FL algorithm.
We show empirically that FURL significantly improves the performance of models in the FL setting. The improvements are 8% and 51% on two real-world datasets. We also show that performance in the FL setting closely matches the centralized training with small reductions of only 0% and 4% on the datasets.
Finally, we analyze user embeddings learned in FL and compare with the user representations learned in centralized training. showing that both user representations have similar structures.
RELATED WORK
Most existing work on collaborative personalization in the FL setting has focused on FL-specific im- plementations of personalization. Multi-task task formulations of Federated Learning (MTL-FL) (Smith
ct al. 2017 Sebastian Caldas] 2019) present a general way to leverage the relationship among users learn personalized weights in FL.. However, this approach is not scalable since the number of parameters increases quadratically with the number of users. We leverage existing, successful tech- niques for on-server personalization of neural networks that are more scalable but less general, i.c., they satisfy the splii-personalization constraint 12
Transfer learning has also been proposed for personalization in FL (Hartmann) 2018), but it requires alternative freezing of local and global models, thus complicating the FL training process. Moreover, some versions (Zhao al. 2018) need access to global proxy data. Chen et al. (2018) uses a two- level meta-training procedure with separate query set to personalize models in FL.
FURL scalable approach to collaborative personalization that does not require complex multi- phase training, works empirically on non-convex objectives, and leverages existing techniques used to personalize neural networks in the centralized setting. We show empirically that user representa- tions learned by FURL are similar to the centralized setting. Collaborative filtering (Ammad-ud-din et al. 2019) can be seen as a specific instance of the generalized approach in FURL. Finally, while fine-tuning individual user models after FL training (Popoy & Kudinov] 2018) can be effective, we focuses on more powerful collaborative personalization that leverages common behavior among users.
2

---

Under review as a conference paper at ICLR 2020
3 LEARNING PRIVATE USER REPRESENTATIONS
The main constraint in preserving privacy while learning user embeddings is that embeddings should not be transferred back to the server nor distributed to other users. While typical model parameters are trained on data from all users, user embeddings are very privacy-sensitive (Resheff et al. 2018) because user's embedding is trained only on that user's data.
FURL proposes splitting model parameters into federated and private parts. In this section, we show that this parameter-splitueee has no effect on the FL training algorithm, as long the FL training algorithm satisfies the split-personalization constraint. Models using common personalization tech- niques like collaborative filtering, personalization via embeddings user-specific weights satisfy the split-personalizattoon constraint when trained using Federated Averaging.
3.1 SPLIT PERSONALIZATION CONSTRAINT
FL algorithms typically have two steps:
1. Local Training: Each user initializes their local model parameters to be the same as the latest global parameters stored on the server. Local model parameters are then updated by individual users by training on their own data. This produces different models for each user []
Global Aggregation: Locally-trained models are "aggregated" together to produce an improved global model on the server. Many different aggregation schemes have been proposed, from a sim- ple weighted average of parameters (MeMahan et al. 2016), to a quadratic optimization (Smith) et al. 2017).
To protect user privacy and reduce network communication, user embeddings are treated private parameters and not sent to the server in the aggregation step. Formal conditions under which this splitting does not affect the model quality are described as follows.
Suppose we train a model on data from n users, and the k-th training example for user i has features ri. and label v/. The predicted label is 84 = f ; wy, wp.... WF). where the model has federated parameters wy E IR and private parameters vo E RP vi E (1....,,).
In order to guarantee model quality loss from splitting of parameters, FURL requires the split- personalization constraint to be satisfied, i.e., any iteration of training produces the same results irrespective of whether private parameters are kept locally, shared with the server in the aggre- gation step. The two following constraints are sufficient (but not necessary) to satisfy the split- personalization constraint: local training must satisfy the independent- -oa---llllll--inngg constraint ( and global aggregation must satisfy the independent independen--aggreeattin constraint (3
3.2 INDEPENDENT LOCAL TRAINING CONSTRAINT
The independent local- rrcal-training constraint requires that the loss function used in local training on user i is independent of private parameters for other users ws. Vj # i. A corollary of this constraint that for training example on user i, the gradient of the local loss function with respect to other
users' private parameters is zero:
a vtt (f(f wy, w.. wp.................. ................................ w vi
0, Vjj ti
(I)
Owh
Owh
Equation T is satisfied by most implementations of personalization techniques like collaborative
filtering, personalization via user embeddings or user-specific model weights, and -TL.-FL. (Smith) ct al. 2017 Sebastian Caldas 2019). Note that ([ is not satisfied if the loss function includes a norm of the global user representation matrix for regularization. In the FL setting, global regularization of the user representation matrix is impractical from a bandwidth and privacy perspective. Even in centralized training, regularization of the global representation matrix slows down training lot, and hence is rarely used in practice (Ni et al.) 2018). Dropout regularization does not violate I Neither does regularization of the norm of each user representation separately.
A lhough not required, for expositional clarity we assume local training uses gradient descent, or stochas- tie variant of gradient descent.
3

---

Under review as a conference paper at ICLR 2020
3.3 INDEPENDENT AGGREGATION CONSTRAINT
The independent-aggress ated parameters is independent of private parameters. In particular, the global update for federated parameters depends only on locally trained values of federated parameters, and optionally, on some summary statistics of training data.
constraint requires, informally, that the global update step for feder-
Furthermore, the global update step for private parameters for user i is required to be independent of private parameters of other users, and independent of the federated parameters. The global update for private parameters for user i depends only on locally trained values of private parameters for user i, and optionally, on some summary statistics.
constraint implies that the aggregation step has no interaction terms between private parameters of different users. Since interaction terms increase quadratically in the number of users, scalable FL approaches, like Federated Averaging and its derivatives (McMahan ctt al., 2016 Leroy et al. 2018) satisfy the
The independent-aggres
assumption. However, MTL-FL formulations (Smith et al. 2017 Sebastian Caldas 2019) do not.
More formally, at the beginning of training iteration let w E R denote federated parameters and we/ E I2 denote private parameters for user i. These are produced by the global aggregation step the end of the training iteration t
Local Training At the start of local training iteration 4, model of user i initializes its local feder- ated parameters as u := wf. and its local private parameters as u, wht where represents a local parameter that will change during local training. up denotes private parameters of user stored locally on user i's device. Local training typically involves running few iterations of gradient descent on the model of user which updates its local parameters v',, and u.f p.3
1.f
Global Aggregation At the end of local training, these locally updated parameters u are sent to the server for global aggregation. Equation 2 for federated parameters and Equation 3 for private parameters must hold to satisfy the cerppendent-aggrgartte update rule for federated parameters Wy E R must be of the form:
constraint. particular, the global
with a u uj............. f.n $1, $ 8x)
where u/y,: is the local update of wy from user in iteration t, s; R is summary information about training data of user i (e.g., number of training examples), and a is a function from IR /t+nf+ns + RR/
Also, the global update rule for private parameters of user w E RP, must be of the form: r++1 = (w $1.5881..... S1,...,8p) "
where ppii
is the local update of w from user in iteration f, 8; IR is summary information about training data of user i, and ap is a function from 22 + RP.
In empirical evaluation of FURL,,
Federated Averaging the function ay. while the function ap is the identity function w == t (more details in Section 3.4). However, FURL's
approach of splitting parameters is valid for any FL. algorithm that satisfies and (
3.4 FURL WITH FEDERATED AVERAGING
FURL works for all FL algorithms that satisfy the split-personalizattonn constraint. Our empirical evaluation of FURL uses Federated Averaging (McMahan et al. 2016), the most popular FL algo-
rithm.
The global update rule of vanilla Federated Averaging satisfies the independdent-oggrregation straint since the global update of parameter w after iteration t is:
1= ufc
w
1-
(4)
Σ Ci
4

---

Under review as a conference paper at ICLR 2020
Server
MLP
User
BLSTMM
User
Character emboddings
BLSTM
param
emb
emb
Character
Character
Privato
embeddings
param
Figure 1:
Personalized Document Model FL.
where o the number of training examples for user i, and uf is the value of parameter after local training on user in iteration l. Recall that uf is initialized to w/ at the beginning of local training. Our implementation uses a small tweak to the global update rule for private parameters to simplify implementation, described below.
In practical implementations Federated Averaging (Bonawitz et al. 2019). instead of sending trained model parameters the server, user devices send model deltas, i.e., the difference between the original model downloaded from the server and the locally-traine model: di := us w', or uf di w.. Thus, the global update for Federated Averaging in Equation 4 can be written as:
Σ
(d) + w*)a
Σ dic; E. w'c
dic
w 1+1
w²
(5)
E
E Ci
EE
Since most personalization techniques follow Equation I the private parameters of user i, wy. don't change during local training on other users. Let d be the model delta of private parameters of user i after local training on user in iteration I, then di 0, Vj + i. Equation 5 for updating private parameters of user can hence be written as:
E/=1 d.t.
i.1+1 wh + up
w 1.t
+
wht + 2
(6)
44
E Gi
where 23
The second term in the equation above is multiplied by noisy scaling factor 24, an artifact of per- user example weighting Federated Averaging. While is not an essential part of FURL, our implementation ignores this scaling factor for private parameters. Sparse-gradien approaches for learning representations
centralized training (Abadi al.) 2015 Paszke et al.) 2017) also ignore a similar scaling factor for efficiency reasons. Thus, for the private parameters of user i, we simply retain the value after local training on user (i.e., 2; = 1) since it simplifies implementation and does not affect the model performance:
"
where ve. is the local update of w from user i in iteration f.. In other words, the global update rule for private parameters of user is simply keep the locally trained value from user
3.5 FURL TRAINING PROCESS
While this paper focuses on learning user embeddings. our approach is applicable to any personaliza- tion technique that satisfies the splii-personalizationn constraint. The training process is as follows:
1. Local Training: Initially, each user downloads the latest federated parameters from the server. Private parameters of user w. are initialized to the output of local training from the last time user participated training, or to default value if this was the first time user i was trained. Federated and private parameters are then jointly trained the task in question.
2. Global Aggregation: Federated parameters trained in the step above are transferred back to, and get averaged the central server in vanilla Federated Averaging. Private parameters (e.g., user embeddings) trained above are stored locally on the user device without being transferred back the server. These will be used for the next round of training. They may also be used for local inference.
Figure I shows example of federated and private parameters, explained further in Section 4.1
5

---

Under review as a conference paper at ICLR 2020
Datasct # Samples (Train/Eval/Test)
Users (Train/Bval/Testt
Sticker
940K (750K/94K/96K)
3.4K (3.3K/3.0K/3.4k)
Subreddit
942K (752K/94K/96K)
3.8K (3.8K/3.8K/3.88)
Table I: Dataset statistics.
EVALUATION
We evaluate the performance of FURL document classification tasks that reflect real-world data distribution across users.
4.1 EXPERIMENTAL SETUP
Datasets
We two datasets, called Sticker and Subreddit. Their characteristics follows.
1. In-house production dataset (Sticker): This proprictary dataset from a popular messaging app has randomly selected, anonymized messages for which the app suggested Sticker reply. The features are messages; the task to predict user action (click or not click) on the Sticker suggestion, i.e., binary classification. The messages and annotated; they were not read or labeled by human annotators.
automatically collected, de-identified,
2. Reddit comment dataset (Subreddit): These are comments the top 256 subreddits reddit.com. Following Bagdasaryan et al. (2018), we filter out users who have fewer than 150 or more than 500 comments, that each has sufficient data. The features comments; the task is to predict the subreddit where the comment was posted, i.e., multiclass classification. The authors are not affiliated with this publicly available dataset (Baumgartner 2019).
Sticker dataset has 940K samples and 3.4K users (274 messages/user on average) while Subred- dit has 942K samples and 3.8K users (248 comments/user average). Each user's data is split (0.8/0. 1/0. 1) to form train/eval/test sets. Table I presents the summary statistics of the datasets.
Document Model We formulate the problems document classification tasks and use the LSTM-based (Hochreiter & Schmidhuber 1997) neural network architecture in Figure The text is encoded into input representation vector by using character-level embeddings and a Bidirectional LSTM (BLSTM) layer. A trainable embedding layer translates each user ID into a user embedding vector. Finally, an Multi-Layer Perceptron (MLP) produces the prediction from the concatenation of the input representation and the user embedding. All the parameters in the character embedding, BLSTM and MLP layers are federated parameters that are shared across all users. These parameters are locally trained and sent back to the server and averaged as in standard Federated Averaging. User embedding is considered private parameter. but kept privately on the device. Even though user embeddings are trained independently on each de- vice, they evolve collaboratively through the globally shared model, i.e., embeddings are multiplied by the same shared model weights.
jointly trained with federated parameters,
Configurations We ran configurations evaluate the performance of the models with/without FL. and personalization: Global Server, Personalized Server, Global FI., and Personalized FL.. Global is a synonym for non non-personalized, Server is a synonym for centralized training. The ex- periment combinations are shown in Table
Model Training Server-training uses SGD, while FL training uses Federated Averaging com- bine SGDD-raiined client models (McMahan et al. 2016). Personalization in FL uses FURL training as described section The models were trained for 30 and 40 epochs for the Sticker and Sub- reddit datasets, respectively. One epoch in FL means the all samples in the training set were used once.
We ran hyperparameter sweeps find the best model architectures (such as user embedding dimen- sion, BLSTM and MLP dimensions) and learning rates. The FL configurations randomly select 10 users/round and rum 1 epoch locally for each user in each round. Separate hyperparameter sweeps for FL and centralized training resulted in the same optimal embedding dimension for both config- urations. The optimal dimension was 4 for the Sticker task and 32 for the Subreddit task.

---

Under review as a conference paper at ICLR 2020
Config
With personalization
With FL
Global Server
No
No
Personalized Server
Yes
No
Global FL
No
Yes
Personalized FL
Yes
Yes
Table 2: Experimental configurations.
0.6
0.5
0.4
0.58
0.3
0.2
0.1
Sticker (AUC)
Subreddit (Accuracy)
Dataset Model Performance
Global Personalized Global Personalized
Figure 2: Performance of the configurations.
Metries We report accuracy for experiments on the Subreddit dataset. However, we report AUC instead of accuracy for the Sticker dataset since classes are highly unbalanced.
4.2 EVALUATION RESULTS
Personalization improves the performance significantly. User embeddings increase the AUC on the Sticker dataset by 7.85% and 8.39% in the Server and FL configurations, respectively. The improvement is even larger in the Subreddit dataset with 37.2% and 50.51 % increase in the accuracy for the Server and FL settings, respectively. As shown in Figure 2 these results demonstrate that the user representations effectively learn the features of the users from their data.
Personalization in FL provides similar performance server training. There is AUC re- duction on the Sticker dataset while the accuracy drops only 3.72% on the Subreddit dataset (as shown in Figure expected and consistent with other results (McMahan et al., 2016). The learning curves on the evaluation set Figure 3 show the performance of HL models asymptotically approaches the server counterpart. Therefore, FL. provide similar performance with the centralized setting while protecting the user privacy.
Furthermore, the small decrease of FL compared to centralized training is
User embeddings learned in FL have similar structure to those learned in server training. Recall that for both datasets, the optimal embedding dimension was the same for both centralized and FL training. We visualize the user representations learned in both the centralized and FL settings using t-SNE (yan der Maaten & Hinton Nov 2008). The results demonstrate that similar users are clustered together in both settings.
Visualization of user embeddings learned in the Sticker dataset in Figure shows that users having similar (e.g., low high) click-through rate (CTR) the suggested stickers are clustered together. For the Subreddit dataset, we highlight users who comment a lot on a particular subreddit, for the top 5 subreddits (AskReddit, CFB, The Donald, nba, and politics). Figure[jjindcateee submit their comments to the same subreddits are clustered together, in both settings. Hence, learned user embeddings reflect users' subreddit commenting behavior, in both FL and Server training.
that users who
CONCLUSION AND FUTURE WORK
This paper proposes FURL, a simple, scalable, bandwidth-eeffiieent technique for model person- alization in FL. FURL improves performance over non-personalized models and achieves similar performance to centralized personalized model while preserving user privacy. Moreover, represen- tations learned in both server training and FL show similar structures. In future, would like to evaluate FURL on other datasets and models, learn user embeddings jointly across multiple tasks, address the cold start problem and personalize for users not participating in global FL aggregation.

---

Under review as a conference paper at ICLR 2020
AUC (Sticker)
Accuracy (Subreddit)
Config
Global Server
57.75%
28.93%
Personalized Server
65.60%
66.13%
Global FL
57.24% 65.63%
11.90%
Personalized FL
62.41% %
Table Performance of the configurations.
Eval Auc (Sticker)
Eval Accuracy (Subreddit)
0.70
0.70
0.60
0.66
Global FL
0.500
Global Server
0.62
0.40
Clobal FL
Personalzed FL
Personalized Server
0.58
0.30
0.20
0.54
0.10
0.50
0.00
10
10
15
20
25
30
35
40
Number of Epochs
Number Epochs
(a) Sticker dataset.
(b) Subreddit dataset.
Figure 3: Learning curves the datasets.
Server User Embeddings
FL User Embeddings
High
CTR
40
20
20
0
-20
-20
-40
-60
-40
-80
Low
50
-25
50
CTR
Figure 4: User embeddings in Sticker dataset, colored by user click-through rates (CTR).
Server User Embeddings
FL User Embeddings
60
60
40
40
20
20
0
0
-20
-20
-40
-60
-60
-50
-25
25
50
-50
-25
0
25
50
Other
CFB
nba
AskReddit
The_ Donald
politics
Figure 5: User embeddings in Subreddit dataset, colored by each user's most most-posted subreddit.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving. Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg. Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, llya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-scale machine learn- ing on heterogencous systems, 2015. URL hitps: /twp:://wwww.. www.orfessr.....oomm org/ Software available from tensorflow.org.
Muhammad Ammad- Ammad-ud-din, Elena Ivannikova, Suleiman A. Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System. arXiv:1901. 109888 (es, stat], 2019. URL http:
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How To Backdoor Federated Learning. vrXiv:/807.004955 [cs], July 2018. /ttppp///aaxxiv.oo arXiv: 1807.00459.
abs/18073 00459
Jason Baumgartner. Reddit Comments Dumps.
comments/ 2019. Accessed: 2019-09-03.
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chlo M Kiddon, Jakub Konen, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards Federated Learning at Scale: System Design. In SysML 2019, 2019. URL ltttps://arxiv/ org/ abs/192.. [01466
Sebastian Caldas, Jakub Koneny, H. Brendan McMahan, and Ameet Talwalkar. Expanding the Reach of Federated Learning by Reducing Client Resource Requirements, 1rXiv:1812.072.10 [es, stat], December 2018. URL htRL:ttp:/ www.xxi...rgggb.e..
07210
arXiv: 1812.07210.
Fei Chen, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. Federated meta-learning for recommen- dation, 2018.
Mihajlo Grbovic and Haibin Cheng. Real-time Personalization Using Embeddings for Search Rank- ing at Airbnb. In Proceedings of the 24th ACM SIGKDD International Conference Knowl- edge Discovery and Data Mining, KDD '1, pp. 311-320, New York, NY, USA, 2018 ACM. ISBN 978-1-4503-5552-0, doi: 10.1145//21181199221988
org/ URL http: acm. arm.org/
10.1144/32199 3219885
Florian Hartmann. Federated Learning. Master's thesis, Freien Universitat, 2018.
Sepp Hochreiter and Jürgen Schmidhuber. Long Short-Term Memory. Neural Comput., 97): November 1997. ISSN 0899-7667. doi:
1997/9............. URL
10. 1162/necoocooo 1991 .9....9.. 19.8.1735
Aaron Jaech and Mari Ostendorf. Personalized language model for query auto-completion. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Pupers), pp. 700-705, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 10.18853/v1ln 1111665//1//188221 1/118-211111111111111111 URL URLLttps. https: www. aclweb. org/ /azzg///tthoog///
P18 2111
Jakub Konen, H. Brendan McMahan, Daniel Ramage, and Peter Richtrik. Federated Optimiza- tion: Distributed Machine Learning for On-Device Intelligence. arXiv:1610.02527 [cs), October 2016a. wtt.http:///axxiv.coom
02527,
arXiv: 1610.02527.
Jakub Konen, H. Brendan McMahan, Felix X. Yu, Peter Richtarik, Ananda Theertha Suresh, and Dave Bacon. Federated Learning: Strategies for Improving Communication Efficiency. In NIPS Workshop on Private Multi-Party Machine Learning, 2016b. URL https: http///aavvv...//
abs/ 61 0. 05492

---

Under review as a conference paper at ICLR 2020
H. Lee, B. Tseng, T. Wen, and Y. Tsao. Personalizing Cecurrent-Neural-eeere Based Language Model by Social Network. /EEE/ACM Transactions on Audio, Speech, and Language Processing, 25(3):519-533 March 2017. ISSN 2329-9290. doi: 10.1 10.1109TAASLP20012266444
Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, and Alex Large-scale Graph Embedding System. In Proceedings of
Peysakhovich. FTorch-Biggrran the 2nd SysML. Conference, Palo Alto, CA, USA, 2019.
David Leroy, Alice Coucke, Thibaut Lavril, Thibault Gisselbrecht, and Joseph Dureau, Federated learning for keyword spotting, 2018.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated Optimization for Heterogeneous Networks. arXiv: 11/2.06/12 [cs, stat), December 2018.
http://wwwgga///////.oeae/ C6127
arXiv: 1812.06127.
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated Learning: Challenges, [cs, stat], August 2019a. URL http://
Methods, and Future Directions.
argiv.org/abe 1908. 07873
arXiv: 1908.07873.
Tian Li, Maziar Sanjabi, and Virginia Smith. Fair Resource Allocation in Federated Learning.
arXiv: 1047 [cs, star), May 2019b. URL htDp://aaxxx...rr//////99999 arXiv: 1905.10497.
Yang Liu, Tianjian Chen, and Qiang Yang. Secure Federated Transfer Learning, arXiv:1812.03337 [cs, stat]. December 2018. URL http://arxiv.org http./oorrr/bbb//2203333333 arXiv: 1812.03337.
lan McGraw, Rohit Prabhavalkar, Raziel Alvarez, Montse Gonzalez Arenas, Kanishka Rao, David Rybach, Ouais Alsharif, Hasim Sak, Alexander Gruenstein, Franoise Beaufays, and Carolina Parada. Personalized speech recognition on mobile devices. 2016 IEEE International Conference 2016.
on Acoustics, Speech and Signal Processing (ICASSP), pp.
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication Effiiceent Learnnng of Deep Networks from Decentralized Data. In Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017. JMLR: W&CP volume 54, 2016.
Yabo Ni, Dan Ou, Shichen Liu, Xiang Li. Wenwu Ou, Anxiang Zeng, and Luo Si. Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery &
Data Mining, KDD '18, pp. 996-605, New York, NY, USA, 2018. ACM. ISBN 978-1-4503-5555 URL http:://ol acm. http::///.......o///.........1.. 1ttp:/10.115/3221190
0. doi: 10. 211111////219999/ 3219828.
321 9823
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In NIPS Autodiff Workshop, 2017.
Vadim Popov and Mikhail Kudinov. Fine-tuning of language models with discriminator, 2018.
Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, and Oren Sar Shalom. Privacy and faimess in recommender systems via adversarial training of user representations, 2018.
Ameet Talwalkar Sebastian Caldas, Virginia Smith. Federated Kernelized Multi-task Learning. In SysML 2018, 2019.
Virginia Smith, Chao-Kaa Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. In Proceedings of the 31st International Conference Neural Information Processing Systems, NIPS' 17, pp. 4427 4437, USA, 2017. Curran Associates Inc. ISBN URL http: / //11 or.. / ci witiaaiiiiii on. wttp:ccmmmmmmmmmmm 02099966.2295.5 32951
10

---

Under review as a conference paper at ICLR 2020
Jiaxi Tang and Ke Wang. Personalized Top-N Sequential Recommendation via Convolutional Se- quence Embedding. In Proceedings of Eleventh ACM International Conference on Web Search and Data Mining, WSDM 18, pp. 565-573, New York. NY, USA, 2018. ACM. ISBN 978-1- 4503-5581-4 doi: 10.1 10.1145//15996231996666
URL http: /ttp:://do... acm. wtt..aacm.oo//.1110/0911 1145/
3159652 315666
L.J.P van der Maaten and G.E. Hinton. Visualizing high-Dimensiinal Data Using t-SNE. Journal of Machine Learning Research, 9: 25792605, Nov 2008.
Jan Vosecky, Kenneth Wai-Ting Leung, and Wilfred Ng. Collaborative personalized twitter search with topic-language models. In Proceedings of the 37th International ACM SIGIR Conference Research & Development in Information Retrieval, SIGIR 14, pp. 53-62, New York, NY, USA, 2014. ACM. ISBN 978- 777-1-44533-2277777 doi: 01L144/2260428.66695555 URL http: 7/doi am. org / 10. 11 45/ 2ttp://0004022222 2609584
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated Machine Learning: Concept and Applications. ACM Trans. Intell. Syst. Techmol. 10(2):12:1112) January 2019. ISSN 2157-6904. doi: 10.1 145/2298228 URL http: //cii acm. org/10 /C 1145/ 3298981
X. Yao, Huang, C. Wu, R. Zhang, and Sun. Towards faster and better federated learning: A feature fusion approach. In 2019 IEEE International Conference Image Processing (ICIP), pp- 175-179, Sep. 2019. doi: 10.1 11110//CI.2111.88833300
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated Learning with Non-IID Data. arXiv: 1.Xiv:/806.000 005.00582 (cs, stat], June 2018. URL http:://ttttt:tt org/abs/18066 00582 arXiv:
11


Reviews:
[{"review_id": "SJeGpxFFYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors proposed a formal training scheme (FURL) for personalized federated models. The claimed benefits of such models are 1) preservation of user privacy by keeping the personalized parameters locally on each user's device, and 2) reduced data exchange to make the training complexity grow linearly with the number of users. Authors approached the problem with defining the constraint of split personalization, and argued that common FL setting such as Federated Averaging could satisfy this constraint. \n\nAuthors designed a personalized classification deep network for two data sets, namely Stickers and SubReddit. Both tasks could benefit personal preference in addition to textual features: Stickers CTR depends on user's adoption of the feature; SubReddit categorization depends on user's past activities in each sub-Reddit. A clearly conducted experiment showed that personalization has a significant contribution in non-federated setting, and using FURL in federated setting achieved similar performance while non-personalized FL may suffer bigger loss (in the case of SubReddit). Authors also compared the conversage curve and visualized final embeddings to show that federated learning produces acceptable convergence and equally reasonable embeddings.\n\nThe paper is well written and all claimed contributions are well articulated. Reviewer didn't find any significant problems.\n\nReviewer has limited knowledge of previous work in the personalized FL field, thus is only able to confirm the novelty from Authors' related work section.\n\nOne comment about formatting: in Figure 5, the color dots in the legend could be larger for easier identification. Please also consider some color-independent label/description to help readers with difficulties in color perception. For example, you can name the color in the legend (i.e. \"Red\") and provide some text labels in the embedding chart to tell which part is mostly red.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HJx8Dzey5S", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes the use of Federated Averaging for achieving personalised user embedding. Federated Learning is used whether they propose a particular split of model parameters with user embedding (private) and the overall BLSTM model (shared). Federated Averaging is used for the global update.\n\nThe key contribution of this paper is not clear. It seems to be the introduction of the notion of split-personalisation-constraint, and it shows that the modeling each user with a “private” embedding that feeds to a global MLP with a global BLSTM as another input (named as FURL) can achieve the constraint so that FL can be used. The originality is limited.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BkgTC6L6KS", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose using federated learning (FL) to train personalized models, which improves the scalability and privacy preservation of the existing personalization techniques. The empirical results show good performance.\n\nHowever, in general, I think the contribution is limited. The reasons are as follows:\n\n1. The proposed algorithm, FURL, is a direct and simple combination of personalized model and FL. Although the authors claim that there is significant improvement in the performance, such improvement comes from the personalization. And, the personalization itself is not a novel thing (I think the personalized model used in this paper is similar to [1] or some other references. Please correct me if the personalized model used in this paper is new, since I'm not an expert in personalization.) Thus, in general, this paper simply use FL to replace fully synchronous SGD in the training of the personalized models. All the benefits claimed in the introduction, including scalability, privacy preservation, and improvement of performance, come from either vanilla personalization or vanilla FL. I fail to find any new contribution in this combination.\n\n2. The authors emphasize a lot on the \"independent aggregation constraint\". Although it sounds like such constraint is designed especially for FL + personalization, it is actually a feature only for personalization, which has nothing to do with FL. Note that when doing inference/prediction, each user uses his/her own private part of the model. Different users' private part of models will never affect each other. It is equivalent to training a global model, which concatenates the private parts of models into a big model, and each user update the global model in a sparse manner. Thus, we can also train such personalized model with fully synchronous SGD with sparse gradients, which also does not synchronize the private parts. The private part is never shared by different users, no matter trained by fully synchronous SGD or FL.\n\n\n------------\nReferences\n\n[1] Jaech, Aaron, and Mari Ostendorf. \"Personalized language model for query auto-completion.\" arXiv preprint arXiv:1804.09661 (2018).", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This manuscript personalization techniques to improve the scalability and privacy preservation of federated learning. Empirical results are provided which suggests improved performance.

The reviewers and AC agree that the problem studied is timely and interesting, as the tradeoffs between personalization and performance are a known concern in federated learning.  However, this manuscript also received quite divergent reviews, resulting from differences in opinion about the novelty and clarity of the conceptual and empirical results. Reviewers were also unconvinced by the provided empirical evaluation results. 
Decision: Reject