Dual personalization for federated recommendation on devices
Authors: Chunxu Zhang
Guodong Long
Tianyi Zhou
Zijian Zhang
Bo Yang
Venue: Submitted to ICLR 2023 (2023)
Keywords: federated learning
personalization
recommmendation system

Abstract: Federated recommendation is a new Internet service architecture that aims to provide privacy-preserving recommendation services in federated settings. Existing solutions are used to combine distributed recommendation algorithms and privacy-preserving mechanisms. Thus it inherently takes the form of heavyweight models at the server and hinders the deployment of on-device intelligent models to end-users. This paper proposes a novel Personalized Federated Recommendation (PFedRec) framework to learn many user-specific lightweight models to be deployed on smart devices rather than a heavyweight model on a server. Moreover, we propose a new dual personalization mechanism to effectively learn fine-grained personalization on both users and items. The overall learning process is formulated into a unified federated optimization framework. Specifically, unlike previous methods that share exactly the same item embeddings across users in a federated system, dual personalization allows mild finetuning of item embeddings for each user to generate user-specific views for item representations which can be integrated into existing federated recommendation methods to gain improvements immediately. Experiments on multiple benchmark datasets have demonstrated the effectiveness of PFedRec and the dual personalization mechanism. Moreover, we provide visualizations and in-depth analysis of the personalization techniques in item embedding, which shed novel insights on the design of RecSys in federated settings.

27 setting and adding privacy-preserve algorithms with guaranteed protection. However, these im- plementations of federated recommendations still inherit the traditional service architecture, which 29 is to deploy large-scale models at servers. Thus it is impractical and inconsistent with the newly Under review as a conference paper at ICLR 2023
DUAL PERSONALIZATION ON FEDERATED RECOMMEN- DATION
Anonymous authors Paper under double-blind review
ABSTRACT
Federated recommendation is a new Internet service architecture that aims to provide preaeecirrs recommendation services in federated settings. Ex- isting solutions are used to combine distributed recommendation algorithms and privacy-preservin mechanisms. Thus inherently takes the form of heavyweight models at the server and hinders the deployment of on-device intelligent models to end-users. This paper proposes novel Personalized Federated Recommen- dation (PFedRec) framework to learn many user-specific lightweight models to be deployed on smart devices rather than a heavyweight model on server. Moreover, we propose new dual personalization mechanism to effectively learn fine- fine-graieed personalization on both users and items. The overall learning process is formulated into unified federated optimization framework. Specifically, unlike previous methods that share exactly the same item embeddings across users in a federated system, dual personalization allows mild finetuning of item embeddings for each user to generate user-specific views for item representations which can be integrated into existing federated recommendation methods gain improvements immediately. Experiments on multiple benchmark datasets have demonstrated the effectiveness of PFedRec and the dual personalization mechanism. Moreover, we provide visualizations and in-depth analysis of the personalization techniques item embedding, which shed novel insights on the design of RecSys in federated settings.
2
6
10
12
13
14
15
16
17
19
21
INTRODUCTION
Federated recommendation is a new service architecture for Internet applications, and it aims to provide personalized recommendation service while preserving user privacy in the federated set- tings. Existing federated recommendation systems (Ammad- Ud-Din Ud-Dii et al. 2019: Chai et al. 2020 Muhammad et al. 2020 Perifanis & Efraimidis] 2022; Singhal et al. 2021) are usually to be an adaptation of distributed recommendation algorithms by embodying the data locality in federated
23
25
30 raised on-device service architecture, which deploy a lightweight model on the device to pro-
31 vide service independently without frequently communicating with the server. Given the challenge
32 of implementing data locality on devices federated settings, the personalization mechanism needs
33 to be reconsidered to better capture fine-grained personalization for end-users.
34 Personalization is the core component of implementing federated recommendation systems. Inher-
35 ited from conventional recommendation algorithms, existing federated recommendation frameworks
36 are usually composed three modules: user embedding to preserve the user's profile, item embed- ding to maintain proximity relationships among items, and the score function to predict the user's 38 preference or rating for a given item. They usually preserve user-specific personalization in the user embedding module while sharing consensus item embeddings and score functions.
39
40 This paper proposes a new dual personalization mechanism designed to capture fine mhe-grained two- fold personal preferences for users in the federated recommendation system. Inspired by human
42 beings' decision logic, we believe all modules in the recommendation framework should be used preserve part of personalization rather than
embedding only. For example, the score

---

Under review as a conference paper at ICLR 2023
f
Score Function
Score Function
User Embedding
Item Embedding
User Embedding
Item Embedding
Item
Embedding
u
(a) FodMF/FedRecon
(b) FedNCF
(c) PFedRee
Figure 1: Different frameworks for the personalized federated recommendation. The green block represents personalized module, which indicates the purt of model is to preserve user preference. Our proposed model will preserve dual personalization on two modules.
44
function is to mimic the user's personal decision logic that is natural to be diverse across clients. Furthermore, given an itemset, different people may have a different view to measure their prox- imity relationships. Therefore, personalized item embedding could be essential to capture people's personal preferences further.
45
46
47
To implement the aforementioned ideas in federated settings, propose federated recom- mendation framework to implement fine-grained personalization on multiple modules which are illustrated in Figure (c). First, use personalized score function capture users' preferences, and it could be implemented using a multi-layer neural networks. Second, we remove the user embedding from the framework because the current neural-based personalized score function has enough representation capability to preserve the information of user embeddings. Third, we imple- ment light tine-tuning learn personalized item embeddings in federated settings. This proposed decentralized intelligence architecture is natural simulation of human beings' decision decision-making that each person has relatively independent mind to make decisions.
49
(c).
51
58
54
55
56
57
The learning procedure is also carefully tailored in a federated setting. A personalized score function will be learned using own data on the device, and then it won't be sent the server for model 58
59
aggregation that usually generates general view for all devices. Moreover, the personalized item embedding will be implemented through light fine-tuning a federated learning framework, thus can leverage both the general view from server and the personalized view from its own data.
60
62 In summary, we propose a novel federated recommendation framework that integrates both the per- sonalized score function and personalized item embedding via light finetuning from the shared item embedding. Our key contributions are summarized as follows.
64
85
We propose novel federated recommendation framework which is more naturally consis- tent with layer-wise neural architecture which can better fit federated learning.
66
67
We design a novel dual personalization to capture user preferences using a personalized score function and fine-grained personalization on item embeddings. It can be integrated with other baselines to improve their performances.
88
89
70
We formulate the proposed federated recommendation learning problem into a unified fed- erated optimization framework with the meta-learning objective.
71
72
Our method can significantly outperform existing federated recommendation baselines.
2
RELATED WORK
74
2.1
PERSONALIZED FEDERATED LEARNING
75
Federated Learning (FL) new machine learning paradigm that server orchestrates a large
76
number of clients to train model without accessing their data (Kairouz et al. 2021 Li et al. 2020 Bonawitz al. 2019 Yang al. 2019). The vanilla federated learning method, FedAwg (McMahan
77
al. 2017), is to learn robust model at the server while embodying data locality for each device
When the items one-hot encoding vectors, can simply equivalent use item embedding network and item embedding representations.
2

---

Under review as a conference paper at ICLR 2023
with non-llD data. Personalized Federated Learning (PFL) is to learn a personalized model for each device to tackle the non-lID challenge. A simple PFL method could train a global model with FedAvg, then conduct a few steps of finetuning on each client (Cheng al. 202T). In this framework, knowledge sharing is model aggregation, and model personalization is local finetuning. Per-FedAAg (Fallah et al. (020)) adds finetuning as regularization term to the objective learning function of the global model. Ditto (Li et al. 2021) proposes a bi-level optimization framework for PFL while constraining the distance between the local and global models. Investigations by (Sham- sian et al. 2021 Chen et al. 2018) aim to train a global hyper-network or meta-learnet instead of 86
80
81
82
83
84
85
87
a global model before sending to clients for local optimization. SCAFFOLD (Karimireddy et al.) 2020) proposes to learn personalized control variate that corrects the local model accordingly. Fe- method that preserves local model for each client and trains a global model collaboratively with FedAvg. Layer-wise personalization (Arivazha-
88
89
dRecon (Singhal et al. 202T) is veeta-lerring-bassee
90
91
gan et al. 2019 Liang et al. 2020) is also a simple and effective technique in PFL.
92
2.2
FEDERATED RECOMMENDATION SYSTEMS
93
Federated recommendation has attracted much attention recently due to the rising concern about privacy. Some recent works focus on using the interaction matrix only. FCF (Ammad- Ud-Din et al.) 2019) is the first Fl.-based collaborative filtering method, which employs the stochastic gradient ap- proach to update the local model, and FedAvg is adopted to update the global model. Improving on user privacy protection, Chai et al. present FedMF (Chai et al.) 2000), which adapts distributed ma- trix factorization to FL setting and introduces the homomorphic encryption technique on gradients before uploading to the server. MetaMF (Lin et al. 2020b) is a federated meta-learning framework where a meta-netwee network is adopted to generate the rating prediction model and private item embed- ding. (Wu et al., 2021> presents FedGNN where each user maintains a GNN model to incorporate high-order user-item information. However, the server in both methods preserves all the recommen- dation model parameters which can be used to infer the user's interaction information, resulting in the risk of user privacy leakage. FedNCF (Perifanis & Efraimidis (002)) adapts Neural Collabora-
94
95
96
97
98
99
100
101
102
103
104
105
live Filtering (NCF) (He et al. (017) to the federated setting which introduces neural network to learn user-item interaction function enhance the model learning ability. Federated recommen-
108
107
dation using rich information considers multiple data sources or modalities in modeling. Fed-
108
Fast (Muhammad et al., (0200) extends FedAvg (McMahan et al., 2017) with an active aggregation method to facilitate the convergence. Efficient-FedRee (Yi et al. 2021) decomposes the model into large news model on the server and a light user model on the client, and reduces the computation and communication cost for users. Both works rely more data sources, such as user features news
110
112
attributes rather than an interaction matrix. (Lin et al.] [2200a] Du et al.] [022]] Yang et al. 1021
Minto et al.] 2021 Lin et al.] (02T) are endeavors that focus enhancing privacy of FedRec. There
114
are also attempts for other applications in FedRec, such as federated attack (Wu et al. 20226] Zhang
115
al.
2022). , social recommendation (Liu et al. 2022b). Click-Through Rate (CTR) prediction (Wu
116
et al. 2022a). fair recommendation (Liu et al. 2022a and payload optimization (Khan et al. 202
3
PROBLEM FORMULATION
118
Federated Learning is to learn a global model parameterized by A to serve all clients whose data are private. The optimal solution should minimize the accumulated loss of all clients. That is,
119
N
min
0z (0)
(1)
120
where (0) is the supervised loss on the i-th client with dataset D,, and all clients share the global parameter 8. The a; is a weight for the loss of the i-th client. For example, the conventional federated learning algorithm, FedAvg (McMahan et al. 2017). defines a as the fraction of the size
121
122
of the client's 's training data, i.e., a; := D.I/ E [Dyl. Once the global model is trained, it can be used for prediction tasks on all clients.
123
124
125
Personalized Federated Learning simultaneously leverages common knowledge among clients and learns a personalized model for each client. The learning objective usually formulated as
126
N
min 0, L.N
L,L,(0,0))
(2)
3

---

Under review as a conference paper at ICLR 2023
where each client has unique personalized parameter 8, and is the global parameter as mentioned in Eq. For example, |aallah et al. (2020) leverage 0 as initialization of f,, i.e., 6, := B (4( where (0) is the loss of vanilla model on the i-th client. The (, 0,) is then formulated
128
(.) = 4(0 VI(())
130
Recommendation on Neural Networks This work focuses on a fundamental scenario where rec- ommendation only relies on user-item interaction matrix. The recommendation task is then fulfilled by the Neural Collaborative Filtering (NCF) model (He et al.] 2017), which consists of three parts: score function S, a user embedding module & and an item embedding module E. We denote these modules' parameters as 0 == (0", 0", (m)) and formulate the learning objective in Eq.
131
132
133
134
min r. = min L(0 E(()))
(4)
135
where e" and e are one-hot encoding representing users and items. r is a user's rate to the given item and f is prediction from the score function ((E(e). E(e")) )) is the loss evaluating predic- tion performance. It could be point-wise loss as used in (Wang et al. 2016 He et al. (017), or pair-wise loss as in (Rendle et al. 2012, Wang et al. 2019). is worth noting that conventional Ma- trix Factorization (MF) methods could be viewed as special case of the NCF (He et al. 2017), i.e.,
136
137
136
140
the conventional MF is a model where the score function S is simplified as the multiplication opera- tor without learnable parameters, and the embedding of user/item is obtained by the decomposition of the user-item interaction matrix.
142
143
METHODOLOGG
144
In this section, we propose a novel Personalized Federated Recommendation (PFedRec) framework, which aims to simultaneously learn many user-specific NCF models deployed on end devices.
145
148
4.1
OBJECTIVE FUNCTION
147
Federated Learning Objective We regard each user as a client under FL settings. The on-device recommendation task then depicted a PFL problem. Particularly, the item embedding mod- ule E, is assigned to be a global component which learns common item information and the score function S, maintained locally to learn personalized decision logie. To further capture the dif-
148
149
150
ference between users and achieve a preference-preserving item embedding, we devise a bi-level optimization objective,
151
152
min 4m (0.) N
Σ a₁ (O,Tr)
(5)
s.t.
0, := (0mh
V Lu;, 0.)
153
: (0thh, 02) is the personalized parameter for E, and S,, and L; will be evaluated on the i-th client local data D,. Under this framework. PFedRec first tunes E into personalized item embedding module E,, and then learns a lightweight local score function S; to make personalized predictions. Different from the conventional recommendation algorithms, the user embedding mod- ule & is depreciated since the personalization procedure on client will automatically capture the client's preference. There is no use to learn extra embeddings to describe clients.
where 0=
154
155
156
157
158
Loss for Recommendation Equipped with the item embedding module and score function, we formulate the prediction of j-th item by i-th user's recommendation model as,
160
Fgz = (EE(()))
(6)
Particularly, we discuss the typical recommendation task with implicit feedback, that is, Tij = i-th user interacted with j-th item; otherwise rij = 0. With the binary-value nature of implicit feedback, we define the loss function of i-th user the binary cross-end voss-entropy loss,
162
L,(Bar,f)
Σ log fsz
Σ (1,"))
log(1 f.ge)
(..))D,
4

---

Using partial-based federated model aggregation to learn personalized user score function on each device. Our proposed model is composed of a neural-based score function parameterized by 0 and an item embedding module parameterized by 8T The coordinator/server of federated sys- Under review as a conference paper at ICLR 2023
where D," is the negative instances set of user. Notably, other loss functions can also be used, and here we choose the binary cross- coss-eytropy loss to simplify the description. Particularly, to construct Dj efficiently, first count all the uninteracted items set as,
185
I III
where I denotes the full item list and Z is the interacted item set of i-th user. Then, uniformly sample negative instances from Z by setting the sampling ratio according to the number of observed interactions and obtain D
188
189
170
4.2
DUAL PERSONALIZATION
We have implemented dual personalization mechanism enable the proposed framework can preserve fine-grained personalization for both user and item.
172
173
174
175
178
tem will iteratively aggregate model parameters or gradients collected from each participant/device.
177
Due to the concern of personalization and privacy, we could implement a partial model aggregation strategy by keeping the score function as a private module on devices while sharing the item em- bedding to the server. Therefore, the server only aggregates the gradients or parameters t from the item embedding network. The user's personalized score function network 0* won't be sent to the server and thus won't be aggregated. More discussion about learning efficiency of personalized score function can be found in Appenidx
179
181
182
183
Fine-tuning the item embedding module to generate personalized representations for items on each device. According to Eq. 5 the learning objective of 4TH could be viewed searching for a "good initialization" that could be fast adaptive to the learning task on different devices. shares similar ideas with meta-leamiing based methods (Fallah et al., 2020) which have local loss
185
186
187
in Eq. 3 However, our proposed method takes a different optimization strategy we call post-tuming.
188
Specifically, rather than directly tuning global model on clients' local data, it first learns the local score function with the global item embedding network, and then replaces the global item embedding with personalized item embedding obtained by fine-tuning fmM Details of the learning process illustrated in Algorithm. [ and extensive experiments show it will achieve superior performance
189
190
192
to the vanilla meta-learning based methods. Furthermore, we discuss the effectiveness of one-step fine-tuning in Appendix A.2
194
4.3 ALGORITIM
195
O Otimiization To solve the optimization problem as descripted in Sec. 4.1 objective function, we conduct an alternative optimization algorithm train the model. As illustrated in Algorithm [ when client receives the item embedding network from server, it first replace its embedding with global one, and then updates the score function while keeping item embedding network fixed. Then the client updates the item embedding based on the updated personalized score function. Finally, the updated item embedding would be uploads to server for global update.
197
199
200
201
The overall workflow of the algorithm could be summarized into several steps as follows. The server is responsible for updating shared parameters and organizing all clients to complete collaborative training. At the beginning of federated optimization, the server initializes the model parameters, which would be used initial parameters for all client models. In each round, the
Workflow.
203
204
205
server selects a random set of clients and distributes the global item embedding 0m to them. When local training is over, the server collects the updated item embedding network from each client to perform global aggregation. We build on the simplified version of FedAvg, a direct average of locally uploaded item embedding network. The overall procedure is summarized in Algorithm
206
207
208
DISCUSSIONS
209
5.1 PRIVACY ON FEDERATED RECOMMENDATION
211
trivacy-preserving is an essential motivation to advance existing cloud-centric recommendation to client-centric recommendation service architecture. In general, the federated learning's 's decen-
212
5

---

Under review as a conference paper at ICLR 2023
Algorithm Dual Personalization for Federated Recommendation
ServerExecute:
1: Initialize item embedding 1 + 08" and score function 0 + 0
2: for each round 1, do
Global communication rounds
3: St + (select a set of size n. randomly from all N clients)
4: for each client index E St in parallel do
5: HTh +- ClliemUpdaate(i, ())
> Distribute global item embedding to client for update
6: fm
Global aggregation over local item embedding network
ClientUpdate:
1: Initialize 6TTh with 4T
2: Initialize of with the latest updates
3: Retrieve user positive feedback D, according to user index i
4: Sample negative feedbcak D; from unobserved instances
5: B +- (split D; U DJ into batches of size B)
6: for from 1 E do
Local training epochs
7: for batch b E B do
8: Compute (0,,,, () with Eq. r, 9: 0- - 8- V Li
Model loss of batch data b > Score funtion update
Compute (, F) with Eq.
Model loss with the updated 6 post-tuning for personalized item embedding network
11:
0g +- 0
V Vam Li
Return 0Thh server
213
tralized framework can embody data locality and information minization rules (GDPR) that could greatly mitigate the risk of privacy leakage (Kairouz et al. 2019). To provide service with privacy guranttee, the FL framework should be integrated with other preaaccpreserving methods, such as Differential Privacy and secure communication. Our proposed framework derives the same decen- tralized framework from vaniall FI. to preserve data locality. For example, to tackle the privacy leakage risk caused by sending item embedding network the server, could simply apply dif- ferential privacy to inject noise into the gradients so that the server cannot simply infer the updated
215
217
219
220
items by watching the changes of gradients. More discussion can be found in Appendix D.4
221
5.2
GENERAL FRAMEWORK FOR FEDERATED RECOMMENDATION
The proposed framework in Figure I (c) could be general form of federated recommendation. Because our framework could be easily transformed to an equivalent form other frameworks. For example, if assign the score function
223
one-layer neural network, PFedRec is equal to FedMF and FedRecon in Figure [ (a). Moreover, we change the personalized score function from full personalization to partial layer personalization, our method could be equivalent to FedNCF in Figure (b) which has shared score function across clients. Furthermore, our proposed framework's s architecture could be naturally aligned the classic neural network architecture, thus has bigger potential to achieve better learning efficiency and is more flexible to extend. More discussion about communication efficiency and time complexity can be found Appendix
225
226
227
228
229
230
B. B.I B.2
231
6
EXPERIMENTS
232
6.1
EXPERIMENTAL SETUP
233
We evaluate the proposed PFedRec on four real-world datasets: MovieLens-100s MovieLens- IM, Lastfm-2K and Amazon-Video. They are all widely used datasets in assessing recommendation models. Specifically, two MovieLens datasets were collected through the MovieLens website, con- taining movie ratings and each user has at least 20 ratings. Lastfm-2K is music recommendation dataset, and each user maintains list of her favorite artists and corresponding tags. Amazon- Video was collected from the Amazon site, containing product reviews and metadata information. We ex- cluded users with less than 5 interactions in Lastfm-2K and Amazon- Video. The characteristics of datasets shown in Appendix C.1 For dataset split, We follow the prevalent leave-one-out eval-
235
237
239
241
uation (He et al. 2017). We evaluate the model performance with Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) metrics. Details can be referred Appendix C.2

---

Under review as a conference paper at ICLR 2023
6.2
BASELINES AND IMPLEMENTATION DETAILS
244
Our method is compared with baselines in both centralized and federated settings. Fo- cusing on the performance improvement of the infrastructure of recommendation models that all others derive from, we select the general and fundamental baselines that conduct recommendations based on the interaction matrix.
Baselines
245
246
247
248
Matrix Factorization (MF) (Koren et al. 2009): This method is a typical recommendation al- gorithm. Particularly, it decomposes the rating matrix into two embeddings located in the same latent space to characterize user and item, respectively.
249
250
251
Neural Collaborative Filtering (NCF) (He et al. 2017): This method models user-item interac- tion function with an MLP, and is one of the most representative neural recommendation models. Specifically, we apply the interaction function with a three-layers MI.P for comparison, adopted in the original paper.
252
253
254
255
FedMF (Chai et al. 2020): It is a federated version of MF and a typical FedRec method. updates user embedding locally and uploads item gradients to the server for global update.
257
FedNCF (Perifanis & Efraimidis 2022):: It is a federated version of NCF. Specifically, each user updates user embedding locally and uploads item embedding network and score function to the server for global update.
258
259
260
Federated Reconstruction (FedRecon) (Singhal et al., 2021]: is a ofate-of-thhearr PFL frame- work, and we test it under the matrix factorization scenario. Between every two rounds, this method does not inherit user embedding from the previous round but trains from scratch.
262
263
Implementation details We randomly sample 4 negative samples for each positive sample fol- lowing (He et al. 2017). For a fair comparison, we keep the same latent user (item) embedding size for all methods, i.e,, 32 and set other model details of baselines according to their original papers. For our method, we assign the score function with a one-layer MLP for simplification, which can be regarded as an enhanced FedMF with the dual personalization mechanism. We implement the methods based on the Pytorch framework and run all the experiments for 5 repetitions and report the average results. Parameter configuration can be found in Appendix C.3
264
286
268
269
6.3
COMPARISON ANALYSIS
We conduct experiments on four datasets for performance comparison.
MovieLems- 100K
MovieLem-1h IM
Lastfm-2K
Amazon- Vidao
HR
NDCG@10
HX6
NUCG8 10
HK
ND0G
HK
NDCG8 10
NCF
14 0.98
37.91
0.3
64.17
0.99
57.85 + 0.68
82.44
0.42
67.43
2 0.89
60.66 = 0.45
58.97 0.14
MF
64.43 + 1.02
38.95
0.56
08.45
0.34
41.37 + 0.18
82771
0.54
71.04
+: 0.62
46.69 = 0.65
29:33
0.45
FedNCF
60.2 + 0.59
33.25
1.35
00.54
1.46
2: 0.40
81.35
0:338
61,03
0.63
0.07
36,86
0006
FedRes
FedReeco
64.45 + 0.81
37.78
0.38
63.28
0.15
36.39
4: 0.33
82.06
0.38
61.58
4: 0.38
59.00 +: 0.14
38.87
4 0.13
FedMP
65.15 4: 1.16
39.38
1.08
67.72
0.14
40,90
1: 0.14
81.51
0.448
69.36
1: 0.42
59.67
:: 0.19
38.55
0:021
PFedRee (Ours)
71.42 + 0.83
43.44
0.89
73.26
0.20
44.36
+ 0.16
82.38
4: 0.92
73.19
0.38
60.08
0.08
39.12 + 0.09
Table 1: Performance of HR@ 10 and NDCG@ 10 on four datasets. Rec and FedRec represent centralized and federated methods, respectively. The results are the mean and standard deviation of five repeated trials.
Results & discussion As shown in Table I (1) PFedRec obtains better performance than central- 272
ized methods in some cases. In centralized scenario, only user embedding is regarded as personal- ized component to learn user characteristics, and other components are totally shared among users. 275 In comparison, our dual personalization mechanism considers two forms of personalization, which can further exploit user preferences. (2) PFedRec realizes outstanding advances on the two Movie- Lens datasets. These two contain more samples for training, which supports the personalization depiction. (3) PFedRec consistently achieves the best performance on all settings. In FedRec, the
273
274
277
279
common item embedding helps transfer the shared information among users, which facilitates col-
laborative training of individual user models. However, different users present rather distinct prefer-
281
ences for items. Our dual personalization mechanism offers fine- grained personalization which fits the local data. It filters out the interference of redundant information and obtains better performance.
282
283
We also analyse spacial and computational efficiency of our proposed model and compare with baselines, including parameter volume, training epoch and running time. Specifically, our proposed
7

---

Under review as a conference paper at ICLR 2023
method achieves best performance with relatively low level of running time consumption. It shows that our dual personalization mechanism will not increase spatial and computational compexity. Detailed analysis could be found in Appendix [.I] Besides, we present the convergence curves of PFedRec and all baselines on all datasets in Appendix [D22 It shows PFedRec is the consistently the fastest to converge, which emphasizes method's advanced efficiency.
286
288
289
6.4
INTEGRATING BASELINES WITH OUR DUAL PERSONALIZATION MECHANISM
291
This paper proposes a lightweight dual personalization mechanism to enhance personalization han- dling, which can be easily transferred to nearly-all federated learning methods. FedRec baselines to exhibit its efficacy.
282
We apply it to
Method
MovicLans, 100K
MovicLans-Ih IM
Lasthu-2K 3K
Amazon-Vidl
10
NDCGG-10
HR
NDCG0
FR@10
NDCG8
10
NDCG@100
FedNCF
60,62 0.59
33,25 1: 1.35
60.54 1: 0.16
36,17 1: 0.40
81.55 0:38
61.03 1: 0.63
57.77 1 0,07
36.86 = 0.06
w/ DualPer
68.82 +: 1.35
39.33 +: 0.85
68.17 +: 0.55
39.56 + 0.29
82:31 + 0.56
71.64 +: 0.43
59.57 +: 0.57
38.73 - 0.62
Improvement
13.3%
18.29%
12.60%
15,77%
0.933%
t 17.38%
1 3.1%%
5.07%
FedRecon
64.45 0.81
37.78 +: 0.38
63.28 + 0.15
36.59 + 0.33
82.06 + 0.38
67.58 + 0.35
59.80 + 0.14
28,87 = 0.13
u/ DualPer
70.20 0.90
41,83 0.71
+ 0.26
+ 0.16
83.51 0:23
74:83 + 0.44
+ 0.16
0.12
Improvement
+ 8.92%
1 10.72%
8.87%
9.43%
f 1.77%
t 10.73%
1 0.72%
1 0.85%
FedMF
65.15
67.72 + 0.14
+ 0.14
0..4
69.36
= 0.42
= 0.19
0.21
w/ DualPer
71.62 + 0.83
43.44 + 0.89
73.26 + 0.50
44.36
0.16
82:38 0.92
73.19 0.38
60.08 = 0.08
19.12 0:09
Improvement
9.93%
10.31%
8.18%
8.46%
0.91%
5.52%
0.69%
1.48%
Table 2: Performance improvement for integrating our dual personalization mechanism to baseline algorithms. Improvement denotes the performance gain beyond the baselines due incorporating our dual personalization mechanism (DualPer). The results are the mean and standard deviation of five repeated trials, and the significant improvemments (over 5%) highlighted.
293
294
Results& & discussion According to Table 2 all the baselines significantly improved by inte- grating our dual personalization mechanism since our mechanism enhances their modeling of user personalization. The highest HR and NDCG increases exist at FedNCF on MovieLens- 100K, i.e,, 13.5 13.53% and 18.29%. The enhancement of FedNCF attains the most remarkable boost, which em- phasizes the necessity of learning personalized item embedding for each user and the capacity of our dual personalization mechanism. Comparing with Lastfm-2K and Amazon- Video, the improve- ment of this mechanism is more evident on the two MovieLens datasets, almost around 10%, where each user has more samples locally. In summary, our proposed dual personalization mechanism can help the local model to learn more information when training personalized item embedding, which benefits the recommendation system prominently.
285
296
297
298
299
300
302
304
Impact Factor Analysis To further analysis the efficacy of PFedRec, we design comprehensive impact factor influence analysis, including latent embedding size, negative sample size and clients volume participationg in each round. In a nutshell, the best embedding size for all datasets is 32. Generally, the performance grows gradually as the negative samples increases. PFedRec able to
305
306
307
308
obtain the best performance regardless of different volume of client samples, while more clients facilitate the convergence. For more details please refer to Appendix D.3
309
310
6.5
A CLOSE LOOK PERSONALIIZATION PFEDREC
311
In our method, we learn the personalized item embedding network for each user based on the global item embedding to learn -ne-grained personalization. To further verify and analyze the role of personalized item embedding, we conduct empirical experiments to answer the questions:
312
313
314
Q1: Why personalized item embeddings benefit recommendation more than the global one?
315
Q2: How specific are the personalized item embeddings between users?
318
To answer Q1, We first discuss its straightforward insight, then we present visualization to demon- strate our claim. The recommendation system supposed to provide user-specific recommendations by exploiting historical interactions. In the FedRec setting, item embedding is consistently consid- ered to maintain the common characteristics among users, and its role in depicting user-specific preferences has been neglected. On the other hand, describing users with common item embedding
317
318
319
321
leads in noisy information, which may incur unsuitable recommendations. Through personalizing
item embedding, enhance personalization modeling in federated learning methods, which de-
323
picts the user-specific preference. We will demonstrate how precise the personalized embedding
describes user preference of our method compared with baselines.
8

---

Under review as a conference paper at ICLR 2023
We compare the item embedding learned by baselines with method. Particularly, select user randomly from the MovieLens-100 100K dataset and visualize the embeddings by mapping them 326
into 2-D space through t-SNE (Maaten & Hinton 2008). In this paper, mainly focus the implicit feedback recommendation, so each item is either a positive or negative sample of the user.
328
As shown in Figure - the item embeddings of positive (blue) and negative (purple) samples mixed together in baselines, However, they can be obviously divided into two clusters by PFedRec. We can easily conclude that our model learns which items the user prefers.
330
68
20-
20-
-20
-40-
-40-
-40-
-20
48-
-60-
(a) FedNCF
(b) FodRecon
(c) FedMF
(d) PFodRec
Figure 2: 7SNE visualization of item embeddings learnd by baselines and our method.
To answer Q2, compare three usages of item embedding follows:
335
Another random user: Each client is assigned with item embedding from another random user,
L.., every client runs with its score function and item embedding from another random user.
335
Global: We assign each client with globally shared item embedding, i.e., every client runs with its score function and global item embedding.
337
Own: It follows our setting that every client runs with its own score function and item embedding.
Specifically, we first train PFedRec, then assign the learned item embeddings the above three ways for inference. Experimental results are shown in Figure 3 Clients with their item embedding achieve the best performance, and clients with item embedding from others degrade significantly. Item embedding from another user contains a relatively low level helpful information for in- ference, even less than the common characteristics in global item embedding. The personalized item embedding learned by PFedRec has been adapted to the client preference, and different clients achieve rather distinct item embeddings, which depict the user-specific information.
339
341
343
85
80
Random
Random
80
Global
70
Global
Own
Own
H100:10
70
NDCC@00
60
50
40
60
55
30
50
20
ML-100K
ML-1M Lastfm-2K
Amazon
ML-100K
ML-1M
Lastim-2K
Amazon
Performance HR
Performance NDCG
Figure 3: Client inference using different item embeddings.
CONCLUSION
348
This paper proposes novel personalized federated recommendation framework to learn many on- device models simultaneously. We are the first to design the dual personalization mechanism that can learn fine-graine fine-nrained personalization on both users and items. This work could be fundamental work to pave the way for implementing a new service architecture with better privacy preservation, fine-grained personalization, and on-device intelligence. Given the complex nature of modern rec-
347
349
351
ommendation applications, such as cold-start problems, dynamics, using auxiliary information, and proposed framework is simple and flexible enough to be extended to tackle many new challenges. Moreover, the proposed dual personalization a simple- but-effective mechanism to be easily integrated with existing federated recommendation systems.
processing multi-modalin contents,
353

---

Under review as a conference paper at ICLR 2023
REFERENCES
356
Muhammad Ammad-Ud-Din, Elena Ivannikova, Suleiman A Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. Federated collaborative filtering for privacy-preserving personalized recommendation system. arXiv preprint vrXiv:1901.09888 2019.
357
359
Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary, Fed- erated learning with personalization layers. arXiv preprint arXiv:1912.0088) 2019.
361
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, et al. Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems, 1:
363
374 388, 2019.
365
Di Chai, Leye Wang, Kai Chen, and Qiang Yang. Secure federated matrix factorization. JEEE Intelligent Systems, 2020.
366
387
Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiugiang He. Federated meta-learning with fast convergence and efficient communication. arXiv preprini arXiv:1802.07876 2018
388
389
Gary Cheng, Karan Chadha, and John Duchi. Fine-tuning is fine in federated learning. arXiv preprint MXXx:2108.003113 2021.
370
Woo-Seok Choi, Matthew Tomei, Jose Rodrigo Sanchez Vicarte, Pavan Kumar Hanumolu, and Rakesh Kumar. Guaranteeing local differential privacy on altra-low-power systems. In 2018 ACM/IEE 45th Annual International Symposium on Computer Architecture (ISCA), pp. 561-
372
373
374
574. IEEE, 2018
Yongjie Du, Deyun Zhou, Yu Xie, Jiao Shi, and Maoguo Gong. Federated matrix factorization for privacy-preserving recommender systems. Applied Soft Computing, 11 111:107700, 2021.
376
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with the- oretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33:3557-3563 2020.
378
380
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural col- laborative filtering. In Proceedings of the 26th international conference world wide web, pp. 173-182, 2017.
382
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Ad- vances and open problems federated learning. arXiv preprint
384
385
2019.
386
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, al. Ad- vances and open problems in federated learning, Foundations and Trends@ in Machine Learning,
387
388
309
14(1 1):1-2):112 1200 2021.
390
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference Machine Learning, pp. 5132-5143, PMLR, 2020.
391
393
Farwa
Khan, Adrian Flanagan, Kuan Eeik Tan, Zareen Alamgir, and Muhammad Ammad-Us Din. A payload optimization method for federated recommender systems. In Fifieenth ACM Conference on Recommender Systems, pp. 432-442 2021.
395
Yehuda Koren. Factorization meets the neighborhood: multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 426 434, 2008.
397
399
Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, 2009.
10

---

Under review as a conference paper at ICLR 2023
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50 60, 2020.
402
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, pp. 6357- 6368. PMLR. 2021.
404
406
Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis- Phiis-Philippe Moreney. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.0152. 2020.
408
Guanyu Lin, Feng Liang, Weike Pan, and Zhong Ming. Fedrec: Federated recommendation with explicit feedback. IEEE Intelligent Systems, 2020a.
410
Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Dongxiao Yu, Jun Ma, Maarten de Rijke, and Xiuzhen Cheng. Meta matrix factorization for federated rating predictions. In Proceedings of the 43rd International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 981-990, 2020b.
412
414
415
Zhaohao Lin. Weike Pan, and Zhong Ming. Fr-fmss: federated recommendation via fake marks and secret sharing. In Fiffeenth ACM Conference on Recommender Systems, pp. 668-673, 2021.
418
417
Shuchang Liu, Yingqiang Ge, Shuyuan Xu, Yongfeng Zhang, and Amelie Marian. Faimess-awaae federated matrix factorization. In Proceedings of the 16th ACM Conference on Recommender Systems, pp. 168-178, 2022a.
418
419
420
Zhiwei Liu, Liangwei Yang. Ziwei Fan, Hao Peng, and Philip S Yu. Federated social recommenda- tion with graph neural network. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4):1-22 24, 2022b.
422
423
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-266) 2008.
424
425
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effcceentt learning of deep networks from decentralized data. In Artificial Intelli- gence and Statistics, pp. 1273-1288 PMLR, 2017.
426
427
428
Lorenzo Minto, Moritz Haller, Benjamin Livshits, and Hamed Haddadi. Stronger privacy for feder- ated collaborative filtering with implicit feedback. In Fifteenth ACM Conference on Recommender Systems, pp. 342-350, 2021.
429
431
Khalil Muhammad, Qinqin Wang, Diarmuid O' O'Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor. Fedfast: Going beyond average for faster training of federated recommender systems. Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Data Mining, pp. 1234-1242, 2020.
435
435
Vasileio Perifanis and Pavlos Efraimidis. Federated neural collaborative filtering. Knowledge- Based Systems, 242:108441, 2022.
437
Krishna Pillutla, Kshitiz Malik, Abdel-Rahman Mohamed, Mike Rabbat, Maziar Sanjabi, and Lin Xiao. Federated learning with partial model personalization. In International Conference Machine Learning, pp. 17716-17758. PMLR, 2022.
439
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars -himidd Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.26.. 2012.
441
Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypemetworks. In International Conference on Machine Learning, pp. 1489-9502 PMLR, 2021.
443
Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, and Sushant Prakash. Federated reconstruction: Partially local federated learning. Advances in Neural Information Processing Systems, 34, 2021.
445
11

---

Under review as a conference paper at ICLR 2023
Meng Wang, Weijie Fu, Shijie Hao, Dacheng Tao, and Xindong Wu. Scalable semi- eemi-supervised learning by efficient anchor graph regularization. IEEE Transactions on Knowledge and Data Engineering, 28(7):184-11877, 2016.
448
450
Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165-174, 2019.
451
452
453
Chuhan Wu, Fangzhao Wu, Yang Cao, Yongfeng Huang, and Xing Xie. Fedgnn: Federated graph neural network for
recommendation. arXiv preprint xxxi:21102.04225, 2021.
455
Chuhan Wu, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Fedetr: Federated native ad ctr prediction with cross platform user behavior data. ACM Transactions on Intelligent Systems and Technology (TIST), 2022a.
457
458
Chuhan Wu, Fangzhao Wu, Tao Qi, Yongfeng Huang, and Xing Xie. Fedattack: Effective and covert poisoning attack on federated recommendation via hard sampling. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4164 4172, 2022b.
459
460
Liu Yang, Ben Tan, Bo Liu, Vincent W Zheng, Kai Chen, and Qiang Yang. Practical and secure federated recommendation with personalized masks. arXiv preprint arXiv:2109.02466 2021.
462
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.
464
465
488
Jingwei Yi, Fangzhao Wu, Chuhan Wu, Ruixuan Liu, Guangzhong Sun, and Xing Xie. Efficient- fedrec: Efficient federated learning framework for arXiv preprint arXiv: 2rXiv:2009005444: 2021.
467
news recommendation.
489
Shijie Zhang, Hongzhi Yin, Tong Chen, Zi Huang, Quoc Viet Hung Nguyen, and Lizhen Cui. Pi- pattack: Poisoning federated recommender systems for manipulating item promotion. In Pro- ceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp- 1423,, 2022.
471
472
473
A
MORE DISCUSSION ABOUT DUAL PERSONALIZATION
474
A.1
LEARNING EFFICIENCY OF PERSONALIZED SCORE FUNCTION
475
Due to the design of dual personalization, the user's critical information is decomposed into person- alized item embeddings and personalized score function. Therefore, in most scenarios, including inactive users, the score function module does not demand a large neural network. The simple and swift multi-layer neural network is capable of tackling most scenarios. For example, in our imple- mentation, we use a one-layer MLP as the score function, which achieves significant performance improvement again baselines. Particularly,
478
477
478
479
conduct empirical experiments to analyze the model's effectiveness on inactive users and details can be found D.6 Moreover, to further improve the learning efficiency, we could pre-train a neural score model for each user group considering their demographic features.
481
483
EFFECTIVENESS OF ONE- FINE-TUNING A.2
485
The one-step gradient fine-tuning is necessary for our proposed PFedRec, and our design is sup- ported by the theoretical analysis in Pillutla et al. (002). A significant consideration is that the PFedRec alternately tunes one gradient step each for the global item embedding module and the
486
487
488
-lient-specific score function. This learning process is consistent with the partial personalization scheme studied in Pillutla et al. (2022). where theoretical analysis shows that one gradient step is sufficient and the algorithm's 's convergence is guaranteed when the following assumptions hold: (1)
490
The gradient of local loss function is L-Lipschitz smooth. (2) The gradient of local loss function has bounded variance. (3) the variance of the gradient of local loss functions on different clients is bounded. As the three assumptions universal and hold for most functions, the conclusions from
492
12

---

Under review as a conference paper at ICLR 2023
Pillutla et al. (2022) also hold in our proposed PFedRec, i.e., the one-step fine- tinettuning is sufficient and guaranteed to converge.
495
B
KEY ISSUES DISCUSSION IN FEDERATED RECOMMENDATION
497
B.1
COMMUNICATION EFFICIENCY
498
Due to the nature of federated learning, multiple rounds of parameter transfers are required between the server and the clients complete the training process, and communication efficiency is impor- lant consideration in federated learning modeling. For the FedRec model, there are three key factors that determine the communication cost, including the parameter volume M transmitted between server and clients, the number of sampled clients S in each round, and the total communication rounds T. The overall communication cost can be formulated as,
500
502
503
C-M-ST
504
Since S and T are constant among different models, the communication cost is positively corre- lated to .M, i.e., C x M. Generally, item embedding is regarded as the shared component in FedRec research. For example, FedMF Chai et al. (2020) and FedRecon Singhal et al. (202T) only transmit item embedding whose M [ d, where [I] is the number of items and d denotes the embedding size. Besides, there are also some works that take the score function or user embed- ding common components transmitted between server and clients together with item embedding. FedNCF Perifanis & Efraimidis (2022) and MetaMF Lin et al.] (2020b) transmit both item embed- ding and score function whose M [Z| d + FF,, where [F] is the parameter volume of score function, such as three-layer MLP. FedGNN Wu et al. (202T) also share user embedding whose [ + [Fl + [24] d, where [24] is the number of users. In our method, we only share item embedding among users, which results in the minimal transmission parameter volume, i.e.,
506
508
509
510
511
512
513
M
514
515
M [Z|-d.
In our implementation, each client does not need to maintain the full item embedding table. Since each user only trains her local model with historical interactions and randomly sampled negative instance e.g., 5 times of historical interactions. As result, the number of items processed by the user is much less than the complete item list, which further improves the communication efficiency of the proposed method. Take Lastfm-2K average size of historical interactions is 16, which is further less than transmitting the full item set. Moreover, in this paper, mainly focus on the fundamental recommendation task that only relies on the interaction matrix, and it is necessary to instantiate the item embedding module with the item embedding table. When implementing our framework under the scenario where item attributes are available, we can replace the item embedding table with lightweight neural network similar to the score function. Then the local model on each device can be more lightweight, which results in less communication cost. More empirical results about communication efficiency can be found in Efficiency Comparison D.I Convergence Analysis D.2 and Hyper-Paramen Study-Effect of client samples participating in each round D.3
517
519
example, the size of the item set is 12,454, and the
521
523
524
525
526
527
528
529
530
B.2
TIME COMPLEXITY
We analyze the time complexity of PFedRec. For clarity, we restate the notation that [Z] d is the item embedding table, where [Z] is the number of items and d is the latent embedding size; 0 is the score function, and we instantiate with single-layer MLP whose size is d; let denotes the local training epochs, S is the sampled clients in each round and T is the total communication rounds. Since is usually a small constant, e.g., e 1 in our implementation, then the time complexity of PFedRec is O(IId(
532
538
534
535
536
C EXPERIMENTAL SETUP DETAILS
537
538
C.I MORE DETAILS OF THE DATASETS
Forr practical implementation, the server could randomly sample ordinary instances rather than negative instances only, and then send it to the client to judge whether it is negative instance. The server's item set size is usually much bigger than the number of interacted items by user. Thus most randomly selected instances will be negative instances.
13

---

Under review as a conference paper at ICLR 2023
We filter the users whose historic interactions less than 5 and the characteristics of the four datasets used experiments are shown in Table B Since we focus on the implicit feedback recom-
540
mendation scenario in this paper, transform the ratings in each dataset into implicit data, that is, let mark that the user has rated an item.
Dataset
Interactions
Users
Items
Sparsity
Moviel e00K
100,000
943
1,682
93.70%
MovieLenssIM
1,000,209
6,040
3,706
95.53%
Lastfm-2K
185,650
1,600
12,454
99.07%
Amazon- Video
63,836
8,072
11,830
99.93%
Table 3: Datasets statistics.
542
543
C.2
EVALUATION PROTOCOLS
544
For dataset split, we adopt prevalent leave-one-out evaluation, following He al. (2017). For each user, we take her last interaction the test data and remain other interactions for training. Addition- ally, we regard the latest raction in the training set as the validation data to find nyper-parameters. For the final evaluation,
546
follow the regular strategy Koren (2008); He al. (2017) that randomly samples 99 unobserved items for each user, ranking the test item among the 100 items. We evaluate the ranked list with Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) metries.
548
549
550
In particular, HR measures whether the test data is in the top-K list, while NDCG considers the test
551
data's 's position in the list. The two evaluation metrics could be formulated follows,
HR
hits(u)
(10)
552 where is the number users and hit = indicates that the test data of user u is in the top-K recommendation list, otherwise hits(u) = 0.
NDCG
1 N log(p. + 1)
Σ
log 2
(11)
where Pa denotes the position of test data of u in the recommendation list and Pa do when the test data is not in the top-K K recommendation list. Here we set K = 10 for all experiments.
555
C.3 PARAMETER CONFIGURATION
557
In the implementation, we set the latent embedding size as 32 and the batchsize is fixed as 256 for all baselines and our method. The total number of communication rounds is set to 100, and this value enables all methods to be trained to converge through experiments. For learning rate, we search in [0.0001; 0.0005: 0.001; 0.005; 0.01; 0.5) and the specific setting four datasets summarized in Table 4 For other empirical experiments of our method, we directly used the same parameters without researching.
559
561
Method
1L01
ML-IM
Lastfm-2K
Amazon
NCF
0.005
0.005
0.005
0.005
MF
0.0005
0.0005
0.001
0.001
FedNCF
0.5
0.5
0.5
FedNCF DualPer
0.5
0.5
FedRecon
0.1
0.1
0.1
0.05
FedRecon w/ DualPer
0.1
0.1
0.1
0.1
FedMF
0.1
0.1
0.05
0.05
PFedRec (Ours)
0.1
0.1
0.05
0.01
Table 4: Learning rate configuration for all methods on four datasets. w/ DualPer indicates the model enhanced with our proposed dual personalization mechanism.
562
14

---

Under review as a conference paper at ICLR 2023
D MORE EXPERIMENTAL RESULTS
564
D. EFFICIENCY COMPARISON
585
In federated learning, space and time efficiency are prominent factors for application. We compare the model's efficiency, including parameter volume, training epochs and running time, as shown in Table[5| According to the results, (/) FedNCF has the largest volumes of parameters. Parameters of recommendation systems consist of several parts, i.e., user and item embedding and score function. The embedding size is the same for all methods in each dataset. For score function, FedNCF FedNCFemploys three-layers MLP, which leads to much more parameters than one-dimensional embedding in other methods. (2) FedRecon takes the most training epochs to converge. The reconstruction mechanism
566
567
569
571
572
in FedRecon demands retraining the local module from scratch in each round, which results in more training epochs. Taking Moviel 100K as an example, our method converges in 61 training epochs, while FedRecon requires 465 training epochs, approaching times ours. FedNCF takes the
573
575
second longest training epochs and training time. On MovieLens-10000 the total training time of
578
FedNCF is about 2 times as long as ours. (3) The space and time efficiency of FedMF is at the same level as our method. Our method enhances the personalization modeling and achieves the best performance without extra computational complexity. our experiments, we found that just one local gradient descent step to learn personalized item embedding can yield advanced performance.
577
578
MovieLens-Io00
MovieLens-IN -M
Lastfm-2K
Amazon- Video
Method
parameters
epochs
time (8)
parameters
epochs
time (5)
parameters
opochs
time (6)
parameters
epochs
time (6)
FedNCF 86,753
90
1,800
314,625
98
18.718
452,481
4,750
639,617
60
5,820
finetune
86,751
314,625
452,481
4,680
639.617
4,120
FedRecon
9,486
123,892
398.560
7.154
378,592
3
w/ linetune
53,856
444
9,380
118.624
476
125,230
398.560
6,740
378.592
4,760
FeaMF
53,856
72
936
1186224
93
12,927
398,560
82.
3,280
378,592
56
3,920
PFedRee
53,857
61
854
118.625
95
13.585
398,561
60
2,340
378,593
75
5,100
Table Efficiency comparison results on four datasets, including model parameter volume, training epochs and running time.
579
580
D.2
CONVERGENCE ANALYSIS
581
We show the convergence curves of the two evaluation metrics for our method and baselines on four datasets. As shown in Figure our method achieves the fastest convergence on all datasets and met- rics almost all the time. followed by FedMF and FedRecon. FedNCF has the slowest convergence speed because has the most parameters and requires a longer number of iterations.
582
FoENCI
FedNCE
FodNCF
FodNCF
Pedieecon
FedRacen
Pedllocen
FedRocen
FedMF
PedM[
PedMf
Currs
Ours
Curz
Dure
189
Number Rovands
Number Rovands
Number
Sumber
(5) MovieLenns L00K
(b) MovieL.as-11 M
(c) Lastfm-2K
(4) Amazon
Nocrogts
30
Fedlluccn
Fedilaccn
Fedllocon
189
Nuntor Rouads
Nonther of Rounds
Number of Rounds
Number of Reunds
(6) MovieLens- 1O0K
(f) MovieLens-IN -M
Lastim 2K
Amazon
Figure 4: Model convergence comparison. The horizontal axis the number of federated optimiza- tion rounds, and the vertical axis is the model performance, where (a)-(d) are HR @ 10 metric and (e)-(h) are NDCG@ 10 metric.
15

---

Under review as a conference paper at ICLR 2023
D.3
RYPER-PARAMETER STUDY
585
Effeet of latent embedding size We tune the latent embedding size from cordingly, the architecture of the score function, i.e., one-layer MLP, is 16 1, 32 + 1, 64
64, 128). Ac-
586
587
and 128 + 1, respectively. Experimental results are shown in Table 6 Almost all four datasets achieve the best results when the latent embedding size is 32. Due to the limited data of a single user, increasing the model parameters did not obtain further performance improvement. Generally, setting the dimension to 16 or 32 can achieve advanced performance. A small volume of parameters helps to build lightweight on-device recommendation model.
586
580
Dataset
Metries
16
32
64
128
HR@10
72.81 + 0.90
71.62 + 0.83
71.64 + 0.44
71.75 + 0.80
Movielens-000K
NDCG@10
43.32 1. 0.43
43.44
0.89
44.70 1.01
45.42 + 0.88
1R
72.70 i 0.18
73.26 + 0.20
71.91 +: 0.41
70.67 0.15
Movielens-IM
1DCG@10
43.04 1 0.19
44.38 + 0.16
44.19 1: 0.15
43.74 1: 0.36
$1.93
0.92
81.93
82.01
Lastfm-2K
1D 10
72.47 = 0.65
73.19 + 0.38
72.41 + 0.88
72.63 + 0.29
HR@10 10
59.96 0.18
60.08 0.08
59.75 0.15
59.60 0.16
Amazon-Video
1DCG@10
39.14 + 0.07
39.12 + 0.09
39.07 +: 0.14
38.99 0.11
Table 6: Performance of different latent embedding sizes on four datasets. The results are the mean and standard deviation of the five repeated trials. Each number has an order of magnitude of 1e-2.
582
Effect of negative sample size We set the negative sample size from 10 and observe the effect on model performance. Experimental results are shown in Figure 5 For two Movielens datasets, model performance improves significantly as the number of negative samples increases.
593
594
595
Since these two datasets have more user data than Lastfm-2K and Amazon, they contain richer user preference information to learn. Sampling more negative instances help the model to further identify user preferences. On the other two datasets, 4 negative samples are enough to obtain ideal model performance.
596
587
95
95
NDCrig1t BRgin
Illiri NDCrag19
IR
IlRta 10
1D
NDCGg10
25
Performnnce
Performance nace
Performance
65
65
76
80
45
5
Number of Negatives
Number of Negeativve es
Number Neggativees
Number of Negatives
(aj MovieLenns-1000
(b) MovieLenns-1M
(e) Lastfm-2K
(d) Amazon
Figure 5: Performance under different negative sample numbers for each positive sample.
598
599
Effeet of client samples participating in each round There is a trade-off between client sam- pling ratio and communication efficiency in federated optimization. Generally, the more clients are selected to participate in the global aggregation, the faster the model converges in each round. How- ever, in the physical scenario, it is difficult for the server to collect the complete model information from all the clients. Particularly, there are a large number of user clients in the recommendation scenario, which further increases the difficulty. To verify the relationship between the model's con- vergence and the clients' participation each round, we conduct experiments on four datasets with various client samples. To create consistent validation environment for all datasets, we set the number of users selected in each round as 100, 200, 300, 400 and 500, respectively. Experimental results represented in Figure 0
600
601
602
603
605
607
609
We run the model until convergence and report the best validation performance with the corre- sponding epoch. According to the experimental results, can observe that PFedRec could reach consistently advanced performance all settings on all datasets, even only with 100 clients selected in each round during model training. On the other hand, it is obvious that more clients participating
611
16

---

Under review as a conference paper at ICLR 2023
100
BiRg10 NDCGggia
HIGUU NDOGEH
-A-
Bilkel NDCG
Hikgl0 NOCCaio
+
Perfomaanc
Perfocmanes
Egodte
Eppoctt
Eiposhs
108
158
280 30
281
00:
(1)
Moviee.enss-1000
100K
(b)
Moviel.mm Lens-
1M
(c)
Lastimn22K
(d)
Amazon
Figure 6: Performance under different client numbers participating in each round.
613
in each round of training lead to a quicker convergence. PFedRec supports the server to update with insufficient clients accessible, which is ubiquitous in physical cireumstances.
615
D.4
PROTECTION WITH DIFFERENTIAL PRIVACY
616
In addition to data locality inherited from the FL framework, introducing privacy-preservin meth- ods into our method can further enhance privacy protection FedRec. In our model, item embed- ding is the shared component in the federated optimization process. There is the risk of user interac- tion information exposure when the client uploads the updated item embedding to the server. Adding noise the item embedding is general method to defend against the privacy leakage attack, such as introducing differential privacy and homomorphic encryption into the model. Moreover, design- injection method is also potential solution to further enhance
617
618
619
621
ing reasonable privacy protection, which can be discussed in future work.
623
Here integrate the local differential privacy technique Choi et al. (2018) into our method as example. Particularly, we add the zero-mean Laplacian noise to the client's item embedding before uploading the server,
625
628
0 1th + Laplace (0, A)
(12)
where À is the noise strength. We set À = [0, 0,,, 0.3, 0.4, 0.5] to test our method's performance 627
and the results are shown in Table 7 We can see that the performance declines slightly the noise strength grows, while the performance drop is still acceptable. For example, when we set À = 0.4, 630 the performance is also better than baselines in most cases. Hence, a moderate noise strength is desirable to achieve a good balance between recommendation accuracy and privacy protection.
629
Dataset
Noise strength
A=0
A=0.1
A=0,2
A=0.3
A=0.4
A=0.5
HR@10
0..6
71.45=1.01
71.26::0.2
71.13::1.1
70.8420.96
70.88:0097
ML-100K
NDCG@10
03.44100.8
43.36=0.85
43.30+00.1
43.22:00.5
13.14#0.75
43.2140.69
HRe10
0.22
73.13=0.11
73.
13.191:0.21
73.05::0 211
73. 73.1820.29
73.0840.19
ML-1M
1D
0..16
44.16-0018
14.25:10.32
04.11410.14
44.23
44.18110.40
HR@10
82.3810..2
82.04.0.69
81.91
81.85.10.23
81.9810.52
81.8810.34
Lastfm-2K
NDCG@10
03.19
72.11=0.39
72.23::0.4 19
72.13:+0 68
72. 72.3920.27
72.36+0.44 42
HR@10
60.08
59.31=0.12
59. 00.0
59.21
59. 1320.66222
59.06+0.71
Amazon
NDCG@10
0.01
37.97 37.97=0.12
37.92+0.03
37.83+0.00 .0.
37.81=0.08
37.31+0.11
Table 7: Results of applying differential privacy technique into our method with various Laplacian noise strength X.
632
D.5 EVALUATION BASED ON FULL RANKING LIST
As illustrated in in the main experiments, we adopt the efficient sampled metric. To provide 634 a more comprehensive comparison, we conduct experiments to evaluate the test item in the full ranking list, which is more challenging than the sampled metric. As shown in Table 8 our proposed method still outperforms other baselines in all benchmark datasets, which emphasizes its outstanding efficacy.
636
17

---

Under review as a conference paper at ICLR 2023
Method
MovieLeas- 100K
MorieLeus-IM
Lasthm-2K
Video
1R
NDCG@10
IRR6
NDCGB10
1RR110
NDCG@10
UR
NDCG@
NCF
7:23 0.53
3:37
30.33
0.34
MF
15.76 + 0.88
8.35 + 0.4
8.00 = 0.19
4.38 2 0.14
37.06 0.4
25.88
0.50
7.19 0.23
2.74
FedNCF
5700::::
1543 1.4441333
0:6:::000
FedRee
FedRecon
13.10:00.33 0.75
6.92 0.31
6.00 = 0.21
3.25 2 0.11
31.63 + 0.94
20:00
22 2.76
6:70 2: 0.09
2.8 + 0.12
FedMF
4: 0.5
0.37
8:11
38.24 0.16
19.00
3.07
0.99
PFedRec (Ours)
19.19 1: 0.32
10.57 022
9.75
0.23
0.15
56.63
0.40
33.11
5.52
7.28 0.12
2,22 1 004
Table 8: Performance of HR@ 10 and NDCG@ 10 on full ranking list.
638
D.6
MODEL. PERFORMANCE ON INACTIVE USERS
639
In the recommendation task, there are usually some users with fewer available interactions, and call them inactive users, which poses great challenge for model training. Here we show the model performance on inactive users verify the effectiveness of our method. Particularly, count the number of ratings from all users, then extract inactive users and calculate their performance. The inactive users' statistics of each dataset are summarized in Table We omit the Amazon- Video due to its high sparsity, where the average size of historical interactions for each user is 8.
640
642
643
Dataset
Interactions range
Inactive users volume
Ratio of inactive users
ML-100K
20~30
199
21.10%
ML-1M
20~~30
751
12.43%
Lastfm-2k
5--20
641
40.06%
Table 9: Inactive users' statistics of each dataset.
The comparison results of performance inactive and full users shown in Table TO Our method 646 achieves similar or even better performance on the inactive users set than the full users set, indicating that our method can effectively tackle inactive users.
MovieLens-100K
MovieLens-LM
Lastim-2K
User group
HR 10
NDCG@10 10
HR@10
NDCG@ 10
HR@10 10
NDCG@ 10
Full users
71.79
43.39
73.10
44.24
82.50
73.27
Inactive users
79.40
52.51
77.76
53.24
78.94
69.08
Table 10: Comparison results of HR@ 10 and NDCG@ 10 inactive users and full
647
18


Reviews:
[{"review_id": "rL4DyIFjnf", "reviewer": "Reviewer_EAiQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new personalized federated training model architecture that does not contain user embeddings explicitly. The model relies on personalized item embedding and scoring function to achieve the personalized recommendation. When conducting federated training (global), it learns a shared embedding parameter for the items. And, during the personalization step, it conducts 1 step gradient fine-tuning for each client. The term \"Dual\" personalization simply comes from separately learned  scoring function that never been federated trained. Experiment results show promising performance improvements.", "summary_of_the_review": "The idea seems straightforward and somewhat novel in terms of reducing modeling cost. But the cost reduction is not significant (as least not demonstrated to be significant).\nThe description of this paper is a bit lack polishing. And, I have to guess what the authors try to deliver\n- 1. does \\theta^m contains item embedding? why do you use E and \\theta^m together? are they somewhat different?\n- 2. why is one-step gradient for each client sufficient? any justification or assumption?\nThe scoring functions are never transferred and there is no default scoring function. In this case, how do you handle users who lack interactions? I think this need further discussion. \nI am also concerning the risk of overfitting of this model. As the model personalizing (fine-tuning) ALL of the client parameters. The strong collaborative regularization is missing. Considering the datasets used in this paper are often showing very coherent preference (lack diversity) of each user, it is possible that the client model is simply learned to remember the user's historical interactions. ", "strength_and_weaknesses": "The idea seems straightforward and somewhat novel in terms of reducing modeling cost.\nThe description of this paper is a bit lack polishing. \nI am also concerning the risk of overfitting of this model. \n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "_KsjdWTDlyW", "reviewer": "Reviewer_kuz5", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "To learn a personalized and lightweight recommendation model for each client, this paper proposes to learn user-specific lightweight models that are deployed on the device side. Furthermore, it introduces a dual personalization mechanism to learn personalized item embedding tables and the score function for each user. Experimental results on four real-world datasets demonstrate the effectiveness of the proposed method.", "summary_of_the_review": "Due to the limited novelty and missing important related work,  I recommend reject. ", "strength_and_weaknesses": "Pros:\na)\tThe motivation of this paper, i.e., learning a personalized recommendation model for each user, is important.\nb)\tEmpirical results demonstrate the effectiveness of the proposed method.\nc)\tThe paper is well-written and easy to follow.\n\nCons:\na)  This paper argues that the existing federated recommendation models are cumbersome, so it proposes to learn a LIGHTWEIGHT local score function to make personalized predictions on the client side. However, under the recommendation scenario, the item embedding table generally dominates the vast majority of parameters of the model instead of the parameters of the score function. Thus, the proposed method is meaningless.\nb)  The training of the personalized score function and the fine-tuning of the item embedding table are performed locally on the device. However, the training data on each device is very limited, which makes it hard to learn an accurate model.\nc)  The technical novelty of this paper is very limited. Learning the personalized item embedding table and score functions has been studied in [1]. The authors should clarify the key difference between their work and [1].\nd)  Important baselines are missing, such as [1,2]. The authors should compare the proposed method with them.\ne)  There is a duplicate phrase, “optimization framework” in the third contribution on page 2.\n\nReference:\n[1] Lin, Yujie, et al. \"Meta matrix factorization for federated rating predictions.\" Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020.\n[2] Wu, Chuhan, et al. \"A federated graph neural network framework for privacy. preserving personalization.\" Nature Communications 13.1 (2022): 1-10.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Gg6s8_8c_1", "reviewer": "Reviewer_rUEH", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a dual personalization strategy for federated recommendation, which can better handle non-iid user data and improve recommendation performance. The authors propose to use personalized scoring modules and personalized item embeddings on devices, where the model first updates the scoring module and then updates the item embeddings. Although the proposed method is simple, it is surprisingly effective. The authors also provide some discussions on its insight.", "summary_of_the_review": "This paper presents a good empirical study on federated recommendation. The proposed method is quite simple, general, and effective. Thus, my recommendation is weak accept, though its technical novelty is not very big.", "strength_and_weaknesses": "Strengths:\n- This paper is clearly written and easy to follow.\n- The proposed method is very simple and easy to use.\n- The performance improvement is quite impressive.\n\nWeaknesses:\n- It seems that the devices need to maintain a large set of item embeddings, which may be somewhat unscalable. Some discussions on how to reduce the number of locally maintained item embeddings are recommended.\n- The authors seem to use sampled metrics over the top recommendation results. In fact, it may not be consistent with the results on the full ranking list (see [1]). \n- It is not clear whether the proposed method still works well for inactive users. Since the scoring module may contain much more parameters than the user embedding, it would require a larger number of samples to train.\n\n[1] Krichene, W., & Rendle, S. (2022). On sampled metrics for item recommendation. Communications of the ACM, 65(7), 75-83.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-icz62SNPF6", "reviewer": "Reviewer_2m4p", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors study the problem of federated recommendation with implicit feedback, and propose a new framework with personalized score function and personalized item embedding (without user embedding), which is illustrated in Figure 1(c).", "summary_of_the_review": "The authors study an important problem of federated recommendation with implicit feedback, and propose a new framework with personalized score function and personalized item embedding.\n\nMy major concern is that the technical contribution compared with existing works on federated recommendation is limited.\n\nMinors:\noptimization framework optimization framework\nIt is a federated version of FedNCF\n", "strength_and_weaknesses": "Strength:\n1 The idea of using personalized item embedding in federated recommendation is new (though it is not new in traditional recommendation).\n\nWeakness:\n1 For key issues in federated recommendation, the authors do not contribute/discuss much, e.g., communication cost, privacy protection, time complexity.\n\n2 The technical contribution is limited. For example, the contents of Section 4 are not about a formal and principled solution, but most about heuristics.\n\n3 For the studied problem, there are many recent works, which are not studied in the experiments.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject