D UAL PERSONALIZATION ON FEDERATED RECOMMENDATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Federated recommendation is a new Internet service architecture that aims toprovide privacy-preserving recommendation services in federated settings.
Existing solutions are used to combine distributed recommendation algorithms andprivacy-preserving mechanisms.
Thus it inherently takes the form of heavyweightmodels at the server and hinders the deployment of on-device intelligent modelsto end-users.
This paper proposes a novel Personalized Federated Recommendation (PFedRec) framework to learn many user-speciﬁc lightweight modelsto be deployed on smart devices rather than a heavyweight model on a server.
Moreover, we propose a new dual personalization mechanism to effectively learnﬁne-grained personalization on both users and items.
The overall learning processis formulated into a uniﬁed federated optimization framework.
Speciﬁcally,unlike previous methods that share exactly the same item embeddings acrossusers in a federated system, dual personalization allows mild ﬁnetuning of itemembeddings for each user to generate user-speciﬁc views for item representationswhich can be integrated into existing federated recommendation methods togain improvements immediately.
Experiments on multiple benchmark datasetshave demonstrated the effectiveness of PFedRec and the dual personalizationmechanism.
Moreover, we provide visualizations and in-depth analysis of thepersonalization techniques in item embedding, which shed novel insights on thedesign of RecSys in federated settings.
I NTRODUCTION
Federated recommendation is a new service architecture for Internet applications, and it aims toprovide personalized recommendation service while preserving user privacy in the federated settings.
Existing federated recommendation systems (Ammad-Ud-Din et al., 2019; Chai et al., 2020;
Muhammad et al., 2020; Perifanis & Efraimidis, 2022; Singhal et al., 2021) are usually to be anadaptation of distributed recommendation algorithms by embodying the data locality in federatedsetting and adding privacy-preserving algorithms with guaranteed protection.
However, these implementations of federated recommendations still inherit the traditional service architecture, whichis to deploy large-scale models at servers.
Thus it is impractical and inconsistent with the newlyraised on-device service architecture, which is to deploy a lightweight model on the device to provide service independently without frequently communicating with the server.
Given the challengeof implementing data locality on devices in federated settings, the personalization mechanism needsto be reconsidered to better capture ﬁne-grained personalization for end-users.
Personalization is the core component of implementing federated recommendation systems.
Inherited from conventional recommendation algorithms, existing federated recommendation frameworksare usually composed of three modules: user embedding to preserve the user’s proﬁle, item embedding to maintain proximity relationships among items, and the score function to predict the user’spreference or rating for a given item.
They usually preserve user-speciﬁc personalization in the userembedding module while sharing consensus on item embeddings and score functions.
This paper proposes a new dual personalization mechanism designed to capture ﬁne-grained twofold personal preferences for users in the federated recommendation system.
Inspired by humanbeings’ decision logic, we believe all modules in the recommendation framework should be usedto preserve part of personalization rather than use user embedding only.
For example, the score
[Figure]
Our proposed model will preserve dual personalization on modules.
function is to mimic the user’s personal decision logic that is natural to be diverse across clients.
Furthermore, given an itemset, different people may have a different view to measure their proximity relationships.
Therefore, personalized item embedding could be essential to capture people’spersonal preferences further.
To implement the aforementioned ideas in federated settings, we propose a new federated recommendation framework to implement ﬁne-grained personalization on multiple modules which areillustrated in Figure 1 (c).
First, we use a personalized score function to capture users’ preferences,and it could be implemented using a multi-layer neural networks.
Second, we remove the userembedding from the framework because the current neural-based personalized score function hasenough representation capability to preserve the information of user embeddings.
Third, we implement a light ﬁne-tuning to learn personalized item embeddings 1 in federated settings.
This proposeddecentralized intelligence architecture is a natural simulation of human beings’ decision-making thateach person has a relatively independent mind to make decisions.
The learning procedure is also carefully tailored in a federated setting.
A personalized score functionwill be learned using its own data on the device, and then it won’t be sent to the server for modelaggregation that usually generates a general view for all devices.
Moreover, the personalized itemembedding will be implemented through light ﬁne-tuning in a federated learning framework, thus itcan leverage both the general view from server and the personalized view from its own data.
In summary, we propose a novel federated recommendation framework that integrates both the personalized score function and personalized item embedding via light ﬁnetuning from the shared itemembedding.
Our key contributions are summarized as follows.
• We propose a novel federated recommendation framework which is more naturally consis- tent with layer-wise neural architecture which can better ﬁt federated learning.
• We design a novel dual personalization to capture user preferences using a personalized score function and to ﬁne-grained personalization on item embeddings.
It can be integrated with other baselines to improve their performances.
• We formulate the proposed federated recommendation learning problem into a uniﬁed fed- erated optimization framework with the meta-learning objective.
• Our method can signiﬁcantly outperform existing federated recommendation baselines.
R ELATED W ORK2. P ERSONALIZED FEDERATED LEARNING
Federated Learning (FL) is a new machine learning paradigm that a server orchestrates a largenumber of clients to train a model without accessing their data (Kairouz et al., 2021;
Li et al., 2020;
Bonawitz et al., 2019; Yang et al., 2019).
The vanilla federated learning method, FedAvg (McMahanet al., 2017), is to learn a robust model at the server while embodying data locality for each device with non-IID data.
Personalized Federated Learning (PFL) is to learn a personalized model foreach device to tackle the non-IID challenge.
A simple PFL method could train a global modelwith FedAvg, then conduct a few steps of ﬁnetuning on each client (Cheng et al., 2021).
In thisframework, knowledge sharing is model aggregation, and model personalization is local ﬁnetuning.
Per-FedAvg (Fallah et al., 2020) adds ﬁnetuning as a regularization term to the objective learningfunction of the global model.
Ditto (Li et al., 2021) proposes a bi-level optimization framework for PFL while constraining the distance between the local and global models.
Investigations by (Shamsian et al., 2021; Chen et al., 2018) aim to train a global hyper-network or meta-learner instead ofa global model before sending it to clients for local optimization.
SCAFFOLD (Karimireddy et al.
,2020) proposes to learn personalized control variate that corrects the local model accordingly.
FedRecon (Singhal et al., 2021) is a meta-learning-based method that preserves a local model for eachclient and trains a global model collaboratively with FedAvg.
Layer-wise personalization (Arivazhagan et al., 2019;
Liang et al., 2020) is also a simple and effective technique in PFL.
2. F EDERATED R ECOMMENDATION S YSTEMS
Federated recommendation has attracted much attention recently due to the rising concern aboutprivacy. Some recent works focus on using the interaction matrix only.
FCF (Ammad-Ud-Din et al.
,2019) is the ﬁrst FL-based collaborative ﬁltering method, which employs the stochastic gradient approach to update the local model, and FedAvg is adopted to update the global model.
Improving onuser privacy protection, Chai et al.
present FedMF (Chai et al., 2020), which adapts distributed matrix factorization to FL setting and introduces the homomorphic encryption technique on gradientsbefore uploading to the server.
MetaMF (Lin et al., 2020b) is a federated meta-learning frameworkwhere a meta-network is adopted to generate the rating prediction model and private item embedding.
(Wu et al., 2021) presents FedGNN where each user maintains a GNN model to incorporatehigh-order user-item information.
However, the server in both methods preserves all the recommendation model parameters which can be used to infer the user’s interaction information, resulting inthe risk of user privacy leakage.
FedNCF (Perifanis & Efraimidis, 2022) adapts Neural Collaborative Filtering (NCF)
(He et al., 2017) to the federated setting which introduces neural network tolearn user-item interaction function to enhance the model learning ability.
Federated recommendation using rich information considers multiple data sources or modalities in modeling. FedFast (Muhammad et al., 2020) extends FedAvg (McMahan et al., 2017) with an active aggregationmethod to facilitate the convergence.
Efﬁcient-FedRec (Yi et al., 2021) decomposes the model into alarge news model on the server and a light user model on the client, and reduces the computation andcommunication cost for users.
Both works rely on more data sources, such as user features or newsattributes rather than an interaction matrix.
(Lin et al., 2020a; Du et al., 2021; Yang et al., 2021;
Minto et al., 2021; Lin et al., 2021) are endeavors that focus on enhancing privacy of FedRec.
Thereare also attempts for other applications in FedRec, such as federated attack (Wu et al., 2022b; Zhanget al., 2022), social recommendation (Liu et al., 2022b), Click-Through Rate (CTR) prediction (Wuet al., 2022a), fair recommendation (Liu et al., 2022a) and payload optimization (Khan et al., 2021).
P ROBLEM FORMULATION
Federated Learning is to learn a global model parameterized by θ to serve all clients whose dataare private.
The optimal solution should minimize the accumulated loss of all clients.
That is,
[Equation]
where L i ( θ ) is the supervised loss on the i -th client with dataset D i , and all clients share the globalparameter θ .
The α i is a weight for the loss of the i -th client.
For example, the conventionalfederated learning algorithm, FedAvg (McMahan et al., 2017), deﬁnes α i as the fraction of the sizeof the client’s training data, i.e., α i :=
| D i | / (cid:80) N j =1 | D j | .
Once the global model is trained, it can beused for prediction tasks on all clients.
Personalized Federated Learning simultaneously leverages common knowledge among clientsand learns a personalized model for each client.
The learning objective is usually formulated as
[Equation]
where each client has a unique personalized parameter θ i , and θ is the global parameter as mentionedin Eq.
For example, Fallah et al.
(2020) leverage θ as initialization of θ i , i.e., θ
i :
= θ − ∇ l i ( θ ) ,where l i ( θ ) is the loss of a vanilla model on the i -th client.
The L i ( θ, θ i ) is then formulated as
[Equation]
Recommendation on Neural Networks This work focuses on a fundamental scenario where recommendation only relies on user-item interaction matrix.
The recommendation task is then fulﬁlledby the Neural Collaborative Filtering (NCF) model (He et al., 2017), which consists of three parts:a score function S , a user embedding module E and an item embedding module E .
We denote thesemodules’ parameters as θ :=
( θ s , θ u , θ m ) and formulate the learning objective in Eq.
[Equation] [Equation]
where e u and e m are one-hot encoding representing users and items.
r is a user’s rate to the givenitem and ˆ r is a prediction from the score function S ( E ( e u ) , E ( e m )) .
L is the loss evaluating prediction performance.
It could be a point-wise loss as used in (Wang et al., 2016;
He et al., 2017), or apair-wise loss as in (Rendle et al., 2012; Wang et al., 2019).
It is worth noting that conventional Matrix Factorization (MF) methods could be viewed as a special case of the NCF (He et al., 2017), i.e.,the conventional MF is a model where the score function S is simpliﬁed as the multiplication operator without learnable parameters, and the embedding of user/item is obtained by the decompositionof the user-item interaction matrix.
M ETHODOLOGY
In this section, we propose a novel P ersonalized Fed erated Rec ommendation (PFedRec) framework,which aims to simultaneously learn many user-speciﬁc NCF models deployed on end devices.
4. O BJECTIVE FUNCTION
Federated Learning Objective We regard each user as a client under FL settings.
The on-devicerecommendation task is then depicted as a PFL problem.
Particularly, the item embedding module E i is assigned to be a global component which learns common item information and the scorefunction S i is maintained locally to learn personalized decision logic.
To further capture the difference between users and achieve a preference-preserving item embedding, we devise a bi-leveloptimization objective, [Equation]
where θ i := ( θ mi , θ si ) is the personalized parameter for E i and S i , and L i will be evaluated on thei -th client local data D i . Under this framework, PFedRec ﬁrst tunes E into a personalized itemembedding module E i , and then learns a lightweight local score function S i to make personalizedpredictions.
Different from the conventional recommendation algorithms, the user embedding module E is depreciated since the personalization procedure on a client will automatically capture theclient’s preference.
There is no use to learn extra embeddings to describe clients.
Loss for Recommendation Equipped with the item embedding module and score function, weformulate the prediction of j -th item by i -th user’s recommendation model as,
[Equation]
Particularly, we discuss the typical recommendation task with implicit feedback, that is, r ij = 1if i -th user interacted with j -th item; otherwise r ij = 0 .
With the binary-value nature of implicitfeedback, we deﬁne the loss function of i -th user as the binary cross-entropy loss ,
[Equation]
where D − i is the negative instances set of user.
Notably, other loss functions can also be used, andhere we choose the binary cross-entropy loss to simplify the description. Particularly, to construct D − i efﬁciently, we ﬁrst count all the uninteracted items set as,
[Equation]
where I denotes the full item list and
I i is the interacted item set of i -th user.
Then, we uniformlysample negative instances from I − i by setting the sampling ratio according to the number of observedinteractions and obtain D − i . 4. D UAL P ERSONALIZATION
We have implemented a dual personalization mechanism to enable the proposed framework canpreserve ﬁne-grained personalization for both user and item.
Using partial-based federated model aggregation to learn personalized user score function oneach device.
Our proposed model is composed of a neural-based score function parameterized byθ s and an item embedding module parameterized by θ m .
The coordinator/server of federated system will iteratively aggregate model parameters or gradients collected from each participant/device.
Due to the concern of personalization and privacy, we could implement a partial model aggregationstrategy by keeping the score function as a private module on devices while sharing the item embedding to the server.
Therefore, the server only aggregates the gradients or parameters θ m fromthe item embedding network.
The user’s personalized score function network θ s won’t be sent tothe server and thus won’t be aggregated. More discussion about learning efﬁciency of personalizedscore function can be found in Appenidx A.1 .
Fine-tuning the item embedding module to generate personalized representations for itemson each device.
According to Eq. 5, the learning objective of θ m could be viewed as searchingfor a ”good initialization” that could be fast adaptive to the learning task on different devices.
Itshares similar ideas with meta-learning based methods (Fallah et al., 2020) which have a local lossin Eq. 3.
However, our proposed method takes a different optimization strategy we call post-tuning .
Speciﬁcally, rather than directly tuning a global model on clients’ local data, it ﬁrst learns the localscore function with the global item embedding network, and then replaces the global item embeddingwith personalized item embedding obtained by ﬁne-tuning θ m .
Details of the learning process isillustrated in Algorithm.
1 and extensive experiments show it will achieve superior performanceto the vanilla meta-learning based methods. Furthermore, we discuss the effectiveness of one-stepﬁne-tuning in Appendix A.2 .
4. A LGORITHM
Optimization To solve the optimization problem as descripted in Sec.
4.1 - objective function, weconduct an alternative optimization algorithm to train the model.
As illustrated in Algorithm 1, whenclient receives the item embedding network from server, it ﬁrst replace its embedding with globalone, and then updates the score function while keeping item embedding network ﬁxed.
Then theclient updates the item embedding based on the updated personalized score function.
Finally, theupdated item embedding would be uploads to server for global update.
Workﬂow .
The overall workﬂow of the algorithm could be summarized into several steps as follows.
The server is responsible for updating shared parameters and organizing all clients to completecollaborative training.
At the beginning of federated optimization, the server initializes the modelparameters, which would be used as initial parameters for all client models.
In each round, theserver selects a random set of clients and distributes the global item embedding θ m to them.
Whenlocal training is over, the server collects the updated item embedding network from each client toperform global aggregation.
We build on the simpliﬁed version of FedAvg, a direct average oflocally uploaded item embedding network.
The overall procedure is summarized in Algorithm 1.
D
ISCUSSIONS 5. P RIVACY ON F EDERATED R ECOMMENDATION
Privacy-preserving is an essential motivation to advance existing cloud-centric recommendationto client-centric recommendation service architecture.
In general, the federated learning’s decen- tralized framework can embody data locality and information minization rules (GDPR) that couldgreatly mitigate the risk of privacy leakage (Kairouz et al., 2019).
To provide service with privacyguranttee, the FL framework should be integrated with other privacy-preserving methods, such as Differential Privacy and secure communication.
Our proposed framework derives the same decentralized framework from vaniall FL to preserve data locality.
For example, to tackle the privacyleakage risk caused by sending item embedding network to the server, we could simply apply differential privacy to inject noise into the gradients so that the server cannot simply infer the updateditems by watching the changes of gradients. {+More discussion can be found in Appendix D.
+}
5. A GENERAL FRAMEWORK FOR FEDERATED RECOMMENDATION
The proposed framework in Figure 1 (c) could be a general form of federated recommendation.
Because our framework could be easily transformed to an equivalent form of other frameworks.
Forexample, if we assign the score function as a one-layer neural network, PFedRec is equal to FedMFand FedRecon in Figure 1 (a).
Moreover, if we change the personalized score function from fullpersonalization to partial layer personalization, our method could be equivalent to FedNCF in Figure1 (b) which has a shared score function across clients.
Furthermore, our proposed framework’sarchitecture could be naturally aligned to the classic neural network architecture, thus it has a biggerpotential to achieve a better learning efﬁciency and is more ﬂexible to extend. {+More discussion aboutcommunication efﬁciency and time complexity can be found in Appendix B.1 and B.
+}
E XPERIMENTS6. E XPERIMENTAL S ETUP
We evaluate the proposed PFedRec on four real-world datasets: MovieLens-100K, MovieLens-1M,
Lastfm-2K and Amazon-Video.
They are all widely used datasets in assessing recommendationmodels.
Speciﬁcally, two MovieLens datasets were collected through the MovieLens website, containing movie ratings and each user has at least 20 ratings.
Lastfm-2K is a music recommendationdataset, and each user maintains a list of her favorite artists and corresponding tags.
Amazon-Videowas collected from the Amazon site, containing product reviews and metadata information.
We excluded users with less than 5 interactions in Lastfm-2K and Amazon-Video.
The characteristics ofdatasets are shown in Appendix C.1 .
For dataset split, We follow the prevalent leave-one-out evaluation (He et al., 2017).
We evaluate the model performance with Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) metrics.
Details can be referred to Appendix C.2 .
6. B ASELINES AND I MPLEMENTATION D ETAILS
Baselines Our method is
compared with baselines in both centralized and federated settings. Fo-
cusing on the performance improvement of the infrastructure of recommendation
models that all others derive from, we select the general and fundamental baselines that conduct recommendationsbased on the interaction matrix. • Matrix Factorization (MF) (Koren et al., 2009):
This method is a typical recommendation algorithm.
Particularly, it decomposes the rating matrix into two embeddings located in the samelatent space to characterize user and item, respectively.• Neural Collaborative Filtering (NCF)
(He et al., 2017):
This method models user-item interaction function with an MLP, and is one of the most representative neural recommendation models.
Speciﬁcally, we apply the interaction function with a three-layers MLP for comparison, adoptedin the original paper.• FedMF (Chai et al., 2020):
It is a federated version of MF and a typical FedRec method.
Itupdates user embedding locally and uploads item gradients to the server for global update.• FedNCF (Perifanis & Efraimidis, 2022):
It is a federated version of NCF.
Speciﬁcally, each userupdates user embedding locally and uploads item embedding network and score function to theserver for global update.• Federated Reconstruction (FedRecon) (Singhal et al., 2021):
It is a state-of-the-art PFL framework, and we test it under the matrix factorization scenario.
Between every two rounds, thismethod does not inherit user embedding from the previous round but trains it from scratch.
Implementation details We randomly sample 4 negative samples for each positive sample fol- lowing (He et al., 2017).
For a fair comparison, we keep the same latent user (item) embedding sizefor all methods, i.e., 32 and set other model details of baselines according to their original papers.
For our method, we assign the score function with a one-layer MLP for simpliﬁcation, which canbe regarded as an enhanced FedMF with the dual personalization mechanism.
We implement themethods based on the Pytorch framework and run all the experiments for 5 repetitions and report theaverage results. {+Parameter conﬁguration can be found in Appendix C.
+}
6. C OMPARISON ANALYSIS
We conduct experiments on four datasets for performance comparison
[Table]
Results & discussion As shown in Table 1, (1) PFedRec obtains better performance than centralized methods in some cases.
In centralized scenario, only user embedding is regarded as personalized component to learn user characteristics, and other components are totally shared among users.
In comparison, our dual personalization mechanism considers two forms of personalization, whichcan further exploit user preferences.
(2) PFedRec realizes outstanding advances on the two MovieLens datasets.
These two contain more samples for training, which supports the personalizationdepiction.
(3) PFedRec consistently achieves the best performance on all settings.
In FedRec, thecommon item embedding helps transfer the shared information among users, which facilitates collaborative training of individual user models.
However, different users present rather distinct preferences for items.
Our dual personalization mechanism offers ﬁne-grained personalization which ﬁtsthe local data.
It ﬁlters out the interference of redundant information and obtains better performance.
We also analyse spacial and computational efﬁciency of our proposed model and compare withbaselines, including parameter volume, training epoch and running time.
Speciﬁcally, our proposed method achieves best performance with relatively low level of running time consumption.
It showsthat our dual personalization mechanism will not increase spatial and computational compexity.
Detailed analysis could be found in Appendix D.1 .
Besides, we present the convergence curves of
PFedRec and all baselines on all datasets in Appendix D.2 .
It shows PFedRec is the consistentlythe fastest one to converge, which emphasizes our method’s advanced efﬁciency.
6. I NTEGRATING B ASELINES WITH O UR D UAL P ERSONALIZATION M ECHANISM
This paper proposes a lightweight dual personalization mechanism to enhance personalization handling, which can be easily transferred to nearly-all federated learning methods.
We apply it to
FedRec baselines to exhibit its efﬁcacy.
[Table]
Results & discussion
According to Table 2, all the baselines are signiﬁcantly improved by integrating our dual personalization mechanism since our mechanism enhances their modeling of userpersonalization.
The highest HR and NDCG increases exist at FedNCF on MovieLens-100K, i.e.,13.53% and 18.29%.
The enhancement of FedNCF attains the most remarkable boost, which emphasizes the necessity of learning personalized item embedding for each user and the capacity ofour dual personalization mechanism.
Comparing with Lastfm-2K and Amazon-Video, the improvement of this mechanism is more evident on the two MovieLens datasets, almost around 10%, whereeach user has more samples locally.
In summary, our proposed dual personalization mechanism canhelp the local model to learn more information when training personalized item embedding, whichbeneﬁts the recommendation system prominently.
Impact Factor Analysis To further analysis the efﬁcacy of PFedRec, we design comprehensiveimpact factor inﬂuence analysis, including latent embedding size, negative sample size and clientsvolume participationg in each round.
In a nutshell, the best embedding size for all datasets is 32.
Generally, the performance grows gradually as the negative samples increases.
PFedRec is able toobtain the best performance regardless of different volume of client samples, while more clientsfacilitate the convergence.
For more details please refer to Appendix D.3 .
6. A C LOSE L OOK OF P ERSONALIZATION IN PF ED R EC
In our method, we learn the personalized item embedding network for each user based on the globalitem embedding to learn ﬁne-grained personalization.
To further verify and analyze the role ofpersonalized item embedding, we conduct empirical experiments to answer the questions:
• Q1 : Why personalized item embeddings beneﬁt recommendation more than the global one?
• Q2 :
How speciﬁc are the personalized item embeddings between users?
To answer Q1 , We ﬁrst discuss its straightforward insight, then we present visualization to demonstrate our claim.
The recommendation system is supposed to provide user-speciﬁc recommendationsby exploiting historical interactions.
In the FedRec setting, item embedding is consistently considered to maintain the common characteristics among users, and its role in depicting user-speciﬁcpreferences has been neglected.
On the other hand, describing users with common item embeddingleads in noisy information, which may incur unsuitable recommendations.
Through personalizingitem embedding, we enhance personalization modeling in federated learning methods, which depicts the user-speciﬁc preference.
We will demonstrate how precise the personalized embeddingdescribes user preference of our method compared with baselines.
We compare the item embedding learned by baselines with our method.
Particularly, we select auser randomly from the MovieLens-100K dataset and visualize the embeddings by mapping theminto a 2-D space through t-SNE (Maaten & Hinton, 2008).
In this paper, we mainly focus on theimplicit feedback recommendation, so each item is either a positive or negative sample of the user.
As shown in Figure 2, the item embeddings of positive (blue) and negative (purple) samples aremixed together in baselines.
However, they can be obviously divided into two clusters by PFedRec.
We can easily conclude that our model learns which items the user prefers.
[Figure]
331 To answer Q2 , we compare three usages of item embedding as follows:
– Another random user : Each client is assigned with item embedding from another random user,i.e., every client runs with its score function and item embedding from another random user.
– Global : We assign each client with globally shared item embedding, i.e., every client runs with itsscore function and global item embedding.
– Own : It follows our setting that every client runs with its own score function and item embedding.
Speciﬁcally, we ﬁrst train PFedRec, then assign the learned item embeddings as the above threeways for inference.
Experimental results are shown in Figure 3.
Clients with their item embeddingachieve the best performance, and clients with item embedding from others degrade signiﬁcantly.
Item embedding from another user contains a relatively low level of helpful information for inference, even less than the common characteristics in global item embedding.
The personalizeditem embedding learned by PFedRec has been adapted to the client preference, and different clientsachieve rather distinct item embeddings, which depict the user-speciﬁc information.
[Figure]
C ONCLUSION
This paper proposes a novel personalized federated recommendation framework to learn many ondevice models simultaneously.
We are the ﬁrst to design the dual personalization mechanism thatcan learn ﬁne-grained personalization on both users and items.
This work could be a fundamentalwork to pave the way for implementing a new service architecture with better privacy preservation,ﬁne-grained personalization, and on-device intelligence.
Given the complex nature of modern recommendation applications, such as cold-start problems, dynamics, using auxiliary information, andprocessing multi-modality contents, our proposed framework is simple and ﬂexible enough to beextended to tackle many new challenges.
Moreover, the proposed dual personalization is a simplebut-effective mechanism to be easily integrated with existing federated recommendation systems. A M ORE D ISCUSSION ABOUT D UAL P ERSONALIZATIONA. L EARNING E FFICIENCY OF P ERSONALIZED S CORE F UNCTION
Due to the design of dual personalization the user’s critical information is decomposed into personalized item embeddings and a personalized score function.
Therefore, in most scenarios, includinginactive users the score function module does not demand a large neural network.
The simple andswift multi-layer neural network is capable of tackling most scenarios.
For example in our implementation we use a one-layer MLP as the score function, which achieves signiﬁcant performanceimprovement again baselines.
Particularly, we conduct empirical experiments to analyze the model’seffectiveness on inactive users and details can be found in D.6.
Moreover, to further improve thelearning efﬁciency, we could pre-train a neural score model for each user group considering theirdemographic features.
A. E FFECTIVENESS OF O NE -S TEP F INE -T UNING
The one-step gradient ﬁne-tuning is necessary for our proposed PFedRec and our design is supported by the theoretical analysis in Pillutla et al.
A signiﬁcant consideration is that the
PFedRec alternately tunes one gradient step each for the global item embedding module and theclient-speciﬁc score function.
This learning process is consistent with the partial personalizationscheme studied in Pillutla et al.
(2022), where theoretical analysis shows that one gradient step issufﬁcient and the algorithm’s convergence is guaranteed when the following assumptions hold: (1)
The gradient of a local loss function is L-Lipschitz smooth.
The gradient of a local loss functionhas bounded variance.
(3) the variance of the gradient of local loss functions on different clients isbounded.
As the three assumptions are universal and hold for most functions, the conclusions from
Pillutla et al.
(2022) also hold in our proposed PFedRec, i.e., the one-step ﬁne-tuning is sufﬁcientand guaranteed to converge.
B K EY I SSUES D ISCUSSION IN F EDERATED R ECOMMENDATIONB. C OMMUNICATION EFFICIENCY
Due to the nature of federated learning, multiple rounds of parameter transfers are required betweenthe server and the clients to complete the training process, and communication efﬁciency is an important consideration in federated learning modeling.
For the FedRec model, there are three key factorsthat determine the communication cost, including the parameter volume M transmitted betweenserver and clients, the number of sampled clients S in each round, and the total communicationrounds T .
The overall communication cost can be formulated as,
[Equation]
Since S and T are constant among different models, the communication cost is positively correlated to M , i.e., C ∝ M .
Generally, item embedding is regarded as the shared component in
FedRec research.
For example, FedMF Chai et al.
(2020) and FedRecon Singhal et al.
(2021) onlytransmit item embedding whose M = |I| · d , where |I| is the number of items and d denotes theembedding size.
Besides, there are also some works that take the score function or user embedding as common components transmitted between server and clients together with item embedding.
FedNCF Perifanis & Efraimidis (2022) and MetaMF Lin et al.
(2020b) transmit both item embedding and score function whose M = |I| · d + |F| , where |F| is the parameter volume of scorefunction, such as a three-layer MLP.
FedGNN Wu et al.
(2021) also share user embedding whose
M = |I| · d + |F| + |U| · d , where |U| is the number of users.
In our method, we only shareitem embedding among users, which results in the minimal transmission parameter volume, i.e.,
M = |I| · d .
In our implementation, each client does not need to maintain the full item embedding table.
Sinceeach user only trains her local model with historical interactions and randomly sampled negativeinstances 2 , e.g., 5 times of historical interactions.
As a result, the number of items processed by theuser is much less than the complete item list, which further improves the communication efﬁciencyof the proposed method.
Take Lastfm-2K as an example, the size of the item set is 12,454, and theaverage size of historical interactions is 116, which is further less than transmitting the full item set.
Moreover, in this paper, we mainly focus on the fundamental recommendation task that only relieson the interaction matrix, and it is necessary to instantiate the item embedding module with the itemembedding table.
When implementing our framework under the scenario where item attributes areavailable, we can replace the item embedding table with a lightweight neural network similar tothe score function.
Then the local model on each device can be more lightweight, which results inless communication cost.
More empirical results about communication efﬁciency can be found in
Efﬁciency Comparison D.1, Convergence Analysis D.2 and Hyper-Parameter Study–Effect ofclient samples participating in each round D.3.
B. T IME C OMPLEXITY
We analyze the time complexity of PFedRec.
For clarity, we restate the notation that |I| · d is theitem embedding table, where |I| is the number of items and d is the latent embedding size; θ s is thescore function, and we instantiate it with a single-layer MLP whose size is d ; let e denotes the localtraining epochs, S is the sampled clients in each round and T is the total communication rounds.
Since e is usually a small constant, e.g., e = 1 in our implementation, then the time complexity of
PFedRec is O ( |I| d 2 ST )
C E XPERIMENTAL SETUP DETAILS
C. M ORE D ETAILS OF THE D ATASETS
We ﬁlter the users whose historic interactions are less than 5 and the characteristics of the fourdatasets used in experiments are shown in Table 3.
Since we focus on the implicit feedback recommendation scenario in this paper, we transform the ratings in each dataset into implicit data, that is,let 1 mark that the user has rated an item.
[Table]
C. E VALUATION P ROTOCOLS
For dataset split, we adopt prevalent leave-one-out evaluation, following He et al.
For eachuser, we take her last interaction as the test data and remain other interactions for training.
Additionally, we regard the latest raction in the training set as the validation data to ﬁnd hyper-parameters.
For the ﬁnal evaluation, we follow the regular strategy Koren (2008);
He et al.
(2017) that randomlysamples 99 unobserved items for each user, ranking the test item among the 100 items.
We evaluatethe ranked list with Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG) metrics.
In particular, HR measures whether the test data is in the top-K list, while NDCG considers the testdata’s position in the list.
The two evaluation metrics could be formulated as follows,
[Equation]
where N is the number of users and hits ( u ) =
1 indicates that the test data of user u is in the top-Krecommendation list, otherwise hits ( u ) = 0 .
[Equation]
where p u denotes the position of test data of user u in the recommendation list and p u → ∞ whenthe test data is not in the top-K recommendation list.
Here we set K = 10 for all experiments.
C. P ARAMETER C ONFIGURATION
In the implementation, we set the latent embedding size as 32 and the batchsize is ﬁxed as 256 for allbaselines and our method.
The total number of communication rounds is set to 100, and this valueenables all methods to be trained to converge through experiments.
For learning rate, we searchit in [0 . 0001; 0 . 0005; 0 . 001; 0 .
and the speciﬁc setting on four datasets aresummarized in Table 4.
For other empirical experiments of our method, we directly used the sameparameters without researching.
[Table]
D M ORE E XPERIMENTAL R ESULTSD. E FFICIENCY C OMPARISON
In federated learning, space and time efﬁciency are prominent factors for application.
We comparethe model’s efﬁciency, including parameter volume, training epochs and running time, as shown in
Table 5.
According to the results, (1) FedNCF has the largest volumes of parameters.
Parameters ofrecommendation systems consist of several parts, i.e., user and item embedding and score function.
The embedding size is the same for all methods in each dataset.
For score function, FedNCF employsa three-layers MLP, which leads to much more parameters than one-dimensional embedding in othermethods.
(2) FedRecon takes the most training epochs to converge.
The reconstruction mechanismin FedRecon demands retraining the local module from scratch in each round, which results inmore training epochs.
Taking MovieLens-100K as an example, our method converges in 61 trainingepochs, while FedRecon requires 465 training epochs, approaching 8 times ours.
FedNCF takes thesecond longest training epochs and training time.
On MovieLens-100K, the total training time of
FedNCF is about 2 times as long as ours.
(3) The space and time efﬁciency of FedMF is at thesame level as our method.
Our method enhances the personalization modeling and achieves the bestperformance without extra computational complexity.
In our experiments, we found that just onelocal gradient descent step to learn personalized item embedding can yield advanced performance.
[Table]
820 w/ ﬁnetune 86,  1, 314,   452,  4, 639,  4,120 FedRecon 53,  9, 118, 470 123,892 398,  7, 378,  3,819 w/ ﬁnetune 53,  9, 118, 476 125,230 398,  6, 378,  4,760 FedMF 53,   118,  12, 398,  3, 378,  3,920 PFedRec 53,   118,  13, 398,  2, 378,  5,
D. C ONVERGENCE A NALYSIS
We show the convergence curves of the two evaluation metrics for our method and baselines on fourdatasets.
As shown in Figure 4, our method achieves the fastest convergence on all datasets and metrics almost all the time, followed by FedMF and FedRecon.
FedNCF has the slowest convergencespeed because it has the most parameters and requires a longer number of iterations.

   
 1XPEHURI5RXQGV       
  + 5 #  D0RYLH/HQV.
)HG1&))HG5HFRQ)HG0)2XUV 
   
 1XPEHURI5RXQGV       
  + 5 #  E0RYLH/HQV )HG1&))HG5HFRQ)HG0)2XUV 
   
 1XPEHURI5RXQGV     
 
 + 5 #  F/DVWIP.
)HG1&))HG5HFRQ)HG0)2XUV 
   
 1XPEHURI5RXQGV        + 5 #  G
$PD]RQ )HG1&))HG5HFRQ)HG0)2XUV 
   
 1XPEHURI5RXQGV     
 1'& * #  HH
0RYLH/HQV 00K )HG1&))HG5HFRQ)HG0)2XUV 
         
 1'& * # 
1XPEHURI5RXQGV I
I0RYLH/HQV -1M )HG1&))HG5HFRQ)HG0)2XUV 
   
      
 
1'& * #  1XPEHURI5RXQGV JJ/DVWIP -2K )HG1&))HG5HFRQ)HG0)2XUV 
         
 1'& * # 
1XPEHURI5RXQGV KK )HG1&))HG5HFRQ)HG0)2XUV $PD]RQ
D. H YPER -P ARAMETERS S TUDY
Effect of latent embedding size We tune the latent embedding size from { 16 , 32 , 64 , 128 } .
Accordingly, the architecture of the score function, i.e., one-layer MLP, is 16 → 1 , 32 → 1 , 64 → 1and 128 → 1 , respectively.
Experimental results are shown in Table 6.
Almost all four datasetsachieve the best results when the latent embedding size is 32.
Due to the limited data of a singleuser, increasing the model parameters did not obtain further performance improvement.
Generally,setting the dimension to 16 or 32 can achieve advanced performance.
A small volume of parametershelps to build a lightweight on-device recommendation model.
[Table]
591 Effect of negative sample size We set the negative sample size from 1 to 10 and observe theeffect on model performance.
Experimental results are shown in Figure 5.
For two MovieLensdatasets, model performance improves signiﬁcantly as the number of negative samples increases.
Since these two datasets have more user data than Lastfm-2K and Amazon, they contain richer userpreference information to learn.
Sampling more negative instances help the model to further identifyuser preferences.
On the other two datasets, 4 negative samples are enough to obtain ideal modelperformance.
[Figure]
Effect of client samples participating in each round There is a trade-off between client sampling ratio and communication efﬁciency in federated optimization.
Generally, the more clients areselected to participate in the global aggregation, the faster the model converges in each round.
However, in the physical scenario, it is difﬁcult for the server to collect the complete model informationfrom all the clients.
Particularly, there are a large number of user clients in the recommendationscenario, which further increases the difﬁculty.
To verify the relationship between the model’s convergence and the clients’ participation in each round, we conduct experiments on four datasets withvarious client samples.
To create a consistent validation environment for all datasets, we set thenumber of users selected in each round as 100, 200, 300, 400 and 500, respectively.
Experimentalresults are represented in Figure 6.
We run the model until convergence and report the best validation performance with the corresponding epoch.
According to the experimental results, we can observe that PFedRec could reachconsistently advanced performance in all settings on all datasets, even only with 100 clients selectedin each round during model training.
On the other hand, it is obvious that more clients participating
     D0RYLH/HQV.       ( SR F K V    
 
  3 H UI R U P D Q FH +5#1'&*#      E0RYLH/HQV0      ( SR F K V    
 
  3 H UI R U P D Q FH +5#1'&*#      F/DVWIP.      ( SR F K V
  

 +5#1'&*#      G$PD]RQ            3 H UI R U P D Q FH +5#1'&*# 3 H UI R U P D Q FH ( SR F K V in each round of training lead to a quicker convergence.
PFedRec supports the server to update withinsufﬁcient clients accessible, which is ubiquitous in physical circumstances.
D. P ROTECTION WITH D IFFERENTIAL P RIVACY
In addition to data locality inherited from the FL framework, introducing privacy-preserving methods into our method can further enhance privacy protection in FedRec.
In our model, item embedding is the shared component in the federated optimization process.
There is the risk of user interaction information exposure when the client uploads the updated item embedding to the server.
Addingnoise to the item embedding is a general method to defend against the privacy leakage attack, suchas introducing differential privacy and homomorphic encryption into the model.
Moreover, designing a reasonable pseudo-interactions injection method is also a potential solution to further enhanceprivacy protection, which can be discussed in future work.
Here we integrate the local differential privacy technique Choi et al.
(2018) into our method as anexample.
Particularly, we add the zero-mean Laplacian noise to the client’s item embedding beforeuploading to the server,
[Equation]
where λ is the noise strength.
We set λ = [0 , 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5] to test our method’s performanceand the results are shown in Table 7.
We can see that the performance declines slightly as the noisestrength λ grows, while the performance drop is still acceptable.
For example, when we set λ = 0 .
4 ,the performance is also better than baselines in most cases.
Hence, a moderate noise strength isdesirable to achieve a good balance between recommendation accuracy and privacy protection.
[Table]
λ
[Table]
λ =
[Table]
λ =
[Table]
D. E VALUATION BASED ON F ULL R ANKING L IST
As illustrated in C.2, in the main experiments, we adopt the efﬁcient sampled metric.
To providea more comprehensive comparison, we conduct experiments to evaluate the test item in the fullranking list, which is more challenging than the sampled metric.
As shown in Table 8, our proposedmethod still outperforms other baselines in all benchmark datasets, which emphasizes its outstandingefﬁcacy.
[Table]
[Table]
ERFORMANCE ON
I NACTIVE U SERS
In the recommendation task, there are usually some users with fewer available interactions, and wecall them inactive users, which poses a great challenge for model training.
Here we show the modelperformance on inactive users to verify the effectiveness of our method.
Particularly, we count thenumber of ratings from all users, then extract inactive users and calculate their performance.
Theinactive users’ statistics of each dataset are summarized in Table 9.
We omit the Amazon-Video dueto its high sparsity, where the average size of historical interactions for each user is 8.
[Table]
The comparison results of performance on inactive and full users are shown in Table 10.
Our methodachieves similar or even better performance on the inactive users set than the full users set, indicatingthat our method can effectively tackle inactive users.
[Table]