Cluster-driven Personalized Federated Recommendation with Interest-aware Graph Convolution Network for Multimedia

Authors: xingyuan  mao
yuwen  liu
lianyong  qi
li  duan
xiaolong  xu
xuyun  zhang
wanchun  dou
amin  beheshti
xiaokang  zhou
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-03 11:17:56

Abstract: Federated learning addresses privacy concerns in multimedia recommender systems by enabling collaborative model training without exchanging raw data. However, existing federated recommendation models are mainly based on basic backbones like Matrix Factorization (MF), which are inadequate to capture complex implicit interactions between users and multimedia content. Graph Convolutional Networks (GCNs) offer a promising method by utilizing the information from high-order neighbors, but face challenges in federated settings due to problems such as over-smoothing, data heterogeneity, and elevated communication expenses. To resolve these problems, we propose a Cluster-driven Personalized Federated Recommender System with Interest-aware Graph Convolution Network (CPF-GCN) for multimedia recommendation. CPF-GCN comprises a local interest-aware GCN module that optimizes node representations through subgraph-enhanced adaptive graph convolution operations, mitigating the over-smoothing problem by adaptively extracting information from layers and selectively utilizing high-order connectivity based on user interests. Simultaneously, a cluster-driven aggregation approach at the server significantly reduces communication costs by selectively aggregating models from clusters. The aggregation produces a global model and cluster-level models, combining them with the user's local model allows us to tailor the recommendation model for the user, achieving personalized recommendations. Moreover, we propose an adversarial optimization technique to further augment the robustness of CPF-GCN. Experiments on three datasets demonstrate that CPF-GCN significantly outperforms the state-of-the-art models.

Main paper: 
 introduction: 
 With the explosive growth of multimedia content in recent years, navigating the vast ocean of available videos, images, and music has become increasingly challenging. Recommender systems have risen to this challenge [12] , helping users discover their interested content across various domains [19, 20, 31] . As deep learning technology has become more widely applied [23, 34] , it has further enhanced the capabilities of these systems. However, recommender systems typically process data and train models on central servers, requiring the collection of users' personal data. This exposes users to significant privacy risks. Federated recommender systems have emerged to solve this problem by allowing participants to collaboratively train the model through the exchange of intermediate parameters instead of raw data [17] , thereby preserving privacy.The backbones of existing federated recommender systems are mostly basic, such as Matrix Factorization (MF) [4] and Neural Collaborative Filtering (NCF) [30] . In these methods, updates are primarily according to explicit user-item interactions, overlooking the complex implicit interactions represented by a bipartite graph of users and items. Graph Neural Networks (GNNs) introduce a novel approach to leverage this graph structure [1] . As a prominent GNN variant, Graph Convolutional Networks (GCNs) can effectively gather collaborative information from high-order connections via the propagation of embeddings across the graph [11] . In a federated setting, each client keeps a private graph that is not shared with the server or other users, ensuring privacy protection. Some GCN-based federated recommendation models have been introduced recently [24, 26] , where users enhance their local graphs to access high-order neighbor information by uploading encrypted embeddings to the server. However, these frameworks primarily focus on low-order connectivity, limiting their ability to use high-order information as effectively as centralized models. Besides, GCN-based models often encounter the over-smoothing issue, where multiple graph convolution layers cause node embeddings to grow overly similar [18] , thereby reducing model effectiveness.In addition to the issues mentioned above, federated recommender systems also face problems of heterogeneity and high communication costs. Heterogeneity arises because the data owned by various clients is rarely independent and identically distributed (IID) [16] , complicating the task of generating only one unified model to effectively serve all users and achieve personalized recommendations, as described in the top left part of Fig. 1 . Furthermore, achieving optimal performance in federated recommender systems needs frequent exchanges between the server and all clients. Given that modern recommender systems often rely on complex deep learning architectures with millions of parameters, this results in significant communication burdens.Motivated by the considerations above, we propose a Clusterdriven Personalized Federated Recommender System with Interestaware Graph Convolution Network (CPF-GCN). CPF-GCN comprises two key components: a local interest-aware recommendation module and a cluster-driven global server. For the local client, the interest-aware GCN module divides users into different subgraphs based on their interests, then applies high-order graph convolution operations within these subgraphs. Thus, the learning of a user's embedding can only be facilitated by neighbors who share interests, avoiding the propagation of irrelevant negative noise. For the graph convolution on the subgraphs, we refer to and improve upon the simplified graph convolution proposed in LightGCN [9] , introducing an adaptive graph convolution method. This method can adaptively extract information from the ego layer and learn node representations for the next layer, avoiding the inclusion of excessive high-order information and further alleviating the oversmoothing problem. Moreover, we employ an adversarial optimization technique to enhance the robustness of our GCN model. For the global server, we propose a cluster-driven federated learning framework. The server first clusters users based on their representations. Unlike traditional federated methods that select all clients or randomly choose some clients for aggregation, our method adaptively selects a certain number of clients from each cluster based on its size, ensuring that the selected clients are representative. In this way, only a relatively small number of clients need to be selected to achieve good performance, reducing communication costs. Subsequently, our model aggregates the chosen clients, generating a global model and several cluster-level models. To safeguard user privacy, a Local Differential Privacy (LDP) technique is used during this procedure. Through the integration of the generated global model, cluster-specific model, and previously obtained local model, a personalized recommendation model can be produced for each user, achieving personalized recommendations while protecting user privacy. As illustrated in the top right part of Fig. 1 , users upload their distinct multimedia preferences to the server, then the server tailors a personalized model for each user.To summarize, our key contributions are outlined as follows:• We propose a subgraph-enhanced adaptive graph convolution method for the local GCN model to mitigate the issue of over-smoothing by generating subgraphs from the user-item bipartite graph, and adaptively learning node representations by weighing similarities between hidden and ego layers.• We design a cluster-driven federated framework that selects representatives from each user cluster for model aggregation and generates cluster-level and global models for personalized recommendation, addressing heterogeneity while boosting efficiency.• We integrate the concept of adversarial training into model optimization by designing an adversarial loss, thereby enhancing the model's robustness.• Extensive experiments on three multimedia datasets indicate that our proposed CPF-GCN model significantly outperforms existing federated recommendation models. 
related work 2.1 gnns for recommendation: 
 Graph Neural Networks (GNNs) [33] emerge as an effective tool in recommender systems that exploit the interactive information in user-item interaction graphs. Early adaptations of GNNs in recommender systems focused on learning embeddings by capturing the structure of the user-item bipartite graph [6, 15, 21] . For example, [15] designs a removal algorithm to filter redundant information while utilizing GNNs to mine information from high-order connectivity, enhancing the model's generalization capabilities. In addition, models like Graph Convolutional Networks (GCNs) [14] have laid the foundation for subsequent advances in this field. [22] highlights the dynamic nature of both users and points-of-interest, proposing a time-sensitive and interaction-augmented GCN for continuous point-of-interest recommendations. Moreover, over-smoothing is a notorious issue with GNN-based methods, leading to advances that modify or simplify GCN components [9, 18, 27] . LightGCN [9] is an important milestone that simplifies GCNs by eliminating feature transformations and nonlinearities. [18] further proposes IMP-GCN, which addresses the over-smoothing problem by ensuring that high-order graph convolution only involves users with similar interests. [40] designs a layer-refined GCN known as Lay-erGCN, which reduces the noise introduced by external factors by refining layer representations during each propagation. These works enhance information utilization, dynamic preference learning, and training efficiency. Despite these advancements, privacy protection in recommender systems remains underexplored.Federated Learning (FL) trains models without exchanging raw data between decentralized devices or servers, keeping user data private. Therefore, FL can be well applied in recommender systems and achieve practical privacy protection [2, 13, 28] . [28] introduces the Federated Averaging algorithm, marking the beginning of its application in various domains, including recommender systems.[2] first combines FL with collaborative filtering for personalized recommendations using implicit user feedback. To address privacy leakage during training and gradient transmission, methods like perturbing data and encrypting gradients are used in some works. [13] randomly selects some unrated items for each user and then assigns some dummy ratings so that the server cannot easily identify the set of scores and rated items during server-client interactions. FedMF [4] uses homomorphic encryption to encrypt item gradients before transferring them to the server. Recently, many advances have begun to combine FL with GNNs to protect users' privacy while capturing higher-order information. FGC [38] is a federated method based on GCNs, using the potential overlap of services across different clients to guide embedding aggregation and sharing, optimizing the local training results. PerFedRec [25] clusters users into different groups, enabling personalized model learning for each cluster. It integrates LightGCN with a FL framework, achieving personalized federated recommendations under privacy protection. However, these methods merely use basic GCN models, which often suffer from the over-smoothing problem, into FL without unique customization for the federated setting. 
problem definition: 
 We denote U = {𝑢 1 , 𝑢 In federated settings, historical interactions between users and items are stored on individual user devices rather than centrally on a server. Regarding the graph structure, the user-item bipartite graph G adopts a decentralized approach where every user maintains a private graph, which includes their own interactions with items. However, interactions from a single user alone are insufficient to train the model. To build a graph with multiple user nodes and their interacted items for training the local GCN model, we adopt the strategy employed in [36] . Each user uploads their privacypreserved embedding along with encrypted item IDs to the server. The server then returns the encrypted user embeddings and item IDs to all users. This allows each user to receive multiple users' embeddings without disclosing identities, enabling them to access neighboring users and expand their local interaction graph G. 
methodology: 
 The framework of the proposed CPF-GCN model is illustrated in Fig. 2 . CPF-GCN consists of a local interest-aware recommendation module and a cluster-driven global server. The local GCN model can allocate users into different subgraphs and perform adaptive graph convolution. The server can cluster users and implement aggregation based on these clusters to update the model. They work collaboratively to achieve personalized recommendations.For the local client, we propose an interest-aware GCN module that performs subgraph-enhanced adaptive graph convolutions based on user interests. Additionally, we introduce adversarial optimization to further improve the performance and robustness of our module.4.1.1 Subgraph Extraction. This section details the generation of subgraph 𝐺 𝑠 from the user-item interaction graph 𝐺, aiming to group users by interests. User grouping operates as an unsupervised classification task, with users being characterized through feature vectors and allocated to clusters. Specifically, each user's feature vector combines information from the graph structure and their ID embedding. We refer to this process as feature fusion:EQUATION𝑢 + e(1) 𝑢EQUATIONLocal Client where F 𝑢 represents the user feature obtained through feature fusion. The user embedding of the ego layer is denoted by e (0)𝑢 denotes the user embedding following the first layer of propagation within full graph 𝐺. The weight matrix and bias vector for the fusion method are respectively symbolized by 𝑾 1 ∈ R 𝑑 ×𝑑 and 𝒃 1 ∈ R 1×𝑑 , where 𝑑 indicates the size of the embedding. The activation function 𝜎 is chosen to be LeakyReLU, given its efficacy in processing either positive or slightly negative information. To assign users to distinct subgraphs, F 𝑢 is further transformed through a two-layer neural network as follows:EQUATIONwhere P 𝑢 is the prediction vector indicating the user's group assignment based on the highest value's position within the vector. W 2 ∈ R 𝑑 ×𝑑 and W 3 ∈ R 𝑑 × N 𝑠 are weight matrices for the two layers of the neural network, while b 2 ∈ R 1×𝑑 and b 3 ∈ R 1×N 𝑠 represent the bias vectors for these layers. Users with similar embeddings will produce similar prediction vectors via Eq. ( 2), thereby being classified into the same group. The goal of creating subgraphs is to form a matrix that maps user-item relationships within each subgraph. For the matrix of each subgraph, user-item adjacencies from the original graph are excluded if the corresponding user does not belong to the group, based on the group information obtained. EQUATION)where T is the transition matrix, formulated as T = D -1/2 AD -1/2 . Here, A represents the previously defined adjacency matrix, and D represents the corresponding diagonal matrix where each element D 𝑖𝑖 indicates the count of non-zero entries within the 𝑖-th row of A. This equation signifies that the node embeddings for the 𝑙-th layer are obtained through the aggregation of data from the (𝑙 -1)-th layer with the transition matrix T.However, aggregating features iteratively from higher-order neighbors can cause the over-smoothing problem and a dilution of unique node features. This process overlooks the variety of higherorder features and allows noise from neighbors to degrade the learning of embeddings, consequently diminishing model effectiveness. To address this problem, our proposed model groups users into subgraphs and performs the high-order propagation within these subgraphs. Similarly, the convolution process within a subgraph employs a symmetric transition matrix T𝑠 = D-1/2 𝑠 A 𝑠 D -1/2 𝑠 .The adjacency matrix A 𝑠 for the subgraph 𝐺 𝑠 is derived from the interaction matrix R 𝑠 ∈ R N 𝑠𝑢 × N 𝑠𝑖 , with N 𝑠𝑢 and N 𝑠𝑖 representing the counts of users and items in the subgraph 𝐺 𝑠 , respectively. Specifically, A 𝑠 is structured as follows:EQUATIONGiven that the immediate connections linking users to items significantly reflect user preferences, in our model, the first-order graph convolution operation incorporates all first-order neighbors. In other words, the first layer embedding propagation acts within the full graph 𝐺, described by the following formulation:EQUATION)For the high-order graph convolution, we perform each convolution operation within subgraphs, and we design an adaptive graph convolution mechanism that adaptively extracts the information from the ego layer to mitigate the introduction of extraneous noise. In this context, a node within a subgraph is restricted to utilizing data solely from its neighbor nodes within the same subgraph. This ensures that users maintain access to information from all interconnected items, as these items are part of the user's subgraph, while keeping the noise from the high-order neighbors out. We formulate the high-order graph convolution within subgraph 𝐺 𝑠 as:EQUATIONwhere 𝑙 ⩾ 2, E (𝑙 ) 𝑠 represents the embedding of the nodes in subgraph 𝐺 𝑠 after the 𝑙-th convolution layer. The similarity vector m (𝑙 ) ∈ R N 𝑠 captures node similarity between the current layer and the ego layer, aiming to preserve crucial features while adaptively extract information from the ego layer. The small quantity 𝜉 serves to ensure there is no zero vector in E (𝑙 ) 𝑠 . The similarity measurement m (𝑙 ) is determined by:EQUATION)In this context, cosine similarity serves as the sim function which calculates the similarity between two vectors e 𝑖 = E where 𝜖 represents a tiny constant introduced to prevent division by zero. The adaptation of embeddings enhances the integration of hidden layers akin to the ego layer while diminishing the impact of those layers deviating from it, preserving the uniqueness of each node's embedding so that they do not become increasingly similar to other nodes. Thus further alleviating the over-smoothing issue.After getting node embeddings at the 𝑙-th layer for each subgraph, the ultimate node embeddings for the 𝑙-th layer are constructed by combining all embeddings for the 𝑙-th layer from various subgraphs as follows:EQUATIONTo obtain the final embeddings for users and items, we combine embeddings from all layers, similar to the approach used in LightGCN, which can be expressed by the following equation:EQUATIONwhere 𝛼 𝑙 ≥ 0 denotes the weight given to the 𝑙-th layer. In line with LightGCN, this weight is uniformly set to 1 𝐿+1 , ensuring an equal contribution from each layer to the final embedding.In our system, each user benefits from a uniquely tailored recommendation model 𝜃 𝑡 𝑛 to achieve personalized multimedia recommendations. Details on formulating 𝜃 𝑡 𝑛 will be provided in Section 4.2.2. Using this personalized model, we can acquire the final representations for users and items. Then, to get personalized predictions, we dot product the final embeddings for each user and item:EQUATIONwhere e 𝑢 ∈ R 𝑑 and e 𝑖 ∈ R 𝑑 are the final embeddings of user 𝑢 and item 𝑖, respectively. After getting the prediction, we use an adversarial method to optimize our model. One commonly used loss function in recommender systems is the pairwise Bayesian personalized ranking (BPR) loss [32] , designed to promote the model to assign higher scores to observed interactions than those unobserved. The BPR loss can be defined by:EQUATIONwhere 𝜃 represents the model parameters. 𝜎 is the sigmoid function, and I + 𝑢 denotes the items with which 𝑢 has interacted before. However, BPR lacks robustness against small data perturbations, leading to potentially poorer generalization on unseen data [10] . To address this issue, we design a loss function that introduces adversarial perturbations to the model parameters during training. We formulate the adversarial loss as follows:EQUATION)where 𝜆 𝐿 2 controls the 𝐿 2 regularization strength to prevent overfitting and 𝜆 𝑎𝑑𝑣 controls the strength of the adversarial component.Δ denotes the perturbations on model parameters, specifically can be formulated as follows:EQUATION)where 𝜆 𝛿 is a hyper-parameter controlling the magnitude of the perturbations and ∇ 𝜃 L 𝐵𝑃𝑅 (𝜃 ) represents the gradient of the BPR loss with respect to the model parameters 𝜃 . The denominator here normalizes the gradient to ensure that the perturbation is solely directional, keeping its magnitude controlled by 𝜖.The training process of our proposed adversarial loss can be seen as engaging in a minimax game:EQUATIONwhere the optimization of model parameters 𝜃 serves as the minimizer, while the perturbations Δ act as the maximizer, seeking to find the most challenging perturbations against the current model. This game iteratively proceeds until achieving convergence. By compelling the model to learn more robust and distinctive features, our proposed adversarial optimization method can not only bolster the model's resilience to slight variations in data but also enhance its generalization ability.For the global server, we propose a cluster-driven federated learning framework. This framework performs privacy-preserving model and embedding aggregation within different scopes to generate global and cluster-level models, which are used for creating the previously mentioned personalized recommendation model.User Clustering and Sampling. Similar to the earlier process of extracting local subgraphs, the global server clusters users into different classes based on their embeddings e 𝑢,𝑛 , with user 𝑛 being assigned to cluster 𝐶 (𝑛). Standard clustering techniques like K-means are suitable here. The node representation e 𝑢,𝑛 benefits from a combination of characteristic and joint features unique to every user, thereby enriching the representation.In federated recommender systems, while conventional client selection techniques randomly pick clients from the complete client set, we propose a novel approach that is particularly beneficial under conditions where communication costs are high. Specifically, we introduce a cluster-based client sampling method that adaptively selects a handful of random users from each cluster, relative to the size of the cluster, for participation in the model aggregation. By choosing clients that accurately represent their clusters, our federated model can operate more effectively and efficiently.(LDP) as a key privacy-preserving mechanism in our system. LDP, an extension of differential privacy, ensures data confidentiality by encrypting uploaded gradients. This encryption involves clipping gradients whose 𝐿 ∞ -norm exceeds a certain threshold 𝛿 and adding zero-mean Laplacian noise to protect privacy. For each client's gradient 𝑔, we modify it through the equation:EQUATIONwhere 𝑔 ′ represents encrypted gradient, 𝑐𝑢𝑡 (•, 𝛿) refers to the cutoff operation at threshold 𝛿, and 𝐿𝑎𝑝 (0, 𝜆) adds Laplacian noise characterized by a scale parameter 𝜆.Using the encrypted gradient, our framework executes aggregation for model parameters and embeddings. During aggregation at epoch 𝑡, the server creates a global model 𝜃 𝑡 𝑔𝑙𝑜𝑏𝑎𝑙 for all users, as well as cluster-level models 𝜃 𝑡 𝐶 (𝑛) , with 𝐶 (𝑛) signifying the cluster that includes user 𝑛. These models are formed using a weighted EQUATION)where 𝜂 represents the learning rate, and ḡ is the aggregated gradient for all selected clients S, calculated as a weighted sum of the encrypted gradients 𝑔 ′ 𝑛 :EQUATIONwhere 𝐷 𝑛 indicates the local data amount of client 𝑛. Similarly, the update of the cluster-level model 𝜃 𝑡 𝐶 (𝑛) are formulated as:EQUATIONwhere ḡ𝐶 (𝑛) is the aggregated gradient for the cluster 𝐶 (𝑛), calculated as a weighted sum of the encrypted gradients 𝑔 ′ 𝑛 from the selected clients within the cluster.In the following step, we tailor a personalized recommendation model for each user as described in Section 4. 𝑛 for user 𝑛 using the formula:EQUATIONwhere 𝛽 𝑛,1 , 𝛽 𝑛,2 , 𝛽 𝑛,3 are the weights that balance the contributions of the local, cluster-level, and global models, respectively. These coefficients can be either fixed or learnable parameters. 
experiment 5.1 experimental setup: 
 5.1.1 Datasets. We use three datasets enriched with multimedia information, including Foursquare [37] , a dataset capturing user check-ins; Lastfm-2K [3] , a collection of music listening histories from the Last.fm platform; and MovieLens-100K [8] , a dataset of movie ratings by users. All these datasets are publicly available and commonly used to evaluate multimedia recommender system models, the key characteristics of which are shown in Table 1 .5.1.2 Baseline Methods. We conduct comparisons between our CPF-GCN and several representative or cutting-edge models to prove its efficacy. These methods include centralized approaches as well as federated methods, specifically as follows:The centralized models:• BPR [32] : It is a typical matrix factorization algorithm optimized through Bayesian personalized ranking (BPR) loss. The federated models:• FedMF [4] : This model uses federated matrix factorization and employs stochastic gradient descent while safeguarding gradients through a secure aggregation method. • FedNCF [30] : This is a federated recommendation model built upon the neural collaborative filtering (NCF) framework. It can train NCF models while preserving user privacy. • FedGNN [36] : It utilizes the GNN for privacy-preserving federated recommendation, using LightGCN as the backbone model for local clients.• PerFedRec [25] : It is a federated recommendation model enhanced with a joint representation learning technique to achieve personalized recommendations. • FedIS [5] : It is an efficient federated recommendation model that can reduce user feature dependencies through the training of a global item-based collaborative filtering model. • PFedRec [39] : It proposes a dual personalization approach to achieve detailed customization for both users and items, allowing slight adjustments to item embeddings.We apply the commonly used leaveone-out method for evaluation [35] . We sort the users' behaviors by time. For each user in the dataset, their most recent interaction is designated for testing, while all prior interactions serve as training data. Following previous settings [25, 29] , we select 100 uninteracted items randomly and place these items in the test set, then rank the test item within this set. The evaluation protocols we used are Recall@𝑘 and NDCG@𝑘, where 𝑘 is chosen to be 5 or 10, using the top-5 and top-10 results to compute the performance.Hyper-parameter Settings. Our CPF-GCN model is implemented in PyTorch. The number of training epochs is set to 500 with an early stopping strategy. The embedding size 𝑑 is fixed to 64Table 3 : Ablation study on three datasets. R@5 and N@5 refer to Recall@5 and NDCG@5, respectively.Model Foursquare Lastfm MovieLens R@5 N@5 R@5 N@5 R@5 N@5 for both users and items, and the embeddings are initialized using the Xavier method [7] . The learning rate is initially configured at 0.01, and we apply a dropout ratio of 0.3. For the adversarial settings, the coefficients 𝜆 𝑎𝑑𝑣 and 𝜖 are both set to 0.1 to control the strength of adversarial perturbations. There are 128 users that take part in each training round, and we defaultly divide these users into 3 clusters. The parameters 𝛽 𝑛,1 , 𝛽 𝑛,2 , 𝛽 𝑛,3 , which are used in generating personalized recommendation model, are all fixed to 1 3 .The performance comparison outcomes are shown in Table 2 . The results show that compared with the baseline models, our proposed model consistently provides better performance. The most significant improvement is seen in the MovieLens dataset, since each user in this dataset has interacted with a large number of items, which helps us tailor personalized recommendation models for them. Specifically, we can find that CPF-GCN achieves a significant lead over other models in terms of NDCG, indicating that our personalized model excels in capturing relevant items and optimizing their ranking order. Although the lead in recall is not as high as that in NDCG, there is still a noticeable improvement. Additionally, for different top-k values, our model shows a more significant lead at 𝑘 = 5 compared to 𝑘 = 10. This suggests that our model is particularly effective at ranking the most relevant items at the top of the recommendation list, allowing us to recommend the multimedia content that users are most interested in.To prove the impact of some important modules of CPF-GCN, we perform ablation experiments on three datasets. We implement three variants of our model: CPF-GCN-s replaces the subgraphenhanced adaptive graph convolution method with the primary simplified graph convolution introduced in LightGCN. CPF-GCNp modifies CPF-GCN by removing the user clustering step and personalized recommendation module, thereby creating a unified model for all users. CPF-GCN-a removes the adversarial optimization method, using the BPR loos function instead. Table 3 displays the outcomes of ablation experiments in terms of Recall@5 and NDCG@5. We can find that the full CPF-GCN model outperforms all its variants. Specifically, the CPF-GCN-p model shows the poorest performance, proving that our proposed methods can effectively customize personalized recommendation models for users, achieving more accurate recommendations. The performance also declines in CPF-GCN-s, demonstrating the superiority of our proposed local interest-aware GCN model over existing GCNs. The performance of CPF-GCN-a also decreases compared to the full CPF-GCN, indicating that adversarial optimization contributes to the improvement of model robustness and generalization performance.In this section, we explore how some key hyper-parameters affect the performance of the CPF-GCN. These parameters include the number of layers and subgraphs, the coefficient 𝜆 𝑎𝑑𝑣 , which controls the strength of the adversarial loss, and the number of user clusters.Effect of Number of Layers 𝐿 and Subgraphs N 𝑠 . We implement CPF-GCN with varying numbers of layers 𝐿 and subgraphs N 𝑠 , as shown in Fig. 3 . From the perspective of 𝐿, CPF-GCN performs best with 𝐿 = 2 or 𝐿 = 3 on two datasets. As we stack more than three layers, the performance of models with N 𝑠 = 1, i.e., only one graph, significantly declines, indicating the issue of over-smoothing. In contrast, models with multiple subgraphs show more stable performance, suggesting our method can effectively mitigate the over-smoothing problem. Regarding N 𝑠 , models with N 𝑠 = 2 and N 𝑠 = 3 perform better, while the N 𝑠 = 4 model, despite being the most stable with increased layers, also cuts off many important connections between short-distance nodes. As a result, its performance after stacking more layers is even worse than that of models with fewer layers and subgraphs, and additionally leads to high communication costs due to the larger number of subgraphs.Effect of Adversarial Coefficient 𝜆 𝑎𝑑𝑣 . 𝜆 𝑎𝑑𝑣 is a coefficient that controls the strength of the adversarial loss. The performance of CPF-GCN with different 𝜆 𝑎𝑑𝑣 is shown in Fig. 4 . From the results, we can see that the two datasets have a significant difference in sensitivity to 𝜆 𝑎𝑑𝑣 . For the Foursquare dataset, CPF-GCN performs best when 𝜆 𝑎𝑑𝑣 = 10. In contrast, for the MovieLens dataset,  
conclusion: 
 In this paper, we introduce a GCN-based personalized federated multimedia recommendation framework named CPF-GCN, which addresses the challenges of over-smoothing, data heterogeneity, and high communication costs associated with existing GCN-based federated recommender systems. By integrating the local interestaware GCN module with the global cluster-driven federated framework, CPF-GCN can effectively capture user preference, achieving personalized multimedia recommendations while preserving user privacy. Extensive experiments on three multimedia datasets demonstrate that CPF-GCN significantly outperforms state-of-theart federated recommendation models. 
 
 Appendix:  


Decision: The motivation is clear, and a comprehensive and innovative framework is developed with subcomponents (subgraph extraction, personalized recommendation, and advisory optimization) designed to address the issues. High-order information exploitation by integrating GCN architecture also provides insights of user-content interactions. In general, this paper is easy to follow and interesting.

Reviews: review: paper_summary: paper_summary: The paper introduces a novel multimedia recommender system called Cluster-driven Personalized Federated Recommender System with Interest-aware Graph Convolution Network (CPF-GCN), which leverages federated learning to address privacy concerns while enhancing recommendation accuracy. By utilizing Graph Convolutional Networks (GCNs), CPF-GCN aims to capture complex implicit interactions between users and multimedia content without the need to share raw data. Traditional models like Matrix Factorization (MF) are limited in their ability to model these interactions, which CPF-GCN overcomes by employing a local interest-aware GCN module. This module optimizes node representations through subgraph-enhanced adaptive graph convolution operations, effectively mitigating issues such as over-smoothing by selectively utilizing high-order connectivity based on user interests.
strength_weakness: strength_weakness: The integration of GCNs allows the system to exploit high-order neighbor information, providing a deeper understanding of user-content interactions.

The model adapts to individual user preferences by incorporating local user data into the global learning process, leading to more accurate and personalized recommendations.
limitations: limitations: The experiments are conducted on a relatively small dataset, which may not convincingly demonstrate the model's effectiveness across diverse and larger multimedia environments.

Despite focusing on multimedia recommendation, the model does not explicitly incorporate multimedia-specific information, which could limit its ability to fully understand and utilize the content features.
paper_summary: paper_summary: The paper introduces CPF-GCN, a federated learning framework for multimedia recommendation that addresses privacy concerns by training models collaboratively without sharing raw data. CPF-GCN utilizes an interest-aware graph convolutional network (GCN) module to capture complex user-item interactions and reduce over-smoothing by adaptively extracting information from user subgraphs based on interests. Additionally, it employs a cluster-driven aggregation approach on the server to decrease communication costs and generate personalized recommendations by combining global, cluster-level, and local user models. Adversarial optimization further enhances the robustness of the model. Experiments on three multimedia datasets show CPF-GCN significantly outperforms state-of-the-art federated recommendation models, achieving personalized and privacy-preserving recommendations.
strength_weakness: strength_weakness: 1. The motivations are clear. This work aims to address the issues, such as over-smoothing (the perspective of GNN), data heterogeneity (the perspective of FL), and elevated communication expenses (the perspective of FL)
2. The proposed framework is comprehensive. Several subcomponents are designed to address these issues.
3. The presentation is good. The organization facilitates readability. Figures 1,2 describe the settings and proposed frameworks.
limitations: limitations: 1. The issues that motivate this work are general in the field of GNN or FL. In the multimedia settings, what more the particular or new aspects of these issues?
2. Some details should be clarified. In Section 4.1.1 Subgraph Extraction, how to optimize this unsupervised classification task? Please provide the loss function or optimization objective.
paper_summary: paper_summary: Graph Convolutional Networks (GCNs) face challenges in federated settings due to problems such as over-smoothing, data heterogeneity, and elevated communication expenses. This paper proposes a Cluster-driven Personalized Federated Recommender System with Interest-aware Graph Convolution Network (CPF-GCN) for multimedia recommendation. CPF-GCN comprises a local interest-aware GCN module that optimizes node representations through subgraph-enhanced adaptive graph convolution operations, mitigating the over-smoothing problem by adaptively extracting information from layers and selectively utilizing high order connectivity based on user interests. Simultaneously, a cluster-driven aggregation approach at the server achieves personalized recommendations by selectively aggregating models from clusters. Experiments on three multimedia datasets demonstrate that CPF-GCN significantly outperforms the state-of-the-art models.
strength_weakness: strength_weakness: (1) Federated recommendation represents a promising research direction. Its design, particularly in addressing issues of heterogeneity and high communication costs, stands out as the crucial challenges to tackle.

(2) This paper introduces an innovative approach to address the challenge of over-smoothing in local Graph Convolutional Network (GCN) models, a crucial issue in the field of graph learning.

(3) The experiments of this paper seem to be very solid and convincing.
limitations: limitations: (1) Using a cluster-driven aggregation approach to achieve personalization is not so novel. It is essential to note that many previous researches have explored clustering methods for personalized federated learning. Regrettably, the paper lacks thorough elucidation regarding the distinctions between the clustering method proposed herein and those from prior works, as well as fails to highlight the unique advantages of clustering offered by this paper.

(2) Some parts of the paper are pretty vague. For example, (1) Section 3 is not doing a good job of explaining federated recommendations. The relationship between the clients in a FL system and the user nodes in the graph remains ambiguous. The paper appears to suggest the existence of a graph $G$ on each client, housing multiple user nodes. However, this premise contradicts real-world scenarios. For example, a user's phone (i.e. client) typically only accesses one user's account and cannot aggregate information from other users. (2) The paper lacks clarity regarding the combination of embeddings between subgraphs when different subgraphs contain distinct users and items in Section 4.1.2.

(3) The paper would benefit from elucidating the intrinsic connection and components interaction between Subgraph Extraction, Personalized Recommendation, and Advisory Optimization. At present, these three components appear as a stack of seemingly unrelated technologies.

(4) The adversarial optimization method proposed in Section 4.1.4 bears striking similarities to the technology outlined in "BPR: Bayesian personalized ranking from implicit feedback." It would be beneficial for you to elucidate the distinct innovative features of your method.