Self-derived Knowledge Graph Contrastive Learning for Recommendation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-09 12:53:08

Abstract: Knowledge Graphs (KGs) serve as valuable auxiliary information to improve the accuracy of recommendation systems. Previous methods have leveraged the knowledge graph to enhance item representation and thus achieve excellent performance. However, these approaches heavily rely on high-quality knowledge graphs and learn enhanced representations with the assistance of carefully designed triplets. Furthermore, the emergence of knowledge graphs has led to models that ignore the inherent relationships between items and entities. To address these challenges, we propose a Self-Derived Knowledge Graph Contrastive Learning framework (CL-SDKG) to enhance recommendation systems. Specifically, we employ the variational graph reconstruction technique to estimate the Gaussian distribution of user-item nodes corresponding to the graph neural network aggregation layer. This process generates multiple KGs, referred to as self-derived KGs. The self-derived KG acquires more robust perceptual representations through the consistency of the estimated structure. Besides, the self-derived KG allows models to focus on user-item interactions and reduce the negative impact of miscellaneous dependencies introduced by conventional KGs. Finally, we apply contrastive learning to the self-derived KG to further improve the robustness of CL-SDKG through the traditional KG contrast-enhanced process. We conducted comprehensive experiments on three public datasets, and the results demonstrate that our CL-SDKG outperforms state-of-the-art baselines.

Main paper: 
 introduction: 
 Recommendation systems [16, 21, 50] play a crucial role in helping online users discover relevant information by suggesting items that align with their interests. Considering the interaction between users and items as first-order neighbors of users in the user-item graph neural networks (GNNs) [18, 24, 27, 58] have become widely employed in the field of recommendation. Numerous studies have demonstrated that GNNs can capture high-order implicit connectivity in user-item interaction graphs, thereby improving the performance of collaborative filtering [31, 42, 47] . However, when dealing with a vast user-item interaction diagram [13, 35] , unrelated items may obscure the user's genuine interests. In an attempt to mitigate this selection error, some researchers often utilize message drop techniques [2, 5, 7] , randomly discarding outbound messages based on the user-item diagram. Nevertheless, this approach can lead to a sparser user-item interaction graph in extremely high sparsity recommendation scenarios [15, 17, 23, 53] , resulting in diminished robustness of GNNs.Subsequently, contrastive learning (CL) [19, 54, 55] offers a selfsupervised manner to learn user and item embeddings, demonstrating its effectiveness in enhancing the robustness of GNNs. Specifically, for each node in the user-item graph, its subgraph undergoes random perturbation, involving the random discarding of a subset of its subgraph edges. Notably, techniques like SGL [38] and SLRec [46] leverage contrastive self-supervised tasks to improve node representations in GNNs through self-discrimination [39, 40, 49] . Then VGCL [45] samples the input graph to reconstruct it and performs contrastive learning at both node and cluster levels. Finally, CGCL [9] takes similar semantic structure representations to design contrastive learning at three levels: neighbor structure, candidate structure, and candidate neighbor, resulting in high-quality node representations. However, in the process of generating contrast subgraphs, the above methods disrupt the original graph structure, potentially erasing valuable information and consequently diminishing the representational power of nodes.Recently, researchers have turned to Knowledge Graph (KG) assisted GNNs [1, 6, 20, 25] to elevate the performance of recommendation systems, which can overcome the potential disruption caused by contrastive learning and enhance node representations. Utilizing a relationship graph converter, RGTN [14] propagates information between nodes and employs various encoders to capture both the structural and semantic features of nodes. Moreover, it uses the attention fusion module to obtain the importance values of knowledge graph nodes. Notably, KGCL [44] addresses noise in knowledge graphs by incorporating additional supervised signals during the KG enhancement process, guiding cross-view contrastive learning paradigms that play an important role in unbiased user-item interactions during gradient descent. Additionally, When KGRec [43] conducts contrastive learning, it maintains consistency between the two signals of knowledge and user-item interaction, which employs the masking of important knowledge through high rationalization scores. Despite these strengths, current KG-based methods heavily depend on the quality of the used KG. Moreover, an excessive reliance on KG may introduce a learning bias, where models become dominated by KG features, potentially overlooking valuable user-item connections in the original graph.To address the challenges mentioned above and achieve excellent performance and robustness, we present the Self-Derived subgraphs Contrastive Learning framework (CL-SDKG). In the initial step, we use the graph from each GNN layer as input for the variational graph reconstructor, allowing us to estimate the Gaussian distribution of user-item nodes corresponding to the GNN aggregation layer. This innovative approach enables the generation of multiple knowledge graphs, referred to as self-derived knowledge graphs, which can reduce undue reliance on the knowledge graph and mitigate noise associated with it through the consistency of the estimated structure. Moreover, the self-derived knowledge graphs not only facilitate a better understanding of relationships between user-item nodes but also serve as a mechanism for the model to comprehend and utilize valuable connections within the original graph. This unique feature empowers the CL-SDKG method to comprehensively leverage information from the original graph, avoiding biases toward knowledge graphs. Additionally, we incorporate signals during the original knowledge graph contrastive enhancement process to conduct contrastive learning on the self-generated subgraphs. This step further fortifies the model's robustness, enabling it to adapt more effectively to various user-item interaction scenarios and suppress noise during gradient descent.In summary, the main contributions of our paper are:• We propose a novel recommendation algorithm CL-SDKG, which has excellent robustness and better recommendation performance, i.e., it can effectively reduce the noise information brought by knowledge graphs and reduce excessive dependence on knowledge graphs. • We use a variational estimator to reconstruct the input graph as a learnable version of the traditional knowledge graph, which can effectively correct and supervise the model's behavior during the learning process and enable the model to learn effective knowledge connections of the data, without overly relying on the traditional knowledge graph. • We validated the effectiveness of CL-SDKG on three realworld datasets. The experimental results strongly indicate that CL-SDKG outperforms other advanced methods in terms of performance. 
related work 2.1 gnn in recommendation: 
 Recommendation systems are increasingly gaining popularity among users seeking personalized items. Traditionally, Relying on historical user-item interactions to uncover user preferences and interests, collaborative filtering models [30, 56] have dominated the realm of recommendation systems. However, these methods encounter challenges stemming from intricate user behavior or data input complexities. In response to these challenges, GNN-based recommendation methods have garnered the attention of researchers.Graph recommendation methods have proven to deliver impressive recommendation performance, primarily owing to their adeptness in effectively leveraging graph structures to capture meaningful connections between nodes during the iterative process. For instance, AutoGCL [52] utilizes multiple graph view generators to learn the probability distribution of input graphs. And it also introduces sufficient enhanced variance in the contrastive learning process. To generate CL views, XSimGCL [48] utilizes invalid graph enhancements, employing a simple and effective noise-based embedding approach. LightGCN [10] enhances recommendation performance by eliminating nonlinearity and obtaining the initialization of embeddings by computing the network embedding on the compressed graph for each node. This approach also alleviates the over-smoothing phenomenon associated with sparse interaction data in GNNS. However, the high sparsity of the user-item interaction graph presents a challenge, as GNNs may lack sufficient generalization ability, leading to an ineffective perception of the dependency relationship between users and items.Enhanced knowledge graph recommendation involves two ways: embedding enhancement and path enhancement. In the embedded recommendation methods, the emphasis is on leveraging the auxiliary information inherent in the knowledge graph. For instance, Kopra [29] derives the corresponding user representation by pinpointing relevant entities in the knowledge graph based on the user's click history. CKE [51] , On the flip side, captures both structural and semantic representations of items by extracting information from the heterogeneity of nodes and the knowledge graph (KG), covering aspects such as structure, text, and vision. In a similar vein, KGTORE [26] employs the knowledge graph to learn potential representations of semantic features. This enables the interpretation of user decisions as a personal sublimation of item feature representations.Enhancing path recommendation methods primarily focus on the representations of meta-paths. In their work, [12] introduces a three-way neural interaction model utilizing priority sampling techniques to select better path instances. The model is based on meta-paths, enhancing the representation ability of context, user, and item, with the three elements reinforcing each other. RippleNet [32] automatically extends users' potential interests along links in the KG, encouraging the propagation of user interests among knowledge entities. In a similar vein, KGCN mines pertinent attributes on the knowledge graph to efficiently capture correlations between items. While path-based recommendation methods demonstrate superior performance compared to embedding-based methods, as they can capture higher-order knowledge perception dependencies, they heavily rely on domain knowledge-based meta-path design. Additionally, the exploration of different meta-paths involves significant time costs.Graph Contrastive Learning (GCL) has emerged as a prominent trend in recent years within the research community. Many GCL approaches [33, 38] incorporate additional supervisory signals from raw data for recommendation. [44] introduces a universal Knowledge Graph Contrastive Learning framework (KGCL) to reduce noise information within KG-enhanced recommendation systems. This approach utilizes the knowledge graph expansion pattern to mitigate noise during information aggregation, leading to a more resilient representation of item perception. KGRec [43] introduces a novel self-supervised rationalization method, producing rationalization scores for knowledge triplets. Specifically, it integrates generative and contrastive self-supervised tasks to enhance model reconstruction by assigning high rationalization scores to crucial knowledge while masking it. However, GNNs combined with KG recommendation methods [34, 36] heavily rely on the quality of knowledge graphs. Moreover, excessive reliance on knowledge graphs may cause the model to learn features biased towards the KG, potentially overlooking useful connections between nodes in the original graph. In comparison, our method CL-SDKG achieves superior performance and robustness by utilizing self-generated subgraphs from the original graph for contrastive learning. 
preliminaries: 
 In this section, we mainly introduce the key symbols and notations used in our paper and formalize the definitions of our research problem. We adopt 𝜕 = (U, V), 𝑢 ∈ U and 𝑣 ∈ V represent individual users and items, respectively. We construct a user-item interaction diagram G u = (𝑢, 𝐼, 𝑣) to represent whether the user and the item have the interaction. Here, the presence of an interaction between a user and an item is expressed as 𝐼 = 1. Conversely, the absence of interaction is denoted as 𝐼 = 0. We use knowledge graphs containing triples to represent the actual knowledge of related items, represented asG kg = (𝑒 1 , 𝑟 𝑒 1 ,𝑒 2 , 𝑒 2 ),𝑒 1 , 𝑒 2 ∈ E 𝑚, where E is the 𝑚dimensional vector set of knowledge entities, 𝑟 𝑒 1 ,𝑒 2 represents the relationship between two entities such as (engineer, build, house). It's crucial to emphasize that the entity set is not exactly equal to the item set. Indeed, it is a subset of the entity set, enabling the reconstruction of meaningful connections between items and entities within the knowledge graph.We employ a variational estimator 𝐹 (G i kg |𝑢, 𝑣, G i u , G kg ), to estimate the distribution of nodes corresponding to the iterated useritem graph. This process yields multiple KGs, represented as G i kg * .Leveraging these KGs enhances our model's ability to learn the representation of the input graph, resulting in heightened robustness.Finally, our task can be formally defined as follows: considering a user-item interaction graph denoted by G u , and a knowledge graph represented by G kg , Our objective is to develop a recommendation model 𝐻 (𝑢, 𝑣 |G u , G kg , G i kg * ), 𝜔), where 𝜔 represents learnable parameters. The model outputs a value within the range of [0, 1], indicating the likelihood of user 𝑢 interacting with item 𝑣. 
methodology: 
 In this section, we will provide a detailed overview of CL-SDKG. The overall framework of our CL-SDKG is illustrated in Figure 1 . The CL-SDKG comprises three main parts: (1) Knowledge aggregation enhanced recommendation. (2) Variational graph reconstruction and enhanced representation of self-derived KGs. (3) Contrastive learning of self-derived KGs in user-item view. For initialization, we use free embeddings 𝑢 ∈ U and 𝑣 ∈ V to represent user and item nodes. The vectors 𝑢 𝑖 and 𝑣 𝑗 correspond to user 𝑖 and item 𝑗 respectively. Subsequently, we will explore each component for a more comprehensive understanding.In dealing with a complex knowledge graph, we employ a fundamental principle involving the weighted learning of the probability of knowledge triplet existence and the basic tenet of collaborative interaction. The function applies a weighted graph attention mechanism to each knowledge triplet, incorporating learnable content:EQUATIONwhere 𝑒 1 , 𝑟 𝑒 1 ,𝑒 2 , and 𝑒 2 are entity representations of the head, relationship and tail. W Q and W k are the trainable weights of attention, with a size of 𝑑 × 𝑑. To capture the relevant entities, we rotate 𝑒 1 to the potential space of 𝑒 2 by using the relationship 𝑟 𝑒 1 ,𝑒 2 . We use a knowledge triplet with a basic principle score of G kg (𝑒 1 , 𝑟 𝑒 1 ,𝑒 2 , 𝑒 2 ) to assist the recommendation system in enhancing recommendations.To ensure comparability of the scores, we normalize the scores with the following formulas:𝜔 𝑒 1 , 𝑟 𝑒 1 ,𝑒 2 , 𝑒 2 = exp G kg 𝑒 1 , 𝑟 𝑒 1 ,𝑒 2 , 𝑒 2 𝑒 1 ,𝑟 ′ 𝑒 1 ,𝑒 2 ,𝑒 ′ 2 ∈𝜏 𝑒 1 exp G kg 𝑒 1 , 𝑟 ′ 𝑒 1 ,𝑒 2 , 𝑒 ′ 2 (2)where 𝜏 𝑒 1 is the is the neighbors of 𝑒 In the process of reconstructing variational graphs, we use VAE to estimate the Gaussian distribution of nodes and edges in the input graph to achieve graph reconstruction, thereby generating new subgraphs. VAE Brief. Given a training set, the variational autoencoder [22] operates under the assumption that each sample 𝑥 𝑖 is part of it. Samples are constructed through a random process, where 𝑥 ∼ 𝑝 𝛿 (𝑥 |𝑘). Additionally, we can derive the maximum likelihood function:EQUATIONwhere 𝑝 (𝑘) is the prior distribution of potential variables 𝑘 . However, since it is impossible for us to know all potential variables, VAE adopts variational inference [8] and chooses to use inference models 𝑞 𝜎 (𝑘 |𝑥) to model posterior distributions 𝑝 𝛿 (𝑥 |𝑘). Furthermore, variational autoencoders are optimized by minimizing evidencebased lower bounds to obtain the best results:EQUATIONwhere 𝑞 𝜎 (𝑘 |𝑥) and 𝑝 𝛿 (𝑥 |𝑘) represent the encoder and decoder of the parameterized neural network; KL is the Kullback-Leibler divergence [41] between 𝑞 𝜎 (𝑘 |𝑥) and 𝑝 (𝑘), introducing constraints to align it with the vicinity of a prior Gaussian distribution. Then, we conveniently use Gaussian distribution of input graph nodes and edges for graph reconstruction.Graph Reconstruction. We aim to refactor the input useritem interaction diagram and the initialized node representation. The probability distribution of the input graph structure 𝑘 : of user-item graphs more effectively, we estimate the mean and variance of node distributions by utilizing GNN. After stacking 𝑁 multiple convolutional layers, we fuse the calculated mean and utilize a Multi-Layer Perceptron (MLP) to calculate its variance. Once the approximate posterior mean and variance are obtained, we generate potential subgraphs 𝑘 𝑖 by sampling in 𝑁 (𝜇 𝑖 , 𝜎 2 𝑖 ). Since this sampling process is non-differentiable, we adopt the reparameterization techniques to replace the sampling process:𝑦 ′ ∼ 𝑝 𝛿 (𝑦|𝑘) is learnable. Each nodeEQUATIONEQUATION𝑘 𝑖 = 𝜇 𝑖 + 𝜎 𝑖 * 𝜉 (7) where 𝜉 ∼ 𝑁 (0, 1) is the noise that follows a normal distribution. Once the probability distribution of variables is estimated, our primary objective is to reconstruct the original graph. Therefore, we employ the inner product to calculate the probability score of nodes 𝑖 and 𝑗 for connections:EQUATIONwhere intr 𝑖 𝑗 indicates whether there is interaction between nodes.After deriving the subgraph 𝜁 1 , 𝜁 2 , we defined the consistency con 𝑖 of the item's knowledge graph structure based on the consistency between graph encoded representations:EQUATIONwhere 𝑓 𝜁 (•) represents the knowledge aggregation function for relationship perception, and 𝑠 (•) is the cosine function for estimating the similarity between them. The consistency of the derived knowledge structure can serve as a guide for each item, avoiding certain deviations in the derived knowledge graph and alleviating user-item interaction noise caused by random values of normal distribution.After obtaining the self-derived knowledge graph comparison subgraph corresponding to each layer, we process it as a knowledge graph. We can predict user interests more effectively by using higher consistency in the knowledge graph structure to reduce noise. To reduce noise in the subgraph, we entails identifying and eliminating weakly correlated edges. Additionally, we incorporate 𝑐𝑜𝑛 𝑖 into the data augmentation process for the user-item interaction graph to mitigate the limitation of contrastive learning due to pure drop operations. The specific formula for knowledge-guided enhancement is as follows:EQUATIONEQUATIONwhere 𝑝 𝑢,𝑖 denotes the probability of the edge connected to the discarded user 𝑢 and item 𝑖. 𝑤 𝑢,𝑖 represents the attractiveness of item 𝑖 to user 𝑢. con 𝑖 is the structural consistency score corresponding to item 𝑖 . Then we normalize the maximum and minimum truncation probabilities of 𝑤 𝑢,𝑖 , where 𝑝 𝑥 is used to influence the strength of the mean. Subsequently, we generate two mask vectors 𝑚𝑎𝑠𝑘 1 𝑢 , 𝑚𝑎𝑠𝑘 2 𝑢 based on the Bernoulli distribution:EQUATIONwhere 𝑓 𝜙 (•) represents the operational symbols that enhance the input graph and discard user-item interactions from the edge set 𝑉 based on inferred probabilities 𝑝 𝑢,𝑖 .After obtaining the enhanced self-derived knowledge graph, we aim to preserve interactive connections that distinctly reflect user interests with high rationality and effectiveness. To achieve this, we opt to assign weights to each interaction edge based on the average rationality score of the knowledge triplet from the original knowledge graph. This approach effectively mitigates the impact of interaction noise.EQUATIONEQUATION)where the lower 𝑣 indicates a weaker correlation between the knowledge entities adjacent to the item and the corresponding recommendation task in the knowledge graph. This correlation may introduce deviations to the model. Consequently, we filter out these weak connections with lower scores to enhance the model's performance.To address overfitting on user and item representations, we implement a strategy involving polynomial distribution sampling to eliminate edges originating from recommended connections. By using the strategy, we effectively enhances the generalization and robustness. We use a predefined feature fusion algorithm to acquire the attempt-specific nodes of the item as contrastive embeddings.We adopt iteratively LightGCN to represent the 𝐺 𝑢,𝑖 :EQUATIONAfter obtaining the acquisition of the representation for the useritem interaction view, we aggregate the representations of each layer to obtain the final comparison embedding. Then, we feed enhanced comparison subgraphs into an MLP to further map them into the same latent space:EQUATION)where W and 𝑏 are learnable weights and deviations. Furthermore, to ensure alignment of item representations across different views and enhance the robustness of the model, we incorporate a modified InfoNCE loss for optimization. We set 𝜅 = exp(𝑠 (𝑒𝑚𝑏 * 𝑣 𝑖 , 𝑒𝑚𝑏 * 𝑣 𝑗 )/𝛽), and 𝛾 = exp(𝑠 (𝑒𝑚𝑏 * 𝑧 𝑖 , 𝑒𝑚𝑏 * 𝑣 𝑗 )/𝛽)). This loss will assign a random sample as a negative sample to each subgraph, and the specific formula is as follows:EQUATIONIn the comparison loss, 𝑣 ′ and 𝑣 ′′ serve as negative samples randomly drawn from sampled items. The similarity measurement 𝑠 (•) is defined as the cosine similarity of the normalized vector. The temperature hyperparameter 𝛽 regulates the difficulty of comparison compared to the target. A higher 𝛽 might make the comparison more relaxed, with even slight differences being treated as similar, while a lower 𝛽 would make the comparison more restrictive. Furthermore, We have enriched the perceptual representation of the existing knowledge graph, while combining self-derived knowledge graphs and existing knowledge graphs to enhance the recommendation system. Similarly, for the primary task, we utilize 𝑦 ′ 𝑢𝑣 = 𝑒𝑚𝑏 ⊤ 𝑢 𝑒𝑚𝑏 𝑣 as the prediction result for recommendation, 𝑒𝑚𝑏 𝑢 is the enhanced user vector and 𝑒𝑚𝑏 𝑣 is the enhanced project vector. To refine the model parameters, we apply the widely embraced Bayesian Personalized Ranking (BPR) loss. We employ a holistic loss function to optimize all losses:EQUATION)EQUATION)where 𝐿𝑜𝑠𝑠 𝑘𝑔 represents the loss corresponding to the knowledge graph, 𝐿𝑜𝑠𝑠 𝑚𝑎𝑠𝑘 represents the mask loss. 𝜆 1 and 𝜆 2 are weights associated with the loss of the knowledge graph and self-derived subgraphs, respectively. These weights control whether the model is more inclined to learn the representation of its nodes or the representation of the knowledge graph nodes, and the sum of the two is equal to 1. In addition, because 𝐿𝑜𝑠𝑠 𝑚𝑎𝑠𝑘 enables the model to reconstruct effective connections between relational contexts, we add a mask loss [43] in CL-SDKG.In this section, we analyze the time complexity of the three main modules in our CL-SDKG. ( 1  
datasets.: 
 To thoroughly validate the effectiveness of CL-SDKG and ensure a comprehensive evaluation of diversity, we employed three distinct real-world datasets: Last FM, focusing on music recommendations, MIND for news recommendations, and Alibaba-iFashion, which caters to shopping recommendations. Furthermore, to enhance the performance of recommendation systems and mitigate noise introduced by knowledge graphs, we utilized the approach outlined in [43] to preprocess the dataset. Table 1 summarizes statistical details regarding user-item interactions and knowledge graphs for the three evaluation datasets. Collaborative Filtering Method Recommendation.• BPR [28] . It is an optimization standard directly aimed at personalized ranking, which obtains the estimator of the maximum posterior distribution through Bayesian analysis.• NCF [11] . It is a universal framework rooted in neural network collaborative filtering. Multi-layer perceptrons are employed to learn the interaction function between users and items.• GC-MC [3] .It is a graph autoencoder framework based on bipartite interactive graph differentiable message passing. This algorithm utilizes message passing to generate potential features of users and item nodes. • LightGCN [10] . It is a streamlined Graph Convolutional Network (GCN) model, obtaining user and item representations through linear propagation on the user-item interaction graph. The final representation is determined by the weighted sum of representations learned across all layers. • SGL [29] . It is a multi-to-many seq2seq architecture trained using MNMT targets, which has excellent scalability for tasks such as cross-language AMR parsing.Embedding-based Knowledge-aware Recommendation.• CKE [51] . It is an integrated framework that utilizes collaborative learning to filter latent representations and semantic representations of items in a knowledge base. • KTUP [4] . It considers various preferences when translating users into items, and then combines multiple transmission schemes with KG to complete model joint training.GNN-based KG for Recommenders.• KGNN-LS [33] . It entails a knowledge-aware Graph Neural Network incorporating label smoothing regularization. The knowledge graph transforms into a user-specific weighted graph, and a graph neural network is then applied to compute personalized item embeddings. • KGCN [34] . It is a comprehensive framework designed to efficiently capture the correlation between items by extracting pertinent attributes from a knowledge graph.• KGAT [36] . It pertains to knowledge graph attention networks, explicitly modeling high-order connections in the knowledge graph in an end-to-end manner.• KGIN [37] . It represents a novel information aggregation scheme that recursively integrates relational sequences of distant connections. This algorithm extracts valuable information related to user intent and encodes it into representations of both users and items.Self-Supervised KG for Recommenders.• MCCLK [57] . It considers three graph views for knowledge graph recommendation: a global-level structural view, a locallevel collaborative view, and a semantic view.• KGCL [44] . It is a universal knowledge graph contrastive learning framework. This framework can reduce the information noise in knowledge graph-enhanced recommendation systems.• KGRec [43] . It integrates generative and contrastive selfmonitoring tasks, providing recommendations through rational blocking.We adopt commonly used Recall@20 and NDCG@20 as evaluation indicators to measure the performance of our CL-SDKG and baselines. Furthermore, for a fair performance evaluation, we partition the datasets into three segments for each algorithm. Specifically, 70% is allocated for training, 10% for tuning hyperparameters, and 20% for testing.Using PyTorch as our deep learning framework, we have implemented several baseline models by utilizing either official or third-party code. To further refine our proposed algorithm, we performed a hyperparameter search on the weight ratios associated with self-derived knowledge graphs and external knowledge graphs. Moreover, we search the weight of 𝐿𝑜𝑠𝑠 𝐶𝐿-𝑆𝐷𝐾𝐺 in the range of {0.1,. . . ,0.5,. . . ,0.9}, search the temperature of the CL in the range of {0.1,. . . ,0.5,. . . 0.9}, and we set masking sizes in the range of {128, 256}, maintaining proportions from from {0.2, 0.4, 0.6, 0.8}. Additionally, for all GNN-based methods, the number of GNN layers is fixed at 2.As shown in Table 2 , we present a comparison between our CL-SDKG and baselines on the three real datasets. Based on the results, we draw the following observations. Our proposed CL-SDKG algorithm outperforms others in both Recall and NDCG evaluation metrics. This superiority can be primarily attributed to two factors. Firstly, we employ a variational estimator to estimate the distribution of each node and introduce noise, significantly enhancing the model's robustness during the reconstruction process. Additionally, the contrastive learning of self-generated subgraphs diminishes the model's reliance on Knowledge Graphs (KG), thereby placing greater emphasis on discerning meaningful connections between user-item interactions and more effectively capturing user interest representations.The approach of combining knowledge graphs does not consistently outperform the Collaborative Filtering Method. It is evident that, in both datasets, CKE and KTUP demonstrate superior recommendation performance compared to BPR, NCF, GC-MC, and LightGCN. However, the performance of SGL stands out even more, precisely highlighting the limitations of Knowledge Graphs (KG) in recommendation tasks. This phenomenon is particularly pronounced in datasets with complex knowledge graphs and sparse interactions.The knowledge-aware recommendation models based on GNNs outperform the embedding-based models. It is evident that, in comparison to BPR and NCF, the knowledge-aware recommendation model based on GNN exhibits superior performance. This advantage arises from the linear propagation modeling utilized in the embedding model, whereas GNNs can capture more intricate and high-order information within the knowledge graph.In the comparison between contrastive learning models (such as MCCLK and KGCL) and unsupervised models (such as KGIN), no model consistently excels on all datasets. The inconsistency observed could stem from the limitations of random graph enhancement or visually handcrafted cross-view pairing. These methods may fall short in fully uncovering genuinely valuable knowledge graph information for encoding user interests.To evaluate the influence of weights 𝜆 2 and 𝜆 1 associated with KG and self-generated subgraphs on our CL-SDKG, we vary the corresponding weight values within the range of (0.2, 0.8). As shown in Figure 2 , it is observed that the impact of 𝜆 2 and 𝜆 1 on CL-SDKG is not very significant. And when 𝜆 2 = 0.6 and 𝜆 1 = 0.4, we can see that our CL-SDKG obtains the best performance. The model performs better when the weights of KG and self-derived knowledge graphs are evenly matched in contrastive learning, indicating good robustness. Additionally, when 𝜆 2 is varied within the range of (0.2, 0.4) and (0.6, 0.8), there is no substantial improvement in the overall performance of the model. This observation precisely underscores the limitations of KG in optimizing recommendation tasks, affirming the effectiveness of our work. In this section, we conduct ablation experiments about losses to evaluate the impact of different losses on our CL-SDKG.Based on the results shown in Table 3 , 𝐿𝑜𝑠𝑠 𝑟𝑒𝑐 is the loss we only calculate for the recommendation task and use that loss to update the model. 𝐿𝑜𝑠𝑠 𝑟𝑒𝑐 + 𝐿𝑜𝑠𝑠 𝑚𝑎𝑠𝑘 is the addition of the loss on the MASK task to update the model based on the previous ones. 𝐿𝑜𝑠𝑠 𝑟𝑒𝑐 + 𝐿𝑜𝑠𝑠 𝑚𝑎𝑠𝑘 + 𝐿𝑜𝑠𝑠 𝑘𝑔 is the experiment where we do not use 𝐿𝑜𝑠𝑠 𝐶𝐿-𝑆𝐷𝐾𝐺 . And the last is the full version of our CL-SDKG. It is evident that when employing different losses to optimize the model, CL-SDKG with all losses in this paper achieves the best performance. In comparison to other variants such as our CL-SDKG without 𝐿𝑜𝑠𝑠 𝐶𝐿-𝑆𝐷𝐾𝐺 , the full version of our CL-SDKG demonstrates the most effective recommendation performance on three real datasets. This underscores the significance of our proposed self-generating subgraph contrastive learning module, which significantly enhances the recommendation performance of our model.In this section, we delve into the robustness of our CL-SDKG from two perspectives: the number of layers and time parameters. Initially, we investigate the impact of graph inference layers on the performance. Subsequently, we further demonstrate the robustness of our CL-SDKG by examining the influence of temperature on contrastive learning. As shown in Figure 3 , we compare our CL-SDKG with two state-of-the-art models based on the GNN.The impact of GNN-Layers. As Figure 3 shows, in the Last-FM and Alibaba-iFashion datasets, we notice that as the number of graph inference layers increases, the performance of recommendations gradually improves and then reaches a plateau. Specifically, when the layer value is 1, the performance of all comparison methods is the worst. As the number of layers increases, other comparison methods vary widely, while our method consistently performs the best and demonstrates stable improvement. This suggests that when the number of layers is small, the graph network struggles to accurately estimate node distribution, while when the number of inference layers is excessively high it can lead to instability in estimation quality due to over-smoothing. It is evident that CL-SDKG outperforms other models overall and exhibits stronger robustness.The impact of Temperature. Furthermore, as depicted in Figure 4 , owing to the incorporation of contrastive learning in CL-SDKG, we investigated the influence of temperature on contrastive  
conclusion: 
 In this paper, we propose a novel self-derived knowledge graph contrastive learning (CL-SDKG) for recommendation. Our CL-SDKG employs a variational estimator to estimate the distribution of input graph nodes for deriving subgraphs that align with the original input graph. We utilize contrastive learning on user-item views to optimize the recommendation task. Experimental results on multiple datasets demonstrate the remarkable advantages of our CL-SDKG compared to state-of-the-art methods. In future work, we aim to further explore effective ways to integrate self-generated subgraphs with knowledge graphs, such as through graph adversarial generative learning and hypergraphs. This direction holds the potential to offer more possibilities in combining self-generated subgraphs and knowledge graphs.  
 
 Appendix:  


Decision: All the reviewers agree that the paper is a high-quality paper. After reading the paper, I am confident that the paper is above the bar. Thus I recommend acceptance.

Reviews: review: paper_summary: paper_summary: In this paper, a self-derived Knowledge Graph Contrastive Learning Framework (CL-SDKG) is proposed to enhance the recommendation system. The variational graph reconstruction technique is used to estimate the Gaussian distribution of user item nodes. Finally, the contrastive learning is applied to KG to improve the robustness. And good results have been achieved in multiple dataset experiments.
strength_weakness: strength_weakness: 1. In the process of generating knowledge graphs, the authors innovatively generate multiple knowledge graphs, which can reduce the over-reliance on knowledge graphs and reduce the impact of noise by estimating the consistency of the structure. It can solve the problem of over-reliance on graphs in the past.
2. The article is full of experiments and analysis of inter-complexity analysis, with rigorous logic and good writing.
limitations: limitations: 1. Please add a more detailed explanation about Figure 1 for better understanding.
2. Why did you choose a Gaussian distribution and have you tried other distribution in experiments?
3. Is the space complexity too high when generating multiple graphs due to memory space resources?
4. The effectiveness of generating subgraphs can be expressed with visualization analysis.
paper_summary: paper_summary: The authors propose a Self-Derived Knowledge Graph Contrastive Learning framework (CL-SDKG) to enhance recommendation systems. The framework employs the variational graph reconstruction technique to generate self-derived Knowledge Graphs (KGs) by estimating the Gaussian distribution of user-item nodes. These self-derived KGs provide robust perceptual representations and allow models to focus on user-item interactions, minimizing the negative effects of dependencies from conventional KGs. Additionally, contrastive learning is applied to these self-derived KGs to enhance the robustness of the framework. Experiments conducted on three public datasets demonstrate that CL-SDKG outperforms state-of-the-art baselines.
strength_weakness: strength_weakness: The paper is easy to follow;
The tecnical contribution is clear.
limitations: limitations: I find the following aspects need to be addressed in the current version:
	Section 3 is too short; please consider merging it with Section 4.
	The paper contains formal symbolization errors, where vectors are not bold, such as u_i and v_i in Section 4 METHODOLOGY.
	Could not find the calculation formula for 〖Loss〗_mask. Please provide the corresponding formula.
	In Section 5.1.1, please add an explanation of the dataset preprocessing methods to facilitate reader understanding.
	In Section 5.2, please consider using underscores to label suboptimal baseline methods for easier comparison of performance across different methods.
	The field datasets used in this paper and the code is not available to the research community. Hence, the results are rather hard to reproduce.
   Consider adding more evaluation metrics.
paper_summary: paper_summary: This paper proposes a model (CL-SDKG) that uses variational graph reconstruction technology to improve the quality of knowledge graphs in GNN-based recommendation methods. In addition, it combines contrastive learning to enable CL-SDKG to model a more robust entity representation and enhance the ability to capture the interaction relationships between different items and entities.
strength_weakness: strength_weakness: 1. Through variational graph reconstruction technology, the original graph structure is preserved as much as possible while enhancing the node representation ability in the graph.
2. Enhance GNN robustness through contrastive learning.
limitations: limitations: 1. How is the time complexity of the proposed model compared to the baseline models?     
2. How does the model's recommendation performance change if the self-derived knowledge graph proposed in this article is not used?
3. What is the `Rationale-aware Heterogeneous Knowledge Aggregation` mentioned in Figure 1?
4. In  robustness analysis section, only parameter experiment analysis is carried out. it does not verify the contribution mentioned in "it can effectively reduce the noise information brought by knowledge graphs and reduce excessive dependence on knowledge graphs”
5. Issues of Expression
   1) How to obtain $Loss_{mask}$?
   2) As defined in `PRELIMINARIES`, $\omega$ is the learnable parameter of the model, so what does $\omega$ in formula (14) mean?
6. Some Typos:
   1) Case issues: `We` in line 527, and `Our` in line 276;
   2) Punctuation issue: `.` in line 558 should be `,`;
   3) Statement consistency issue: Line 299 is expressed as `potional space`, while line 506 is called `latent space`;
7. Formula issues:
   1) Why is Q in superscript and k in subscript in $W^Q$ and $W_k$ in formula (1)? Is there any special difference between them?
   2) Is `v` in line 487 the same as `v` in formula (13)? (The font is different)
paper_summary: paper_summary: This paper introduces a novel framework for enhancing recommendation systems using a self-derived Knowledge Graph (KG) and Contrastive Learning (CL) termed as CL-SDKG. The approach leverages variational graph reconstruction to estimate the distribution of user-item node pairs, generating multiple self-derived KGs that focus on reducing dependencies and enhancing user-item interaction analysis. Comprehensive experiments on three public datasets demonstrate that CL-SDKG outperforms standard baselines, showcasing its effectiveness in improving the robustness and accuracy of recommendations.
strength_weakness: strength_weakness: 1. Innovation: The paper proposes a novel recommendation system framework, namely self derived knowledge graph contrastive learning (CL-SDKG), which combines variational graph reconstruction technology and contrastive learning to generate subgraphs that are consistent with the original input graph structure. The paper uses variational autoencoder (VAE) to estimate the Gaussian distribution of nodes and edges, achieving graph reconstruction. This method is theoretically reasonable and well founded. In addition, the robustness of the model was further improved through contrastive learning.
2. Clear structure: The methodology in the paper is based on the current research frontiers of Graph Neural Networks (GNN) and Knowledge Graph (KG), with clear technical details, clear and correct algorithm processes, and thorough consideration of details.
3. Full evaluation: The author conducted experiments on three different public datasets, covering a wide range of application scenarios, to verify the effectiveness of the proposed method and its practicality.
4. Research motivation: The article points out that existing knowledge aware recommendation models rely on the quality of knowledge graphs. Even worse, the model may overfill the KGs with information unrelated to recommendations.
limitations: limitations: 1. Related work can be further introduced in the field of combining graph contrastive learning and knowledge graphs
2. The color matching in the experimental renderings can be improved
3. The fluency of English can be further enhanced