HAS IT REALLY IMPROVED? KNOWLEDGE GRAPH BASED SEPARATION AND FUSION FOR RECOMMENDATION
Authors: Ying Tang
Jintian Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: recommendation
knowledge-graph
graph neural network

Abstract: In this paper we study the knowledge graph (KG) based recommendation systems. We first design the metric to study the relationship between different SOTA models and find that the current recommendation systems based on knowledge graph have poor ability to retain collaborative filtering signals, and higher-order connectivity would introduce noises. In addition, we explore the collaborative filtering recommendation method using GNN and design the experiment to show that the information learned between GNN models stacked with different layers is different, which provides the explanation for the unstable performance of GNN stacking different layers from a new perspective. According to the above findings, we first design the model-agnostic Cross-Layer Fusion Mechanism without any parameters to improve the performance of GNN. Experimental results on three datasets for collaborative filtering show that Cross-Layer Fusion Mechanism is effective for improving GNN performance. Then we design three independent signal extractors to mine the data at three different perspectives and train them separately. Finally, we use the signal fusion mechanism to fuse different signals. Experimental results on three datasets that introduce KG show that our KGSF achieves significant improvements over current SOTA KG based recommendation methods and the results are interpretable.

Under review as a conference paper at ICLR 2023
HAS IT REALLY IMPROVED? KNOWLEDGE GRAPH BASED SEPARATION AND FU- SION FOR RECOMMENDATION
Anonymous authors Paper under double-bling review
ABSTRACT
In this paper study the knowledge graph (KG) based recommendation systems. We first design the metric to study the relationship between different SOTA mod- els and find that the current recommendation systems based on knowledge graph have poor ability to retain collaborative filtering signals, and higher-order connec- tivity would introduce noises. In addition, explore the collaborative filtering recommendation method using GNN and design the experiment to show that the information learned between GNN models stacked with different layers is differ- ent, which provides the explanation for the unstable performance of GNN stacking different layers from a new perspective. According to the above findings, we first design the model-agnostic Cross-Laye Fusion Mechanism without any parame- ters to improve the performance of GNN. Experimental results on three datasets for collaborative filtering show that Cross-Layer Fusion Mechanism is effective for improving GNN performance. Then we design three independent signal ex- tractors to mine the data at three different perspectives and train them separately. Finally, we use the signal fusion mechanism to fuse different signals. Experi- mental results three datasets that introduce KG show that our KGSF achieves significant improvements over current SOTA KG based recommendation methods and the results are interpretable.
INTRODUCTION
The recommendation system is the im- portant technique for information fil- tering, which can help users find the data they want in large amount of data. Collaborative filtering algorithm is classical recommendation method, and its main idea is to make recom- mendations by mining the collaborative signals between users and items. As a deep learning method, graph neural net- works (GNN) have been used to effec- tively mine users' collaborative signals, such 2020). Recent works (Wu et al., 2021 Yu et al. 2022 use LightGCN as the backbone to introduce contrastive learning and achieve better performance.
IUA
5/%
Figure 1: Two categories of improvement
NGCF (Wang et al. 2019d). LightGCN (He et al.
From LightGCN (He et al. 2020) to SimGCL (Yu et al. 2022). the performance is constantly im- proving. For the convenience of description, we assume that there are two models, M1 and M2. The overall performance of M1 is better than that of M2. In practical applications, for some specific users, the recommendation effect of M1 may be inferior to that of M2. problem that may be overlooked is, is M1 really improvement over M2 That is, what is the relationship between M1 and M2? Figure shows two possible relationships between MI and M2. (i ~ is are commodities, the circle "Test" represents the range of the test set, the circle "M1" represents the range of top K commodi- ties given by the M1 model, the cricle "M2" represents the range of the test the top K commodities given by the M2 model). In Figure a)) M1 learns new things the basis of retaining the infor-

---

Under review as a conference paper at ICLR 2023
mation of M2, while in Figure []bb) the improvement of M1 to M2 is to lose part of the information of M2 and learn more new information. To objectively measure these two cases, we design a new
metric, Intersecti ion@N, to measure the differences between two models. Based on this indi- cator, we conduct experiments between different collaborative filtering models (Wu et al. 2021 Yu et al.] (002), and between different stacked layers of the same collaborative filtering model ( (which are described in Section and Appendix (C. and obtain two findings:( (1) The relationship between different collaborative filtering methods using GNN is shown in Figure lb). (2) The relationship
between models based on collaborative filtering that uses the same GNN model but stacks different
layers is shown in Figure lb). i.e., models that stack higher layers cannot fully "include" models that stack lower layers.
Many studies Wang et al., 2019d He et al. 2020) pointed out that within a certain range, the more GNN layers are stacked, the higher the model performance is. The performance decreases after exceeding this range. Numerous studies (He et al. 2020; Zhao & Akoglu 2019) attribute the poor performance to the over-smoothing of nodes caused by multi-layer stacking, and based this rea- son, many methods are designed to alleviate the over-smoothing. A common feature of these works is that they all choose model with fixed stacked L layers as the final model. The default assump- tion in doing so is that the model with good performance (stacked L layers) is an improvement over the model with stacked T layers of poor performance and the improvement is understood in Fig- [a). However our experiments show that this assumption is invalid and instead their relationship should be as shown in Figure [[)) (i.e. The first finding). The first observation of this paper is that the recommendation method based on collaborative filtering using GNN does not fully exploit the performance of GNN.
Accompanying GNN, knowledge graphs (KG) are introduced into the recommendation systems to improve their performances with auxiliary information. The popular KG based recommendation methods are KGAT (Wang et al., 2019c) and KGIN (Wang et al. 202I), which connect KG and user-item bipartite graphs through items, thus unify the two into one graph structure.
According to KGAT and KGIN, we believe that the KG based recommendation system includes item-based collaborative signals, content signals and attribute-based collaborative signals. These three signals are mined in KGAT and KGIN. The first two signals are mined in the User- Item bipartite graph and the KG respectively, and the third signal is mined in the unified graph by higher-order connectivity (Wang al. 2019c). However, we have two observations with this unified graph structure: (1) Poor preservation for collaborative filtering signals. We use the Intersect i @N metric to measure the KG-based recommendation method and the collabora- tive filtering recommendation method, and find the relationship between the two methods are as shown in Figure [b). Existing methods that introduce KG discard half of the information based on collaborative filtering and learn more information introduced by KG, so the performance of the former is higher than that of the latter. (2) Unnecessary information are introduced by higher- order connectivity which makes the propagation path too long. Taking user U2 recommending
03 3 is, and the semantic in- formation of this path is that U2 likes items with as attribute. However, the path given by KGAT is like U2 + ig 3 a3 3 i4. In this path, the information of node in includes content signal and user-based collaborative signal, which is not helpful for the original semantic information and will introduce unnecessary information. In addition, longer propagation paths also introduce noise.
4 in Figure 5 as an example, a possible path of a path is U2
like
Based on above three observations, in this paper we propose a general knowledge graph based separation and fusion model. It consists of three core parts to meet the three challenges mentioned above:
Cross-Layer Fusion Mechanism. We find that there are differences in the information learned between models that stack different layers, so we cannot directly select a model that stacks N layers. We design a model-agnostic, general-purposs Cross-Layer Fusion Mechanism without any trainable parameters, which fuses models stacked with different layers and can preserve the information of different models.
Signal Extractor. We design three independent, separately trained signal extractors to extract the three kinds of signals in the data mentioned before, which can avoid the mutual influence of each signal. We use the existing collaborative filtering method for item-based collaborative signal ex- traction and the Cross-Layer Fusion Mechanism is applied to further improve the performance. For
2

---

Under review as a conference paper at ICLR 2023
attribute-based based collaborative signals, we process the original bbee-Item-Attrbbuue graph into User- Attribute-Iten graph and apply LightGCN to extract signals from the User-Attribute graph. The Cross-Layer Fusion Mechanism is also applied. The extraction of the content signal draws the idea of Transformer [Vaswani et al. 2017). which uses the user as the Query Vector and the attribute as the Key Vector to obtain the user's interest in each attribute to get fine-grained explanation.
Signal Fusion Mechanism. The three signal extractors get different scores. We draw on the idea of ensemble learning and design Signal Fusion Mechanism, which is essentially weighted summation.
We conduct extensive experiments on three real datasets. Experimental results show that our Cross- Layer Fusion Mechanism improves collaborative filtering better than state-of-the-art methods, such as SGL Wu ct al. (202I), SimGCL Yu et al. (2022). Our designed KGSF (Knowledge Graph based Separate and Fusion model) also outperforms tate-of-the-art methods in knowledge graph-based recommendation, such as KGAT (Wang et al., 2019c), KGIN (Wang et al., 2021). KGCL (Yuhao al. 2022). HAKG (Du et al. 2022). The contributions of this paper are summarized as follows: (1) The Intersection indicator is designed to measure the relationship between different models. Using this indicator we give a new explanation for the unstable performances of GNNs stacking different layers, and find that the improved relationship between most models belongs to Figure (2) The Cross-Layer Fusion Mechanism can effectively improve the performance of GNN, which modellindependdnt, general, and without any training parameters. (3) A highly interpretable and extensible KGSF framework is proposed. (4) Experimental studies on three datasets demonstrate the superiority and effectiveness of Cross-Layer Fusion Mechanism and KGSF.
2 PRELIMINARIES
triggered a research boom since NGCF (Wang al.) 2019d) introduced GNN into the collabo- rative filtering method. LightGCN (He et al., 2020) removes activation and projection in NGCF and achieves remarkable results. Afterwards, contrastive learning was introduced and the SGL (Wul al. 2021) framework
proposed. Then there is the SimGCL (Yu et al. (2022) method, which removes graph augmentations and adds perturbation on the basis of contrastive learning. The per- formance of the above method gradually increases.
However, there are two problems to be studied, First, from NGCF and LightGCN to SGL and SimGCL, is their relationship shown in Figure (Ta) Figure [[b) Second, the increase in the number of GNN layers will degrade the performance. Many studies attribute the reason to the learned embedding tends to be smooth. Is the relationship between these different layer shown in Figure [a) or Figure [b)) Here, we choose two SOTA methods SGL (Wu et al.) 2021 and SimGCL (Yu et al. 2022) for comparative experiments, and give conclusions in Appendix and
2.I
respectively. Meanwhile, a solution is given in subsection 2.2 In addition, we suggest readers read concepts including "Intersection", "UA Graph", "IA Graph" etc. in Appendix B
2.1 THE RELATIONSHIP BETWEEN DIFFERENT LAYERS THE SIMGCL AND SGL
A large number of GNN-based (Wang et al. 2019d He et al. 2020) collaborative filtering methods believe that the performance will increase with the increase of the number of layers in small range. However, the performance will decrease when the stacking reaches cer-
tain level. Table 2 also confirms this. Many works (He et al. 2020) Zhao & Akoglu 2019 Rong et al. 2019) attribute the poor performance to the fact that as the number of layer in- creases, the node representations learned by GNN become smoother and thus lack discrimination. The default premise of this view is that model
Table 1: Intersection @ 20 between different layers.
with better performance is an extension of a model with poor performance, in which the former per- fectly preserves the lat- ter's information. To ver- ify this, we still use In- tersection@21 20 metric
Yelp2018
List-MM
Amazon Book
L-1
L-2
L-3
L-1
1.2
L-3
L-1
1-2
L-3
0.8384
0.7528
1.0000
0.71%
0.6.9%
1,0000
0.8101
0.7330
SGI
1-2
0.7791
1,0000
0.7838
0.7296
1.0000
0.6355
0.7027
1.0000
0.7371
0,6873
0.6000
0.5755
0.6122
0.7151
1-1
1,0000
0.7945
0.7558
1.0000
0.7206
0.6661
1,0000
0.8179
0.8190
SinGiCL
L22
0,7583
1.0000
0.8194
0.7251
1,0000
0.7459
0.7922
1.0000
0.8751
L-3
0.7381
0.8462
1,0000
0.7126
0.8040
1.0000
0,7657
0.8463
1,0000
3

---

Under review as a conference paper at ICLR 2023
conduct experiments between different layers of the same method, and the results are shown in Table [ We summarize our observation and conclusions as follows: (1) We found that on the whole, the similarity between different layers of the same method and the same dataset is between 57% and 85%. This shows that the information learned by different layers is also partly different and the relationship between them is intertwined shown in Figure In other words, the layer with better performance is not an improvement on the layer with worse performance. One possible reason for the poor performance the model with higher layers that the total amount of high-level information in the dataset is smaller than low-level information.
(2) We also found that the minimum of similarity between layers occurs between layers 1 and 3, except that the minimum of the SGL on the Last-FM dataset occurs between layers 2 and 3. This shows that with the increase of the number of layers, the retention ability the upper layer to the lower layer gradually reduced, and the new information learned by the upper layer is greater than the reduced. Therefore, the effect of the upper layer is better than that of the lower layer.
2.2 CROSS-LAYER FUSION MECHANISM
Heretofore, many methods Akoglu 2019 Rong et al.
Zhao & 2019 Liu
Table 2: Performance comparison between SGL and SimGCL.
et al. 2020 Feng et al. [2020]
Chen et al.
2020 have been proposed to mitigate the performance degradation caused by increasing the number of layers. However, this is based on the premise that the better- performing model retains the information of the pooter-perfor peofer-performing model. Our periments prove that this premise is wrong. According to the experiments in section 2. we found that the information learned be- tween different layers of the same model independent. Therefore, it's naive to think that simply retaining the information between different layers improves performance without mod- ifying the model structure. We designed a method that is model-agnostic and can be applied to any graph-based model that consists of user and/or item embedding. More importantly, no trainable pa- rameters are introduced. Its structure is shown in Figure Ta). We will describe the detailed process the next paragraph.
Yelp2018
Las:FM
Amazon: Book
Recall
NUOG
Kocall
NDCG
Kecall
NDOG
Layer-1
(L0723
0.0722
8,0700
0.125
0,0702
Layer-2
0.0773
0.0508
0.0834
0.0731
0.1431
0.0803
Layer-3
0.0796
0.0718
0.4881
Fusion
0,0799
0.0522
0
0.0810
0.1599
0.0895
Inpri
0.38%
0.35%
10.97%
10.81%
7.97%
8,35%
Layer-1
0,0789
0.0815
0.0990
0.0501
0.3445
0,0777
Layen-
0.0824
0.0541
0.0701
0.0611
0.1500
0.0815
SinGCL
Layer-3
0,0828
0.0543
0.0999
0.0501
0.1550
0.0845
Fusion
0,0839
0.0550
0.0763
0.0669
0.1603
0.0869
Impri
1.33%
1.29%
8.84%
9.49%
3.42%
First, we need to separately train models with different numbers of layers (thre models are trained using the green box frame shown in the Figure 4 with one, two and three layers stacked respec- tively). Second, each model can get an embedding for a user and an item, and we multiply them to get the score. Three models get three different scores. Since the range of scores obtained by different models may be different, we apply maximum and minimum normal- ization to limit the scores to the range of 0 to 1. Finally, we fuse the different scores using weighed summation and define that the value of weight is positively related to the performance of a single model.
Figure 2: Components of the fused model
In order to verify the effectiveness of this method, we use the model trained in section 2. and then set the weights of the three mod- els be 1. The experimental results are shown in Table 2 where 8 8mpp denotes the relative improvement of the best performing method(hold) over the strongest models(underlined) excluding our method. Our method achieves the best performance. From the point of datasets, our method has obvious improvement effect in Last-FM and Amazon-Bood Book.. From the point of the model, SGL has obvious improvement. This illustrate the feasibility and effectiveness of our method.
Figure 3: Components of the fused model(
We conduct two experiments to further explore how our method fuses information from different layers and whether new information is
4

---

Under review as a conference paper at ICLR 2023
May
Lower
Mtree
Kihee-OWeed Choigall Emsacenth
ge Laree
0.0.0
0
Figure 4: Illustration of the proposed KGSF framework.
generated. The first experiment calculates the Intersection@20
between the fused model and model from different layers. The results
shown in Table in Appendix A the Fusion column indicates that the single model on the left column is the base, and the Fusion= column indicates that the Fusion model is the base). In the second experiment, statistics are made on which model from different layers the fused model information comes from. The results are shown in Figure 2 and Figure
From the results, we can get the following conclusions: (1) We can find that the information shared by three models accounts for the highest proportion after fusion, accounting for about 45% to 69%. New information accounts for the least after fusion, accounting for about 0.5% to 3.9%. shows that the cross-layer fusion mechanism tends to preserve the information shared by three models and will generate a small amount of new information. (2) In terms of the proportion of retained information, the information shared by the three models are larger than those shared by the models (about 4.4% to 14.2%), and the latter is larger than the unique information a single model (about 2.2% to 10.0%). This phenomenon is the same as ensemble learning. (3) The retention rate of the method for a single model information is about 78% to 93%, which means that can not retain all the information of the three models. However, the amount of information lost less than the sum of the unique information retained and the new information generated, so a better effect is achieved.
3 METHODOLOGY
In this section, we introduce our proposed Knowledge Graph-based Separate and Fusion model (KGSF) in detail. First, decoupling the UIA which used by (Wang al.) 2019c 2021) into three graph including UI Graph, UA Graph and IA Graph. Different signals can be found in each graph. Item-based collaborative filtering signal(i.e., item-user-item co-occurrence) can be found in UI Graph, attribute-based collaborative filtering signal(i.e., attribute-user-attriine can be found in UA Graph and content signal(i.e., items with similar attributes are similar) can be found in IA Graph. We designed three different signal extractors to mine information. Finally, we designed fuser to fuse different signals. It is important to note that this is not an end-to-end framework. Different signal extractors are trained separately.
co-occurrence)
3.1
ITEM-BASED CF-SIGNAL EXTRACTOR
It hot filed to extract collaborative signals in UI Graph for recommendation. Lots of work (Wang
ct al.
2019d; He et al.] 2020 Wu et al. 1021 Yu et al. 2022 Wang et al., (0220b) has shown that
GNN
can effectively extract item-based collaborative signals. Here we adopt the LightGCN (He)
al.
2020), which is typical and effective model. Its graph convolution operation is defined as:
Curru
(k+1)
Σ
1
(K) Gurah
(++1) Gurs
Σ NEN
(R) Gursu
(1)
/
N
V
/N
where GuraD Gerza E Rdder are embedding of item and user u at layer k, deu, is embedding dimensions, specially Gorr and Serau are ID embedding (i.e., trainable parameters). We define
(k)
(k)
NGOs {u| (u, 1) E Gur} to represent the set of users u who have interacted with item i and
5

---

Under review as a conference paper at ICLR 2023
Ng = {i (u. E Gui) to represent the set of items that have interacted with user Each layer will get embeddings corresponding to items and users. After K layers stacking, LightGCN further combines the embeddings obtained at each layer to form the final representation of user (k) (an item): efu- = से Σx Gurz
Gur L
K+1 Ex (k)
Guyan
According to the conclusion in section 2.2 there are differences in the information learned by the models that stack different layers. Therefore, we first train models that stack different layers sepa- rately, and then use a cross-layer fusion mechanism for fusion.
3.2 ATTRIBUTE-BASED CF-SIGNAL EXTRACTOR
One of the purpose of introducing KG into recommen- dation system is to extract attribute-based collabora- tive filtering signals. Experiments (Wang et al. 2019c 2021) show that such collaborative signals exist in UIA Graph. As shown in Figure [5] a possible connec- tion path from user u2 to item can be expressed as like U2 - 12 " a3 3 i4. Each node in this path contains the information of other nodes (e.g.. the information of node i2 may contain the information of node @2, ag express that user U2 like items containing
1
Figure 5: High-
High-Order Order Connectivity
and 01). But the meaning of introducing this path attribute ag, so this path will introduce noise that is hard to estimate. In order to extract this collab- orative signal of users more efficiently, we put the extraction of this collaborative filtering signal on like
the UA Graph, as shown in Figure 5 At this point, the path can be simplified to u2 -> This removes iz from the original path to avoid introducing unnecessary information, resulting in a purer embedding.
03 23 i4-
Here, as shown in Figure d) we also apply the LightGCN to extract the collaborative filtering (U)
(0) and GGu in layer O, signal from the UA Graph. User and attributes are represented by egu A:t.
they are ID embedding (i.c., trainable parameters). According to Formula I we can easily define the graph convolution operation in UA Graph as follows:
(k)
(k) GUA
(k+1) Coaa
Σ
(k+1) Coai
Σ GENSUA
(2)
V V|NrseN
GENOUA
where A Au E Rd dou is embedding dimensions. We define Ngea = ful(u, e
(R)
G to represent the set of users who have interacted with attribute and AfSua {al(u, a) A to represent the set of attribute a that have interacted with user u. Each layer will get embed- dings corresponding to attributes and users. After K layers stacking. LightGCN further combines the embeddings to obtained at each layer to form the final representation of a user or an attribute:
K eGua K+1 Σ Gorssa eaeA K+1 Σk=o C Now have the embedding of user and attribute which contain collaborative signals between users and attributes. In the task descrip- tion, we talked about getting the user's rating for each item, thus we also need to give the definition of item embedding: &GUA EuEN.GA CuAA, Likewise, we first train models that stack different layers separately, and then use a cross-layer fusion mechanism for fusion.
(k)
3.3 CONTENT SIGNAL EXTRACTOR
The second reason for the introduction of knowledge graph is to provide richer information for the embedding of items. The content signal extractor we designed consists of two components. The first is the knowledge graph embedding layer, shown in Figure Tc). which uses the knowledge graph embedding method like RotatE (Sun et al. 2018a) to obtain the attribute embedding. The other is the user interest mining layer, as shown in Figure 4 f), its input is the attribute embedding obtained by the first layer, and then using the user as the query vector and the attribute as the key vector, and finally the embedding of the user and item obtained, denoted as eGra and Gr. respectively. For a detailed introduction to these tow layers, please refer Appendix D
i,u

---

Under review as a conference paper at ICLR 2023
3.4 SIGNAL FUSION MECHANISM
In the previous three subsections, we proposed three different signal extractors. Instead of con- catenating between the three different items embedding and users embedding, train three dif- ferent signal extractors separately because the three extractors are independent of each other. Here, SA are used represent the scores of item-based CF-signal extractor, attribute- CF-signal extractor and content signal extractor, respectively, where S a S E R M|x[z], As shown in Figure b), since the three different scores have different ranges, we use max-min normalization to constrain their range between 0 and denoted as StS SU and S1 respectively. The final score is defined as 5 = TT. + SU + 72S where To: T 72 E (0, 1]. The value of To, T1, 72 depend on the performance of the three signal extractors. The stronger the performance of the signal extractor, the greater the corresponding weight.
3.5 MODEL PREDICTION AND MODEL OPTIMIZATION
In section
3.4 we mentioned that each signal extractor is trained separately. so within each signal extractor, user u's rating for item i is defined the dot product of the corresponding embeddings. The following are the score definitions for each of the three signal extractors.
gGul
-U. Eu Gos,
Guy = eu 9T AUAAGGA
e,A,
GIAA eu
GiA 2,
(3)
should be noted that the scores for item-based cf-signal extractor and attribute-based ef-signal extractor here refer to model, shown in the green box in Figure
Here, we opt the BPR RRedlle et al. 20122 loss to optimize our model. Since the three signal extrac- tors are trained separately, give the loss functions of the three signal extractors in Appendix E should noted that the training method in content signal extractor to train knowledge graph embedding layer first, and then train interest mining layer alone after the training of knowledge graph embedding layer is completed.
4 EXPERIMENTS
We present empirical results to demonstrate the effective- ness of our proposed KGSF framework. The experiments are designed to answer the following research questions: RQ1: How does KGSF perform, comparing the state-of-the-art knowledge-awar aware recommend models? RQ2: How do different components of KGSF (e.g., the attention guiding mechanism, the cross-layer fusion mech- anism, the independence and completeness of three signal extractors, effectiveness of signal fusion mechanism) affect the performance of KGSF? RQ3: Can KGSF provide meaningful insights on user intents and give an intuitive impression of explainability? Only some experiments and results are shown here. Please refer to Appendix F for more details.
Table 3: Overall performance comparison.
Yelp2018
Last-FM
Amazon-Baen Book
Recall
NDCG
Recall
NDCG
Recall
NDCG
MF
6.27%
4.13%
7.24%
6.17%
13.00%
6.78%
CKE
6.53%
4.23%
7.32%
6.30%
13.42%
6.98%
KGNN-LS
6.71%
4.22%
8.80%
6.42%
13.62%
5.60%
KGAT
7.05%
4.63%
8.73%
7.44%
14.87%
7.99%
CKAN
6.46%
4.41%
8.12%
6.60%
14.42%
6.98%
KGIN
6.98%
4.51%
9.78%
8.48%
16.87%
9.15%
KGCL.
7.54%
4.92%
6.26%
5.64%
14.86%
7.84%
HAKG
7.78%
5.01%
10.08%
9.31 %
KGSF(uur
8.59%
5.60%
10.76%
9.22%
17.26%
9.46%
4.1 SETUP
Datasets, Baselines and Evaluation Metries. We use three benchmark datasets: Amazon-
Last-FM and Ye1p2018, which are extensively evaluated by the SOTA methods (Wang et al., 2019cc
Du et al. 2022 Yuhao et al.) 2022) and vary in iterms of domain, size and sparsity. Table presents the overall statisties information of our experimented datasets. We adopt two widely-
2021
evaluation protocols Krichene & Rendle (2022) recall@ and ndcg K. where K=20 by default. To demonstrate the effectiveness of KGSF, we compare it with the state-of-the-art methods, covering
7

---

Under review as a conference paper at ICLR 2023
KG-free method (Rendle et al. 2012). embedding-based method (Zhang et al. 2016). propagation-
based methods (Wang et al. 2019a 9a c 2021 2020c) and multiview based methods (Du et al. 2022
Yuhao et al. 2022). See Appendix F2 for more details.
4.2 PERFORMANCE COMPARISON(RQU)
We report overall performance evaluation of all methods in Table 3 and performance comparison of
Table 4: Performance comparison of KGSF components.
KGSF components in Table 4 (UT means item-based cf-signal extractor, UA means attribute- based cf-signal extractor, IA means content signal extractor). should noted that dataset Yelp2018 and Amazon-Book apply SimGCL (Yu et al.] 2022) in UI components, dataset
Yelp2018
Last-FM
Amazon-Boote
Recall
NDCG
Recall
NDCG
Recall
NDCG
UI
8.39%
5.50%
9.31%
8.10%
16.03%
8.68%
UA
3.08%
1.88%
9.13%
7.30%
7.46%
4.00%
IA 2.82%
1.72%
6.01%
4.69%
4.74%
2.27%
UI&UA
8.56%
5.59%
10.35%
8.83%
16.76%
9.05%
UL&IA
8.44%
5.52%
10.36%
8.98%
16.71%
9.05%
UA&IA
3.34%
2.04%
9.91%
8.10%
8.59%
4.66%
Last-FM ap-
plies SGL (Wu et al.
2021) in UI component. From the results, we summarize the following
observations:
(1) KGSF consistently yields the best performance on the dataset, except for ndeg metric on the Last-FM dataset. particular, achieves signif- icant improvement even over the strongest baselines w.r.t. recall @ 20 by 10.41 %, 6.75% and 2.31 Yelp2018, Last-FM and Amazon-Bonn Book, re- spectively. This demonstrates the rationality and effectiveness of KGSF. We attribute these improvements to fusion mechanism and the decou- pling of graphs. (2) Jointly amalyzing the performance of KGSF across the three datasets, we find that the improvement on Yclp2018 dataset is significant than that other dataset. One possible reason is that collaborative signal is more significant to the dataset than the con- tent siganl. This confirms that KSGF has strong adaptability to differ- ent types of signals.
Figure 6: Components of the KGSF.
(3) From the perspective of a single component of KGSF, UI component work best, and its performance existing SOTA methods that introduce KG. Its performance on Amazon Amazon-Book dataset higher than the SOTA methods except KGIN. Therefore, the proportion nals in these three dataset is higher than that of the other two signals. If the KG can not be modeled correctly, the introduction of it may weaken the collaborative signal and introduce noise, such as learning unified graph structure in KGAT and KGIN, and using gating mechanism in HAKG to fuse the decoupled embedding, etc.. (4) From the fusion of any two components of KGSF, there no performance degradation and the performance is better than that of single component. The fusion performance of UI component and UA component is better in most cases. This proves that the introduction of UA graph is effective and it can extract information that not be extracted by Ul graph. The effectiveness of the fusion mechanism also illustrated.
Yelp2018 dataset is much higher than the
fised collaborative filtering sig-
4.3 STUDY OF KGSF(RQ2)
In these section, we present part of our results to explore KGSF, We refer readers to Appendix F for more experiments.
Independence
Table 5: Intersection @20 between different components.
of three sig- nal extractors.
Yelp2018
Last-FM
Amazon- Amazon-Book
We judge the
UA
IA
U1
UA
IA
UI
UA
IA
UI
independence
UA
1,0000
0.2008
0.3178
1.0000
0.2750
0.5314
1,0000
0.2694
0,5052
between each
IA 0.2320
1.0000
0.3358
0.5335
1.0000
0.5334
0.4389
1.0000
0.4833
signal extractor,
U1
0.1198
0.1104
1.0000
0.4941
0.2570
1.0000
0.2366
0.1417
1.0000
that is, whether
the information
8

---

Under review as a conference paper at ICLR 2023
extracted by each signal extractor is different. We adopt Intersection @ @20 measure independence. Specifically, we calculate the Intersection 20 between the different signal extractors that have been trained. The results are shown in TableB5 (the value of cach row is based on the extractor, e.g., 0.2008 in the Yelp2018 indicating that UA component is used as the base, that is WVAnAANT] 0.2008). [WAAz]]
We observe: (1) Overall, the value of Intersection between any two component is less than 0.54, indicating that least 46% of the information extracted between any tow signal extractors is different. It proves that one of the three signal extractors are independent.
(2) Comparing the three datasets, it can be found that the three extractors have the highest independence on Yelp2018 and the lowest independence on the Last-FM. Comparing the performance of each extractors in Table we find that the performance of UA component and IA component is the worst on Yelp2018, and the performance is the best on Last-FM. Therefore, the independence between individual signal extractors is negatively correlated with the performance between individual signal extractors. (3) In the same dataset, comparing each row, we can find that the independence among the three signal extractors is "IA&UA", "UI&IA" and "UI&UA" in order from strong to weak. At the same time, combined with the results in Table 4 it can be found that the effect of fusion is also in this order from low to high in most cases. This shows that the stronger the independence, the less the effect will be improved after fusion.
Effectiveness of Signal Fusion Mechanism. We will explore the preservation effect of the signal fu- sion mechanism and whether new information is generated. The exper- imental step is to select benchmark method and then analyze the com- ponent proportions of the bench- mark method to the three indepen- dent signal extractors. The bench- mark method selected here is KGSF. The experimental results here are shown in Table 6 and Fig- ure [(c.. From the results, we summarize the following observation: (1) From the perspective of retention ability, the retention ability of UI component in all datasets is the strongest (about 79% ~ 91%), followed by UA component (about 42% ~ 78%), and the IA component is the worst (about 39% 68%). One possible reason is the setting of 7; in the signal fusion mechanism. Within a certain range, the higher the value, the better the retention effect for this component. (2) According Figure 6 we find that about 6% 9% of the new information generated in the three datasets. It shows that the signal fusion mechanism will generate new information.
Table 6: Intersection @ 20 between KGSF and three compo- nents.
Yelp2018
Last-FM
mmazonnBook
KGSF
KGSF*
KGSF
KGSF*
KGSF
KGSF
UI
0.9076
0,8884
0.7874
0,6827
0.8547
0.8147
UA
0.4155
0.1544
0.7773
0.6293
0.6949
0.3606
IA
0.3920
0.1264
0.6777
0.2866
0.6443
0.1757
RELATED WORK
Existing recommendation methods that introduce knowledge graph can be mainly classified into four categories, namely, embedding-basee methods, path-based methods, propagation-base -based methods and multiview-based methods. We give brief introduction in the Appendix G
6 CONCLUSION AND FUTURE WORK
In this paper, we first propose the Intersection metric to measure the relationship between differ- ent models. Experiments verify that the relationship between different collaborative filtering (CF) methods using GNN is shown in [ In addition, the relationship between models using the same CF method but stacking different layers also the same. We then design a model-agnostic cross-layer fusion mechanism that does not introduce any training parameters, and conduct extensive experiments on three real datasets to demonstrate its effectiveness. Then we analyze the current chal- lenges of introducing KG recommendation methods, and design an extensible KGSF framework to improve the recommendation performance through independent signal extractors and fusion mech- anisms, and demonstrate the performance of KGSF on three datasets effectiveness. Experiments show that direct fusion between different signals will interfere with each other, and the current dif- ficulty is how to effectively fuse different features. In the future, we will explore whether the above problems exist in other tasks using GNN, object detection in computer vision, etc., and how to design an efficient, end-to-end feature fusion mechanism.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. Learning heterogeneous knowledge base embeddings for explainable recommendation. Algorithms, 11(9):137, 2018.
Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. Unifying knowledge graph learning and recommendation: Towards better understanding of user preferences. In The world wide web conference, pp. 151-161, 2019a.
Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. Unifying knowledge graph learning and recommendation: Towards better understanding of user preferences. In The world wide web conference, pp. 151-161, 2019b.
Rose Catherine and William Cohen. Personalized recommendations using knowledge graphs: A probabilisticc logic programming approach. In Proceedings of the 10th ACM conference rec- ommender systems, pp. 325-333-3333 322-332, 2016.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph con- volutional networks. In International Conference on Machine Learning, pp. 1725-1735, PMLR, 2020.
Yuntao Du, Xinjun Zhu, Lu Chen, Baihua Zheng, and Yunjun Gao. Hakg: Hierarchy-aware knowl- edge gated network for recommendation. arXiv preprint arXiv:2204. 224222944.4999 2022.
Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi-supervised learning graphs. Advances in neural information processing systems, 33:22092-22103, 2020.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. Journal of Machine Learning Research, 2010.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastie estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings, 2010.
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 639-648, 2020.
Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S Yu. Leveraging meta-path based context for top-n recommendation with neural co-attention model. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1531-1540, 2018.
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. Improving se- quential recommendation with anowledge-enhaneee memory networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 505-514, 2018.
Jiarui Jin, Jiarui Qin, Yuchen Fang, Kounianhua Du, Weinan Zhang, Yong Yu, Zheng Zhang, and Alexander Smola. An efficient neighborhood-baae based interaction model for recommendation on heterogeneous graph. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 75 75-84, 84, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv: Learning, 2014.
Walid Krichene and Steffen Rendle, On sampled metrics for item recommendation. Communica- tions of the ACM, 65(7):75-83, 2022.
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence, 2015.
10

---

Under review as a conference paper at ICLR 2023
Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards deeper graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp- 338-348, 2020.
Weizhi Ma, Min Zhang. Yue Cao, Woojeong Jin, Chenyang Wang, Yiqun Liu, Shaoping Ma, and Xiang Ren. Jointly learning explainable rules for recommendation with knowledge graph. In The world wide web conference. pp. 1210-1221, 2019.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thiemme Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint 1rXiv:/205.2611 2012.
Yu Rong. Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification, arXiv preprint 1907 10900, 2019.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg. Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European semantic web conference, pp. 593-607. Springer, 2018.
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. Rotate: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations, 2018a.
Zhu Sun, Jie Yang, Jie Zhang, Alessandro Bozzon, Long-Kai Huang, and Chi Xu. Recurrent knowl- edge graph embedding for effective recommendation. In Proceedings of the 12th ACM conference on recommender systems, pp. 297-305, 2018b.
Ke Tu, Peng Cui, Daixin Wang. Zhiqiang Zhang, Jun Zhou, Yuan Qi, and Wenwu Zhu. Conditional graph attention networks for distilling and refining knowledge graphs in recommendation. In Pro- ceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1834-1843, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
Chenyang Wang, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Make it a chorus: knowledge-and and time-aware item modeling for sequential recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 109-118, 2020a.
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM international conference on information and knowledge manage- ment, pp. 417-426 2018a.
Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. Dkn: Deep knowledge-aware network for news recommendation. In Proceedings of the 2018 world wide web conference, pp. 1835- 1844, 2018b.
Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. Knowledge-aware graph neural networks label smoothness regularization for recommender systems. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining, pp. 968-977, 2019a.
Hongwei Wang. Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. Knowledge graph convolutional networks for recommender systems. In The world wide web conference, pp. 3307-3313, 2019b.
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 950-958, 2019c.
Xiang Wang. Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165-174, 2019d.
11

---

Under review as a conference paper at ICLR 2023
Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. Explain- able reasoning over knowledge graphs for recommendation. In Proceedings of the AAAI confer- ence on artificial intelligence, volume 33, pp.
2019e.
Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. Disentangled graph collaborative filtering. In Proceedings of the 43rd international ACM SIGIR conference on research and development information retrieval, pp. 1001-1010, 2020b.
Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. Learning intents behind interactions with knowledge graph for recommendation. In Proceedings of the Web Conference 2021, pp. 878-887, 2021.
Ze Wang, Guangyan Lin, Huobin Tan, Qinghong Chen, and Xiyang Liu. Ckan: collaborative knowledge-awee attentive network for recommender systems. In Proceedings of the 43rd In- ternational ACM SIGIR conference on research and development in Information Retrieval, pp. 219-228, 2020c.
Zhigang Wang, Juanzi Li, Zhiyuan Liu, and Jie Tang. Text-enhanced representation learning for knowledge graph. In Proceedings of International joint conference on artificial intelligent (IJ- CAI), pp. 4-17, 2016.
Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. Self- supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp. 726-735, 2021.
Junliang Yu. Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Pro- ceedings of the 45th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, pp. 1294 1303, 2022.
Yang Yuhao, Chao Huang. Lianghao Xia, and Chenliang Li. Knowledge graph contrastive learning for recommendation. SIGIR, 2022.
Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowl- edge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 353-362, 2016.
Lingxiao Zhao and Leman Akoglu. Pairnorm: Tackling oversmoothing in gnns. arXiv preprint
arXiv: 12223, 2019.
Ding Zou, Wei Wei, Xian-Ling Mao, Ziyang Wang, Minghui Qiu, Feida Zhu, and Xin Cao. Multi- level -rews-view contrastive learning for knowledge-awee aware recommender system. arXiv preprint 200
12

---

Under review as a conference paper at ICLR 2023
A
PARTIAL FIGURES AND TABLES
Table 7: Statistics of the datasets.
Yclp2018
Last-FM
Amazon-Boo
#Users
45,919
23,566
70,679
#ltems
45,538
48,123
24,915
#lnteractions
1,185,068
3,034,796
847,733
#Entities
90,961
58,266
88,572
#Relations
42
39
#Triplets
1,853,704
464,567
255,746
Table 8: Intersection @20 between fusion, SGL and SimGCI.
Yelp2018
Last-FM
Amazon-Book
Fusion
Fusion
Fusion
Fusion *
Fusion
Fusion *
L-1
0.8871
0.8138
0.8714
0.7341
0.9032
0.7051
SGL
L-2
0.9026
0.8800
0.8633
0.7177
0.9009
0.8097
L-3
0.8396
0.8442
0.7779
0.6992
0.8414
0.7789
L-1
0.8746
0.8192
0.8461
0.7578
0.9106
0.8282
SimGCL
L-2
0.9065
0.8897
0.9004
0.8107
0.9343
0.8660
L-3
0.8910
0.8779
0.8940
0.7487
0.9151
0.8797
B PROBLEM FORMULATION
We first introduce the data structures related studied problem, and then formulate our task.
Table 9: Different relations of Intersection
Intersection@NNNe Intersection @ N(M2)
Relation(M1I
Figure (a)
1
Similar
User-Item Bipartite Graph (UI). In this paper, we focus on learning user's preference from im- plicit feedback including click and purchase. We define as = {t}. The user-item bipartite graph is defined as Gur = {(u, i)lu E U, E I}. has interacted with item the pair (u, i) will be in Gur.
as l {u} and item set
Knowledge
Graph(IA). KG stores many real facts, which can press the relationship between entities. They are usually stored in the form of triples. of triplets, E as set of entities, and R as set of relations. Let GIA = {(h, r, h,, E E,r E R} be a collection of triplets, where each (h, r,t) E T For exam-
We define T
means that there is relation between head entity and tial entity ple, triplet (Wolf Warriors, director, Jason Wu) indicates that the movie Wolf Warriors is directed by Json Wu. Here assume that all items appear in KG as entities (i.e., I E), which is the common assumption of all existing knowledge-aware recommendation systems. We connect the items in user-item graph with entities in KG to offer auxiliary semantics to interactions.
User-Attribute Bipartite Graph(UA), User-Attribte bipartite graph is actually a combination of
User-Item bipartite graph and Item-Attribute graph. Let A {ala E, & I} be collection of attributes, which is a supplement to items. Let GU {(u, a)lu E U a e A, (u, Gui, (i, GIa). If user u has been interacted with item and relation
13

---

Under review as a conference paper at ICLR 2023
of item i is attribute a, the pair (u, a) is in the User-Attribte bipartite graph G,U For example, if user u has watch the movie Wolf Warriors, and the director of it is Json Wu, so it is said the user u has interacted with Json Wu, which can be expressed by (u, Jason Wu).
This is the same the Collaborative Knowledge Graph definition in KGAT (Wang et al. 2019c).
Inersection@N, @N. This is a new metric we propose to measure the difference between two models. Take a user as an example, we consider two models, they will make two ordered lists of items sorted by rating for the user u. We remove the ones that appeared in the training set from these two ordered lists and then take out the top N constituent sets, which are recorded
M, and M2 respectively. We denote the test set as T. So let Intersection (M.)
[M, [M
where the number of elements in the set. The meaning of it is shown in
Table
Task Description. Given a user-item graph Gur. user-attribute graph U a and item- aett-attibbute graph IΛ our task is to predict how likely that a user would adopt an item that she or has never engaged with.
C THE RELATIONSHIP BETWEEN SIMGCL AND SGL IN THE SAME LAYER
Table 10: Performance comparison of SGL, SimGCL and their Fusion
Yelp2018
Last-FM
Amazon-Book
SGL
SimGCI. *
SGI.*
SimGCI. *
SGL.* *
SimGCL *
SGL-1
SimGCL-1
0.6786
0.6231
0.5214
0.5742
0.6767
0.5834
SGL-2
SimGCL-2
0.7243
0.6762
0.4965
0.5308
0.7045
0.6768
SGL-3
SimGCL-3
0.7219
0.6925
0.5080
0.6320
0.7376
0.7045
First, we introduce SGL (u et al. 2021) and SimGCL Yu al. 2022) briefly. SGL proposes three graph argumentation including CoWNod--Drop) ED(Edge- Drop) and RW (WRnddomm- Walk) and then applies the InfoNCE (Gutmann & Hyvärinen 2010) loss function to maximize the similarity of the same node and minimize the similarity of different nodes. SimGCL believes that the graph argumentation of SGL is not necessary. It adopts a simpler method, which is adding noise to nodes obtain different representations of the same node. Finally InfoNCE is also applied.
Then, we conducted two different experiments under the same conditions (such as dataset, batch size, etc.). should be noted that graph argumentation method adopted by SGL in the experiment is ED because of its best performance. The first experiment is using recall @ 20 and ndcg @ 20 metrics test the performance of SGL and SimGCL and the results are shown in Table 2 The second experiment is using Intersection @2 metric to measure the similarity of the two methods in the same layer. The results are shown Table 10 ("Take 0.6786 in the dataset yelp2018 as an example, it is represented in the first layer and based on SGL, the similarity between them is 0.6786, that is [SGLnSimGCLLT] 0.6786). |SGLAT
From the result, we find that SimGCL. performs better on Yelp2018 and Amazon-Bouu datasets. SGL is better on the Last-FM dataset. In addition, we observe that in Yelp2018 8 and Amazon Amazon-Book datasets, the similarity between the two methods around 70% in any layer, while the similarity in Last-FM dataset is around 50%. This means that they learned a part of the information that the other did not. From the observation, we can draw the following conclusions: is not improvement over SGL and the relationship between them is intertwined shown in Figure (a), rather than inclusive or juxtaposed. (2) Both graph argumentation and node perturbation necessary. This contradicts the conclusions of the SimGCL. (Yu et al. 2022).
(1) SimGCL
14

---

Under review as a conference paper at ICLR 2023
D CONTENT SIGNAL EXTRACTOR
Knowledge Graph Embedding Layer. the Graph, the movies Wolf Warrior, Wolf Warrior and Mermaid are independent of each other three ID embeddings. The introduction of knowledge graph tells us that Wolf Warrior and Wolf Warrior 2 are more similar than Wolf Warrior and Mermaid, because the former has the same director, genres and actors, while the latter does not. A large number of methods such as TransR (Lin et al. 2015), RotatE (Sun et al. 2018a), etc. have excellent performance in capturing this similarity.
Here we apply the RotatE, which projects entities and relations to the complex plane (i.e., h,t E Cdesa, de,a is embedding dimension). Specifically, given a triple (h, t) GIΛ, the scoring function d(h, r, t) will be as high as possible. the triple (k', r, t²) $ GIA, the scoring function d(h', r, (') will be low as possible. The scoring function is defined as d(h,r,t) = -Ih o t||, where o is multúplication in the complex field, || is LI norm. RotatE's training method is to
consider that the score of valid triplets is higher than broken triplets. Its loss function is defined as follows:
n
C = d E p (16 r, ()) log o (-d (1,, -))
(4)
exp ad (h)y.
(5) where y, a is pyppee,aramettr, (-)) is sigmoid function, (h, () E 9TA, (h, 1) & Gra. The purpose of introducing p here is to distinguish easy negative samples from difficult negative samples, thereby improving performance.
r, | (1,
Ex r,
After training, we will get the embedding representation of the entity eç, where eș E E. Since A C E, we take out the entity that satisfies eq E AA in the entity set, denoted as e. Beacuse E Cd contains two parts, real part and imaginary part, which we denote as Re(e) and Im(e) respectively. For the convenience of representation, we denote the embedding of entity e as e Re(e) @ (m), where @ is vector concatenation and eGra E Re X2
User Interest Mining Layer. If user u likes the item i, must have intention. KGIN (Wang al, 2021) defines intent as a set of relations, which is coarse -rained. To explore more fine-gramed intent, we not only want to know which relationships users are more interested in, but also which attributes users are more interested in. The attention mechanism is the key to realizing this idea. Our idea about attention mechanism is based on Transformer. We take the user as the query vector and the attribute as the key vector to obtain the user's weight for each attribute, and finally aggregate the attributes to obtain the embedding of items. The network architecture of user interest mining layer is shown in Tigure<Kf)
The inputs to this layer are eQta and other where CiA e other Rd x2_ The reason for in- Cia is that not all information can be provided in IA Graph (e.g., the IA Graph only troducing other provides the director and actor information of a certain movie, but the user likes the movie because of the genre of the movie, so introducing the other attribute can solve this situation).
Considering the same attribute has different relationships with the same item (e.g., the di- rector and actor of Wolf Warrior are both Jason Mu). In order to model it, we de- fine R'' = (id(r) (h,, r) I E J. E A) U {id(-r) (h,r, t) E GIA. E A, A E GTA), where id(r) represents generating a unique, O-based number for relation r. We also define r(a, i) E R', which represents the relation id between item i and attribute where E AU (other), i E J. Especially, r(other, = |R']. It should be noted that if there is no special description, symbol Clia includes e 91
After introducing the introduced symbols, we begin introduce the user interest mining layer in de- tail. According to Transformer (Vaswani et al. 2017]. TransR (Lin et al. 2015) RGGGN (Schlicttrril] al. 2018), we first give the definition of the value vector and key vector of the attribute:
Keya- = Wr((a,), &ra, Value, = (rr(a.)) value Gto,
(6)
key
where a E AU (other), i J, wrlg. E (do x2), R Value, E R2, lo and are the dimension of key vector and value vector of the attribute respec-
(-r(a.))
value E nx "do x2), Keya E
15

---

Under review as a conference paper at ICLR 2023
tively.
Then define query vector and value vector of user as:
Querya gaueryeu Sia Value, W Wealue Sra
where e Rd,, W Wquery E RRxxd Wnatue R" eia is trainable parameters.
IR IRud Queryu R Value,
Here we define No.i = {a (u, i) sent the set of attributes of item i that user have interacted with. Then user u's attention to each attribute of item is defined as
Gur, (i,r,a) E GIA. EE,E,i E A.r E R) to repre-
Attention Softmax ((Query, T Keyo Nu,s))
We multiply the attention by the value vector of the corresponding attribute to model the user's attention to the attribute. Different from the previous two extractors, the embedding of the item hear is related to the user, that is, the embedding of the same is different in different users (e.g,, given the item i, user u1 and u2 have different degrees of attention to the attributes of the item i, that is, the attention is different, the embedding of the item i will also be different). We define the embedding of item as:
Σ ff (Attention,,.()) Value,) ff(ee) WW (LeakyReLU (W]re)). aENu.
(9)
where W3, Roxx [epx,,W], E ERuxP the dimension of hidden layer, Leaky is activation function. Finally, we define the embedding user u as e = Value,. Since we added the other attribute, hope that its attention and other attribute attention mutually exclusive (i.e., hes'''s attention are high, the other attribute low rather than as high as other). To constrain this relationship, introduce the following loss terms (i.e., guiding attention mechanism):
Allenttion,sooleeei 14)2
Lattention exp
(10)
20²
where j and a are papeer-rrameters... This formula is the probability density function of normal distribution. When other's attention value in the middle, its punishment will be greater. When other's attention value tends to both ends, the punishment will be small.
E OPTIMIZATION
This section shows the loss function of the three signal extractors. The first of which consist of BPR and regular terms, and the last one has one more attention constraint than the first two.
LUI = Σ
-In (Tuy
+ G ||090r]
-
(u.i)eGur
(a.i)
LUA=
Σ
Ino (oSUA (pega))
+ Goaa | 2
(1I)
(u.))eurr (u.jiSee
CIA= = Σ
Ino (BIA (811) +
attention Latteation +
Gri ||05ra
(u. (n.j&&er
where 0Ger { Cunas Soru I IE ,00a Ooca
(0)
(0)
Co Au a A, Au
(0)
and 0914 (eg GiA GiAA, Wrla.s) wr((a)) Querynn Value,, W1 W u,,(a, E R'} are the set model parameters. YGur, Xoua, APia and attention are hyper- parameters control the La regularization term and attention loss, respectively.
A,
Cis
16

---

Under review as a conference paper at ICLR 2023
F ADDITIONAL EXPERIMENTAL SETUPS AND RESULTS
F.1
BASELINES
MF
RRenlle et al. 2012) is typical matrix factorization method and doesn't use KG information. It uses ID embeddings of users and items to make the prediction in implementation.
CKE
(Zhang ct al. 2016) is a representative embedding method. It utilizes the TransR (Lin et al. 2015 to encode entities in the KG, which are then used as input to MF framework.
KGNN-LS
Waag et al. 2019a) is a propagation -aased based model, which converts KG into user- specific graphs, and then considers user preference on KG relations and label smoothness in the information aggregation phase, so as to generate the different representation of the same item under different users.
KGAT
(Wang al. 2019c) is propagation- praeatioo---aedd recommend model. It applies unified relation- aware attentive aggregation mechanism in UIA to generate user and item representations. (Wng et al., 2020c) is based on KGNN-LS, which utilizes different aggregation schemes the user-item graph and KG respectively, to mine knowledge association and collabora- tive signals.
CKAN
KGIN
(Wan et al. 2021 a state-of-the-art propagation-base method, which models user inter- action behaviors with latent intents, and proposes relation-aware information aggregation scheme to capture long-range connectivity KG.
HAKG
(Du al. 2022) is also state-of-the-aa multiview- based method which embed users, relations and items in hyperbolic space and use a hyperbolic aggregation scheme. It learns from UI and IA graph generate collaborative signals and knowledge associations, and applies gating mechanism to fuse them.
KGCL
(Yuhao et al., 2022) is a general contrastive learning framework using knowledge graph augmentation schema. Besides, it leverages additional supervision signals to guide cross- view contrastive learning paradigm.
F.2 PARAMETER SETTINGS
For fair comparison, we fix the size of ID embeddings 64 (except that the embedding of the UA component on the Amazon-Book dataset is 128), the optimizer as Adam (Kingma & Ba 2014).
and the batch size as 4096 for all methods. The Xavier (Glorot Bengio 2010) initializer used
to initialize the model parameters, We consider learning rate lr E {10 1,103, 3,10 110- 3). To. E
(0.1, .... 1.0]. AGei, AGUA, GGIA E (10-3, 3,-0 10-4, 10 10-5), 5), = 0.5, = 0.15,, A CIA 10 p = 128.
attention
The parameters for all baseline methods are carefully tuned to achieve optimal performance. Specifi- cally, for KGAT [Wang et al. 2019c), we set the depth to three with the hidden size (64, 32, 16), and use the pre-trained ID embeddings of MF (Rendle et al., 2012) (ct al.] 2020c), KGNN-LS (Wang et al., 2019a). we set the size of neighborhood to 16; for KGIN (Wang al. (021), fix the number of intents to 4; Moreover, early stopping strategy performed for all methods, i.e., premature stopping recall 20 on the test set does not increase for 10 successive epochs.
initialization; for CKAN (Wang)
F.3 ADDITIONAL STUDY OF KGSF(RQ2)
Since graph decoupling, independent training and signal fusion mechanism are the core of KGSF, we conduct extensive experiments explore their effectiveness. Specifically, we first analyze individual components, including the attention guiding mechanism of the IA component and the cross-layer fusion mechanism of the UA component. then delve into the independence and completeness of each signal extractor.
Impact of Attention Guiding Mechanism in IA Graph. Here we verify the effectiveness of the attention guiding mechanism, thus design two variants of the IA component. A variant discards
the attention guiding mechanism, denoted "IA w/o G", where ) S7a retains the attention guiding mechanism, denoted "IA", where UIa attention
0. Another variant
pttention
1. The experimental
17

---

Under review as a conference paper at ICLR 2023
Table 11: Impact of Attention Guiding Mechanism.
Yelp2018
Last-FM
Amazon-Book
Recall
NDCG
Recall
NDCG
Recall
NDCG
IA w/o
2.69%
1.66%
5.81%
4.53%
4.70%
2.23%
IA
2.82%
1.72%
6.01%
4.69%
4.74%
2.27%
results are shown in Table 8. Degraded performance can be observed in all datasets, indicating the necessity of the attention guiding mechanism. Specifically, the attention guiding mechanism is not introduced, the model will preferentially optimize the Other embedding, resulting in the Other's attention close to other attribute's attention, which is not helpful for the interpretability of the model. Introducing the attention guiding mechanism make the model pay more attention to the infor- mation other attribute.
Table 12: Impact of the number layers L and Fusion.
Yelp2018
Last-FM
Amazon-Boon -Bok
Recall
NDCG
Recall
NDCG
Recall
NDCG
UA-1
2.88%
1.77%
8.19%
6.42%
6.97%
3.65%
UA-2
2.98%
1.82%
8.47%
6.75%
6.98%
3.79%
UA-3
2.64%
1.60%
7.86%
6.21% %
5.83%
3.00%
Fusion
3.08 %
1.88 %
9.13%
7.30 %
7.46 %
4.00 %
Impact of model depth and Cross-Layer Fusion Mechanism in UI component. Here, search for L in range 3, and then use cross-layer fusion mechanism to fuse them. We use "UA-i" represent UA component of the stacked layer and "Fusion" represent the model with the Our observations
cross-layer fusion mechanism. The experimental results are shown in Table 12 are follows:
In the three datasets, the performance of "UA-2" is greater than that of "UA-1" and the performance of "UA-1" greater than that of "UA-3". One possible reason for this is that the UA component can simplify high-order connectivity proposed by KGAT. In KGAT, it is necessary stack 3 to layers to achieve better results, while UA component can achieve better results after stacking layers.
According to the conclusion in Section 2.1 the information extracted by UA component at different layers is different. We applied the cross-layer fusion mechanism and found that the performance was better than that of a single model. The improvement was obvious the Last-FM and Amazon-Book datasets. This further verifies the existence of partial information independence in different layers and the effectiveness of the cross-layer fusion mechanism.
Table 13: Intersection @20 between KGIN and three components.
Yelp2018
Last-FM
Amazon-Boo
KGIN
KGIN
KGIN
KGIN
KGIN
KGIN
UI 0.4002
0.5588
0.5725
0.5900
0.5501
0.6144
UA
0.3852
0.2039
0.5212
0.4953
0.5318
0.2725
IA
0.3680
0.1699
0.5558
0.2734
0.5042
0.1617
Completeness of Three Signal Extractors. We judge the completeness of each signal extractor, that is, whether these three signal extractors extract all information after the introduction of KG. Our experimental setup is to select benchmark method and then analyze the component proportions of the benchmark method the three independent signal extractors.
The benchmark method selected here is the SOTA method KGIN (Wang et al. 2021). The experi-
mental results shown in Figure 6(b) and Table 13 The Inverse in the Table 11 means KGIN[9]
18

---

Under review as a conference paper at ICLR 2023
KGIN
base
base,Ui,U
base.UA
base,LUA,
0.30
Imnisscciongoa
0.25
0.10
Yelo2018
Last-FM
Amazon-Book
Figure 7: Components of the KGIN.
is the base, and the KGIN means the column element is the base. Taking 0.5588 as example, 0.5588 0.5588, We can observe as follows:
in Yelp2018 dataset means UUNKGINDE"
KGINGT
According to Figure 7 we find that about 30% of the information is not extracted by the three signal extractors. The UI component and UA component apply the cross-layer fusion mechanism. According to the conclusion in Section 2.2 the cross-layer fusion mechanism will lose some information. Therefore, we don't 't the cross-layer fusion mechanism, so the original signal base extractors are changed from three ("UI", "UA", "IA"" to seven ("UI- 1", "U1-2", "UI-3", "UA-1". "UA-2", "UA-3". "IA"). The experimental results show that about 25% of the information does not exist in the three signal extractors. This shows that the information the three signal extractors cannot extract all information, only convers 75% of the information. Therefore the three signal extractors are not completeness. A possible reason that the relationship is not preserved in the three signal extractors.
According to Table T3 we can find that the components KGIN (Wang et al., 2021) are UI, UA and IA in descending order, among which UI components accounts for about 60% the three datasets. From the perspective of UI component, its retention rate is somewhere between 40% 50%. This shows that the end-to-end method of KGIN cannot completely retain the collaborative filtering signal of user-item. The reason why the performance of KGIN] is better than collaborative filtering is that the discarded information is smaller than the new information learned.
F.4 EXPLAINABILITY OF KGSF (RQ3)
Benefiting from the separate modeling and fusion mechanism of KGSF, we can get more fine- grained explanations than KGIN. We randomly seleeted user usss and related item 44079 (from the test, unseen in the training phase), which book called Never Go Back. The interpretability obtained after visualizing it is shown in Figure g We have the following findings:
From Figure 8 a), we can find that the user U335 likes this book not because of the item- based collaborative signal, but because the attribute-based collaborative signal or content signal.
Figure b)) and Figure (c) are the user's overall preference and preference for a certain attribute of the item 14079 generated by the UA component, respectively. Figure S shows that after the dot product between the user and all attributes, we select the 10 attributes with largest values as the most favorite 10 attributes of users. We can find that all 10 attributes are writers, and the writing styles of writers tends to styles like thriller, crime, mystery, etc., which we can think of as user preferences. Figure (ic) shows the first three attributes obtained by normalizing the attributes of the item 24079 and the user through dot product.
19

---

Under review as a conference paper at ICLR 2023
Tack
Bate
FOX
LEEE CHILD
40%
1:323
Cries wclas
Child
9661
Aatbrr
Fag Schemas
Mystey Noodl
642405
27%
4728
Noloe Deliil
Acli adivatare and
JACK REACHER
Jack Rescher
Slobel
16
1386
Sabbrr
Hates Cotr
Mystery annel theilers
1998
0000
(198)
91530
Thiiker
WEVER 346K
10%
[E001]
Kore
Crter ection
Novel
Myltery
properting
pungustrig
LEE CHILD
34%
24%
JACK REACHER
Books ritree, warkkasthor
NEVER 60 BACK
115
5%
bookd howkchoskegare
parguothor
Figure 8: Explanations of user intents and real cases in Amazon-Book.
It can be found that 40% of the reasons why user Ugss likes item 44079 are the author Lee Child, 27% of the reasons the protagonist "Jack Reacher" in the book, and 10% of the reasons are because the book is a novel.
Figure d) and Figure 8 e) are the user's attention the relationship of this book and the attribute attention of this book generated by IA component, respectively. It can be seen that the reason why user 4335 likes the book because of the type of the book, the character in the book and the author of the book, which is similar to the conclusion in Figure 8 ((.).
The two separately trained models get the same result, shows that this interpretability credible. Figure g e) gives detailed attention. For example, user prefers that the character in the book is "Jack Reacher" in the book (attention for path Nerver Go Back characters Jack Reacher is 0.24), instead of "Jack Reacher" appears in the book (attention for path Jack Reacher appears Nerver Go Back is only 0.01). This very intuitive.
G RELATED WORK
Embedding based methods (Zang et al. 2016; Cao et al. 2019a Ai ct al. 2018 Cao et al.
2019b; Huang et al. 2018) Wang al.) [2020a] 201 8b) use knowledge graph embedding techniques (e.g., TransR (Lin et al. 2015). RotatE (Sun et al. 2018a)) to obtain entity and relation embeddings, and then input these embeddings into subsequent recommendation networks. For example, CKE (Zhang et al.) 2016) uses the TransR (Lin et al. 2015) to learn the structural information of entities from the knowledge graph, and then inputs the learned embeddings into matrix factorization (MF (Rendle et al. 2012)). KTUP (Cao et al. 20 applies TransH (Wang et al. 2016) on both the user-item bipartite graph and the knowledge graph to jointly learn user preferences and complete recommendations. Although these methods can capture the similarity between entities brought by the KG, they ignore the information brought by higher-order connectivity.
Path-based methods (Catherine & Cohen 2016 Hu et al. 2018 Jin et al. 2020 Ma et al. 2019;
Wang al., 2019e] (Snn al.] (018b) find higher-order connectivity for recommendation by finding semantic paths in the KG and then connecting items and users. These paths can input to RNN network (Wang et al. 2019e (unn al. 2018b) or employ an attention mechanism (Hu et al., 2018) to extract user preferences. For example, KPRN (Wang et al.) 2019e) infer the potential high-order connectivity of a user-item interaction by mining the sequential dependence within knowledge-awee requires domain knowledge, which is iabor-intensive and toneccosuming for KG with multiple relationships and various types of entities. At the same time, recommendation systems applied in different fields cannot be transferred to each other, and the generalization ability is poor.
path. But defining the correct meta-path
20

---

Under review as a conference paper at ICLR 2023
Propagation-based (Wang et al. 2021 2019c Tu et al. 2021 Wang et al. 2018a| 2019a b 2020c) methods rely on the information aggregation ability of GNN and the stacking ability of lay- to capture high-order connection in end-to-end manner automatically. For example, KGAT (Wang et al. 2019c) introduces an attention mechanism on unified UKG graph for learning. Based KGAT (Wang et al., 2019c). KGIN (Wang al. 2021) transfers rela- tional information and introduces intent nodes between users and items to achieve better interpretability and performance. Although this unified graph structure captures collabora- tive filtering signals, high-order connectivity and knowledge information. It is experimen- tally proved that only part of this information can be captured. The reason why it is not fully captured is that these information are mixed together when propagating on the graph, which introduces an incalculable noises.
Multiview-based methods (Zou al., 2022 Yuhao al. 2022 Du al. 2022) choose
construct multiple views to learn information from different perspectives, and then learn embeddings about items and users by designing fusion mechanism. For example, HAKG (Du et al., 2022) learns from user-item graph and knowledge graph to generate collaborative signals and knowledge associations, and applies gating mechanism fuse them. Although this method achieves better performance than propagation-basee methods, it still cannot lean the full information these views because different information still represented by one embedding.
21


Reviews:
[{"review_id": "w-R1GpjxxQr", "reviewer": "Reviewer_8G5y", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the knowledge graph-based recommendation problem. The authors firstly study the relationship between different SOTA methods. The authors also develop a model-agnostic cross-layer fusion mechanism to improve the performance of GNN. To demonstrate the effectiveness of the proposed method, the authors have performed experiments on three public datasets.", "summary_of_the_review": "This paper firstly proposes a metric to evaluate the relationship between two models. Then, the authors also propose a cross-layer fusion mechanism to improve the performance of GNN models. Moreover, the authors also propose a knowledge graph based recommendation framework KGSF. However, the connections between these three parts are not very clear. Simply putting these three parts together makes the focus of this paper is not very clear. Moreover, the technical contribution of the proposed KGSF model is limited. The following are some other detailed comments.\n\n1. The font size in Figure 1 is too small. It is better to use a larger font size.\n\n2. Intersection@N is one of the main contributions of this work. The authors need to introduce the definition of Intersection@N and discuss its reasonability in the main content of this work. \n\n3. In this work, the authors study knowledge graph-based recommendation. Knowledge graph is generally a complex graph which does not only include item attributes. However, in this work, the authors only consider item attributes in the knowledge graph. This assumption seems not reasonable enough.\n\n4. It seems that this work should mainly focus on knowledge graph-based recommendation methods. However, the authors have spent many efforts studying traditional GNN-based recommendation methods, e.g., SGL and SimGCL. This makes this work a little confusing. \n\n5. In Table 3, the authors need to include the performance achieved by LightGCN on all the experimental datasets.\n\n6. In Section 4.1, the authors also need to introduce the detailed strategies used to split the train, validation, and test datasets.\n", "strength_and_weaknesses": "Strengths:\n1.\tThe authors propose a new metric, i.e., Intersection@N, to measure the differences between two models. This is an interesting idea.\n\n2.\tThe authors propose a cross-layer fusion mechanism to improve the performance of GNN models.\n\n3.\tThe authors propose a knowledge graph-based recommendation framework, i.e., KGSF, to exploit item knowledge graph for recommendation.\n\nWeaknesses:\n1.\tThe organization of this work is a little confusing. It seems this paper should focus on knowledge graph-based recommendation methods. However, the authors spent lots of efforts analyzing traditional GNN based methods, for example the analysis in Table 1, Table 2, Figure 2, and Figure 3.  The authors need to reorganize the structure of this work.\n\n2.\tThe proposed recommendation framework is just an integration of existing methods. The technical novelty of this work is very limited. \n\n3.\tThe proposed KGSF recommendation framework considers three graphs for recommendation, i.e., UI graph, IA graph, and UA graph. Besides the user-item interaction information, KGSF can only exploit the item attribute information in knowledge graph. Thus, it can not exploit the complex structure information and other entity information. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "uzRfZco2pP_", "reviewer": "Reviewer_3bRV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper first proposes the Intersection metric to measure the relationship between different KG-based recommender system models. The authors then design a model-agnostic cross-layer fusion mechanism, and conduct experiments on three real datasets to demonstrate its effectiveness. At last, the authors design a KGSF framework to improve the recommendation performance through independent signal extractors and fusion mechanisms, which works on user-attribute, item-attribute, and user-item graphs, separately.", "summary_of_the_review": "Overall, the contribution of this paper is limited, and the experimental result does not show significant improvement over baseline methods. In my opinion, the paper does not reach the acceptance threshold of ICLR, and I vote for reject.", "strength_and_weaknesses": "Strength:\n\n1. The idea of separating and fusing KG and recommender systems is interesting;\n\n2. The paper is basically well-written and well-organized.\n\nWeaknesses:\n\n1. The novelty of this paper is limited;\n\n2. Some of the technical details are not clearly presented;\n\n3. The empirical improvement over baseline methods are not significant.\n\nSee below for details of weaknesses.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "eqkxm4TAUk", "reviewer": "Reviewer_wbd8", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies recommendation models based on knowledge graph. The author proposes a metric to measure the candidate model prediction performance compared to a base model. The findings throughout evaluating the metric against a few prior models are used to motivate their own work, in which they design a model-agnostic fusion mechanism to improve GNN performance (with multiple layers). They also propose the idea to use three independent signals extractors and later fuse them together to perform recommendations. Their empirical experiments show improvements over previous KG-based methods.", "summary_of_the_review": "The paper has limited technical contribution and could use more improvement in the writing. It does not meet the bar of acceptance at its current form.\n", "strength_and_weaknesses": "Strength:\n* The proposed metric `intersection@n` is somewhat novel to me. In practice, it could be useful to measure the model improvement when comparing a candidate model with a baseline model if $N$ is evaluated at various levels (e.g, 5, 10, 20, 100). This evaluation is especially useful when we need a validation measure before deploying new model to replace the previous model because the recommendation model often needs an update (either continuously or after a fixed interval of time).\n\nWeakness:\n\n* The technical contribution of this paper is minor, due to the fact that the fusion mechanism introduced in this paper does not seem novel to me. \n* The writing of the paper is not clear in many places. I will list some examples below. \n* The author uses a large portion of the paper to describe their analysis using the proposed \"intersection@n\", which in my opinion, does not seem necessary. Because this part is less technical and not relevant to the modeling contribution they described thereafter, which shall probably be placed in the appendix instead, and more technical discussions could be used for the main text.\n* Most figures (e.g., figure 1, 2, 3, 4) have used very small fonts with illegible texts. I have to zoom to 3x to see clearly the text in the figures. This is not friendly for a hard print on normal A4 paper.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject