ContextGNN: Beyond Two-Tower Recommendation Systems
Authors: Yiwen Yuan
Zecheng Zhang
Xinwei He
Akihiro Nitta
Weihua Hu
Manan Shah
Blaž Stojanovič
Shenyang Huang
Jan Eric Lenssen
Jure Leskovec
Matthias Fey
Venue: ICLR 2025 Poster (2025)
Keywords: graph neural networks
recommendation
relational deep learning
Primary area: learning on graphs and other geometries & topologies

Abstract: Recommendation systems predominantly utilize two-tower architectures, which evaluate user-item rankings through the inner product of their respective embeddings. However, one key limitation of two-tower models is that they learn a pair-agnostic representation of users and items. In contrast, pair-wise representations either scale poorly due to their quadratic complexity or are too restrictive on the candidate pairs to rank. To address these issues, we introduce Context-based Graph Neural Networks (ContextGNNs), a novel deep learning architecture for link prediction in recommendation systems. The method employs a pair-wise representation technique for familiar items situated within a user's local subgraph, while leveraging two-tower representations to facilitate the recommendation of exploratory items. A final network then predicts how to fuse both pair-wise and two-tower recommendations into a single ranking of items. We demonstrate that ContextGNN is able to adapt to different data characteristics and outperforms existing methods, both traditional and GNN-based, on a diverse set of practical recommendation tasks, improving performance by 20\% on average.

Published as a conference paper at ICLR 2025
CONTEXTGNN: BEYOND
Two-TOWER RECOMMENDATION SYSTEMS
Yiwen Yuan, Zecheng Zhang, Xinwei He, Akihiro Nitta, Weihua Hu, Manan Shah, Blaž Stojanovič, Shenyang Huang, Jan Eric Lenssen, Jure Leskovec, Matthias Fey Aumo.Al
ABSTRACT
Recommendation systems predominantly utilize two-tower architectures, which evaluate user-item rankings through the inner product of their respective embed- dings. However, one key limitation of two-tower models is that they learn a pair- agnostic representation of users and items. In contrast, pair-wise representations either scale poorly due to their quadratic complexity or are too restrictive on the candidate pairs rank. To address these issues, introduce Context- basee Graph Neural Networks (CoNTEXTGNNs), a novel deep learning architecture for link prediction in recommendation systems. The method employs pair-wise representation technique for familiar items situated within a user's 's local subgraph, while leveraging two-tower representations exploratory items. A final network then predicts how to fuse both pair-wise and two-tower recommendations into single ranking of items. We demonstrate that CONTEXTGNN is able to adapt to different data characteristics and outperforms existing methods, both traditional and GNN-based, on diverse set of practical recommendation tasks, improving performance by 20% on average.
facilitate the recommendation of
INTRODUCTION
Recommendation systems have emerged important application domain for predictive machine learning over the past decades (Webber) 202T He et al. 2023 Li et al. 2024). Given a set of users and a set of items (e.g., products available for purchase), recommendation systems aim to identify optimal item recommendations for each user (e.g., products most likely to be purchased by the user). Traditionally. this problem is modeled via different variants of swo-tower paradigm (Hu et al., 2008 Koren et al., 2009), where one tower embeds users and the other tower embeds items, which are then matched and ranked via an inner-product decoder. This scheme proves to be highly efficient for scaling up recommendation systems during the inference phase, as it allows to pre-compute user and item representations and to perform the final ranking via fast (approximate) maximum inner product search (Johnson et al., 2019)..
However, one key limitation of two-tower based architectures for recommendation is that they learn pair-agnostic representation for users and items. That is, the user representation is not aware of the item under consideration, and similarly, the item representation is not aware of the user and capturing the uniqueness of user's view the items. As such,
thus item representations neither of the representations on both ends capture knowledge about the pair-wise dependency they making a prediction for. For example, consider a user who restocks their cosmetic products on a regular basis. In this scenario, the fine-grained context of user-cosmetic pairs is crucial, which cannot be adequately captured by two independent user and item representations alone. Such lack of knowledge has severe consequences on the quality of predictions, since, e.g., the model is unable to distinguish between scenarios such as familiar purchases (i.e. users who repeatedly interact with the similar set of items) vs. exploratory purchases (i.e. users who like to explore new items). While pair- wise representations, which incorporate the knowledge about the pair they are making a prediction for, are able to contextualize the prediction, one would need to generate pair-wise representations for all possible user-item pairs, which is ineffable due to its quadratic complexity. Altemattvely, one can pre-filter the set of candidate pairs, e.g., via content- based filtering collaborative filtering Ricci et al.) (2010): Cammaana & Delmastro (2017), but the model's capabilities are then limited by the recall of the candidate generation procedure.

---

Published as a conference paper at ICLR 2025
Here, develop Context-basee Graph Neural Networks (CONTEXTGNNs), novel single-stage Graph Neural Network (GNN)- recommendation system that fuses wiir-wise representations and two-tower representations into single unified architecture. CONTEXTGNN is designed to perform temporal recommendations (predict the next set of items a user will interact with) on a heterogeneous temporal graph, and leverages the best of both worlds: The first model supports pair- wise representations for all items within a user's local interaction graph (Zhu et al.) (021). For distant user-item pairs, i.e. user-item pairs that are not within the k-hop neighborhood of each other, the pair-wise embeddings are supplemented by global, two-tower-basee GNN, which serves as an effective fallback model. A final network then predicts a user-specific, personalized fusion score that dictates how to merge both pair-wise and two-tower recommendations into a single ranking.
They key idea behind CONTEXTGNN is to contextualize the prediction for the area of items for which a user has rich past interactions. Here, pair-wise representations are able to capture fine- grained patterns of past user-item interactions, such as repeat purchases and other prior actions (e.g. clicks, tag-as-favorite, add-to-cart), or through collaborative signals (e.g., whether friends have purchased the same item). The second component, a novel pair-agnostic two-tower model based on shallow item embeddings, allows for modeling exploratory and serendipitous recommendations. For these "distant" items, where no specific user-item context exists, pair-wise representations offer minimal benefits, making the fallback to two-tower representations suitable. Importantly, the two- tower model is designed in such a way that it allows for integration into pair-wise architectures with little computational overhead. Finally, the model learns which users prefer familiar items over exploratory purchases and adjusts the final ranking scores accordingly.
We deploy CONTEXIGNNs in the context of relational deep learning (Fey al.) 2024 for rec- ommendation within relational databases. Relational databases (Robinson et al. (2024) comprise diverse sets of tables with rich multi-modal input features and rich mehtii-bhaaviora, temporal in- teractions, making them the perfect setting for stress stress-testing our model. Furthermore, we examine the challenges in modeling the complex patterns of human behavior. Our analysis and experimental results underscore the need for hybrid architecture, as single model is not sufficient to address the diversity of real-world datasets both within and across tasks effectively. We demonstrate that CONTEXTGNN is able to adapt different data characteristics, while outperforming existing mod- els on realistic and practical recommendation tasks. In particular, CONTEXTGNN improves results by 20% on average compared to the best pair-wise representation baseline, and by 344% on average compared to the best two-tower representation baseline.
2 RELATED WORK
Recommendation systems are long-standing research area in machine learning. The recommenda- tion problem is often formulated as link prediction on a bipartite graph with two sets of nodes, one containing the users and another containing the items. Between these two node sets, past interac- tions are presented as links, and the goal is to predict which links are going to occur in the future. Traditionally, is approached by computing the inner product of user and item embeddings, egg., via matrix factorization (Koren et al.] (209) to discover and utilize similarity between users and items, i.e. collaborative filtering (Hu et al. 2008 Mnih & Salakhutdinoy 2007).
Similar to many other areas, deep learning made an impact in recommendation systems, e.g., by predicting scores with non-linear networks (H. & C., 2017: He al. 2017), by utilizing graph
embedding techniques (Perozzi et al.] [2014: Grover & Leskovec 2016). or by sampling from dis- tributions modeled by a variational autoencoder (Liang et al.) 2018). Recently, the new field of generative recommendation deserves mentioning. utilizing large language models to provide rec- ommendation by text generation (Wu et al. (023). Our work here relates to two lines of research: Sequential recommendation systems and recommendations with GNNs.
Recommendation on Sequences. Recommendation has traditionally been modeled as a sequence prediction problem. Early methods utilize Marcov chains (Rendle et al.] 2010 He & McAuley 2016). which have later been replaced by recurrent neural networks (Liu et al. 2018). temporal attention (Kang & McAuley, 2018 Li et al., (017) and transformer formulations (Sun et al.) 2019 Yang et al. 2022; Xia et al. 2022). They also have been combined with GNNs to additionally
capture collaborative signals (Ma et al. 2020
2

---

Published as a conference paper at ICLR 2025
Recommendation with GNNs. Various noottower-based Graph Neural Networks (Hamilton et al. 2017) have been developed for link prediction tasks, based on the idea of integrating the paradigm of collaborative filtering via refining embeddings on the user-item interaction graph. Early examples use GNN-based autoencoders (Kipf & Welling, 2016 (Schlichtkrull et al., 2018) to obtain prediction on graph links. Since GNNs propagate information locally (and thus cannot reason about the position of nodes inside a graph), it has been shown that propagating shallow user and item embeddings with deep GNNs is crucial capture the full collaborative signal (Wang et al. 2019). LIGHTGCN (He et al. 2020) and ULTRAGCN (Mao et al. (021) further simplity message prop- agation for embedding generation for improved performance on small-scale datasets. Another line of work strengthens GNN-based recommender systems by adding self-supervisio (Wu et al.) 2021
Yu et al. 2022; Cai et al. 2023), e.g. based on contrastive learning via graph augmentations. These findings are orthogonal to our work, but still embrace two-tower architecture under the hood.
The line of work that comes closest to CONTEXTGNN is related to encoding identity- and position- awareness into GNNs in order to relate user-item pairs (You et al., 2019 2021). For example. local subgraphs have been extended to encode positional information of user-item pairs (Zhang & Chen 2018 Teru et al. 2020), but lack efficient inference procedure. NBFNET (Zhu al., 2021 introduces path- based pair-wise representation model that constructs a subgraph centered around the user node, while reading the GNN's representations at the item nodes. This method yields item representations that are conditioned on the specific user, thereby incorporating pair-wise infor- mation without incurring excessive computational cost. However, this method constrains candidates to items within the user's subgraph, which tremendously limits its effectiveness in scenarios in- volving exploratory recommendations or cold-start items. Our local pair-wise representation model builds upon this framework, extending it to fit into the heterogeneous, ral recommendation system context. Most importantly, we analyze and eliminate its main limitation around locality by extending it with an effective fallback model.
muatii-behavioral, and tempo-
While the vast majority of related work focuses on modeling recommendation systems on static graphs, recently multiple benchmarks for temporal recommendation have appeared, such as the TEMPORAL GRAPH BENCHMARK (Huang et al. 2023) and A RELBENCH (Robinson et al., 2024 In this work, we adapt the temporal formulation of recommender systems task and exten- sively evaluate CONTEXTGNN in the practical setting of & RELBENCH and show that our method significantly outperforms previous approaches.
3 TEMPORAL RECOMMENDATION AND WHERE TO FIND THEM
We formulate the temporal recommendation problem on heterogeneous graph snapshot g(-oo.T) (V, E, db, v)) up to timestamp T, with node set V and edge set & C V x V, where each node u E V belongs to node type (b()) and each edge e E & belongs to an edge type vb()). We refer to C c V as the user set and R C V as the item set within our heterogeneous graph. The task then to predict a set of ground-trute -ttth items (T,++)) S R, for which there occurred link for the user E L in the time interval (T, T + +i for a given interval size i. When making the prediction, the model only has access to historical information up to timestamp T.
In the simplest case, g(-o0.21 is given as a bipartite graph, with node set V LUU R. and a single edge type Kv(e)) e E E}] = 1. However, the generalization to heterogeneous graphs allows to incorporate other node types, but most importantly, different edge types of user behaviors (e.g.. click, tag-as- aig-as-fad favorite, add-to-cart) which offer complementary signals for predicting the target behavior (e.g., purchase) (Xia et al. 2022 Nodes and edges may be (partially) annotated with an initial feature representation, e.g. h E Rdd Rdduve V, and with a timestamp indicating their appearance (or disappearance). In the absence of timestamps, the given framework can be easily transformed into a static link prediction problem, as commonly practiced in literature (Kipf & Welling, 2016 Wang jet al. 2019. He et al., 2020p. However, this approach disregards critical information, such as the recency of past activity. Note that the given framework also allows newly appearing users and items over time. However, for simplicity of notation, we assume them to be in L and R at all times.
Locality Score. Developing machine learning solutions for recommendation systems is inherently challenging due to the complex patterns of human behavior (He 2023). Users vary significantly in preferences; some are explorers who constantly seek new experiences, while others are repeaters
3

---

Published as a conference paper at ICLR 2025
who prefer familiarity. Such characteristies especially noticeable across different datasets, e.g-, it is more likely that a user repeats a previous purchase than that the user rates the same movie twice.
To understand these effects better, we introduce the locality score, which measures the fraction of ground-truth links that fall within the k-hop neighborhood N (g-oo.T) of the subgraph centered around a user E L up timestamp T:
(v) n R (2.44))
TT
C
Intuitively, the locality score quantifies how many of its future ground-truth items are local to a user. Depending on the subgraph depth k, the score reflects different interaction patterns. For a shallow subgraph (k 1), it captures whether the user repeats purchases or has previously interacted with the item through actions like clicks or lagging as favorite. As the subgraph depth increases, the score begins to incorporate collaborative filtering signals, such as whether friends have purchased the ground-trun truth item the past (k = 2), or whether the ground-truth item was previously purchased by users that have similar buying behavior as the user under consideration (k: = 3). Note that in practice, it is infeasible to go beyond depth = due to scalability concerns.
Table 1: The locality score for different subgraph depths à E {1,3} on validation/test splits for all recommendation tasks in RELBENCH.
We evaluate the locality score for varving k on A RELBENCH (Robinson et al. 2024), a rela- tional deep learning (Fey et al. 2024) bench- mark that spans a wide variety of real-world recommendation tasks with diverse character- istics (ef. Table D. Interestingly, the locality score does not exceed 0.5 any dataset, indi- cating that the majority of ground-truth items are entirely exploratory and not covered by any path of length I 3 in the graph. However, for h = we observe high locality scores on rel-hm and rel rel-stack, suggesting that users tend to frequently repeat clothing pur- chases or comment multiple times on the same post. Increasing the subgraph depth gener- ally improves coverage as seen in datasets like rel-amazon. While users do not usually re- purchase/rate/eeview the same item twice here, they tend to purchase the same products that "nearby" users bought. Overall, this observa- tion suggests that while pair-wise representa- tion architectures such NBFNET (Zhu al.) 2021 are powerful, they often times fail to offer sufficient coverage within their candidate set, de- creasing their effectiveness tremendously. This raises an interesting question of how we can leverage the benefits of pair-wise representations, while supporting the modeling of exploratory recommen- dations without the need for complex multi-stage approaches.
Dataset
Task
Split
k =
1 k = 3
val 0.002
0.181
user- puzehaaa
test 0.001
0.168
val 0.002
0.183
use5-i -en--iten-aate
test
0.002
0.170
val 0.002
0.185
user-1t seer-tten-reviee
test
0.001
0.175
val 0.045
0.395
user- saae-itta-uuzcaase
test
0.048
0.417
val 0.265
0.308
uses-poat-comment
test
0.256
0.298
rel-stack
val 0.109
0.225
post-post-relsted
test
0.141
0.280
condition-
val
0.000
0.241
cel-trial
sponsor-run
test
0.000
0.242
val
0.000
0.208
aponsos-ouou
test
0.000
0.229
4 RECOMMENDATION WITH CONTEXT-BASED GRAPH NEURAL NETWORKS
Next, we describe Context-based Graph Neural Networks (CONTEXTGNNs), which consist of two separate GNN architectures sitting behind the same GNN backbone. CONTEXTGNN fuses both pair-wise representations and two-tower representations into a single architecture, and is thus naturally able to adapt to diverse dataset and task characteristics. The first model employs pair- wise representations based on the item candidate set given within the local user-centric subgraph (cf. Sec. 4.1.. The second model introduces a novel two-tower architecture based on shallow item representations, which is used to predict ranking for all pairs of users and items outside the user's subgraph (cf. Sec. 4.2). Finally, user-fpecific fusion score, produced by an MLP on the user GNN representation, is added to the scores of the pair-wise representation model, which aligns the distinct
4

---

Published as a conference paper at ICLR 2025
based on uscr-specifie item representations
0.1
Pair-wise
Mart
Final ranking
Fusion
+INDICATOR&
MLP
INDICATORO
0.6
Was
based on shallow Item embeddings
Figure 1: Overview of Context-based Graph Neural Networks. CONTEXTGNN utilizes bidi- rectional GNN to learn user h! and user-speeific item representations (2) within a user's local subgraph. Its message passing scheme is enhanced by additionally propagating shallow item em- beddings wh and seed user IDENTICATOR, representations. Afterwards, item scores are produced depending whether item situates within a user's subgraph. A user-specific fusion score is learned via an MLPa to produce the final ranking by offsetting the contributions of local rankings.
scores of the two models and captures how exploratory a specific user is, thus given either more less weight the respective models (cf. Sec. 4.3 We go over additional considcrations and extensions in Sec. 4.4 An overview of CONTEXTGNN is visualized in Fig.
4.1 PAIR-WISE REPRESENTATIONS
Our local pair-wise representation model builds upon the framework proposed by Zhu et al. (021) extending it to fit into the heterogeneous, multi-behavioral, and temporal recommendation system context. NBFNET is a path-based method that generalizes the Bellman-Ford algorithm (Baras & Theodorakopoulos] 2010). Rather than learning pair-wise representation via two independent node representations h and h! (i.e. via two independent subgraphs), NBFNET only utilizes the user- specific subgraph, but readouts GNN item representations from it. As such, NB ENET abbreviates (k)
the pair-wiss representation h (e...)) via the user-specific item representations (A), Hence, it captures both knowledge about the item w and the user e in its representation. In order to let the GNN differ-
entiate between the seed user and other users being sampled within the subgraph, d-dimensional INDICATORg representation added to the seed node (You et al. 2021). Specifically, our pair-wise representation model then looks follows:
1. Sample a k-hop subgraph 9 + of 00, with node set around user v E L.
2. Add an indicator representation to the user seed node: h2 < !o + INDICATORg.
3. Read both GNN user and item representations at layer k:
( w E VVRR + GNN (() (pair) 4. Compute the final ranking for all items w E in R: V(u,w)
h
+ (1) h
(R)
In contrast to NBFNET, we readout both the user and item representations from the GNN in order to produce the ranking of local items. This lets the model leverage the full k-hop user information, otherwise user representations only computed in intermediate GNN layers based limited subgraph depths.
Further note that sampling k-hop subgraph typically results in directed computation graph to- wards the seed node (Fey & Lenssen 2019). However, to facilitate the extraction of item node representations, the sampled subgraph must be transformed into a bidirectional graph prior to ap- plying the GNN. This approach aligns with previous work that decouples the depth and scope of GNNs (Zeng et al. 2021
5

---

Published as a conference paper at ICLR 2025
This approach proves partieularly effective in the context of temporal, heterogeneous graphs, the GNN naturally learns to integrate multi- multi-behavior signals originating from different edge types. For instance, the pair-wise representation captures all past user-item interactions, such as the recency of the last purchase or whether the item has been previously clicked or tagged. Despite its expres- siveness, this method is also highly efficient, as only a single GNN pass over the user subgraph is required to make predictions for the entire set of related items, Assuming bounded subgraph sizes, training and inference over the full user set can be achieved in O(12)) time, which is in stark contrast to the O(14)- IR)) complexity required by two-tower models. Moreover, as no shallow embeddings are used, this approach naturally extends to newly appearing users and items over time. Nonetheless, as discussed in Sec. 3 this method alone does not fully address the diverse requirements of modern recommendation systems due to its limited set of potential candidates.
4.2 TWO-TOWER REPRESENTATIONS
CONTEXTGNN's 's two-tower model ranks all user-item pairs outside the user's subgraph, serving as an effective fallback mechanism to supplement the pair-wise representations. While drawing inspiration from related two-tower architectures (Wang al. 2019 He et al. 2020), our novel two- tower model enables an efficient integration with pair-wise architectures. The key innovation in our two-tower model is the use of shallow item representations. Specificallyy, we do not deploy a GNN to compute item representations; instead entirely rely shallow embedding matrix W E R ( xd to learn item representations. This design is inspired by multiple key observations:
Limited information gain from applying a GNN on the item side. Item connections are naturally very dense. In extreme cases, popular items may receive over 1M interactions (an item is connected to every user that has previously interacted with it). This leads to significant challenges and uncertaintics in subgraph sampling, and easily exposes the GNN to oversquashing and oversmoothing issues.
Shallow embedding matrices are very effective. Shallow item embedding matrices can capture key signals such as popularity trends, seasonal patterns, demographic preferences and item similarity just as effectively as a GNN.
GNN item representations scale poorly during training. Incorporating both user and item GNN representations limits the model to only use small number of negative sam- ples due to memory constraints. In contrast, shallow item embeddings support training against much larger corpus of negative samples (== 10M), which is critical for improving model performance. This approach also eliminates the need for complex negative sampling strategies, which are often required when the number of negative samples is limited.
Importantly, we also inject the shallow item embeddings within the user's GNN forward pass, such that user representations can better align themselves to the corresponding item representations. Hence, the GNN representation of users follows similar spirit to models such as NGCF (Wang ett al. (011)) or LIGHTGCN (He et al. 2020). although we do not utilize shallow user embeddings as the GNN itself is powerful enough to learn its own rich representation without them. As such, our two-tower representation model can be summarized as follows:
1. Sample a k-hop subgraph g + 9% (o) with node set 10 around user U E L.
2. Add the shallow embedding to all sampled items w E M R:
R: (0) + (0) + Ww.
3. Read out the GNN user representation layer ke h2(k) GNN(()
(k) (Ś, H ()).
4. Compute the final ranking for all items R \ V: v/ flower) (k) Ww-
4.3 CONTEXT-BASED GRAPH NEURAL NETWORKS
Our final CONTEXTGNN architecture fuses both pair-wise representations and two-tower represen- tations into a single unified architecture. That is, for all items w E 1 U R. inside the local user subgraph, we leverage the scores V(o,n) obtained from the wair-wise representations, while we fall-
(pair)
back to the two-tower scores v/ ((o.w)) for all items V R outside the sampled subgraph.

---

Published as a conference paper at ICLR 2025
The observation that user behaviors are diverse is the final element that makes CONTEXTGNN work. To accommodate such diversity across different users, CONTEXIGNN learns a user-specific fusion score predicted from the GNN's user embeddings he via an MLPg. This personalized fusion score aligns the distinct scores by learning which users prefer familiar items over exploratory purchases, and adjusts the final ranking scores accordingly. With this, the final score is given by
(pair)
+ ML.P (n)
if w E VUR.
D V(,,n)
(lower)
otherwise.
In CONTEXTGNN, both pair-wise representations and two-tower representations are derived from the same GNN backbone. Since both models depend solely on the user subgraph, the user em- bedding (*) can be extracted a single GNN forward pass and leveraged for downstream uses in both models. This streamlined approach makes CONTEXTGNN computationally very efficient. CONTEXTGNN is then trained end-to-end, optimizing both types of item scores as well as the fu- sion score altogether to maximize the predictive performance of future user-item interactions. In practice, we utilize the cross entropy loss for optimization, although any other loss formulation is applicable as well (Rendle et al., 2009). During inference, we obtain top scores from the two-tower model via (approximate) maximum inner product search (Johnson et al. 2019), and merge them to the pair-wise scores in a post-procssiing procedure.
4.4 EXTENSIONS
We go over additional considerations and extensions when applying CONTEXTGNNs in practice.
Fitting into the Context of Relational Deep Learning. CONTEXTGNNs nicely align with the framework of relational deep learning (Fey et al. 2024). a blueprint for graph representation learning on relational databases. Specifically, relational deep learning treats relational databases as hetero- geneous temporal graph, in which dimension and fact tables are linked through primary-foreign key relationships. Then, temporal-aware subgraph sampling employed to generate heterogeneous
subgraph snapshots 0% [x) up to timestamp T. which are used to predict future user-item in- teractions. Relational databases (Robinson al., (2024) comprise diverse set of tables with rich multi-modal input features and rich multi-behavioral, temporal interactions, making them the per- fect setting for stress-testing CONTEXTGNNs.
Transductive vs. Inductive Modeling. Although the pair-wise representations in CONTEXTGNN are inductive by design, the usage of the shallow embedding matrix on the item side places it in a transductive setting. This means that while CONTEXTGNN can naturally accommodate newly appearing users over time, it is unable to handle new items at prediction time since their shallow embeddings remain uninitialized. To enable CONTEXTGNN to operate in an inductive setting, we propose to replace the shallow item embedding matrix with a deep neural network on top of its item input features ( This approach allows CONTEXTGNN to scale to tasks such as marketplace or event recommendation, where handling new items essential.
Sampled Softmax Formulation. Since it is infeasible to train against the full item set R when optimizing CONTEXTGNNS, in practice, we rely on a sampled softmax formulation. This basically converts the objective of CONTEXTGNN into a classification problem with C' sampled classes, shared across the entire mini-batch. The utilized sampling procedure is based on a priority queue: we ensure that (I) all ground-truth items and (2) the union of sampled subgraph items of given mini-batch are included in the set of classes. Then, we (3) fill up the remainder of items with unexplored items outside all subgraphs of the mini-bater based a uniform sampling procedure. In practice, we have no issues to scale the number of classes C to x 1M on commodity GPUs (15GB of memory), giving CONTEXTGNN rich signals learn from.
5 EXPERIMENTAL EVALUATION
We perform experiments on six diverse datasets stemming from different domains, including ten different recommendation tasks. We aim to answer the following research questions:
7

---

Published as a conference paper at ICLR 2025
Q1 Which benefits does CONTEXTGNN provide each of its individual component?
Q2 How does CONTEXTGNN perform against state-of ttateoofthhe-rt recommendation system methods?
Q3 How does the locality score of Sec. inffuennce the model performance of CONTEXTGNN?
Q4 How efficient and scalable is CONTEXIGNN compared to the related work?
Our method is implemented in Ô PYTORCH (Paszke et al. 2019) utilizing the PYTORCH GEO-
METRIC (Fey & Lenssen 2019) and & PYTORCH FRAME
(uu et al. 2024) libraries.
5.1 RELATIONAL DEEP LEARNING
Dataset Description. We utilize the recommendation tasks introduced in A RELBENCH (Robin- son et al.) 2024). which consists of eight different realistic and temporal-awar recommendation tasks. A RELBENCH datasets contain rich relational structure, providing a challenging environment for recommendation tasks. The task to predict the top-k items given a user at given seed time. The metric we use is Mean Average Precision (MAP) @ k. where k is set per task (higher is better).
rel- rel-amazon Predict the list of distinet items each user purchases (user-item-purchaan gives a 5-star (eter-ite-rraee or gives a detailed review (user-itee-rrviiev)
rel-hm Predict the list items each user will purchase
rel-stack Predict the list of distinct posts a user will comment (user-post- or
to predict the list of posts that users will link a given post to (psst-post-relatee))
rel-trial Predict whether a condition will have which sponsors
(condition ironditionnssoosorr sponsor-run), or whether sponsor will have a trial in a facility (site- potessssosssssuuee run).
Experimental Protocols.
We compare CONTEXTGNN with the following baseline methods:
LIGHTGBM (Ke et al. 2017) concatenates both user and item features, and feeds them into LIGHTGBM decision tree.
MULTIVAE (Lingg et al. 2018) extends variational autoencoders to collaborative filtering via a user-item interaction matrix for implicit feedback.
GRAPHSAGE (Hamilton et al. 2017) employs heterogeneous GNN on both user and item side, and ranks the produced embeddings via an inner product decoder.
NGCF (Wang et al. 2019) extends GRAPISAGE by propagating both shallow user and item em- beddings inside the GNN.
NBFNET (Zhu et al. 2021) employs pair-wise GNN representations, This is the backbone of our pair-wise representation model in CONTEXTGNN.
SHALLOWITEM describes our two-tower model in CONTEXTGNN, which ranks user GNN repre- sentations and shallow item embeddings via inner product decoder.
Importantly, all GNN- models utilize the same GNN backbone, which guarantees fair compar- ison of training procedures that are agnostic to the underlying model implementation. Specifically, we use a heterogeneous GRAPHSAGE variant as introduced in Robinson et al. (2024). The hyper- parameters tune for each task are: (1) the number of hidden units {32, 64, 128, 256, 512). (2), the batch size E [256, 512, 1024]. and (3) the learning rate E (0.001,0.0))
Discussion. The results are reported in Table 2 We answer Q1 by comparing CONTEXTGNN to its individual components NBFNET and SHALLOWITEM, and answer Q2 by relating CoN- TEXTGNN's performance all reported baselines.
CONTEXTGNN outperforms all competing baselines, often by very significant margins. Notably, one can observe that the two-tower models MULTIVAF, GRAPHSAGH AGE and NGCF all fail to cap- ture the pair-wise signals that both NBFNET and CONTEXTGNN are able to leverage. This shows that two-tower representations are not powerful enough to capture the fine-grained pair-wise de- pendencies that are required to solve these tasks with high precision. Among the two-tower GNN
GitHub: https://tgtt fttpp:://iithub.com cttp:.ccm//uunen htt.://mmaaa//oeteeeeeeee /Cottext.GN
8

---

Published as a conference paper at ICLR 2025
Table Recommendation results (MAP, higher is better, in %) on RELBENCH.
LIGHT GBM
MULTI VAE
GRAPH SAGE
SHALLOW ITEM
CONTEXT
Task
NGCF NBENET
GNN
rel-amazon
user-iten- seeritee-purcchsse
0.16
0.23
0.74
0.88
2.06
0.56
2.93
user-item-rate
0.17
0.24
0.87
0.86
1.24
0.74
2.25
user-iten-nertee
0.09
0.10
0.47
0.55
1.57
0.40
1.63
rel-hm
user-iten-purcharee
0.38
0.28
0.80
0.75
2.81
0.40
2.93
rel-stack
usec-post sesc-posttcomme ccoment
0.04
0.01
0.11
0.13
12.72
0.03
13.34
post-post-related
2.00
0.78
0.07
0.13
10.83
0.82
11.18
rel-trial
condition cppposoorrpn
4.82
2.47
2.89
3.88
11.36
0.85
11.65
site- sponsoz-run
8.40
6.17
10.70
6.54
19.06
10.66
28.02
Average (1)
2.01
1.29
2.08
1.72
7.71
1.81
9.23
models, there is no clear winner between GRAPHSAGE and NGCF, indicating that shallow (user) embeddings do not significantly drive improvements (and may even hinder performance, especially in tasks with inherent temporal dynamics), CONTEXTGNN improves results by 344% on average compared the best two-tower baseline.
Among the baselines, NBFNET performs the best across all tasks, while CONTEXTGNN can con- sistently improve upon these strong outcomes. On average, CONTEXTGNN increases performance by 20% compared to NBFNET, underscoring the importance of incorporating "distant" items into ranking process an aspect that NBFNET overlooks by design.
Our own two-tower SHALLOW ITEM model performs comparably to other two-tower GNN base- lines, despite only using shallow item information in order to improve the overall efficiency of the model. This supports the hypothesis that deep GNN on the item side is not particularly useful on most tasks, as the shallow embeddings can capture most of the key signals on the item side just as well as the GNN. However, there exists specific cases such the condition sponsor task on the rel-trial dataset, where SHALLOW ITEM underperforms relative to GRAPHSAGE and NGCF. On this task, deep GNNs the item side play indeed crucial role to improve results of two-tower models, which is captured in CONTEXTGNN through its pair-wise representation model.
The most noteworthy result is seen in the site seteessposoo-ruu task the sel-trial dataset. Here, both pair-wise models and two-tower models achieve strong initial results. Combining these two paradigms together via CONTEXTGNN improves the final performance by 100%, indicat- ing that each of the individual components of CONTEXTGNN captures orthogonal signals. The fusion of these paradigms yields ranking model that excels in both local and distant item ranking, achieving superior overall performance.
(T,++)) and the
In order to answer Q3, now analyze the relationship between the locality score sg. performance of CONTEXTGNN. We observe that the improvements of CONTEXTGNN compared to NBFNET are notably higher tasks with lower locality scores. This observation aligns intuitively, as CONTEXTGNN needs to rely more heavily on its two-tower component during optimization when locality scores are lower. Specifically, in tasks with the lowest locality scores (e.g., 0.168, 0.170 and 0.229 on the user-ited em pur-ppurrru phase user- asseriiem--e from the rel rel-ama zon and rel-trial datasets), CONTEXTGNN achieves substantial perfor- mance gains of 42% to 80% compared to NBFNET. Conversely, for tasks with higher locality scores (e.g., 0.417, 0.298, and 0.280 on the assr-item purchase, user post- comment, and poost-posts re ratted tasks from the rel-hm and rel rel-steeee stcck datasets), the performance improvements of CONTEXTGNN over NBFNET are more modest, ranging from 3% to 5%. These findings underscore the significant impact of the locality score the performance of NBFNET, whereas CONTEXTGNN demonstrates robustness, achieving stellar performance improvements re- gardless of task-specific characteristics.
and site-sponsor un tasks

---

Published as a conference paper at ICLR 2025
5.2 STATIC LINK PREDICTION
While CONTEXTGNN's main focus to excel on large-scale real-world use-cases which are temporal and heterogeneous, it can also be used in plug-and- play fashion for any link prediction task. To verify, we evaluate CONTEXTGNN on the static link pre- diction task of Amazon-Boon which is small-scale dataset of 52,643 users and 91,599 items, which does not come with input fea- tures, only considers users with at least ten interac- tions, and evaluates on 10% of randomly selected interactions independent of time, cf. Table We can see that CONTEXTGNN able to outperform both NGCF and LIGHTGCN, while it is slightly underperforming compared to, e.g., ULTRAGCN or LIGHTGCL. Given the relatively small size this dataset, much of the progress in GNN-based rec- ommendation systems has centered
Table 3; Results on Amazon-Book.
Model
Recall@20 NDCG@20
NGCF 2019
0.0337
0.0261
LIGHTGCN 2020
0.0410
0.0318
(Wang et al., 2019).
ULTRAGON 202T
0.0681
0.0556
LIGHTGCL 2023
0.0585
0.0436
SimGCL (2022)
0.0478
0.0379
SGL (202T)
0.0468
0.0371
CONTEXTGNN
0.0451
0.0377
key directions: (1) simplifying GNN architectures, and (2) incorporating self-supervised learning techniques. Exploring how CONTEXTGINN can benefit from more lightweight GNN backhone complementary learning signals presents exciting direction for future research.
5.3 TEMPORAL NEXT-ITEM PREDICTION
We use CONTEXTGNN to perform temporal next-item recommendation the ICAI Contest dataset (Xia et al. 2022). which is a common dataset to evaluate sequential recommendation models al. (Sun et al., 2019) Xia et al.] [0200] 2022). As per evalu- ation protocol, we report HitRate@k and NDCG@k over 99 sampled neg- atives per entity. Notably, CON- TEXTGNN excels at incorporating multi-behavioral and temporal signal, of Table 4 We observe that CONTEXTGNN is able out-perform all baselines all metrics on this task (e.g., 170% improvement on HitRate@1).
Table 4: Results on IJCAI Contest.
Model
HR@1
HR @5 NDCG@5 HR@ 10 NDCG@10
DEEPFM
2017
0.138
0.332
0.244
0.469
0.290
BERTAREC
2019
0.141
0.356
0.261
0.467
0.297
CHORUS
20203
0.140
0.345
0.247
0.457
0.283
HyREC
20206
0.137
0.323
0.229
0.442
NMTR 2019
0.141
0.360
0.254
0.481
0.304
MATN 20207
0.142
0.375
0.273
0.489
0.309
MBGCN 2000
0.137
0.332
0.228
0.463
0.277
TGT (2023
0.148
0.399
0,293
0.519
0.330
CONTEXTGNN
0.411
0.603
0.513
0.667
0.534
5.4 EFFICIENCY ANALYSIS
Our hybrid CONTEXTGNN model is designed in such way to have minimal overhead com- pared to its two components in isolation since the vast majority of the model parts shared between the two paradigms. To answer Q4, we report the runtime in seconds reach 1,000 op- timization steps across different models, cf. Ta- 5)e Most importantly, since CONTEXTGNN only requires a single GNN forward pass, it is faster compared to any two-tower GNN by very significant factor (ie., two-tower GNN such as GRAPHS AGE requires to run the GNN on both positive and negative items). In particular, two-tower GNNs scale very poorly when increas- ing the number of negative samples to train against. For training recommendation systems, large number negative examples is important to allow learning of discriminative features. However, In the simplest (one negative pair per positive pair), two-tower GNN needs to be executed three times already. CONTEXTGNN does not have this limitation and can consider up to 1M negatives before running into GPU memory limitations.
Table Runtime [s] of 1,000 optimization steps.
Model
rel-ha re rel-stack
GRAPHS AGE (1 negative)
275s
37s
GRAPHS AGE (10 negatives)
293s
44s
GRAPHS AGE (100 negatives)
00M
00M
NEBNET
77s
21s
SHALLOWITEM
92s
22s
CONTEXTGNN
94s
23s
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank the entire Kumo.AI team for their invaluable support in bringing CONTEXTGNN into production for over dozens of customers and for any dataset scale. Special thanks go to Amitabha Roy, Myungwhan Kim and Federico Reyes Gómez for helpful discussions and feedback.
REFERENCES
Baras and G. Theodorakopoulos, Path problems networks. Synthesis Lectures on Communi- cation Networks, 3(1):1-77, 2010.
X. Cai, C. Huang, L. Xia, and X. Ren. LightGCL: Simple yet effective graph contrastive learning for recommendation. In ICLR, 2023.
M. G. Campana and F. Delmastro. Recommender systems for online and mobile social networks: A survey. Online Social Networks and Media, 3:75-97, 2017.
M. Fey and E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
M. Fey, W. Hu, K. Huang. J. E. Lenssen, R. Ranjan, J. Robinson, R. Ying, J. You, and J. Leskovec. Relational deep learning: Graph representation learning on relational databases. In ICML, 2024.
C Gao, X. He, D. Gan, X. Chen, F. Feng, Y. Li, T. S. Chua, and D. Jin. Neural multi-task recom- mendation from nulti-behavior data. ICDE, 2019.
A. Grover and Leskovec. node2vec: Scalable feature learning for networks. In SIGKDD, 2016.
H. Guo, R. Tang, Y. Yc, Z. Li, and X. He. DeepFM: A factorization-mant machine based neural network for CTR prediction, 2017.
Xiangnan H. and Tat-Seng Neural factorization machines for sparse predictive analytics. In SIGIR, 2017.
W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.
R. He and McAuley, Fusing similarity models with Markov chains for sparse sequential recom- mendation. ICDM, 2016.
X. He, Liao, H. Zhang, Nie, X. Hu, and T. Chua. Neural collaborative filtering. In WWW, 2017.
X. Hc, K. Deng, X. Wang. Y. Li, Y. Zhang, and M. Wang. LightGCN: Simplifying and powering graph convolution network for recommendation. In SIGIR, 2020.
Z. He, W. Liu, W. Guo, J. Qin, Y. Zhang, Y. Hu, and R. Tang. A survey on user behavior modeling in recommender systems. In I/CAI, 2023.
W. Hu, Y. Yuan, Zhang, A. Nitta, Cao, V. Kocijan, J. Leskovec, and M. Fey. PyTorch Frame: A modular framework for multi-modal tabular leaming. CoRR, abs/2404.00776, 2024.
Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In ICDM, 2008.
S. Huang, F. Poursafaei, J. Danovitch, M. Fey, W. Hu, E. Rossi, J. Leskovec, M. Bronstein, G. Rabusseau, and Rabbany. Temporal graph benchmark for machine learning on temporal graphs. NeurIPS, 2023.
B. Jin, C. Gao, X. He, D. Jin, and Y. Li. Multi-hehavior recommendation with graph convolutional networks. In SIGIR, 2020.
Johnson, M. Douze, and H. Jégou. Billion-scall similarity search with GPUs. IEEE Transactions Big Data, 7(3):535-547, 2019.
11

---

Published as a conference paper at ICLR 2025
W. Kang and J. McAuley. Self-attentive sequential recommendation. In ICDM, 2018.
G. Ke, Q. Meng, T. Finley, Wang, W. Chen, Ma, Q. Ye, and T. Liu, LightGBM: A highly efficient gradient boosting decision tree. In NIPS, 2017.
N. Kipf and M. Welling. Varitional graph auto-encoders. In NIPS Workshop on Bayesian Deep Learning, 2016.
Y. Koren, R. Bell, and Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-33) 2009..
J. Li, P. Ren, Z. Chen, Z. Ren, Lian, and Ma. Neural attentive session-based recommendation. New York, NY, USA, 2017.
Y. Li, K. Liu, R. Satapathy, S. Wang, and E. Cambria. Recent developments in recommender systems: A survey, IEEE Computational Intelligence Magazine, 19:78-95, 2024.
D. Liang, R. Krishnan, M. D. Hoffmann, and T. Jebara. Variational autoencoders for collaborative filtering. In WWW, 2018.
Q. Liu, Y. Zeng, R. Mokhosi, and H. Zhang. STAMP: Short-term attentton//memory priority model for session-based recommendation. In SIGKDD, 2018.
C. Ma, L. Ma, Y. Zhang. J. Sun, X. Liu, and M. Coates. Memory augmented graph neural networks for sequential recommendation. In AAAI, 2020.
K. Mao, J. Zhu, X. Xiao, B. Lu, Z. Wang. and X. He. UltraGN: Ultra simplification of graph convolutional networks for recommendation. In CIKM, 2021.
A. Mnih and R. R. Salakhutdinov. Probabilistic matrix factorization. In NIPS, 2007.
A. Paszke, S. Gross, Massa, A. Lerer, Bradbury, G. Chanan, T. Killeen, Lin, Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang. 7. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and Chintala. PyTorch: An imperative style, high-performance deep learning library. In NeurlPS, 2019.
B. Perozzi, Al-Rfou, and S. Skiena. DeepWalk: Online learning of social representations. In SIGKDD, 2014.
Rendle, C. Freudenthaler, Z. Gantner, and I.. Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback. In UAI, 2009.
Rendle, C. Freudenthaler, and Schmidt- Schmid--Thimme Factorizing personalized Markov chains for next-basket recommendation. In wwW, 2010.
Ricci, L. Rokach, and B. Shapira. Introduction recommender systems handbook. In Recom- mender systems handbook. 2010.
Robinson, Ranjan, W. Hu. K. Huang. Han, Dobles, M. Fey, J. E. Lenssen, Yuan, Z. Zhang, X. He, and J. Leskovec. RelBench: A benchmark for deep learning on relational databases. In NeurlPS, 2024.
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling. Modeling relational data with graph convolutional networks. In The Semantic Web, 2018.
Sun, J. Liu, J. WU, C. Pei, X. Lin, W. Ou, and P. Jiang. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM, 2019.
K. K. Teru, E. G. Denis, and W. L. Hamilton. Inductive relation prediction by subgraph reasoning. In ICML, 2020.
C. Wang, M. Zhang, W. Ma, Y. Liu, and Sh. Ma. Make a chorus: knowledge-ad and time-aware item modeling for sequential recommendation. In SIGIR, 2020a.
12

---

Published as a conference paper at ICLR 2025
J. Wang, K. Ding, L. Hong, H. Lui, and J. Caverlee. Next-item recommendation with sequential hypergraphs. In SIGIR, 2020b.
X. Wang, X. He, M. Wang, F. Feng, and T. Chua. Neural graph collaborative filtering. In SIGIR, 2019.
Webber. Powering real-time recommendations with graph database technology. Technical report, Neo4J, 2021.
J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, and X. Xie. Self-supervised graph learning for recommendation. SIGIR, 2021.
L. Wu, Z. Zheng, Z. Qiu, H. Wang, H. Gu, T. Shen, C. Qin, C. Zhu, H. Zhu, Q. Liu, H. Xiong, and E. Chen. A survey large language models for recommendation. CoRR, abs/2305. 19860, 2023.
L. Xia, C. Huang. Xu, Dai, Zhang, and Bo. Multiplex behavioral relation learning for recommendation via memory augmented transformer network. In SIGIR, 2020.
L. Xia, C. Huang, Y. Xu, and J. Pei. Multi-behavior sequential recommendation with temporal graph transformer. In TKDE, 2022.
Y. Yang, C. Huang, L.. Xia, Y. Liang, Y. Yu, and C. Li. Multi-hehavior hypergraph-enhanced trans- former for sequential recommendation. In SIGKDD, 2022.
You, Ying, and J. Leskovec. Position-aware graph neural networks. In ICML, 2019.
You, M. Gomes-Selman R. Ying, and J. Leskovec. Identity aware graph neural networks. In AAAI, 2021.
J. Yu, H. Yin, X. Xia, T. Chen, L. Cui, and Q. V. H. Nguyen. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In SIGIR, 2022.
H. Zeng, M. Zhang, Y. Xia, A. Srivastava, A. Malevich, R. Kannan, V. Prasanna, Jin, and R. Chen. Decoupling the depth and scope of graph neural networks. In NeurIPS, 2021.
M. Zhang and Y. Chen. Link prediction based on graph neural networks. NIPS, 2018.
Z. Zhu, Z. Zhang, L. Xhonneux, and J. Tang. Neural Bellmann-Ford networks: general graph neural network framework for link prediction. In NeurIPS, 2021.
13


Reviews:
[{"review_id": "FbZZ0xrEJr", "reviewer": "Reviewer_GUBd", "summary": "The paper proposes ContextGNNs, a new architecture for graph-based recommendation. Unlike previous similar approaches, such as pair-wise recommendation solutions, the authors propose a pair-wise representation only for exploring the items in the user's local subgraph, while the architecture employs the usual two-tower method for the item side. Finally, a network is designed and trained to fuse the two representations and provide the final recommendations. \n\nOn the one hand, the ContextGNNs framework first generates a pair-wise representation for users and items, where a $k$-hop subgraph for each user is sampled, and a GNN is leveraged to obtain embedding representations and the overall item ranking. On the other hand, a two-tower representation is exploited on the item side, as it can be shown that: 1) GNNs do not learn useful information on the item side as the item interaction matrix tends to be very dense; 2) shallow item embeddings are effective; 3) GNN item representations can scale poorly with large numbers of items in the catalog. The so-learned item embeddings are injected into the GNN forward pass presented above, to allow the user representations to align with the item ones. Finally, an MLP is trained to fuse the two designed contributions. \n\nResults on different tasks and datasets from the RelBench platform demonstrate the efficacy of the ContextGNNs proposed baseline, also when compared to different settings of the same model.", "questions": "I have a question that does not refer to the one weakness raised. As observed by the authors in the locality score study, it is interesting to see that many validation/test links are not in the users' nearest neighborhood. Moreover, we know from the literature, that graph-based recommender systems may be affected by oversmoothing. On such a basis, my question is: how the graph-based model can learn long-range (and, evidently, ground-truth) products connected to users if the model cannot explore on more than ~3 hops (on average)? Do you have any intuitions for that?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "\\+ The proposed approach is well-placed within the existing literature on (graph)-based recommendation, and the motivations behind ContextGNNs are quite solid and sound\n\n\\+ Overall, the paper is well-written and easy to follow\n\n\\+ I appreciated the locality score study, which conceptually and empirically supports the rationale behind the approach\n\n\\+ I also appreciated how the authors discussed possible extensions of the proposed approach in different and interesting new directions\n\n\\+ Even considering the page limitations, I believe the experimental setting is adequately extensive to empirically support the rationale behind the paper", "weaknesses": "\\- To my understanding, it seems that the list of baselines for the experiments is not completely diversified and updated to the latest advances in graph-based recommendation. I would have appreciated seeing more recent and popular approaches being tested against ContextGNNs, such as LightGCN (cited in the paper), SGL [i], UltraGCN [ii], SimGCL [iii], and LightGCL [iv].\n\n**References**\n\n[i] Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, Xing Xie: Self-supervised Graph Learning for Recommendation. SIGIR 2021: 726-735\n\n[ii] Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, Xiuqiang He: UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation. CIKM 2021: 1253-1262 \n\n[iii] Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, Quoc Viet Hung Nguyen: Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation. SIGIR 2022: 1294-1303\n\n[iv] Xuheng Cai, Chao Huang, Lianghao Xia, Xubin Ren: LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. ICLR 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wfXbJkRpCY", "reviewer": "Reviewer_uwX2", "summary": "This paper studies the two-tower architecture for recommender systems. They introduce a context-based GNNs-based framework for link prediction in recommendation by using a pair-wise representation technique for similar items in the user's local subgraph. The proposed method achieves promising performance in their experiments.", "questions": "See the weakness.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The research issue is very important in the industry. \n\n2. Their experimental results demonstrate the effectiveness of the proposed method. \n\n3. The paper is easy to follow.", "weaknesses": "However, there are some concerns in the paper:\n\n1. The novelty of the proposed method is unclear.\nThis paper leverages local interaction graph for representation learning. The idea is very mature and has been well-studied. \n Compared with exciting methods, what are the novelty of the proposed method? Additionally, the related work in the paper can be enhanced. \nThis paper can be improved by discussing more advanced two-tower architectures. Furthermore, the related work cannot demonstrate the novelty of the proposed method.\n\n\n2. The technical contributions of this work are also unclear. I was wondering what are the main challenges they are addressing. \nThe proposed idea looks good. However, the proposed method doesn't have any theoretical analysis.\nThe model design didn't motivate well. \nThis paper considers the temporal factor in modeling recommendation. However, what are the technical contributions for taking advantage of it?\n\n\n3. The experimental section can be enhanced before publication. First, they can compare with some advanced recommendation baselines. There are many advanced recommendation methods for modeling user and item representations. \nSecond, the datasets here are unclear to me. Whether the proposed method can be used for large-scale datasets?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6h9tgHcOo4", "reviewer": "Reviewer_pueu", "summary": "This paper proposes a recommendation framework that integrates pair-wise representations with two-tower representations. Specifically, it leverages pair-wise representations to capture fine-grained patterns of past user-item interactions, while utilizing two-tower representations for \"distant\" items. The authors have validated the effectiveness of their approach on the RELBENCH benchmark.", "questions": "Please refer to the weaknesses", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The motivation behind the proposed method is clear and has significant practical implications for large-scale recommendation systems in the real world.\n2. The authors have validated the effectiveness of the proposed method on the benchmark datasets.", "weaknesses": "1. The authors emphasize in the abstract and introduction that their method is a relatively general and effective strategy in recommendation scenarios. However, the experimental section only tests the method on datasets that include rich multi-behavioral and temporal interactions. This discrepancy between the claims and the experimental validation raises concerns. Therefore, the authors should either revise their overall statements or consider incorporating a wider variety of datasets for comparative analysis.\n2. The experimental section of the paper is quite limited, as it currently only reports the MAP metric and lacks detailed parameter sensitivity analysis. Moreover, the paper does not address how the proposed method's performance may be affected when user behavior information in the dataset is insufficient in type or quantity. Specifically, would we expect a significant decline in performance compared to the baseline algorithms under such conditions? Additionally, I recommend that the authors incorporate a broader range of recent comparison methods to strengthen their empirical evaluation.\n3. The paper does not provide the original code, which raises concerns regarding the reproducibility of the results. Additionally, there are some errors in the equations presented.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "anVy2yahQQ", "reviewer": "Reviewer_U3kG", "summary": "* The method uses a pair-wise representation for familiar items in a user's local subgraph and two-tower representations for recommending exploratory items. A final network combines both pair-wise and two-tower recommendations to create a unified ranking of items.\n\n* The study utilizes CONTEXTGNNs in relational deep learning for recommendations within relational databases, which consist of various tables with rich multi-modal features and complex behavioral interactions. This environment provides an ideal setting for testing the model's effectiveness. Additionally, the research explores the challenges of modeling intricate human behavior patterns.", "questions": "* The paper mentions the complexity issues of pair-wise methods. Could you provide a time complexity analysis and runtime efficiency experiments of this work compared to others?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* This article has a clear structure and is easy to follow.\n\n*Tested different tasks, which is rare in previous work.", "weaknesses": "* It appears that this work has not released the code or dataset, making reproducibility uncertain.\n\n* This work lacks technical innovation and seems quite basic. Such a simple design should emphasize time complexity, runtime efficiency, and scalability, but the paper does not seem to address these aspects.\n\n* This work does not compare with other recent baselines or commonly used models in the industry. Additionally, it does not provide specific data from the dataset. Whether in industry or research, this work does not seem to fit well. Overall, it appears too basic and simple, with relatively little effort involved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1MnD2QUSVP", "reviewer": "Reviewer_7mVo", "summary": "This paper presents ContextGNN, a refined approach that classifies items into user-specific selections within a user-centric local graph, alongside other items. Subsequently, the GNN learns the pertinent user and item embeddings, thereby enabling the derivation of both pairwise and two-tower representations for enhanced recommendations.", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and easy to understand.\n2. The use of both pairwise and two-tower representations for recommendations is quite interesting.\n3. The experiments conducted in this paper seem to demonstrate the model's effectiveness.", "weaknesses": "1. The author should consider open-sourcing the code for the proposed method to enhance reproducibility.\n2. In the comparison of methods, it would be beneficial to include more recent and advanced GNN-based recommender models beyond NGCF.\n3. Given that the author mentions both multi-behavior and temporal graphs, it would be advantageous to illustrate how the GNN backbone is designed to accommodate such graphs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)