Multi-Behavior Dynamic Contrastive Learning for Recommendation
Authors: Wei Wei
Chao Huang
Lianghao Xia
Yanwei Yu
Chuxu Zhang
Venue: Submitted to ICLR 2023 (2023)
Keywords: Multi-Behavior Recommendation
Dynamic Contrastive Learning

Abstract: Dynamic behavior modeling has become an essential task in personalized recommender systems for learning the time-evolving user preference in online platforms. However, most next-item recommendation methods follow the single type behavior learning manner, which notably limits their user representation performance in reality, since the user-item relationships are often multi-typed in real-life applications (e.g., click, tag-as-favorite, review and purchase). To offer better recommendations, this work proposes Evolving Graph Contrastive Memory Network (EGCM) to model dynamic interaction heterogeneity for multi-behavior sequential recommendation. Specifically, we first develop a multi-behavior graph encoder to capture the short-term preference heterogeneity, and preserve the dedicated relation semantics for different types of user-item interactions. In addition, we design a dynamic cross-relational memory network, empowering EGCM to distill the long-term multi-behavior preference of users and the underlying evolving cross-type behavior dependencies over time. To enhance the user representation with multi-behavior commonality and diversity, we design a multi-behavior contrastive learning paradigm with heterogeneous short- and long-term interest modeling. Experiments on several real-world datasets show the superiority of our recommender system over various state-of-the-art baselines.

Under review as a conference paper at ICLR 2023
MULTI-BEHAVIOR DYNAMIC CONTRASTIVE LEARN- ING FOR RECOMMENDATION
Anonymous authors Paper under double-blind review
ABSTRACT
Dynamic behavior modeling has become an essential task in personalized recom- mender systems for learning the time-evolving user preference in online platforms. However, most next-item recommendation methods follow the single type behavior learning manner, which notably limits their user representation performance in ality, since the user-item relationships are often multi-typed in real-life applications (e.g., click, tag-as as-favorite, review and purchase). To offer better recommenda- tions, this work proposes Evolving Graph Contrastive Memory Network (EGCM) model dynamic interaction heterogeneity for multi-behavior sequential recom- mendation. Specifically, we first develop muuti--bhhavir graph encoder to capture the short-term preference heterogeneity, and preserve the dedicated relation seman- ties for different types of user-item interactions. In addition, we design dynamic cross-relational memory network, empowering EGCM to distill the long-terny multi bulti-behhavor preference of users and the underlying evolving cross-tyre type behavior dependencies over time. obtaín robust and informative user representation with multi-behavior commonality and diversity, we design multi-behavin contrastive learning paradigm with heterogeneous short- and long-term interest modeling, and provides theoretical analyses support the modeling of commonality and diversity. Experiments on several real-world datasets show the superiority of our recommender system over various state-of-the-ar baselines.
INTRODUCTION
I Learning user's dynamic preference plays a vital role in recommender systems to predict the next items that users may be interested in |Wan et al. (2019). For example, a family may buy chicken and bread on an online platform for long time because of their daily needs, and also buy turkeys close to Christmas. The recent advances of neural network architectures has inspired many efforts to model the transitions between temporally-ordert of deep learning techniques, e-g., recurrent neural encoder Hidasi et al. (2016), convolution based model Tang & Wang (2018) and attention mechanism Kang & McAuley (2018). More recent sequential recommender systems are built upon the Transformer Sun et (2019)) Graph Neural Networks (GNNs) Wu et al. (201)): Ma et al. (2020); Wang et al. (200c)) provide state-of the-art recommendation performance. Despite their effectiveness, most of existing next-item recommendation approaches rely on only single type of user-item interaction (e.g.. click or purchase data), and thus are limited to capture the item-level multi-behavior interaction patterns.
items, due to the strong representation capability
[ Liu et al. (2021b)
In real-life recommendation scenarios, users often interact with items in various ways, based on their interests which are intrinsically time-evolving and diverse. For instance, different types of user behaviors (e.g.. page view, favoorit, purchase) online retailers may reflect diverse user intentions and heterogeneous user-item relationships Guo et al. (2019); Jin et al. (2020b). Leaving this fact untouched, single type of behavior modeling in previous chronological user embedding functions is insufficient to comprehensively capture diverse user intents with behavior heterogeneity Xia et al. (2021a). Hence, time-evolving multi-bchavior representations can characterize the various latent factors behind user-item interactions, and maintain dedicated embedding space for different types of dynamic user behaviors in recommender systems.
While having realized the importance of modeling behavior-award time-evolving user-item relation- ships in recommendation, some key challenges remain to be carefully tackled. Specifically, (1) How to explicitly preserve the dynamic behavior-specific semantics pertinent to each type of user-item

---

Under review as a conference paper at ICLR 2023
interactions over time then delivering and retaining user preferences, is not trivial multi-behavvor sequential recommendation. It is critical in for the recommender to distill such heterogeneous item- level dependencies with the jointly modeling of short-term and long-term user interests. (2) Learning informative and robust representations of multiplex user-item interactions requires a tailored modeling with performant recommendation paradigm, which towards the encoding of users' multi-hehavior commonality and diversity. While we can embed behavior-specific semantics into individual latent vectors, the understanding of multi-behavior commonality underlying global view of user-specific dynamic preference is critical to multi-behavior modeling.
Contributions. In light of these challenges, we propose an Evolving Graph Contrastive Memory (EGCM) framework that can effectively distill the heterogeneous user intentions over time from multi-behavior data in recommendation. Specifically, we first introduce multi-hehavior graph encoder equipped with temporal context embedding for modeling the behavior-aware short-term interests of users. Furthermore, a dynamic colst-nelaiiona memory network based on self-attention is proposed to incorporate heterogencous cross- wooss-behavior dependencies into learning user dynamic preferences with cross-behavior relational transitions. In a nutshell, this dynamic multi-behavior modeling allows to characterize diverse user intents from the long-term perspective behind the interacted item sequence. To enhance the generalizability and robustness of our recommender, we design our multi- contrastive learning paradigm endow EGCM with the capability of encoding multi-behavior commonality and differentiating the behavior-aware preference of various users. We also provide theoretical analysis of our EGCM model in Supplementary Section.
To summarize, the key contributions of this work are presented as follows:
Emphasizing the importance of jointly learning of dynamic preference heterogeneity with multi- behavior data and diverse user behavior-aware interests for recommendation.
Proposing a new model EGCM, which integrates the dynamic cross-relational dependency model- contrastive learning paradigm, so as to distill the evolving user-item
ing with the relationships at the fine-grained level user preferences. In addition, we perform the theoretical analysis of our proposed EGCM model as presented in the Supplementary Section.
Conducting experiments on three real-world datasets to demonstrate the superiority of EGCM. Further ablation studies and in-depth model analysis justify the rationality of our model design. To support the reproducibility of our experimental results, the model implementations can be found at the anonymous link:
2 RELATED WORK
Next-item/SSqueetial Recommendation, Early studies (e.g., FPMC Rendle et al. (2010) rely on the Markov chain to tackle the sequential recommendation problem. Many recent efforts have been devoted to learning users' dynamic interests with various neural network encodes, such as the RNN-based method GRU4Ree Hidasi al. (2016) and CNN-based approach Caser Tang & Wang] 20188. In addition, several self-attention re relational learning models are introduced to estimate the item correlations, e.g., SASRec Kang & McAuley (2018), BERT4Rec [Sun al. (2019) and
TiSASRec Li et al. 2020). Inspired by the strength of graph neural networks, some recent sequential recommender systems are built over the graph-based based message passing scheme to encode the multi- order dependencies among items, including MA-GNN Ma et al. (2020). SURGE Chang et al. (2021). and GCE-GNN Wang et al. (020c).. Furthermore, self-supervised learning has been used recent sequential recommendation methods for data augmentation, like COTREC Xia et al. (2021c) and DHCN Xia et al. (201cc). However, most of existing methods are built on single type interactions and ignore the heterogeneous behavior-aware user preferences.
Multi-Behavine Recommender System. Recently, considerable attention due to the effectiveness of considering multi-typed user behaviors in boosting the recommendation performance Chen et al. (202T). For example, NMTR [Go et al., (2019) and DIPN Guo et al. (2019) differentiating behavior semantics with multi-task learning schemes. To encode diverse relationships between users and items, some recent studies (e.g., MBGCN Jin et al. (2020b), KHGT Xia et al. (202 and MBGMN Xia et al. (202lb)) attempt to leverage graph neural networks for encoding the melti-behavior patterns, based on their constructed relation- aware heterogeneous user-item interaction graph. One major drawback of existing behavi-o recommender systems is that they mostly focus on the stationary scenarios, while neglect the time- evolving multi-behavior dependencies from diverse user interest representation.
recommendation has gained
2

---

Under review as a conference paper at ICLR 2023
Tenpooal Connatt
E E..
Crass Relation
Atteation Fusion
Message Passing
Cy
fusion
3 click
13 favoriic
10 vO
buy
18]
E. We K = E2-1 w Dynamic Cross-
Short-Term Multi-Behaviine Graph Encoder
Relational Memory
Multi-Behavior Contrastive Learning
The model flow of EGCM framework. The multi-behavior contrastive learning module (right side) augments the graph- ceapheenhanced dynamic memory network (left & middle side) with the auxiliary behavior-award self-supervision signals from both short-term and long-tent mengttem user interests.
Figure I:
Contrastive Representation Learning. As prevalent technique, contrastive learning has shown potential of auxiliary self-supervised signals Liu et al. (201aa] Ho & Nvasconcelos) (2020) generation. For image analysis, many contrastive methods have been proposed to advance the modeling image data with different augmentation techniques /Aberdam et al. (2021) Verma et al.) [: TTia et al. (2020b), such as cropping, horizontal translations and rotations. In addition, cross-view contrastive learning has provided the state-of-the-arr graph representation performance with various augmentation operators, egg., node shuffles in DGI Velickovic et (2019), centrality- awwrre edge dropout in GCA Zhu et al.) (202I) and subgraph sampling in MVGLR Hassani & Khasahmadi (2020). Our work is inspired by the above work and designs a new multi multi-behavior contrastive learning paradigm to simultaneously capture the user-specific behavior commonalities and cross-user behavior diversity.
Liu
METHODOLOGY
We present the technical details of our EGCM model which consists of three key components: Short-term multi-behavior graph encoder that captures the user's short-term interests with multi- relational graph neural networks; ii) Long-term user interest modeling which learns time-evolving multi--ehavior preferences across different time slots; iii) Multi-behavite self-supervised learning which enhances the user representation with the cross-behavior preference commonality and instance self-discrimination.
3.1
SHORT-TERY MULTI-BEHANE GRAPH ENCODER
To capture the dedicated behavior semantics for underlying user's short-term interests, we propose a multi-relational graph encoder handle the heterogeneous item dependencies within each time slot.
3.1.1 SHORT-TERM MULTI-BEHAVION IOR GRAPH
We define the short-term behavior-specifie graph G = E
Eb M)) represent the behavior-aware user-item interactions within the t-th time slot (e.g., day, week, or month) under behavior b. And g is divided from the total interaction graph by time-slot and behavior. In graph G), nodes D = 1dd U Vo and edges& E represent the interactions for specific times and behaviors between user nodes Vh, and item nodes VL, Given the set of edges E we define the user-item interaction matrix M where each entry Mb (. = 1 if user has adopted item under the behavior type of b within the t-th time slot. E B denotes the specific user behavior(e.g.. click, tag-as-favorite, review and purchase). t E denotes the current time slot.
3.1.2 SHHAVIOR-AWARE MESSAGE PASSING
Inspired by recent efforts of utilizing graph neural network to model relational user data He et al. (2020); You et al. (2020). we design behavior- message passing schema to capture the high- order dependencies among items through the recursively embedding propagation across graph layers. Formally, our behavior-awar message passing function from the (1-1 (--1)-th layer the (I)-th layer is
3

---

Under review as a conference paper at ICLR 2023
defined with the following form:
whll)) T(M) M (DD) 2. My (B.) M (DD) -
E a (T(M²) My") E,s 11-11-)
where E7 is the embedding of item i the 1-th graph propagation layer at the t-th time slot under behavior b. (-) is the PReLU He al. (2015) activation function to inject oon-lineristee. r(.) the normalization function to alleviate the large value effects of embeddings during the recursive propagation Wang et al. (020b)) is dedicated for the message passing on user-item heterogeneous bipartite graph, which is different from the symmetrical graph Laplacian of eigenvectors Kippf & Welling 2016). And the two diagonal degree matrices D E x and B² E IR MxX are based on the interaction matrix M.. By stacking multiple embedding propagation layers, the behavior-specific high-order item dependencies can be preserved the generated representations (",(1,, where L denotes the number of propagation layers. And we conduct the concatenation to aggregate the embeddings encoded from each graph layer, the concatenated representation is then sent into a linear layer with normalization E,, = / /|EE,,ll to be reshaped to obtain output item representation E,,, of out short term behavior-aware graph neural network.
After encoding the item embeddings by exploring the high-order dependencies among items, we generate the user representations which maintain the behavior-specific short-term interests:
E (I((M)) E wi,)): r(M") (B²) M T (D.)
where EY,, is corresponding to users' short-term interests at the f-th time slot under the behavior type of b after normalization E EY,,, ( || over adjacency matrix M (u,,)) = 1/(v Nu. (Wi)) with applying r(.).
ll2. And the message passing from item is also
3.1.3 PRIORI-AWARE INITIALIZATION & TEMPORAL CONTEXT INJECTION
In our short-term multi-behavior graph encoder, we rely on the time slot-specific user-item interaction data, which may lead overfitting issue. In light of this limitation, develop priori-aware previous time slot (t 1) into
embedding initialization strategy which injects the representations the embedding initialization of current time slot L, in order to reduce the training difficulty and deterioration Glorot & Bengio (201)). Formally, the priori-aware embedding initialization is defined as follows:
t. = TEE iniit))
(c init - (1- E.11.) W
where Y(-) is the initial function for preprocessing the Xavier initialied embedding E. E,,, for current behavior-specific short-term graph neural network which is apply before Eq. [ Eq. ??It's output Ex, denotes the input embeddings for the short-term behavior -peeiic graph G at the t-th time slot. E E, represents the representations encoded from the previous (t - 1) time slot. Here, and w are learnable aggregation weight and projection matrix, respectively.
After representation learning, inject temporal context signals into representation, temporal context encoder is built over the positional encoding technique Vaswani et al.) (2017). Specifically, the sinusoid functions was introduced generate the relative time embedding for each user-item interaction edge. By doing so, we enhance our short-term multi-behavior graph encoder with the capability of capturing behavior-aware interaction dynamics of users with the temporal context injection.
3.2 DYNAMIC recommender system, long-term multi-behavior dynamics reflect a holistic In the multi-behavior view of diverse preferences of users across different time slots. And different types of user behaviors are inter-dependent among time slots in long-term perspective. For example, customers may first view some products in online retail sites, and make their purchase decisions one day after. To capture and convey such evolving cross-type behavior dependencies across time slots, propose a dynamic cross-relle relationla memory network to learn the evolving cross-type behavior dependencies over time in long-term manner.
MEMORY NETWORK
4

---

Under review as a conference paper at ICLR 2023
In order to form memory network, we first learn the information to be remembered, i.e., learn time-evolving cross-type behavior dependencies. Technically, it can be implemented by explicitly learning the influence weights between behavior-aware representations during specific time slot. Following the scaled dot-product self-attention function Vaswani et al. (017). we take multi-hehavior information í and
query(Q) and key(K), respectively. And value V) is calculated from the embedding at - 1 to ultimately pass the information from - 1. Formally, our designed dynamic cross-relational memory encoder over adjacent time slot-specific behavior representations follows:
Q2 = H W
(2: d/h (
Q2
2 =
Vi-1
K, H W : - -
(4)
VV-1 H
where 1 aggregates
information of current time slot with the incorporation of cross-type behavior dependencies from the previous (t - 1)-th time slot. H and H are three- way tensors to represent the multi-behavior short-term interest for each individual user prepare for self-attention module by stacking type-specific user(E,,,.) and item(E),) embeddings. W E Rddxd
w E [Rdx represent the linear transformation matrices corresponding to the query and key dimension, respectively. The scale /d/h is introduced produce softer attention distribution for avoiding gradient vanishing Vaswani et al. al.l(1221))))) (2017), and represents the number of head representations. the learned self-attention weight matrix. Thus, each entry Φ6,8 E [8|x & indicates the estimated interrelationship between the behaviors of adjacent time slot of specific
In model implementation, we utilize the Broadcast Mechanism Van Der Walt et al. (2011) to improve the computational efficiency of tensor multiplication. Then, with the learned evolving cross-type behavior dependencies, we refine the time slot-specific user and item representations as:
Er, ((Er, + 11,51,0) E = E(EE, @ 21,-1.))
where E,,,, E., the embedding incorporate cross time information in terms of how various behaviors are related to one another. And pu()) indicates the adopted mean pooling operation. With the design of our dynamic cross-relational memory network, EGCM can preserve the dedicated time-evolving behavior dependencies across different types of user interactions.
3.3 MULTI-BEHAVIOR CONTRASTIVE LEARNING
Having encoding the heterogeneous user interest representations corresponding the short-term (E,,,, Fy.s) and long-term (F4,a; Fr,:) perspectives, we propose to jointly capture the diverse multi- behavior dependencies and commonality in the multi-behavior recommendation scenario.
3.3.1 HETEROGENEOUS BEHAVIOR AGGREGATION
We first generate the multi- multi-behavvor representation (E.,, E,) and (E,,,, Er,)) by aggregating type- specific behavior representations with the attention mechanism for short term and long term, respec- tively. Take long-term representations an example, it can be formally presented as follow:
[8]
eep
E,
E
(6)
B ΣA (E
b=1
where wi represents the learned dependency weight of 6-th type of user behaviors. And W, E Rdd the transformation matrix before softmax. With the behavior aggregation attention mechanism, both short-term and long-term multi-behavior preferences of users can be preserved by the fused representations.
3.3.2 CROSS-BEHAVIE BEHAVIOR CONTRASTIVE SELF-SUPERVISION
In our EGCM model, we aim to simultaneously capture the multi-behavior commonality of individual user, and the multi-behavior distinction of different users. To this end, we design the cross-behavior contrastive learning module to enhance the multi- bettiibehavior dependency modeling with the aug- mented self-supervision signals. Specifically, the type-specific behavior semantics and the fused multi-behaviour pattern. Our EGCM performs the behavior-level augmentation by pulling the type-specific behavior embedding (e² Rdx1) and
propose to generate
contrasting views with
5

---

Under review as a conference paper at ICLR 2023
Table I: Statistics of experimented datasets
Dataset
User # Item # Interaction # Sparsity
Interaction Behavior Types
Taobao
31882
31232
167862
99.98%
(View, Favorite, Cart, Purchase]
UJCA
22438
35573
199654
99.98%
(View, Favorite, Cart, Purchase [Browse, Review, Purchase)
E-commerce
31021
1827
370386
99.35%
belti-bebhavor representation (e; E Rdx1) I) of the same user closer as positive pairs, pushing the behavior embeddings of different users away as negative pairs. Formally, the cross-behavior contrastive objective is defined as:
expls(ee Ew)/T) (erp(s(e), ew -xpps(e e,)/T))
F log
exp(s(e Tw) + Σ au
where represents temperature parameter to control the effect of mutual information estimation Chen et al. 2020b). Here, we define the InfoNCE-based similarity function s(e), C,,) ch / || c. which is measured by the doc product between & normalized e, and e... With incorporating the cross-behavior contrastive self-supervision signals into our recommendation framework as auxiliary regularization, EGCMcan jointly capture the commonality of individual user intent and the diversity of different users with respect to their mulli-behavior preferences. And, we talk about the details of commonality and diversity modeling in the supplementary material.
3.4 MODEL INFERENCE PHASE
For the main recommended task, we define our optimized objective with the Bayesian Personalized Ranking (BPR) loss as follows:
LBPR=
Σ
log (sggmoooiid e, - e, -e; 2||@|2
(8)
where
E E (u, E- represents the pairwise training samples. Here, E+ and E- denotes the corresponding observed and unobserved interaction of user To alleviate the overfitting issue, we apply the L2 regularization our BPR loss, and thus, θ denotes the learnable hyperparameters. Finally, our joint optimization objective is given below by integrating BPR loss
[A] [8|
(CBPR) with short-term
(Σ
Eshort) and long-term C clong) contrastive objectives:
B C = Σ CBPR a Σ
18
IAI ΣΣ
Long c
+ 3
Cshort e
where |A| represents the number of time slots. and 3 are regularization strengths of contrastive objectives. We further provide the in-depth analysis of our EGCM model with respect the time complexity analysis and theoretical discussion in the submitted Supplementary material.
4 EVALUATION
4.1 EXPERIMENTAL SETUP
Datasets. We perform the model evaluation on three real-world datasets with the statistics shown in T i) Taobao is benchmark dataset which collects four types of user behaviors from Taobao's recommender system. ii) IJCAI is a released online retailing dataset by IJCAI competition with four types of user online actívities. iii) E-commerce contains browse, review and purchase behaviors of users in real-life online retailer. Following the same settings in Jin et al. (2020a); Xia et al. (2021b), our behavvbb sequential recommender system regard the purchase as the target behaviors and other types of user behaviors auxiliary behaviors.
Evaluation Protocols. In our evaluation, two representative metrics, i.e., Hit Ratio (HR @N) and Normalized Discounted Cumulative Gain (NDCG@N) (N 10 by default), are adopted to measure the recommendation accuracy. To be consistent with the settings existing sequential recommender systems Sun al. (2019), the leave-one-out evaluation strategy is used for evaluation. Additionally,

---

Under review as a conference paper at ICLR 2023
for each user, the last interacted item under the target behavior type is considered as the positive samples in the test set and 99 non-inteonn con-interacted items are randomly sampled as negative instances.
Baselines. We compare our EGCM with various types of state-of-the-arr recommendation methods.
(i) CNN/Attention--ert based Sequential Recommendation Approaches: Caser Tang & Wang (2018) uses the convolutional filters to encode local item dependencies of user sequences. SASRec Kang & McAuley 2018). TiSASRec Li et al. (020)) AttRec Zhang al. (2018) are built self-attention
mechanism to capture correlations between cemporally-ordere items. Bert4Rec [Sun et al. (2019)
trains the bidirectional Transformer model with the cloze task.
(ii) Hybrid Sequential Recommender Systems: HGN Ma et al. (2019) is a hierarchical gating network to learn the item feature relevance. Chorus Wang et al. (2020a) considers both knowledge-
aware item relations and temporal context in sequential recommendation
(iii) GNN-based Sequential Recommendation Models: SR-GNN Wu et al. (2019) first introduces the graph neural networks to model the sequential behavior patterns over short item sequences. MA- MNNMMa al. (2020) leverages GNNs to encode both short-term and long-term item dependencies. Furthermore, HyperRec Wang et al. (2020b) proposes to capture dynamic triadic item relationships using the hypergraph structures. COTREC Xia et al. (222cc) and DHCN Xia et al. (202 d) are two state-of-the-ea sequential recommender systems based on self superviue learning paradigms. (iv) Multi-Behavior Recommender Systems: NMTR |aao et al.] (2019) and DIPN Guo et al. (2019) formalize the multi-behavior recommendation task with multi-task learning frameworks. MBGCN Jin et al. (2020b), KHGT Xia et al. (2021a) and MBGMN Xia et al. (2021 design multi-behavee graph message passing schemes model heterogeneous user-item interactions. EHCF Chen et al. (2020a) and CML Wei et al. (2022) generate additional supervision signals from auxiliary behaviors boost the recommendation performance.
Table 2: Performance comparison of all methods on different datasets in terms of HR & NDCG.
Att
SAS
TISAS
BERT
SR-
MA-
Hyper Rec
Caser
HGN
Chorus
DHCN
Rec
Rec
Rec
4Rec
GNN
GNN
Tmall
H@
0.321
0.328
0.319
0.322
0.329
0.283
0.335
0.318 0.331
0.333
0.321
N@ 10
0.195
0.197
0.184
0.185
0.197
0.172
0.201
0.189
0.202
0.204
0.193
ICCA
H@ 10
0.257
0.261
0.263
0.262
0.281
0.251
0.270
0.280
0.259
0.265
0.271
N@ 10
0.146
0.147
0.148
0.148
0.155
0.136
0.149
0.151 0.149
0.151
0.148
E-commerce
H@ 10
0.627
0.628
0.599
0.599
0.641
0.586
0.639
0.620
0.644
0.648
0.638
N@
0.381
0.387
0.368
0.368
0.392
0.359
0.402 0.373 0.411
0.413
0.391
COTREC NMTR DIPN MBGCN KHGT MBGMN EHCF CML EGCM Impry. p-value
Tmall
H@
0.330
0.362
0.325
0.381
0.391
0.419
0.433
0.543
0.597
9.94%
5.2e
N@ 10
0.201
0.215
0.193
0.213
0.232
0.246
0.260
0.327
0.363
11.01%
4.9c-0
IJCA
H@ 10
0.278
0.269
0.276
0.270
0.278
0.329
0.362
0.410
0,510
24.39%
9.6e
N@
0.153
0.156
0.151
0.138
0.145
0.176
0.207
0.235
0.312
32.77%
2.3e
E-commerce
H@1 10
0.647
0.651
0.655 0.679
0.689
0.690
0.611 0.719 0.763 6. 8.7e 3
N@
0.405
0.408
0.397
0.414
0.434
0.432
0.413 0.427 0.470 10.07% 3.4e-4
save space, we folded the table two parts, Where the bolded columns are our results, followed by the improsment p-value for the best results.
Hyperparameter Settings and Implementation Details. We implement our model in Pytorch and adopt Xavier initializer Glorot & Bengio (010) for parameter initialization. The Cyclical Learning Rate strategy Smith (2017) is used during the model training phase with the AdamW Loshchilov & Hutter (2017) optimizer. For fair comparison, the number of graph layers in all GNN-based models are selected from the range (1,2,3.4) to achieve the best performance. The temperature coefficient tuned from (0.02, 0.035, 0.05, 0.07, O. 0.3, 0.5, 0.7) in our multi-behavior contrastive learning component. We further study the influence of key hyperparameters in our model.
7

---

Under review as a conference paper at ICLR 2023
4.2 RECOMMENDATION PERFORMANCE
We report the performance comparison results in lableeJand summarize the findings: (1) EGCM outperforms various baselines in all cases by achieving significant performance improvements. The "imprv" indicates the relative performance improvement between EGCM and the best performed baseline CML. Through the encoding of the evolving dependencies across different types of be- haviors, EGCM is able to simultaneously capture the dynamics of users' short-term and long-term interests, by distilling the underlying heterogeneous interaction patterss. (2) EGCM outperforms the compared sequential recommender systems by a large margin, which indicates that the incorpora- tion of multi-behavior context is beneficial for disentangling behavior heterogeneity of users. (3) Conducting dynamic contrastive learning with cross-behavior dependencies, EGCM learns better multi-behavior representations compared with state-of-the the-art multi-behavior recommendation ap- proaches, by preserving both the behavior commonality and diversity of users. (4) The consistent performance improvements of our method on datasets with different sparsity degrees, benefits from the incorporation of effective self self-supervision signals generated by our contrastive learning paradigm.
4.3 ABLATION STUDY
We evaluate the efficacy of key components in EGCM with different variants: (1) w/oJBL: Instead of performing jointly learning with multi-typed behavior supervision labels, we directly incorporate auxiliary behavior data as contextual features for user representation. (2) w/o-CL: We further disable our multi-behavior contrastive learning component to capture the behavior commonality and diversity, (3) r/v-GRU: We replace our dynamic relationla memory network with gated recurrent unit (GRU) to encode behavior-specific embedding sequences. (4) w/o-MBG: On the basis of w/o- CL, we remove the behavior-aware graph neural encoder, to model the high-order connectivity over the multi-behavior user-item interaction graph within specific time slot.
From evaluation results shown Table 3 we can observe that our multi-hehavior learning with augmented short- and long-term contrastive objectives improves the generalization of multi- recommender system. In addition, our cross-relational memory network is more effective than GRU to encode long-tern multi muult--ehhavior user interests. Furthermore, the multi-behavior graph encoder brings positive effects to learn short-term heterogeneous user preferences, compared with directly information aggregation over id-corresponding embeddings for collaborative effect modeling.
4.4 IN-DEPTH ANALYSIS OF EGCM MODEL
Commonality and diversity. For cross-behavior user preference commonality, to observe the effect of mutual information maximization, in Figure ( (a)dd), we visualize the encoded behavior-ppecife user embeddings of our EGCM and the variant w/o-CL (without the multi-behavior contrastive learning) on Tmall and IJCAL respectively. Theoretically, limiting contrastive loss is maximizing the lower bound for mutual information and the rationality of multi-view contrastive learning SSupllmmnnary materials 1.1.1), which support this operation. The visualization suggest that our cross- -ehssbb multi-view contrastive paradigm is effective to preserve user's 's multi-behavior commonality by bringing the multiple behaviors semantics close to each other, rather than keep behavioral representations in their own space. For user representation diversity, as detailed in the Supplementary Material.A. 1.2 the gradient of the negative sample sample of the anchor(each user u) is proportional related as follow equation:
dxx) x v1 - (x)² exp (x/7)
(10)
where e(x) is the relationship function of the gradient from the negative samples, x represents the similarity between the anchor node and the paired sample(e.g.. g. cy C. E [-1, And we plot c(x) in right part of Figure 2 (e) when temperature coefficient r {0.02, 0.03, 0. It can be observed that the gradient appears to grow exponentially, when decrease while similarity x increase gradually. Explained from the perspective of the recommendation task, anchor node indicates specific user, and negative samples represent other [Vul user, i.e., the over-smoothing may have occurred and result in indistinguishable user representations. In this case, the contrastive loss will give a larger gradient to those users to pushed them away. Therefore, with cross-behavior contrastive self-supervision, EGCM can better differentiate diverse user preference and enhance the user representations with can be seen from left part of Figure 2 (e) that the temperature decreases which, the more distinguishable the user representation is, which brings the better effect. However, when the gradient is quite large, the 'NaN' value which due to gradient explosion will be observed. as shown in Figure
users. If other users are close to the anchor
diversity. Therefore,
8

---

Under review as a conference paper at ICLR 2023
Table 3: Ablation study the effectiveness of components in EGCM.
Data
Tmall
DCCA
E-commerce
Metrics
HR@10
NDCG@ 10
HR@1O
NDCG@ 10
HR@10
NDCG@ 10
w/o-MBG
0.338
0.219
0.261
0.139
0.630
0.372
r/w LRU
0.361
0.210
0.276
0.148
0.675
0.417
w/o-CI
0.391
0.231
0.300
0.166
0.734
0.461
w/ w//oJBL
0.473
0.291
0.367
0.205
0.747
0.469
EGCM
0.597
0.363
0,510
0.312
0.763
0,470
N@10 Tmail
So
0.0-0.03
0.00
Tempar perature
Constrastive Ratio
L00
Similarity
(1) wio-CL Taobao
(b) EGCM en Taobao
(c) Impact of Multi-echavin Behavior Contrastive Leaming
(c) wlo-Cl. on UCAI
(d) EGCM on UCAI
(f) The Leamnd Dynamic Cross-Behavior Dependencies
Figure 2: (a)-(b): Visualized behavior-aware user representations which preserve multi-behhavor commonality and cross-user behavior diversity. (e): Impact study of contrastive learning terms of temperature parameter 7see Sec. A.1. for details). (f): Learned cross-behavior dependency weights across time slots.
(c), we can notice that a smaller value of temperature value 7 may bring larger gradients for better identifying hard negatives, as to enhance the model discrimination ability in learning personalized user interests.
Case study of memory network. iii) In Figure 2 (f), we visualize the learned cross-behavior dependencies between time slots with 4 x 4 weight matrices Φ E T [ [6] for 400 sampled users on Taobao dataset(with Page View, Favorite, Cart, Purchase four kinds behaviors). Then, we select six cases from the sampled data and combine the real statisties of multi-behavior data to analyze the significance of the learned weight matrices. can be observed in Figure that the weight of each behavior is related to the number of interactions of the behavior. For example, for user 22186 in Figure [] with 111 interactions in page view behavior, the number of interactions in other behaviors 5, 4) which too small relatively. Thus it is difficult to learn differentiated values in the other three rows relative to the first row of the weight matrix. In addition, it can be seen from the thumbnail, most matrices have the darkest diagonal color, which is the characteristic of self-attention.
CONCLUSION
This work studies the problem of multi-hehavior sequential recommendation and propose our model that captures the interaction heterogeneity of each user both the short-term and long-term level interests. In addition, a multi-behavior contrastive learning paradigm is presented to not only model the multi-behavior commonalities of individual user, but also enhances the distinction behavior- aware preference of different users. Experiments on three real- weal-world datasets demonstrate that EGCM significantly advances the recommendation performance compared with many strong baselines. In the future, we plan to extend EGCM to adapt to the cross-domain recommendation to tackle the cold-start problem in multi-behavior recommender systems.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Aviad Aberdam, Ron Litman, Shahar Tsiper, Oron Anschel, Ron Slossberg, Shai Mazor, R Manmatha, and Pietro Perona. paueence-to-sqqeenee contrastive learning for text recognition. In CVPR, pp- 15302-15312, 2021.
Philip Bachman, Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views. NIPS, 32, 2019.
Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. Sequential recommendation with graph neural networks. In SIGIR, pp. 378-387, 2021.
Chong Chen, Min Zhang, Yongfeng Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Efficient hetcrogeneous collaborative filtering without negative sampling for recommendation. In AAAI, volume 34, pp. 19-26, 2020a.
Chong Chen, Weizhi Ma, Min Zhang, Zhaowei Wang, Xiugiang He, Chenyang Wang, Yiqun Liu, and Shaoping Ma. Graph heterogeneous multi-relational recommendation. In AAAI, volume 35, pp. 3958-3966, 2021.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, pp. 1597-1607. PMLR, 2020b.
Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018.
George H Dunteman. Principal components analysis. Number 69. Sage, 1989.
Chen Gao, Xiangnan He, Dahua Gan, Xiangning Chen, Fuli Feng, Yong Li, Tat-Seng Chua, and Depeng Jin. Neural multi-tast Masti recommendation from multi-behavite data. In ICDE, pp. 1554-1557. IEEE, 2019.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, pp. 249-256. JMLR Workshop and Conference Proceedings, 2010.
Long Guo, Lifeng Hua, Rongfei Jia, Binqiang Zhao, Xiaobo Wang, and Bin Cui. Buying or browsing?: Predicting real-time purchasing intent using attention- basudd deep network with multiple behavior. In KDD, pp. 1984-1992, 2019.
Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In ICML, pp. 4116-4126. PMLR, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In CVPR, pp. 1026-1034, 2015.
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In WWW, pp. 173-182, 2017.
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In SIGIR. ACM, 2020.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In ICLR, 2016.
Devon Hjelm, Alex Fedorov, Samuel Lavoie- Marihl-Mr,,,,,oo,, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint
808.006700.0.00 2018.
Chih-Hui Ho and Nuno Nvasconcelos. Contrastive learning with adversarial examples. In NIPS, volume 33, pp. 17081-17093, 2020.
Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. Multi-behavior recommendation with graph convolutional networks. In SIGIR, 2020a.
10

---

Under review as a conference paper at ICLR 2023
Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, et al. Multi-behavior recommendation with graph convolutional networks. In SIGIR, pp. 659-668, 2020b.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In JCDM, pp. 197-206. IEEE, 2018.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint wrXiv:16090099000 2016.
Jiacheng Li, Yujie Wang. and Julian McAuley. Time interval aware self-attention for sequential recommendation. WSDM, pp- 322-330, 2020.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, volume 32, 2018.
Xiao Liu, Fanjin Zhang, Zhenyu Hou, Li Mian, Zhaoyu Wang, Jing Zhang, and Jie Tang. Self- supervised learning: Generative or contrastive. Transactions on Knowledge and Data Engineering (TKDE), 2021a.
Zhiwei Liu, Ziwei Fan, Yu Wang, and Philip S Yu. Augmenting sequential recommendation with pseudo-prior items via reversely pre-training transformer. In SIGIR, pp. 1608-1612, 2021b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, In ICLR, 2017.
Chen Ma, Peng Kang, and Xue Liu. Hierarchical gating networks for sequential recommendation, In KDD, pp. 825 833,833,2222 2019.
Chen Ma, Liheng Ma, Yingxue Zhang, Jianing Sun, Xue Liu, and Mark Coates. Memory augmented graph neural networks for sequential recommendation. In AAAI, volume 34, pp. 5045-5052, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-performance deep learning library. NIPS, 32, 2019.
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In WWW, pp. 811-820, 2010.
Leslie N Smith. Cyclical learning rates for training neural networks. In WACV, pp. 464-472. IEBE, 2017.
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM, pp. 1441- 1450, 2019.
Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM, pp. 565-573, 2018.
Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In ECCV, pp. 776-794. Springer, 2020a.
Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? In NIPS, volume 33, pp. 6827-6839, 2020b.
Aaron Van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv e-prints, pp. 11Xiv 2018.
2018.
Stefan Van Der Walt, S Chris Colbert, and Gael Varoquaux. The numpy array: structure for efficient numerical computation. Computing in science engineering, 13(2):2230, 30, 2011.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems (NIPS), 30, 2017.
Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. ICLRR 2(3:4, 2019.
11

---

Under review as a conference paper at ICLR 2023
Vikas Verma, Thang Luong, Kenji Kawaguchi, Hieu Pham, and Quoc Le. Towards domain-agnosti contrastive learning. In ICML, pp. 10530-10541. PMLR, 2021.
Chenyang Wang, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Make it chorus: knowledge- and time-aware item modeling for sequential recommendation. In SIGIR, pp. 109-118, 2020a.
Feng Wang and Huaping Liu. Understanding the behaviour of contrastive loss. In CVPR, pp- 2495-2504, 2021.
Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. Next-item recommenda- tion with sequential hypergraphs. In SIGIR, pp. 1101-1110, 2020b.
Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. Sequential recommender systems: challenges, progress and prospects. In IJCAI, 2019.
Ziyang Wang. Wei Wei, Gao Cong, Xiao-Li Li, Xian-Ling Mao, and Minghui Qiu. Global context enhanced graph neural networks for ression-based recommendation. In SIGIR, pp- 169-178, 2020c.
Wei Wei, Chao Huang, Lianghao Xia, Yong Xu, Jiashu Zhao, and Dawei Yin. Contrastive meta learning with behavior multiplicity for recommendation. In WSDM, pp. 1120- 1128, 2022.
Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. Self-supervised graph learning for recommendation. In SIGIR, pp. 726-735, 2021.
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang. Xing Xie, et al. Session-base recommendation with graph neural networks. In AAAI, volume 33, pp. 346-353, 2019.
Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Xiyue Zhang, Hongsheng Yang, Jian Pei, and Liefeng Bo. Knowledge-enhanced hierarchical graph transformer network for nulti-behavior recommendation. In AAAI, volume 35, pp. 1186-4493, 2021a.
Lianghao Xia, Yong Xu, Chao Huang, Peng Dai, and Liefeng Bo. Graph meta network for multi- behavior recommendation. In SIGIR, pp. 757-766, 2021b.
Xin Xia, Hongzhi Yin, Junliang Yu, Yingxia Shao, and Lizhen Cui. Self-supervised graph co-training for bession-based recommendation. In CIKM, pp. 2180-2190 2021c
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. Self- supervised hypergraph convolutional networks for session-based recommendation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp.
2021d.
Jiaxuan You, Zhitao Ying, and Jure Leskovec. Design space for graph neural networks. NIPS, 33: 2020. 17009-17021, 2020.
Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. Next item recommendation with self-attention. arXiv preprint arXiv: T 1808. 1888.00411 14, 2018
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In WWW. pp. 2069-2080, 2021.
12


Reviews:
[{"review_id": "ybZS7AHFdz", "reviewer": "Reviewer_rXXA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a graph neural network (GNN) based recommender system for datasets featuring heterogeneous interactions (click, purchase, etc). This is done by performing message passing within behavior and time slot-specific subgraphs. Information is fused between behavior types and across time slots using various standard neural network components. Additionally, the paper proposes to use contrastive learning to encourage representations for the same user to be similar while pushing apart representations for different users. Experiments demonstrate the proposed model outperforms baselines on two small benchmark datasets and a proprietary (I think) e-commerce dataset.", "summary_of_the_review": "While the experiments demonstrate performance improvements over baselines, the large number of seemingly arbitrary design decisions are not sufficiently explored. Instead, the paper reads mostly like a description of random components that happen to perform well in the particular benchmarks chosen. As a result, I believe the paper's contribution is quite limited and do not recommend acceptance.\n\n--------\n\n**Edit**\n\nThe authors have addressed several of my concerns in their replies and I have raised my score accordingly. Furthermore, it should be noted that my original statement that the authors do not provide source code was incorrect. I apologize for having overlooked it initially.\n\n--------\n\n**Edit 2**\n\nI have re-reviewed the most recent revision of the paper. I would like to thank the authors for engaging with reviewers and for their updates to the paper. I believe the paper has improved as a result of the process, but will not be further raising my score at this time, and cannot argue strongly for acceptance of the paper. \n\nWhile I believe there are interesting ideas presented in the paper, I continue to find the disconnect between motivation and experiments the most critical outstanding issue. I would encourage the authors to address this issue in future revisions of the work, as I feel if this issue was addressed, the work has the potential to be a significant contribution. \n\nTo clarify my meaning here, I will reiterate and elaborate on an example I used previously. In Section 3.2 the authors motivate the necessity of the cross-relational memory network component through the need to \"reflect a holistic view of diverse preferences of users across different time slots\" and then go on to give the following example \"customers may first view some products in online retail sites, and make their purchase decisions one day after.\" While the ablation experiments show average performance increases when this component is included, they do not validate that this performance increase is due to the stated motivation. A more convincing experiment would identify examples of such behavior (a product view temporally followed by a purchase) and validate that including this component improves performance on these specific instances (and that performance degrades on these instances when the component is not included). Without this sort of analysis, it is impossible to disentangle the given motivation from other potential sources of performance improvement (regularizing effect, increased capacity, etc).", "strength_and_weaknesses": "**Strengths**\n\n- The method performs well in the experiments.\n- Some ablation studies are included to validate choices.\n\n**Weaknesses**\n\n- The constituent pieces of the proposed model are known and there is limited technical novelty. \n- Many of the design choices seem arbitrary and are not adequately justified. Over three pages of the paper are spent describing the architecture and providing fairly loose intuitive motivation without actually justifying why the specific choice is obvious or should be preferred, e.g., \"both short-term and long-term multi-behavior preferences of users can be preserved by the fused representations,\" \"our EGCM can preserve the dedicated time-evolving behavior dependencies across different types of user interactions.\" The desirability of these properties may be intuitive, but how this maps to the specific choices made is not. \n- The ablation studies are extremely limited, and the experiments are not designed to justify the design decisions made. For example, without designing experiments that explicitly test the model's ability to \"capture such evolving cross-type behavior dependencies across time slots\"  how can the reader be sure this is what the proposed mechanism is actually doing? Indeed, in the results section the authors somewhat tautologically explain the observed performance improvements, \"EGCM outperforms various baselines in all cases by achieving significant performance improvements.\" The lack of rigorous experimental design to validate the reason for performance improvements makes me doubtful that the paper provides a solid foundation on which subsequent research can build.\n- The paper cites [1], although it does not seem to take any lessons from the work. For example, the arguments that applying non-linearities and transformation matrices to non-feature-based representations (e.g. a user or item id) seem to apply to this work as well.\n- The authors use sampled evaluation metrics despite previous work [2] showing this methodology can be misleading. Since the datasets seem relatively small (the largest dataset features 35.5K items), I'm curious if there was a particular reason the paper chose this evaluation.\n- The \"In-Depth Analysis of ECGM Model\" section is only a paragraph, and the conclusions drawn from the figures are not adequately described. I didn't find this section particularly interesting, informative, or in-depth.\n\n**References**\n1. [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://arxiv.org/abs/2002.02126)\n2. [On sampled metrics for item recommendation](https://dl.acm.org/doi/abs/10.1145/3535335)\n3. [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://dl.acm.org/doi/abs/10.1145/3434185)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "yCG3ucku9Q2", "reviewer": "Reviewer_HypT", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the problem of capturing the interaction heterogeneity in modeling dynamic user preference for recommendation. A new evolving graph contrastive learning paradigm is designed to preserve both multi-behavior diversity and commonality. Extensive experiments are conducted to demonstrate the effectiveness of the new method compared with a large number of alternative solutions from different perspectives.", "summary_of_the_review": "The proposed new framework would be beneficial for contrastive learning domain in learning the relational dynamics. The idea of dynamic contrastive learning to behavior-aware recommendation is novel and interesting. Good performance with comprehensive experiments on benchmark datasets are given.", "strength_and_weaknesses": "Pros:\n1.The manuscript is well-organized and easy to follow. \n2. The descriptions of model motivation with technical details are very clear. The authors point out key challenges in learning preference dynamics for multi-behavior recommender systems. In the proposed EGCM method, the multi-behavior dynamic dependency challenges are addressed by integrating the contrastive SSL with cross-relation memory network. \n2.Theoretical discussion of the new contrastive learning is offered to analyze the benefits of the introduced behavior-wise contrastive-based SSL.\n3.Comprehensive experiments are conducted to justify the effectiveness of the new approach by comparing EGCM with 19 alternative solutions. \n4.Component-wise model evaluation as well as the in-depth analysis of EGCM method are provided to show the rationale of the superior performance.\n\nCons:\nIn the model inference phase, the self-supervision loss objectives L_{cl} with both short-term and long-term behavior modeling are incorporated into the original supervision optimized BPR-based loss. Besides the provided pseudo code in the appendix, the details of learning process in EGCM can be elaborated.\n\nFor the experiment of alleviating data sparsity issue, while the variants already achieve the obvious performance gain, the recommendation results of the new method can also be added.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "t4uSYIdfoz", "reviewer": "Reviewer_Usaw", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Summary:\n\nThis paper proposed to care about the sequential multi-behavior recommender systems where the multiple behaviors include a subset from {view, favorite, cart, purchase, browse, review}. To capture the multiple user behaviors, a multi-relation deep GNN is adopted to propagate behavior-aware messages and the graph Laplacian normalized function is incorporated into the message passing. In this way, both item representations and user’s short-term interests can be captured. As for the relations among multiple behaviors, The paper is to learn cross-type behavior dependency matrix by a Transformer, or called self-attention-based memory network in the paper. In this way, both sequential/time-evolving and multi-type behavior dependencies are learned. In the end, the paper proposes a contrastive learning to pull the type-specific behavior and multi-behavior representation of the same user to be closer as positive pairs while to push the behavior of different users away as negative pairs. Experiments are conducted on three datasets by comparing with various kinds of 19 baselines. Ablation and visualization are further shown to understand the components of the proposed EGCM model.\n", "summary_of_the_review": "Comments:\n\nC1: some typos. On page 2, “an” in the statement “we propose a Evolving Graph”. On page 6, the “+” symbol in computing $\\tilde E_{t,u}$; the subscript $e_i$ should be $e_u$ in the statement “by pulling the type-specific behavior embedding…”; the dot product in the statement “measured by the doc product…” \n", "strength_and_weaknesses": "Strong & Weak:\n\nS1: It seems novel to use contrastive learning framework for the sequential & multi-behavior recommender systems. Although both sequential and multi-behavior ResSys methods are proposed in existing works, this paper is the first to adopt contrastive learning technique to construct the two auxiliary tasks to better learn the main task. Furthermore, the ablation study shows that, the proposed model degrades the performance by a large margin if the contrastive learning component is removed, especially on the Tmall and IJCAI datasets, though the impact of contrastive learning is not so big on the E-commerce dataset.\n\nW1: One possible weakness in my own opinion: While the proposed EGCM is a new method for sequential & multi-behavior recommender systems, each of the individual piece of EGCM (multi-relation GNN, Transformer, contrastive learning) is not new in itself and has been widely adopted in existing works as pointed out in the related works. This may be not a big issue in practice, since EGCM demonstrated a strong empirical result on three datasets.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "OwlaMBhTAq", "reviewer": "Reviewer_1ytF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces an Evolving Graph Contrastive Memory (EGCM) framework which effectively models the multiple behaviors in the recommendation. EGCM is comprised of (1) a multi-behavior graph encoder to model the behavior-aware short-term interests of users, (2) a dynamic cross-relational memory network to model cross-behavior relational transitions, (3) a contrastive learning module to enhance the generalizability and robustness. The experimental results show that the proposed method could effectively outperform SOTA methods.", "summary_of_the_review": "The experimental results demonstrate that the proposed method could significantly outperform SOTA methods. However, the novelty is limited. ", "strength_and_weaknesses": "Strengths:\n\n1. The experimental results show that the proposed method could significantly outperform the SOTA baselines.\n2. Code is provided. \n\nWeaknesses:\n1. The novelty of the paper is limited. Many modules are simple applications/extensions of existing methods.\n2. The proposed model is too complex.\n3. The writing is not quite clear, and some parts are difficult to follow. \n\nFor example, Eq. (4) is difficult to understand. Where to use $\\mathbf{m}^b_{u_t\\leftarrow i_t}$? \n\nYou use $f({\\mathbf{e}^b_{i_t}}, {\\mathbf{e}^b_{u_t}})$ in Eq. (4), however, ${\\mathbf{e}^b_{u_t}}$ does not appear on the right hand side.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject