Learning Two-Time-Scale Representations For Large Scale Recommendations
Authors: Xinshi Chen
Yan Zhu
Haowen Xu
Muhan Zhang
Liang Xiong
Le Song
Venue:  (2021)
Keywords: Recommendation System
Large-scale Recommendation
User Behavior Modeling
Long-range sequences

Abstract: We propose a surprisingly simple but effective two-time-scale (2TS) model for learning user representations for recommendation. In our approach, we will partition users into two sets, active users with many observed interactions and inactive or new users with few observed interactions, and we will use two RNNs to model them separately. Furthermore, we design a two-stage training method for our model, where, in the first stage, we learn transductive embeddings for users and items, and then, in the second stage, we learn the two RNNs leveraging the transductive embeddings trained in the first stage. Through the lens of online learning and stochastic optimization, we provide theoretical analysis that motivates the design of our 2TS model. The 2TS model achieves a nice bias-variance trade-off while being computationally efficient. In large scale datasets, our 2TS model is able to achieve significantly better recommendations than previous state-of-the-art, yet being much more computationally efficient. 

Under review as a conference paper at ICLR 2021
LEARNING TWO-TIME-SCALE REPRESENTATIONS FOR LARGE SCALE RECOMMENDATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose surprisingly simple but effective two-time-scale (2TS) model for learning user representations for recommendation. In approach, we will parti- tion users into two sets, active users with many observed interactions and inactive new users with few observed interactions, and we will use two RNNs to model them separately. Furthermore, we design two-stage training method for our model, where, in the first stage, we learn transductive embeddings for users and items, and then, in the second stage, we learn the two RNNs leveraging the transductive embeddings trained in the first stage. Through the lens of online learning and stochastic optimization, we provide theoretical analysis that motivates the design our 2TS model. The 2TS model achieves a nice bias-variance trade-off while being computationally efficient. In large scale datasets, our 2TS model is able to achieve significantly better recommendations than previous state-of titte-of-the-arr the-art, yet being much more computationally efficient.
INTRODUCTION
A hypothetical user's interaction with recommenda- tion systems gives us diminishing returns in terms of its information value in understanding the user. For an active user who has lots of historical in- teractions, she is typically well understood by the recommender, and each new interaction gives rel-
I1 d'11>
Two-time scale convergence of user embeddings as a function of #interactions
inactive
active users
#intoractions
(or time!
atively little new information. In contrast, for an in-
Stage 1: Transductive emboriiing:
active or new user, every additional interaction will provide interesting information for understanding this user. Therefore, the representations for active and inactive users should be updated differently when new interaction occurs. Figure 1 illustrates such information diminishing phenomenon, where the amount of change in embedding from oi to o due to an additional interaction is decay- ing. One can select particular threshold !* for the number of interactions, above which the users can be categorized active users, and below which inactive users. Roughly active users' embeddings evolve slowly function of the number of inter- actions, while inactive users' embeddings evolve fast. Hence woo-imee-cale embedding evolution.
Item
User embeddings
embeddings
User
X1
X2
Xx
Stage 2: Two-timeessale e model for online embedding update
Inactive usens
Active usors
+
+1
42
42
meit
@ d
common initialization
trarsductive embedding
Two-time-scale convergence of user em- heddings motivates design two-stage method, where the first stage estimates transductive embed- ding and second stage learns two different RNNs for active and inactive users respectively.
Figure 1:
Apart from the time-scale difference in temporal dynamics, the simultaneous presence of active and inactive users also presents other modeling and computational challenges. On the one hand, active users lead to long sequences of interactions and high degree nodes in the user-item interaction graph. Existing sequence models, such as RNN models, have some limitations when dealing with long- tong-range sequences, due to the difficulty in gradient propagation. Morcover, graph neural network- based models become computationally inefficient due to the intensive message passing operations through high-degree nodes introduced by active users. On the other hand, predicting preferences of inactive

---

Under review as a conference paper at ICLR 2021
new users (also known the cold-start problem) is a challenging few-shot learning problem. where a decision needs to be made given only a few number of observations. To address various challenges imposed by the presence of two types of users, leverage the different dynamics of these users and propose (i) a two-time-scale (2TS) model and (ii) a two-stage training algorithm.
2TS model. Based on the number of observed interactions, we partition the users into sets: active and inactive users. Our 2TS model (Fig. 1) update the embeddings of active users and inactive users by two RNNs with independent parameters, order respect the two-time-scale nature. Moreover, the initial embeddings of inactive users are represented by a common embedding *b, which is shared across all inactive users. Therefore, the overall model for inactive users is inductive, in the sense that the learned model can be applied to unseen users. In contrast, the initial embedding of each active user a user-specific embedding Φu. which is also called transductive embedding. Such embeddings are very expressive, which can better express users with long history.
Two-stage training. In stage 1, we first learn transductive user embeddings Φu and transductive item embeddings using classical collaborative filtering method. Then we fix these embeddings. and in stage 2, we will learn the parameters of the two RNNs and a common initialization &p for inactive users. It is notable that the transductive embeddings for inactíve users are abandoned in stage 2. Only those for active users are finally used in the 2TS model. Besides, for active users, we do not use all interaction data to learn the RNN since their transductive embeddings have already encoded the information of their history. We only use a small number of last clicked items to learn the adaptation for active users, which improves the efficiency of the training process.
The proposed 2TS model and the two-stage training algorithm lead to a few advantages:
Birs-variance trade-off. The differential use of transductive and inductive embeddings for the two RNN models allows 2TS to achieve a good overall bias-variance trade-off. We theoretically analyze such trade-off in Section 2 through the lens of Marning-to-leamy paradigm for designing online learning (or adaptation) algorithms. Our theory shows that there exists an optimal threshold to split users to achieve the best overall excessive risk.
Encode long-range sequence. The transductive embeddings Φ., for active users are user-specific vectors, so they can memorize the user's s long-range history during the training, without suffering from the difficulty of gradient propagation. The RNN on top of these transductive embeddings is only used for adaptation to recently engaged new items.
Computational efficiency. The efficiency of our method on large-scale problems mainly comes from two designs in the algorithm. First, stage Iearns the transductive embeddings of active users and items, which contain a large number of parameters. However, it is fast since it does not involve any deep neural components and the loss is simply a convex function. Second, stage 2 only learns the RNNs which contain small number of parameters, and the RNN for active users is only trained on a few last engaged items, which cuts off the long sequences. Experimentally, our method reveals to be much more efficient than the baselines on large-scale datasets.
We summarize the contributions of this paper as follows:
To explain the intuition and motivation of the 2TS model, we provide theoretical analysis a simplified setting, which rigorously argues the need for differential use of transductive and inductive embeddings for active and inactive users (Section 2).
Motivated by the analysis, we design the 2TS model and a two-stage training method, for practical use (Section 3). The proposed method is applied to two large-scale benchmark datasets and compared comprehensively to various baseline models, spanning a diverse set of categories, which shows that our method is advantageous in terms of both accuracy and efficiency (Section 5).
2 THEORETICAL MOTIVATION: WHY TWO-TIME-SCALL MODELS?
We will first present the motivation for designing the 2TS model, through the lens of online learning and stochastic optimization. Our analysis quantitatively reveals that (i) the embeddings for active and inactive users evolve in different time scales, and (ii) two different online learning algorithms for active and inactive users respectively can lead to a better overall estimation of user embeddings.
Our analysis will be carried out in learning to-leam setting, where online learning algorithms need to be designed to tackle a family of tasks for estimating the embedding vector of a user. Though this idealized setting can not cover all aspects of real-world recommendation problems, it leads to
2

---

Under review as a conference paper at ICLR 2021
clear insights on the 2TS behavior of active and inactive users and the benefits of using two different online algorithms for these two respective user groups. These insights also motivate our practical implementation of the 2TS model using deep learning in Section 3.
2.1 SETTING: LEARNING-TO-LEARY
Our setting consists of three components: the estimation task for an individual user, the distribution of tasks for family of users, and the online algorithms which we want to design.
Individual Task. We associate each user u with a ground truth embedding 6 E R which can be thought of as a vector representation of user's preference. This embedding is defined by distribution over user's s clicks over items (ær, y), where æ E X' represents the item embedding which we assume is bounded, i.e., || I B., and y E {0, 1) indicates whether the item is clicked. They follow the user-specific distribution (a: y) ~ μ.. More specifically, the ground truth user embedding defined as the minimizer of the expected risk according regularized logistic loss
1 =
arg min seo Ry( where R₂ == 18 (x),)))) æ, y),
x and { x. v) yx Φ + log(1 (1 + exp(2 T x)) + E1101 (1)
where e >> O is some regularization constant and ④ = t E R4: 1 S Ba}.
Typically, we do not have access the distribution μa, but sampled of T observations F|r] := {(x1 yy),,.. ... (aT: yr)) ~ nu=. which can be used as training samples to obtain an initial estimate ((2pr)) of the user embedding. We assume ((2)))) is obtained by applying stochastic gradient descent (SGD) to the loss over 21r in this section, but this training stage is not limited just to the SGD algorithm. It is expected that with more samples the estimation will be closer to the ground truth 67.- The estimation ((2(T)) is modeling the offline training stage of recommendation system.
A Distribution of Tasks. We consider distribution of users, by assuming that the user-specific distribution ji is sampled from a meta-distribution μ ~ Pa. Furthermore, the number of observed interactions for user, denoted by T 18, is random variable that follows power law distribution with density p(T) x (T +7+11 1)- The power law distribution models the fact that there will be lots of users with very few interactions, and very few users with many interactions. A key assumption of our model is that the variance of the ground truth user embeddings is small. That is
Varm Ey-p. || ml| S r, where m Ey-ps $
(2)
The assumption of small variance is critical the sense that will allow us aggregate information from inactive users to obtain better estimations of their user embeddings.
Online Algorithm Design Problem. Our goal to design online adaptation algorithms for this distribution of users, such that the overall excessive risk of the online adaptation across all users small. This to model the online sequential recommendation stage where new user-item click information is incorporated after system deployment. Note that we are not restricted to design single online learning algorithm for all users. In fact, we will show later that online learning algorithms can actually lead to better risk bound than I online learning algorithm.
In this algorithm design problem, each user μl corresponds to an online learning task, where items arrive sequentially. Starting from an initial embedding &) an online algorithm updates the user embedding whenever it observes a new user-item interaction (It, Vt) μt:
(3) and then it applies 1++11p to the next item Ø and suffers the loss ((bb+1 ae41 Me+1) in Eq. The excessive risk of this online algorithm after encountering N sequential samples is
1++ < Update (6',, Bt).
I Ei ((bb) (r,,1) - {( Ft, Vt).
N
The design problem involves (1) the initialization for the algorithm and (2) the update step of the algorithm. One obvious choice is to use the user-speeific embedding ((2(T)) estimated in the offline training phase as the initial embedding Φ) for the online algorithm. However, is this estimation the best choice for the initial embedding Φ) ) Is there better choice? How should the initialization depend on T? We answer these questions in the next subsection.
'This assumption is sensible since Fig. 5 shows that T approximately follow a power law in real-world datascts.
3

---

Under review as a conference paper at ICLR 2021
2.2 ANALYSIS AND ITS IMPLICATIONS
We focus on designing online gradient descent (OGD) algorithms for Eq. 3, where the update step of f Projo ( -? @ zt. ye)] In our theoretical analysis, we will focus on designing only the initial embedding for the algorithmss and the learning rate to gain insights, and deferred the design of more components of the algorithms to later practical neural implementation.
A key aspect of our analysis is that we will consider threshold t" E N that divides the users into two groups: active users with offline training sample T 2 t* before online adaptation and inactive users with 7 < t", Under such split of the users into two groups, we will design 2 online learning algorithms, one for the active user group and the other for the inactive user group. Furthermore
Different initialization. The initial embeddings &h for the two online learning algorithms are designed as: for active users, their learned embeddings Φ(21T) are used as the initialization, and for inactive users, they use a common embedding as the initialization. Mathematically,
Active User: /) = ((2TT) if T 2 1" and Inactive User: & = m if T <t'
(4)
where m represents the common initialization. We will later show Theorem 2.1 that the optimal choice of the common initialization is the mean m T and we can also analytically find the optimal threshold t". The intuition is that, for inactive user, if we start from the estimation Φ0 2(r) and make online adaptation for the new items, it will not perform well due the large discrepancy between (3(r1 and the ground truth o However, since & is distributed around the mean m, we can aggregate the offline observations from inactive users to estimate m. Then may be better off to use the estimated m as the shared initial user embedding for the inactive users. In contrast, for active users, o (2))) can be a better initialization than m since it is already very close to d online Different learning rate. The learning rates y are designed be different for the learning algorithms. As later showed in Theorem 2.1, the optimal learning rate for active users is smaller than that for inactive users, and leads a better overall excessive risk. The intuition that, for an inactive user, each additional observation provides us lot of information about this user, and we want to employ larger learning rate to fully ingest such information. In contrast, for an active user, the additional observation provide diminishing amount of information about this user, and we can use a smaller learning rate.
In the theorem below, we show precisely that different initialization and learning rate for the active and inactive users can lead to a smaller expected excessive risk. Furthermore, there a theoretical optimal choice of the threshold 1. to achieve this. A more detailed statement of the theorem and the proof of the theorem are provided in Appendix A.
Theorem 2.1. Assume the setting and notation in Section 2.1. Assume &-, is updated by OGD with step size 7, With the initialization strategy in Eq. 4, the expected excessive risk is bounded by
E 1 De; Ho) 1(66) De, y1) $ (2yN) (3(t") +,2B2N)
(5)
where 3(t*) = c(a+Le*++1) @ + (la.t"+1)) Varm. B = B + cB Ç is the Hurwitz zeta function, <(a.1)
((a.1)
and Q is constant larger than Varm- Optimizing the step size 7 = B-1 /B((r) N, we have
EE (( vt. Ui) E( : Dt, Ut) S BVB(L)))
(6)
Moreover, the optimal choice of the threshold t* is
arg min,- 200((1) = 1// Varm -
(7)
Implication 1: A differential design for the initial embeddings is better. all users are initialized
by their user-specific embeddings Φ(=(r)). ). then it corresponds the case when 1" = 0. However, is a less optimal choice than !a = 1x* and the gap between them can be quantitatively characterized by 3(0)) - _(t**) = -'
Var,, )t= Since the excessive risk is proportional to / (Btt). the gap shows the advantage of the differential initialization for active and inactive users in Eq. 4. Furthermore, if Var,, is smaller, the gap and the advantage of differential initialization are larger.
Implication 2: The optimal step sizes for active and inactive users are different. In the above theorem, the optimal step size 7 for the online algorithm is optimized over all users. However, if we
4

---

Under review as a conference paper at ICLR 2021
consider using two different step sizes for the group of active users and the group of inactive users, the optimal step sizes for these two groups will be (see derivations in Appendix A.2)
Pact V
/ for active users, and "/in +1)N
1 B
Yar for inactive users. N
is easy to verify that Yact < Yin. which suggests that embeddings of active users should evolve
with smaller step size than inactive users. This is consistent with our intuition for the 2TS model.
3 METHODOLOGY FOR PRACTICE
In this section, we will describe our concrete methodology. including (i) the parameterization of the two-time-scale (2TS) model and (ii) two-stage training algorithm for learning this model efficiently and effectively. The design is motivated and guided by the analysis Section 2, though more advanced techniques are incorporated in order to achieve the state-of-the-ar performance.
Instead of following the OGD algorithm in Section 2, we will use more flexible models for the online update dynamics of user embeddings, in the meta-learning framework (Ravi & Larochelle, 2016). In particular, we will use two different RNNs, which we call two-time-scale (2TS) model, to model the dynamics of active and inactive users respectively, which correspond to the update steps in Eq. 3.
To learn the 2TS model, we first perform a generalized collaborative filtering method to obtain mansductive user and item embeddings, which is similar to the training phase in Section 2, except that item embeddings also need to be learned from data and a more advanced loss function is adopted. The learned transductive embeddings will be fixed in stage 2. Transductive embeddings for active users will be used as the initial embeddings during the training of RNNs, while for inactive users a common initialization will be learned together with the RNNs.
3.1 NEURAL TWO-TIME-SCALE (2TS) MODEL
Consider a set of users u E = {1, 2,, ... ,U} and items i E == in a system. In our model, users are divided into two sets by the number of observed interactions T in the training set. With threshold t" E N, we consider users with more than t" observed interactions as active users Uact, and those with less than interactions as inactive users Unn which include new users.
Our two-time-scale model consists of two RNNs, one for active users and the other for inactive users. They can be thought of as two different online adaptation operators for active and inactive users respectively. Furthermore, these two RNNs will also differ in the way in whether they have user-specific parameters and how they are initialized. Assume for the moment, we have already learned an initial set of transductive user embeddings (Φ. Rd E U} and item embeddings (a;; Rd : E Z}. which will be explained in later two-stage training methods.
The RNN for inactive or new users is purely inductive the sense that it does not contain user-specific parameters. We will not use user-specific transductive embeddings &n, in this RNN. Instead, the RNN will start with a learned common initialization sb E IRd, and then be updated by RNN-based adaptation operators every time a new interaction is observed. More precisely, for an inactive user who clicks on items (i..-..,i ir) sequentially, the embedding will be updated sequentially as:
inactive users Mini & = t² +++ RNNce 1le, ( ...)
For active users, we will view their transductive embeddings as the memories for their long-term history. Different from inactive users, the initial embeddings the RNN for active users will be set to be the user-specific transductive embeddings Φ.. Then the RNN- based adaptation operator will update this initial transductive user embeddings there are more interacted items:
active users u E Uani
Φ 6b.
$ r+1 = RNNce 1e, (6 ).
The inactive and active user models in Eq. 8 and Eq. 9 constitute our two-time-saale model (2TS model). The model consists of several sets of parameters, including the transductive embeddings for active users {Φu Rd u Mact) } and items {z, Rd E I). the two sets of RNN parameters θ and O₂ respectively, and the additional common initialization &b for the inactive user RNN. These parameters will be learned differently using two-stage method described in the next section.
5

---

Under review as a conference paper at ICLR 2021
3.2 TWO-STAGE TRAINING
Let D be the training dataset that records the interactions between users and items as an ordered sequence where the (-th interaction between some user u and item i denoted as et == (u,, it). Our training method for the two-time-scale model consists of two stages. In stage 1, we will learn the transductive embeddings for all users and items from observed interactions D. In stage 2, we will learn the parameters of the two RNNs. In summary, the blue-colored transductive embeddings for active users { : u E Muct) } and items (æ, : i E I} are learned in stage and then fixed in stage 2. The red-colored components that include the common initialization &p for inactive users and the RNN parameters O and O₂ are learned in stage 2. More specifically,
Stage 1. We will first ignore the sequential ordering of the interactions, and learn the transductive user and item embeddings by optimizing the cross-entropy loss (or called softmax loss):
[((ow)i ((t.))(a.))
1 EO..)EDD log () exp(o m5)]
T5 wi,
(10)
where the set Ouu, i) the set of items displayed to user u when he/she clicks on item i. Typically this information is not given, so could randomly sample p items as the non-clicked items, and use these p items together with the clicked item i to constitute the offer set Ou, i).
Stage training is similar to collaborative competitive filtering (CCF) in Yang et al. (2011). is efficient since the objective is a simple function. Besides, since the objective is convex in either &u or 2; it is easier to obtain the global optimum and the results are affected less by the initialization.
Since active users have many interactions, their transductive embeddings are learned well and lead to good predictions in held-out data. However, for inactive users with fewer observed interactions, the learned transductive embeddings could be overfitted. Thus, in the next stage of training RNNs, we will re-use the transductive embeddings for active users but will discard those for inactive users. Furthermore, the learned item embeddings } will also be used in the next stage of training.
Stage 2. We can divide the training set according to active and inactive user as Dact = {(un, ie)
(u,, 4e) E DD A ue E Uaat } and Din = {[u,, it) (u, (1) E D A ue E Um Then we will train the parameters of the two RNNs using the following loss functions:
Cin(O) 26) = Dean E E(untey log Eje((Hn.t) exp(OT, m7:)
- TT. Dig
(11)
Larr = Dess Σ (mm,,))DD
log Ejeo(u),)) exp( a
- T Dig
(12)
where user embeddings (Φ... } in Eq. 11 and Eq. 12 are updated sequentially using the corresponding RNNs in Eq. 8 and Eq. 9, respectively. Furthermore, D C Dact C D contains the last K interac- tions for each active user observed in the training set. That is, for an active user who has more than t* interactions, we will only use the last K (K < t*) interaction of the user to train the RNN. First, this allows us to avoid the direct use of RNN for long sequence modeling, which inefficient. Second, the transductive embeddings for active users have already encoded most information from these users and only a small online adaption is needed to boost the performance further.
Overall, we find that by encoding the history into the transductive embedding and only learn the K step adaptation can largely reduce the computational cost. This reflects another benefit of treating active and inactive using the threshold t*
3.3 IMPLEMENTATION DETAILS
We present two implementation details that are essential for the performance of our model in the experiments but less relevant to the main ideas of this paper.
2TS- We propose a variant of our 2TS model called 2TS-Plus. The only difference from 2TS that, we replace the user embeddings Φu. in 2TS by T.,, + T ,.,_, where Z;,-, is the item embedding of the user u, 's most recently clicked item é 1, and W is a learnable weight matrix. In summary, 2 explicitly incorporates the information of the *last clicked item' to compute the user embeddings. Our experiments show that this can consistently improve performance. In fact, incorporate the 'last clicked item' because we found that the baseline JODIE (Kumar al., 2019)

---

Under review as a conference paper at ICLR 2021
performs particularly well on last Thus we looked into their implementation and found that it is the 'last clicked item' that helps with the performance.
Features. In many datasets. item features {fi hiz are provided. In the experiments. concatenate the transductive item embeddings 2; with the feature-based embedding, I.p (fi). where 9. a simple network with parameter y. The concatenation [a;]_ g ( TT will be used as the item embeddings, and both transductive embeddingg I; and the parameters yp are learned in stage 1.
4 RELATED WORK
Our stage 1 training is most related to methods based on matrix factorization (MF) (Koren et al., 2009). Many methods for MC are collaborative filtering (CF)-based (Koren et al., 2009). Neural models for recommendation broadly fall into the following three categories.
Graph-based models. Monti et al. (2017) first studied recommender systems with graph neural network (GNN). Berg et al. (2017) proposed graph convolutional matrix completion (GCMC) which applies a GNN the user-item bipartite graph learn user and item embeddings. It is transductive model, as it learns user/item-specific embeddings. PinSage (Ying et al., 2018) and IGMC (Zhang & Chen, 2020) are recently proposed inductive GNN-based recommender systems. Despite showing promising results, existing GNN- recommender systems usually have poor scalability, due to their expensive neighborhood sampling procedure for performing graph convolution.
Dynamic graph models. Based on the idea that both users and items will evolve over time via temporal interactions, Dai et (2016); Farajtabar et al. (2017); Kumar et al. (2019); Goyal et al. (2020); Sankar et al. (2020) take the graph temporal evolution into modeling design. However, most existing dynamic graph approaches cannot scale to large interaction graphs. JODIE (Kumar et al., 2019) proposed batching method to make the training process more efficient.
Deep Sequence models. RNN (Hidasi et al., 2015; Jannach & Ludewig, 2017) and LSTM (Devooght & Bersini, 2017; Chen et al., 2019) based deep models are widely used in sequential recommendation. Other methods based on attention models (Zhou et al., 201 8; 2019) have also been explored. However, these models still have difficulties in leveraging the information contained in states located far into the past due to gradient propagation issues (Pascanu et al., 2013). Several recent advances are proposed to deal with long-range sequences (Tang al., 2019; Pi et al., 2019)
Cold-start problem. Traditional approaches to address cold-start problems include content filtering, hybrid models, etc. Vartak al. (2017) proposed a meta-learning perspective for the item cold-start problem, where recommending new items to users is formulated as learning a learning algorithm. Lee al. (2019) proposed use meta-learning to estimate new user's preferences with few consumed items. Bose et al. (2019) proposed Meta- Meta-Graph to perform few-shot link prediction across multiple graphs. Wu al. (2020) proposed to compute embeddings for new users using the embeddings of active users, via attention-based model. They share the same idea of splitting the users into two sets, but their main target is the matrix factorization problem and neither consider the temporal evolution nor the contime--calee difference.
EXPERIMENTS
Table 2: Dataset Statistics
Dataset
#users
#items
#interactions
To evaluate the performance of the 2TS model, we conduct experiments three public datasets, two of which are the largest recommendation benchmark datasets that are closer to industrial scenario, making our results more solid and convincing. By comparing to a diverse set of state-of-the-an (SOTA) methods, we demonstrate the scalability and accuracy of our 2TS model. See detailed configurations in Appendix C.
lastfm-1K
1,000
1,000
1,293,103
ML-25M
162,538
59,048
24.999.849 849
Taobao
987,975
4,111,798 96,678,667
Dataset. We consider 3 public datasets: Taobao dataset (Pi et al., 2019), MovieLens-25M (ML-25M) (Harper& & Konstan, 2015), and lastfm-1K (Celma, 2010). Dataset statisties are summarized in Table 2. Taobao and ML-25M are large-scale. Especially, Taobao contains about 100 million interactions. The results on these two datasets are much more convincing. All datasets provide user-item interaction data where for each interaction, user ID), item ID, timestamp, and item feature are given. For ML-25M, we ignore the ratings and simply use it as interaction data. For each dataset, we sort the interactions by timestamp, and use the first 70% as the training set, the following 10% as the validation set, and the last 20% as the test set. All reported results are estimated on the test set.
7

---

Under review as a conference paper at ICLR 2021
Overall performance for interaction prediction. The best method in each column is colored blue and second best is colored light blue The * symbol indicates that the original implementations released by the authors or the proposed methods are not scalable and will not be able to run on the indicated datasets without our modifications to their methods. The $ symbol indicates that the released implementations have been modified by us to better adapt to the dataset and evaluation metric, so that their results in this table should be expected be better than the original version. The concrete modifications we made are described in Appendix B.
Table 1:
Taobao
ML-25M
lastfm-IK
method
MRR
Rec@ 10
MRR
Rec@ 10
MRR Rec@ 10
CF-based
CCF#
0.402
0.621
0.193
0.392
0.051
0.116
GCMC
0,303
0.542
0.171
0.378
0.081
0.129
GNN-based models
GCMC-SAGE
0.149
0.366
0.109
0.264
0.168
0.208
GCMC-GAT
0.230
0.494
0.185
0.404
0.064
0.118
SumPooling
0,415
0.664
0.302
0.591
0.071
0.180
GRU4REC
0.546
0.777
0.364
0.657
0.152
0.344
Deep sequence modely
DIEN
0.605
0.834
0.356
0.638
0.100
0.213
MIMN
0.607
0.828
0.363
0.653
0.115
0.261
SASReet
0.488
0.702
0,360
0.653
0.147
0.323
Bert4Rec
0.280
0.480*
0.397
0.652
0.216
0.369
Dynamic graph models
dynAERNN
-00m-
-0om-
0.249
0.509
0.021
0.038
JODIE
0.454
0.680*
0.354*
0.634*
0.176
0.325
2TS
0.669
0.844
0.404
0.693
0.151
0.332
Our method
2TS-Plus
0.680
0.844
0.409
0.691
0.203
0.369
Improvement to best baseline
2TS
10.2%
1.2%
1.8%
5.5%
2TS-Plus
12.0%
1.2%
3.0%
5.2%
Baselines. We compare 2TS with 12 models spanning 4 categories: (i) Ca-based methods: Collaborative competitive filtering (CCF) (Yang et al., 2011) is ad- vanced CF model which takes into account the context of user's choice. It is a simple yet very effective method. (ii) GNN-based models: GCMC (Berg et al., 2017) a SOTA graph-based architecture for recommenda- tion. GCMC-SAGE is its stochastic variant which is more efficient. GCMC-GAT is another variant based on graph attention networks (Veličković et al., 2017).
Time to MRR performence on Taobso
0.7
0.6
0.5
0.4
GCMC
GBUAile:
0.2
GCMC-SAGE
MIN
Jonie
2T5-shage1
0.1
SumPPoolig
275ml age?
40
60
80
100
Training Time thist
Time-to-MRR performance on ML-25M
(iii) Deep sequence models: SumPooling is a simple yet effective bascline that is widely used in industry. GRU4REC (Hidasi et al., 2015) is a representative of RNN-based models. DIEN (Zhou et al., 2019) is advanced attention- based sequence model. MIMN (Pi al., 2019) is memory-enhancee to better capture long sequential user behaviors. It is strong baseline and the authors come from the team that publicized Taobao dataset. SASRec (Kang & McAAuley 2018) a 2-layer transformer decoder-like model. Also built on top of transformer architecture, Bert4Rec (Sun et al., 2019) further introduced a BERT type Cloze task with bidirectional attention. (iv) Dynamic graph models: JODIE (Kumar et al., 2019) and dynAERNN (Goyal et al., 2020) are two dynamic temporal graph approaches which learn graph node dynamic embeddings via temporal modeling.
MRR on Test Set
0.2
GEMC
GRIMRer
GEMC-SAGE
MIMN
dyPAERNN
275 stage1
jnbir
2775-shage?
SumPoolinng
RNN architecture
Training Time
Figure 2: Test MRR versus training time. We compare the convergence speeds of different models in wall-clock time. Run-time on lastfm- IK not showed because is small-scale.
We measure the performance of different models in terms of the mean reciprocal rank (MRR), the average of the reciprocal rank, and recall @ 10, which the fraction of interactions in which the ground truth item is ranked in the top 10. For both metrics, higher values better. For every interaction, the ranking of ground truth item is calculated with respect to 500 items where the other 499 negative items are randomly sampled from the set of all items.
Evaluation Metric.
8

---

Under review as a conference paper at ICLR 2021
Overall Performance (Table 1). We summarize the overall performance in terms of MRR and Rec@ 10 in Table 1. On the large-scale datasets Taobao and ML-25M, our models 2TS and 2TS-Plus have consistent and significant improvements compared to all baselines. In terms of MRR, an improvement of 12.0% and 3.0% are achieved by 2TSs- respectively. Note that the lastfm-1K K dataset much smaller than Taobao and MI--5MM, We run experiments that the models have very different behaviors on datasets of different scales. It interesting to see that, for example, GCMC-SAGE, which performs the worst on both two large-scale datasets, achieves the 3rd best performance on lastfm-1K. The real advantages of our models are for large-scale problems where candidate items are sparse.
lastfm-1K K just show
ML-25M
Taobao
Scalability & Efficiency (Fig. 2). To com- pare the training efficiency, we evaluate dif- ferent models' intermediate checkpoint per-
2T5 GONC
10OKE GRUARAC
0.42
GONC-AGGE dAACCIA
MIN
Anorypa MER on Set
0.35
formances on the test set. Fig. 2 shows that the performance of 2TS increases fast. 2TS's first stage transductive training is effi- cient and provides an effective initialization for the second stage inductive training. Be- sides, though not revealed from the figures, the training time of 2TS for each epoch is much smaller than the baselines.
0.3
0.35
0.2
0.2
(102200))
(4.80)
(1L180))
Sinbeam Traiiin
Winernatione Trandhy
Figure 3: Test MRR in different segments of users.
Performance for different users. Fig. 3
shows the MRR performances averaged over users with different numbers of observed interactions (the more interactions the more active the user is). The dash line indicates the threshold that we use to split users into inactive and active groups. 2TS and 2TS-Plus lead to consistent improvements over the entire range of users. On ML-25M, the improvement on inactive users are more obvious, while on Taobao the improvement on active ones more obvious.
Two-time-scale behavior (Fig. 4). Recall 2TS has two RNNs. One updates the embeddings of inactive users and the other updates those of active users. Given the learned RNNs, on the test set, we compute the changes of user embeddings in 2-norm as the user interacts with more items (Fig. 4). The behavior aligns with our intuition. Embeddings of inactive users are changed faster and those of active users are more static.
charge of user embedeings
charge of user embedeingg
2TS on Tapbao
2TS-Plus on Taoboo
Inactive Lisers
Inaetive lisess
Active Users
Active Users
0.75
0.75
0.50
0.50
0.35
0.25
*interactions with items
*interactions with
Figure 4: Two-time-scale behavior of user embeddings.
Ablation Study (Table 3). To show the effectiveness of the two-stage training, we compare 2TS 2TS- SingleStage, which has exactly the same architecture as 2TS but all parameters are trained together in a sin- gle stage. 2TS- Sin-SingleStage performs worse, which means the transductive embeddings are learned very well in Stage 1, benefit from the convexity and easi- ness of optimization. Besides, the training of STS-SingleStage is a lot slower. Furthermore, to show the effectiveness of the 2TS model design, we compare it to 2-GRU, which applies two different RNNs to inactive and active users, but there are no transductive user embeddings for active users.
Table 3: Ablation study.
MI--25M
Tanbao
method
MRR Rec@
MRR Rec@ 10
2TS
0.404
0.693
0.669
0.894
2TS STS-SingleStage
0.373
0.636
0.420
0.533
2-GRU
0.354
0.643
0.597
0.791
CONCLUSION
We proposed to learn two-time-scale representation (2TS) for large recommendation systems to explicitly address the learning efficiency for different user dynamics. We also proposed two-stage training schema, to leverage the transductive embedding as the inductive model initialization for active users. We evaluated 2TS on large scale recommendation ranking tasks, and showed that 2TS is able to outperform several class of of-te-of-the-an art methods including deep sequence models, graph neural network, and dynamic graphs. Our results show that, designing different representation to capture diverse interaction dynamics are desirable. Future work includes separating and updating user dynamics in algorithmic way.

---

Under review as a conference paper at ICLR 2021
REFERENCES
Rianne van den Berg, Thomas N Kipf, and Max Welling. Graph convolutional matrix completion. arXiv preprint arXiv: 772602263, 2017.
Avishek Joey Bose, Ankit Jain, Piero Molino, and William L Hamilton. Meta-graph: Few shot link prediction via meta learning. arXiv preprint arXiv:1912.09800 2019.
O. Celma. Music Recommendation and Discovery in the Long Tail. Springer, 2010.
Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. Generative adversarial user model for reinforcement learning based recommendation system. In International Conference on Machine Learning, pp. 1052-1061. PMLR, 2019.
Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Song. Deep coevolutionary network: Embedding user and item features for recommendation. arXiv preprint wrxiv:1609.036733 2016.
Robin Devooght and Hugues Bersini. Long and short-term recommendations with recurrent neural networks. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personaliza- lion, pp. 13-21, 2017.
Mehrdad Farajtabar, Yichen Wang, Manuel Gomez-Rodriguee Shuang Li, Hongyuan Zha, and Le Song. Coevolve: joint point process model for information diffusion and network evolution. The Journal of Machine Learning Research, (5()):1055-135, 2017.
Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. dyngraph2vec:
Capturing network dynamics using dynamic graph representation learning. Knowledge-Ban Basteeee--aeddd--e Systems, 187:104816 16, 2020.
Maxwell Harper and Joseph Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1-19, 2015.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint 1 2015.
Dietmar Jannach and Malte Ludewig. When recurrent neural networks meet the neighborhood for session-based recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems, pp- 306-310, 2017.
Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pp. 197-206. IEEE, 2018.
Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, 2009.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1269-1278, 2019.
Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. Melu: Meta-learned user preference estimator for cold-start recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1073-1082, 2019.
Federico Monti, Michael Bronstein, and Xavier Bresson. Geometric matrix completion with recurrent multi- gratiigrapphho neural networks. In Advances in Neural Information Processing Systems, pp. 3700- 3710, 2017.
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization, 19(4):1574- 1609, 2009.
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International conference on machine learning, pp. 1310-1318, 2013.
10

---

Under review as a conference paper at ICLR 2021
Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. Practice long sequential user behavior modeling for tlick-through rate prediction. In Proceedings of the 25th ACM SICKDD International Conference Knowledge Discovery & Data Mining, pp. 2671 1- 1-779, 2019.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural rep- resentation learning on dynamic graphs via self-attention networks. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 519-5299222 $19-527, 2020.
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1441-1450, 2019.
Jiaxi Tang, Francois Belletti, Sagar Jain, Minmin Chen, Alex Beutel, Can Xu, and Ed H. Chi. Towards neural mixture recommender for long range dependent user sequences. In The World Wide Web Conference, pp. 7782-1993, 2019.
Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, and Hugo Larochelle. A meta-learning perspective cold-start recommendations for items. In Advances in neural information processing systems, pp. 6904-6914, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint 17Xivv//100 10903, 2017.
Qitian Wu, Hengrui Zhang, and Hongyuan Zha. Inductive relational matrix completion. arXiv preprint arXiv:2007.04833 2020.
Shuang- Yang, Bo Long, Alexander J Smola, Hongyuan Zha, and Zhaohui Zheng. Collaborative competitive filtering: learning recommender using context of user choice. In Proceedings of the 34th international ACM SIGIR conference Research and development in Information Retrieval, pp. 295-304, 2011.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 974 983. ACM, 2018.
Muhan Zhang and Yixin Chen. Inductive matrix completion based on graph neural networks. In International Conference on Learning Representations, 2020, URL htttps://pperreeeeee net/ http://ommmmixxxxxxxaxaa
Guorui Zhou, Xiaoqiang Zhu, Chenru Song. Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1069-1068, 2018
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 5941-5948, 2019.

---

Under review as a conference paper at ICLR 2021
A THEORETICAL MOTIVATION: WHY TWO-TIME-SCALE MODELS?
Here we will provide more details for Section 2. First, we summarize the setting in Section 2.1 formally and mathematically:
Setting (A).
Each item is represented by vector a: in a given bounded space X' : (ar R ||0|||| S Ba
Each user is represented by a distribution over the space X'x {0, 1). Each sample == (ær, y) ~ from this distribution is item and binary label y indicating whether this item is clicked by the user. Further, we assume that user μ is drawn from meta-distribution μ Pa.
We parameterize the user 1 through a logistic regression model so that groundtruth user embed- ding is defined as the minimizer to the expected risk:
$ == arg min Deo (T) where R2 = ( x. v),
and ((Ø, v) == ya T Φ log(1 + exp(2*' x)) ())==(()=--)))))) 1
where c > 0 is a regularization constant. Assume the parameter space is also bounded + = {Φ E
Rd 1|001/2 Ba).
Training samples. Each user is associated with T observed interactions, denoted by [[T]
(111,)1),... (2T. yr)) ~ J²T.. Assume T 2 0 is a random variable that follows a power law
distribution with density p(T) x +T 1)-0, and denote T pg. T independent of j.. Given the observations Z|2g for user μu, estimate of the user embedding ((=12) can be computed using learning algorithms such as stochastic gradient descent (SGD). In this paper, we assume (((1)) is estimated by SGD with initialization Ø 0 and step size 0 1 the loss f. Test scenario. Each user μ can be viewed as an online learning task, where items arrive sequentially. Starting from an initial embedding (), online algorithm updates the user embedding whenever it observes a new data point (æ), Vt) J.:
Update me,ye) ye),
then applies the most updated embedding to of the next item 11411 and suffers from the loss ((
6---1 Fet1, Mt+1). The excessive risk of this online algorithm applying to N test samples is
. E l(6), Bt, M1) E Tt, Mt).
Before the proof of the main theorem, we first introduce the result of stochastic analysis by (Ne- mirovski et al., 2009), which can be adapted to derive the following proposition that tells the error of the estimation (((TT) from the training samples.
Proposition A.1. Assume Setting (A). Given the T observations 2|T| ~ "I"", if we estimate the user embedding using projected stochastic gradient descent (SGD) with initialization 0, step size } and operated on f2-regularized logistic regression loss + E - yy log((1 + exp(x Φ)) + +)) then the expected squared error of the estimation Φ (=(()) can be bounded by
(pp)
(Br + cBa)² [()))
Expyau (1 - 0ill2 <
T 1 where Q(u) = maxt
Proof. Consider the loss E(O,, x. yæ d log(1 exp(x T x)) e | It is easy to verify that E is c-strongly convex, and that the second moment of the gradient is bounded by
exp(x ))
1 BII I+ exp(x T ())
y)æ + c0l12 < E (]|=]12 + 2 & (Bx + cBa)²
Then applying the result by (Nemirovski et al., 2009) (more specifically, Equation (2.9) in the paper), we can obtain the above error bound.
A.1 PROOF OF THEOREM 2.
We first restate Theorem 2.1 as the following theorem, which provides more details. Then we present the proof.
12

---

Under review as a conference paper at ICLR 2021
Theorem A.1 (Detailed version of Theorem 2.1). Assume Setting (A), Assume the online algorithm for updating the embeddings 6i, in Eq. is online gradient descens (OGD) with step size "I, i.e., $1- = Proja & - -d 6;, Tz, vt) Let 2rT) denote the observed training samples and 2Nle denote the sequence of test samples for the online learning task. With the initialization strategy in Eq. 4, the expected excessive risk is bounded by
E (1,T))( xof) E
AADTNNA E
Σ t=1
L Fz Uc) E(o 21, Ue)
N
3(5) + 2 + 2
(13)
24N
where
3(1*) = <(a + 1,0+ Q+ (1 ((a,t+1)) Varm
((a,1)
((a,1)
(Bz + cBa)
Q = Eurpp Q(1)) = Epppp max
11011121
c²
and C(a, =
> (i ( b) b) is the Hurwitz zeta function
1m
Choosing the optimal step size 7 B.tcBa
B(t
the upper bound is
N
3(t))
E N (( xe, y1) E( 21, ve) S
(14)
B3 + cB
N
Besides, the optimal choice of the threshold 1" is
Q
1* arg min (((")
(15)
Varm
20
Proof. Denote the N test samples by N(N) ~ n Following the analysis and results in (Nemirovski al., 2009), one can show that
E Bx | 21 <TE
N 110b
$111
2 MM [(0] - Te(0), ot, vt)] + + + cBa)2.
The above inequality corresponds to equation (2.6) in (Nemirovski et al., 2009). Then, since is convex Φ, then
E [(05 - T Tt, ve) 2 Ex [e(Ø æz, v1) - {( ot, v:)
Denote e
E = 100 27 112 and combine the above two inequalities, we have
2, u [e( Ht, vi) ( xn,11]] S ee el+1 + ( +c
cBa)
Summing over 1, we have
Eelos v1) l(bj vz, vr) S 11 $1.13
A(B, + cBa)²
Eyn N N
2yN
Taking expectation over the training samples 2[T]. T, and μe, we have
E (1TT))~(a Xmg) (xm ~ N
(( Fz, ye)
E( 2z, y1)
| (2(T))
< (( x
1 [ IT 2
2
m 6
((uxTT)((pxxp3)
1 I < <t"
N
(Bn + eBa)2
2
13

---

Under review as a conference paper at ICLR 2021
Regarding the first term the right hand side, by Proposition A.1,
(l, xp4) R (((((()r)
/ [7 2 t*)
Q 1 [ 2
t* B Eunp QT
1 2 t*]
s (ppx)))(pxxpp
T
T+1
1 TT 2 t-
(T+1)-
=QBr~p
Q Σ
T
T ((a,1)
T=t"
1,, 1,, = = +1)
((a,1)
Regarding the second term,
E ( x m - 01.1111 t*]
= Varm Br [T < [*] Varm (1 $1a,(a, 1 +1)
S(a,1)
Combining the above inequalities, we can obtain the first bound in Eq. 13, and Eq. 14 follows. Now we need to find the optimal choice of the threshold t². Note that
Q T
[*] Varm 1 [T' < c-]
Q
B(t") = Er
1 I
2 ET min
Varm
T+1
Since 7+1
monotonely decreases as T increases, is easy to see that
{ Q
Varm: @
when 7 <
min
T [++1 Varm
Varg
when T, Varm
Q
7+1
Therefore, the lower bound of B((*) achieves at
1,
Varm
and it the optimizer.
A.2 DETAILS FOR IMPLICATION SECTION
Regarding Implication in Section 2, here we provide more derivation steps for the optimal step sizes for the two group of users.
The expected excessive risk can be written as the sum of the excessive risks of the two groups of users:
N
E
e ( D1, vI, ve) - E( æe, v.)
N
E
( ju me; y1) e( Te ve) 1[T <('
for inactive users
N
E
Σ
u Tt, we) E( Be, ve. Hx)
1[T 2 (*]
for active users
With similar derivation steps the proof for Theorem A, we that for inactive users, the bound
E(ss pu Lt, vr) - e( ve) IT <19
E
N
1=1
Varm
7(B, + cBa)²
Varm N
r(B, + (cBba²)
E.
1[T <
<t]
Pr[T < <t)].
N
2
14

---

Under review as a conference paper at ICLR 2021
Optimizing the step size gives Yin = B.+oBo V Varma Similarly, for active users, the bound is
N
E
Σ
T De, - -((0), De, l1)
1[T 2 t1
N
Q
(B + ccaB)
1T N t9]
E
(T + 1)2;NN
2
Q5(a + 1,t" +1)
(BB + cB bBB)) S(a,, to +11)
/Slatt" + 1).
N
Optimizing the step size gives Vact = B,tcBa
(lot1,++10 ((a,1+1)N
B BASELINE SPECIFICATION
Some of the compared baseline methods are not directly scalable to large scale interaction graphs or originally designed for ranking task. To make the baselines runnable on large and sparse graphs and comparable to our proposed method, we have made a few adaptions.
CCF:
Since item features are provided in the datasets, to allow CCF to make use of the features, modify it in the same way as how we incorporate features into 2TS model (see Section 3.3).
JODIE: made the following adaptions: 1) replaced the static one-hot representation to 64-dim embeddings because the number of nodes are large; 2) represent the category / tag categorical feature via learnable embedding table; 3) used triplet loss with random negative example rather than original MSE loss, which empirically show improvements.
dynAERNN:
These two methods are originally designed for discrete graph snapshots, while our focused tasks are continues interaction graphs. We manually transformed the interactions graph into 10 snapshots, with equal edge count increments.
For the downstream ranking task, we followed the evaluation method used in dySat (Sankar et al., 2020): after the node embeddings are trained, we train logistic regression classifier to predict link between a pair of user item node embedding using Hadmard operator. The logits on test set are used for ranking. For Taobao dataset, we are not able to get reslt of dynAERNN given the memory constraint.
We changed the data batching from per-user based to per-example based, to better adapt for time ordered interaction data and to better make the full use of the training data. Besides, the implementation of GRU4Rec follows implementation by the authors of MIMN paper, which includes some modifications compared to the original version of GRU4Rec released by their authors, and should be expected to perform better.
GRU4Ree & DIEN & MIMN:
setting for ML- 20M dataset, according to the implementation released by the authors, and changed dimension to 128. For Taobao dataset, to fit the model size into 16GB GPU memory, we changed the embedding dimension to 32 and batch size
Bert4Rec: For ML-25M and lastfm- k dataset, we followed the
GCMC & GCMC-SAGE & GCMC-GAT: We changed the loss function from Softmax over different ratings to Softmax [true item, 10 random selected items), to better adapt to the ranking task.
Besides, we clarify that in the baseline models Bert4Ree, SASRec, and dynAERNN, item features not incorporated, since it's not straightforward include features into their original model design, and it's unclear if adding features will help their setup or not.
C CONFIGURATION OF 2TS AND 2TS-PLUS
We will present some important hyperparameters chosen for the 2TS and 2TS-Plus in the experiments.
15

---

Under review as a conference paper at ICLR 2021
Threshold ta E N: Users with more than t' interactions observed in the training considered active users, and those with less than or equal to t* observed interactions are considered inactive users. For the results that get in Table 1, the choices of the thresholds are given in Table 4.
Table 4: For the results shown in Table 1, the thresholds t* for splitting the users specified as follows.
Taobao
Movielens-25h5
lastfm-IK
2TS
100
200
100
2TS-Plus
100
200
200
Last K interactions for active users: In Stage training, for each active user, we only the last K' interactions to train the RNN. The number K' is chosen to be either 10 or 20, and they actually give similar performances.
Embedding dimension: Both user embeddings and item embeddings set to be of dimension 64. If the item contains features, we will use another 64 dimension for the feature embedding.
Learning rate: the learning rate for training 2TS and 2TSS-llus is searched over and only. Most of the time le 4 works better. The optimizer is Adam.
Histogram user degiee
Hatagram degree
23.9%
65%
30%
5.5%
0.5%
200
600
1800
2000
4000
8000
1200
(a) ML-25M
Histogram user degree
Histogram
30.0%
8.0%
25.0%
20.9%
4.5%
5.5%
0.7%
206
0.0%
(b) Taobao
Figure 5: Distribution of the number of observed interactions in the training dataset.
16


Reviews:
[{"review_id": "utaqoS0dfE0", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main idea in this work is to separately model users with a lot of activity in the systems and users with little activity and data. \nTo this end, two RNN's are trained on the items that the users have interacted with one for active users and one for less active users. User and item embeddings are generated with a matrix factorization process and these are used as input to the RNN's and to initialize its hidden state. For inactive users, a common initialization is used. \n\nWhile this is an interesting idea it is somewhat unclear what the real benefits of this method are. The split between active and inactive users is rather unclear, how many interactions are then left in the inactive users split? \nThe experimental results are also somewhat unclear. it seems that the last interacted item is concatenated to the user profile and item attributes are also concatenated to the item representations learned by matrix factorization. Are there additional data sources included in the other baselines as input somehow? It is also unclear if the improved results are due to inactive or active users. \n\nI also think that it would be interesting if the two models would be linked somehow explicitly rather than 'just' from the joint factorization. Overall an interesting idea but the experimental results I think are not convincing and the split modeling could be justified quantitavly. \n\nminor: In table 1 last column the second-best performing method is GRU4Rec.\n              Figure 2, is not very informative for large timeframes as most methods converge in a few hours (or minutes) on this data\n \n\nAfter reading rebuttal, I think the paper is somewhat improved hence I increase the score. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "qTX7y5DIlPM", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper considers the sequential recommendation problem. The proposed method essentially combines the following two ideas: (i) two-stage learning: using conventional CF to pretrain user/item embeddings, and feed them (fixed, unlearned) into the 2nd stage learning. (ii) two-time-scale: using 2 RNNs to model active users and inactive users respectively.\n\nAlthough the model design makes sense, it seems too ad-hoc and not novel. For example, the idea (i) actually learn the model with pretrained embeddings, which achieves faster training speed. But this has been widely adopted in various domains. The comparing baselines certainly could easily adapt to use the same pretrained embeddings for faster speed. The claimed advantages are not convincing given important related work/baselines are missing. Specifically I have the following concerns:\n\n- Encoding long-range sequence: the proposed method doesn't model long sequences, while seeks to capture user history with a simple MF. This idea: (i) ignores sequential patterns in user previous history; (ii) has been widely adopted in the literature[1][2], especially before we have powerful deep sequential models.\n\n- Computational efficiency: (i) the efficiency advantage mostly comes from the use of pretrained embeddings, which can be adopted by almost all the baselines for acceleration. (ii) Another trick used is to cut off the sequence (only train the last 10/20 user actions). Such an acceleration, again, could be adopted by other methods.\n\n- Baselines: The paper claims SOTA performance on sequential recommendation, but baselines like SASRec(2018), BERT4Rec(2019) are missing. SASRec performed better than GRU4Rec and also showed much faster training speed. The GRU4REC is almost the 2nd best method among all the baselines used in the paper, and it's from 2015.\n\n- table 1 may have some typo. The recall of 2TS on lastfm-1K is worse than GRU4Rec, but the improvement shows 2.2%.\n\nThe proposed mode design, though, makes sense, but lacks novelty as it's a combination of various commonly used tricks. The improved training speed comes from pretrained emb and sequence cut-off, which are not a novel contribution for efficient sequential/recommendation model training. Several strong baselines are missing, which further weaken the paper.\n\n========================brief update after author response==============================\nI decided to maintain my rating due to several key claims in the response are unverified. For example, in Q1 \"We propose that NOT ALL users (but only active users) should use transductive embeddings to memorize their history, which is different from existing methods that treat all users in the same way.\", however, comparison against them is missing, especially for methods like [1] and [3] that uses both inductive and sequential embeddings; in Q2 \"Trivially cutting off the sequence could result in worse accuracy\", however, this is not verified: not clear what's the effect of cutting off on baseline models. I believe these are the core research questions that need to be verified in the paper, which are unfortunately missing.\n\n[1]Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation, ICDM'16\n\n[2]Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding, WSDM'18\n\n[3] Next Item Recommendation with Self-Attention", "extra_scores": {}, "extra_text": {}}, {"review_id": "isg7EtuCQkX", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors propose an interesting application of Recurrent Neural Networks to build a sequential recommender system with a different model for active and inactive users. \n\nIn a real-world recommender system, handling cold-start users separately is a big challenge, and this paper proposes an interesting approach to solve that. \n\nAuthors propose a two-staged method, wherein the first stage, the aggregated user-item interaction matrix is factorized to generate initial transductive embeddings for users and items. These are later re-used in the second stage and for active users, their embeddings are fixed. \n\nSome strong points of the paper:\n\n1) A modular approach is taken to handle active users and inactive users separately. This would be helpful in designing real-world recommender systems, where the cold start problem is critical. \n\n2) The paper is generally well-written with an adequate description of the method and any equation.  \n\n3) The solution is well-motivated with proper justification in Section 3 and in the introduction. \n\n4) Performs well against strong recent methods like JODIE (Kumar et al) and GNNs. \n\nSome questions to authors:\n\n1) How are cold-start items handled in the method? I assume cold-start items is an equally critical problem faced in designing a RecSys system.\n\n2) Why are item embeddings kept constant in the second stage? For frequent items, learning/updating embeddings during RNN training would enhance their quality. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "NxRsp3QLzo2", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Given sequential interactions between users and items, the goal is to predict the next interactions of users.\n\nChallenges\n- For active users, common models such as RNNs have limitations when dealing with long-range sequences due to the difficulty in gradient propagation.\n- For inactive users (or new users), there exists the cold-start problem.\n\nMain idea\n- They partition the users into two sets: active and inactive users. Active users have many observed interactions; Inactive users have very few or zero observed interactions.\n- They pre-train user and item embedding vectors via the matrix factorization.\n- Two independent RNNs are trained for the active and the inactive users, respectively.\n- When training the RNNs, user and item embedding vectors are fixed.\n- For all the inactive users, a common embedding initialization is shared.\n\nStrengths\n- They tried to explain the intuition of the proposed model in detail.\n- The proposed method outperforms some previous methods on real-world datasets.\n- The ablation study shows the effectiveness of the proposed training algorithm.\n\nWeaknesses\n- The idea is very incremental compared to previous methods.\n- There is no theoretical analysis of the proposed method.\n- The presentation of the figures is not good (Colors in Figures 2 and 3).\n\nSummary\n- They suggested the need to separate learning for active and inactive users. The intuition behind the method makes sense, and it has been shown through experiments that it performs well.\n- Authors will be able to increase their contribution through thorough theoretical analysis or developed ideas.\n- Authors also need to improve the presentation of the paper.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received divergent scores (one strong negative and three positives). The positive reviews praise the clear intuition/motivation and strong empirical performance, while the negative review considers the proposed approach ad-hoc with limited novelty. I read the paper myself and found myself leaning more towards the negative score. In more details: 

I think the paper proposed a cleverly-engineered solution to employ two separate RNNs to model two different subsets of the users (active v.s. inactive). To combat overfitting, the authors proposed some tricks: 1) use MF to learn a better initialization and 2) tie some of the parameters together. The ablation study shows that both are quite useful, and not too surprisingly when you have two powerful RNNs and work hard to make sure they don't overfit, they perform better than a single RNN. As I mentioned earlier, this whole approach is quite cleverly-engineered and executed. But it's not clear to me if this is something that the ICLR community can benefit from (maybe except a relatively small proportion). I believe this paper can find a bigger audience in venues like KDD whose deadline is coming up. 

Furthermore, the authors presented some theoretical analysis to justify their intuition, which to me feels a bit forced. To start, a big assumption is that the optimal user embeddings are very concentrated, which is a rather strong assumption and I hardly believe it will hold in practice (what can even be considered as "concentrated" in high-dimensional space?). Following that, the theorem implies that the initial embedding for the inactive users should be the expected optimal embedding, but then later in the paper this point seemingly got completely ignored and instead the authors just proposed to learn a common initialization. Additionally, the theorem suggests that there is an optimal threshold, but later in the paper again this point got completely ignored. I know the theory makes assumptions and builds on simple cases. But my point is that you don't need theorem to show me that two RNNs can perform better than one when trained properly (which, I admit, is non-trivial and is the main contribution of the paper). There has been a lot of discussion around the ML community about the trend of making your paper look "mathy" and I don't think this is a good thing. 

Minor comment: in MF, the objective wrt each embedding is convex, but the whole optimization is not jointly convex and it is not likely that you can get to the global optimum. It is relatively insensitive to initialization though (comparing with neural nets). 
Decision: Reject