Learning Two-Time-Scale Representations For Large Scale Recommendations

1 INTRODUCTION A hypothetical user ’ s interaction with recommendation systems gives us diminishing returns in terms of its information value in understanding the user . For an active user who has lots of historical interactions , she is typically well understood by the recommender , and each new interaction gives relatively little new information . In contrast , for an inactive or new user , every additional interaction will provide interesting information for understanding this user . Therefore , the representations for active and inactive users should be updated differently when a new interaction occurs . Figure 1 illustrates such information diminishing phenomenon , where the amount of change in user embedding from φt to φt+1 due to an additional interaction is decaying . One can select a particular threshold t∗ for the number of interactions , above which the users can be categorized to active users , and below which inactive users . Roughly active users ’ embeddings evolve slowly as a function of the number of interactions , while inactive users ’ embeddings evolve fast . Hence a two-time-scale embedding evolution . Apart from the time-scale difference in temporal dynamics , the simultaneous presence of active and inactive users also presents other modeling and computational challenges . On the one hand , active users lead to long sequences of interactions and high degree nodes in the user-item interaction graph . Existing sequence models , such as RNN models , have some limitations when dealing with long-range sequences , due to the difficulty in gradient propagation . Moreover , graph neural network-based models become computationally inefficient due to the intensive message passing operations through high-degree nodes introduced by active users . On the other hand , predicting preferences of inactive or new users ( also known as the cold-start problem ) is a challenging few-shot learning problem , where a decision needs to be made given only a few number of observations . To address various challenges imposed by the presence of two types of users , we leverage the different dynamics of these users and propose ( i ) a two-time-scale ( 2TS ) model and ( ii ) a two-stage training algorithm . 2TS model . Based on the number of observed interactions , we partition the users into two sets : active and inactive users . Our 2TS model ( Fig . 1 ) update the embeddings of active users and inactive users by two RNNs with independent parameters , in order to respect the two-time-scale nature . Moreover , the initial embeddings of inactive users are represented by a common embedding ψ , which is shared across all inactive users . Therefore , the overall model for inactive users is inductive , in the sense that the learned model can be applied to unseen users . In contrast , the initial embedding of each active user is a user-specific embedding φu , which is also called transductive embedding . Such embeddings are very expressive , which can better express users with a long history . Two-stage training . In stage 1 , we first learn transductive user embeddings φu and transductive item embeddings xi using a classical collaborative filtering method . Then we fix these embeddings , and in stage 2 , we will learn the parameters of the two RNNs and a common initialization ψ for inactive users . It is notable that the transductive embeddings for inactive users are abandoned in stage 2 . Only those for active users are finally used in the 2TS model . Besides , for active users , we do not use all interaction data to learn the RNN since their transductive embeddings have already encoded the information of their history . We only use a small number of last clicked items to learn the adaptation for active users , which improves the efficiency of the training process . The proposed 2TS model and the two-stage training algorithm lead to a few advantages : • Bias-variance trade-off . The differential use of transductive and inductive embeddings for the two RNN models allows 2TS to achieve a good overall bias-variance trade-off . We theoretically analyze such trade-off in Section 2 through the lens of learning-to-learn paradigm for designing online learning ( or adaptation ) algorithms . Our theory shows that there exists an optimal threshold to split users to achieve the best overall excessive risk . • Encode long-range sequence . The transductive embeddings φu for active users are user-specific vectors , so they can memorize the user ’ s long-range history during the training , without suffering from the difficulty of gradient propagation . The RNN on top of these transductive embeddings is only used for adaptation to recently engaged new items . • Computational efficiency . The efficiency of our method on large-scale problems mainly comes from two designs in the algorithm . First , stage 1 learns the transductive embeddings of active users and items , which contain a large number of parameters . However , it is fast since it does not involve any deep neural components and the loss is simply a convex function . Second , stage 2 only learns the RNNs which contain a small number of parameters , and the RNN for active users is only trained on a few last engaged items , which cuts off the long sequences . Experimentally , our method reveals to be much more efficient than the baselines on large-scale datasets . We summarize the contributions of this paper as follows : • To explain the intuition and motivation of the 2TS model , we provide theoretical analysis on a simplified setting , which rigorously argues the need for differential use of transductive and inductive embeddings for active and inactive users ( Section 2 ) . • Motivated by the analysis , we design the 2TS model and a two-stage training method , for practical use ( Section 3 ) . The proposed method is applied to two large-scale benchmark datasets and compared comprehensively to various baseline models , spanning a diverse set of categories , which shows that our method is advantageous in terms of both accuracy and efficiency ( Section 5 ) . 2 THEORETICAL MOTIVATION : WHY TWO-TIME-SCALE MODELS ? . We will first present the motivation for designing the 2TS model , through the lens of online learning and stochastic optimization . Our analysis quantitatively reveals that ( i ) the embeddings for active and inactive users evolve in different time scales , and ( ii ) two different online learning algorithms for active and inactive users respectively can lead to a better overall estimation of user embeddings . Our analysis will be carried out in a learning-to-learn setting , where online learning algorithms need to be designed to tackle a family of tasks for estimating the embedding vector of a user . Though this idealized setting can not cover all aspects of real-world recommendation problems , it leads to clear insights on the 2TS behavior of active and inactive users and the benefits of using two different online algorithms for these two respective user groups . These insights also motivate our practical implementation of the 2TS model using deep learning in Section 3 . 2.1 SETTING : LEARNING-TO-LEARN . Our setting consists of three components : the estimation task for an individual user , the distribution of tasks for a family of users , and the online algorithms which we want to design . Individual Task . We associate each user u with a ground truth embedding φ∗µ ∈ Rd , which can be thought of as a vector representation of a user ’ s preference . This embedding is defined by a distribution µ over user ’ s clicks over items ( x , y ) , where x ∈ X represents the item embedding which we assume is bounded , i.e. , ‖x‖2 ≤ Bx , and y ∈ { 0 , 1 } indicates whether the item is clicked . They follow the user-specific distribution ( x , y ) ∼ µ . More specifically , the ground truth user embedding is defined as the minimizer of the expected risk according to a regularized logistic loss φ∗µ : = arg minφ∈ΦRµ ( φ ) , whereRµ ( φ ) : = E ( x , y ) ∼µ ` ( φ , x , y ) , and ` ( φ , x , y ) : = −yx > φ+ log ( 1 + exp ( x > φ ) ) + c2‖φ‖ 2 2 , ( 1 ) where c > 0 is some regularization constant and Φ : = { φ ∈ Rd : ‖φ‖2 ≤ Bφ } . Typically , we do not have access to the distribution µ , but a sampled set of T observations z [ T ] : = { ( x1 , y1 ) , · · · , ( xT , yT ) } ∼ µT , which can be used as training samples to obtain an initial estimate φ ( z [ T ] ) of the user embedding . We assume φ ( z [ T ] ) is obtained by applying stochastic gradient descent ( SGD ) to the loss ` over z [ T ] in this section , but this training stage is not limited just to the SGD algorithm . It is expected that with more samples the estimation will be closer to the ground truth φ∗µ . The estimation φ ( z [ T ] ) is modeling the offline training stage of a recommendation system . A Distribution of Tasks . We consider a distribution of users , by assuming that the user-specific distribution µ is sampled from a meta-distribution µ ∼ pu . Furthermore , the number of observed interactions for a user , denoted by T ∼ pαT , is a random variable that follows a power law distribution with density p ( T ) ∝ ( T + 1 ) −α . The power law distribution models the fact that there will be lots of users with very few interactions , and very few users with many interactions.1 A key assumption of our model is that the variance of the ground truth user embeddings is small . That is Varm = Eµ∼pu‖φ∗µ −m‖22 ≤ r , wherem = Eµ∼puφ∗µ . ( 2 ) The assumption of small variance is critical in the sense that it will allow us to aggregate information from inactive users to obtain better estimations of their user embeddings . Online Algorithm Design Problem . Our goal is to design online adaptation algorithms for this distribution of users , such that the overall excessive risk of the online adaptation across all users is small . This is to model the online sequential recommendation stage where new user-item click information is incorporated after system deployment . Note that we are not restricted to design a single online learning algorithm for all users . In fact , we will show later that 2 online learning algorithms can actually lead to better risk bound than 1 online learning algorithm . In this algorithm design problem , each user µ corresponds to an online learning task , where items arrive sequentially . Starting from an initial embedding φ1µ , an online algorithm updates the user embedding whenever it observes a new user-item interaction ( xt , yt ) ∼ µ : φt+1µ ← Update ( φtµ , xt , yt ) , ( 3 ) and then it applies φt+1µ to the next item xt+1 , and suffers the loss ` ( φ t+1 µ , xt+1 , yt+1 ) in Eq . 1 . The excessive risk of this online algorithm after encountering N sequential samples is 1 N ∑N t=1 ` ( φ t µ , xt , yt ) − ` ( φ∗µ , xt , yt ) . The design problem involves ( 1 ) the initialization for the algorithm and ( 2 ) the update step of the algorithm . One obvious choice is to use the user-specific embedding φ ( z [ T ] ) estimated in the offline training phase as the initial embedding φ1µ for the online algorithm . However , is this estimation the best choice for the initial embedding φ1µ = φ ( z [ T ] ) ? Is there a better choice ? How should the initialization depend on T ? We answer these questions in the next subsection . 1This assumption is sensible since Fig . 5 shows that T approximately follow a power law in real-world datasets .

Summary: The paper considers the sequential recommendation problem. The proposed method essentially combines the following two ideas: (i) two-stage learning: using conventional CF to pretrain user/item embeddings, and feed them (fixed, unlearned) into the 2nd stage learning. (ii) two-time-scale: using 2 RNNs to model active users and inactive users respectively.