Bridging Recommendation and Marketing via Recurrent Intensity Modeling

1 INTRODUCTION . Many ML applications today involve decision making based on streams of events . In recommender systems ( RecSys ) , reasoning on the stream of events in a user ’ s past history has allowed real-time user-based item recommendations ( ItemRec ) , leading to significant impacts ( Hidasi et al. , 2016 ; Hidasi & Karatzoglou , 2018 ; Ma et al. , 2020 ; Chen et al. , 2019 ; Kang & McAuley , 2018 ) . However , being user-centric , ItemRec has limitations in creating a globally inclusive environment for the item producers . For example , ItemRec tends to over-expose a few popular items to improve the immediate user satisfaction , yet this causes the selected items to get even more popular , discouraging the other producers from creating seminal items with higher potential values in the future . This is an important limitation because , as a RecSys involves the participation of both users and item producers , we must consider the success of the item producers for the long-term richness of the RecSys environment . We consider a UserRec problem which aims to recommend users on behalf of the item providers , based on their likelihood to interact with a given item in the near future . UserRec is a mathematical generalization of several item-marketing scenarios . For example , item providers may choose to bid on search keywords or previously browsed items as a strategy to increase the exposure of their items . These searching or browsing events may be considered as part of the user-history features , with which the likelihood scores can be predicted . As another example , a user may be in a particular state where a nudging via push notifications could lead to positive outcomes . Instead of hand-crafting a filter on the states , we may learn latent representations based on a likelihood objective for UserRec . While UserRec is a great way to communicate the needs of the item providers , this channel can not be abused . Bidding too quickly may deplete the advertisers ’ budgets and sending push notifications too often could turn the users away . Thus , two challenges stand out : how to predict the relevance for UserRec and how to decide with pre-allocated marketing budgets against future users in real time . First , the prediction problem can be challenging because the user state is evolving over time . A past event may stimulate future engagements , whereas sustained inactivity leads to churns . We must redefine user identity to address Ship of Theseus Paradox : whether an object that has had all of its components replaced remains fundamentally the same object . We note that some of these problems are also present in ItemRec in less obvious ways . For example , the state of an item may change after a price change or a phenomenal review ( Wu et al. , 2017 ; Ma et al. , 2020 ) . Nonetheless , state changes are often more critical with users than items . To date , most RecSys have adopted a hidden-state user representation to solve online ItemRec with evolving user states . The user states are based on the user ’ s past event histories , passed through sequence models such as recurrent neural networks ( RNNs ) or Transformers , for the prediction of the item distribution for their next choice ( Cho et al. , 2014 ; Hidasi & Karatzoglou , 2018 ; Ma et al. , 2020 ; Kang & McAuley , 2018 ) . However , the ItemRec formulation may not directly translate to UserRec , because the item choice is formulated as a user-conditional distribution without an identifiable user prior . Specifically , due to the softmax activation across all items for the given user , an inactive user may get similar final scores as an active user of the same type . Further , imputing the user prior term with an incorrect ” Bayesian ” interpretation may lead to a paradox ( Figure 2 ) . The cornerstone in UserRec is to predict the intensity of a user ’ s activity in a given item type in a comparable future time window , based on the user ’ s past data . We observe a clear connection to Marked Temporal Point Processes ( MTPPs ) ( Du et al. , 2016 ; Reinhart , 2018 ) , where a complete temporal model includes the prediction of both event-times and types ( i.e. , items ) . In the analogy , by placing the TPP-intensity parameters learned from the event time variables , we find theoretical consistency in the final scores to be considered for both UserRec and ItemRec . We call our strategy Recurrent Intensity Modeling ( RIM ) . While we discuss some related MTPP work in appendix , most of them stop at the prediction of event time or data likelihood . We claim novelty in the application of the intensity predictions to the UserRec problem , as a solution to the “ Bayesian ” paradox . Our next challenge is online decision under pre-allocated budgets for a distribution of user ( -state ) s , often over a period of time . For example , we may aim to expose an item to the Top-1 % of all users who may visit the website in the next week and be interested in this item . The challenge is that some UserRec decisions must be made in real-time as the user arrives at a suitable state , e.g. , when the user visits the website , despite uncertainties in the user ’ s relevance and/or purchase intents compared with others who may arrive at a later time . This is analogous to online secretary/marriage problem , where at most one candidate can be selected for the whole time and we aim to do so optimally with minimal regrets in hindsight . Further , we focus on the interaction between multiple users and items for their combined benefits . We develop an independent solution that resembles online dual decomposition ( Huang et al. , 2016 ; Mehta , 2013 ; Jenatton et al. , 2016 ) , yet we introduce entropic regularization similar to Sinkhorn ’ s algorithm ( Cuturi , 2013 ) to improve its convergence speed in our experiments . We call this algorithm Online-CVX and it complements our RIM predictions to complete a full story of online user-item matching ( OnlnMtch ) with broader impacts . Additional discussions on related work can be found in the appendix . Our contributions are : 1 . A novel connection between UserRec and ItemRec via Recurrent Intensity Modeling ; 2 . An improved Online-CVX program to handle long-term budget/exposure constraints ; 3 . Empirical results showing RIM ’ s effectiveness in UserRec and ItemRec , and its superiority in managing relevance-diversity trade-offs ; 4 . All RIM model components : RNN , TPP , CVX , can scale to infinite user pools via mini-batching . 2 RECURRENT INTENSITY MODELING ( RIM ) . We are interested in modeling a temporal process represented by a series of event times and types of a given user . Let S = [ ( tj , yj ) : ∀t1 < t2 < · · · < tj < . . . , ∀yj ∈ Y ] to be a random process that generates user behaviors with a continuous time variable and a type variable ( i.e. , the consumed item in the event ) in a large-cardinal set Y . We consider a user-time as a unique instance variable X = S ( t ) = { ( tj , yj ) ∈ S : tj < t } , with observable user history until time t. We use lower-case x = s ( t ) to indicate its realization and x ∈ X to be the set of all possible user-time instances in the recommendation problem . In the ItemRec construct , the sequence models are called upon the arrival of a user event to predict the item choice by a conditional categorical distribution : P ( Y = y|x ) = λ ( x , y ) ∑ y′∈Y λ ( x , y ′ ) , where λ ( x , y ) = exp ( w > y h ( x ) + by +m ( x ) ) , ( 1 ) where h ( · ) is a sequence encoder function , wy is a row in the decoder weight , which is also interpreted as the hidden embedding of a specific item , by is a global bias in the item choices , and m ( x ) represents any extra terms that can not be learned because they cancel between the nominator and the denominator . The m ( x ) term reminds us that the raw ItemRec scores λ ( x , y ) can not be used to compare across users . We aim to repurpose λ ( x , y ) for UserRec by associating m ( x ) with a user prior term that reflects their activeness in the RecSys , an important factor in UserRec . 2.1 MARKED TEMPORAL POINT PROCESSES ( MTPP ) . Given our goal to select ( x|y ) , our first thought is to frame it by Bayes rule p ( x|y ) ∝ p ( y|x ) p ( x ) , i.e. , relating m ( x ) to log p ( x ) and other normalization terms . However , this leads to a paradox shown in Figure 2 . The paradox is because we represent a user as a single variable x , instead of a TPP , s ( t ) . TPP predicts when the next event will happen for a user in addition to what type it is . TPP is different from other temporal models in that it predicts the event time as a moving target , whereas the other models aggregate time as a feature . The temporal concept is best explained from a counting perspective . Here , we model the absolute number of user-item events , ∆N ( x , y ) , in a future time window , ( t , t + ∆t ] , where s ( t ) coincides with x . We adopt a Poisson assumption that the rate at which the desired events happen is smooth in time , given by an intensity parameter : λ ( t ; s ( t ) , y ) = λ ( x , y ) = lim ∆t→0 E [ ∆N ∆t ∣∣∣∣x , y ] = E [ dNdt ∣∣∣∣x , y ] . ( 2 ) Besides matching the expectations , we may further prescribe ∆N ( x , y ) ∼ Poisson ( λ ( x , y ) ∆t ) as a parametric distribution , especially in short time windows.1 The Poisson log-likelihood is : logP ( ∆N ( x , y ) = n|λ ( x , y ) ∆t ) = n log ( λ ( x , y ) ∆t ) − λ ( x , y ) ∆t− log ( n ! ) . ( 3 ) 1Poisson may be over-dispersed in longer horizons , see discussions in Salinas et al . ( 2020 ) . The definition of the intensity parameter is visualized in Figure 1 . The intensity parameter connects three variables of our interests : user sequence x , item y and our prediction target ∆N ( x , y ) in a future time window ( t , t + ∆t ] . Slicing ∆N ( x , y ) in different directions yields different aspects of the problem . Most importantly , we slice by ∆t to connect it to the distribution of future event times . We expect an active user to have shorter intervals between events and higher intensities in unit time . Temporal Point Processes ( TPPs ) . We focus on the connection between the intensity parameter λ ( x ) = λ ( t ; s ( t ) ) and the event-time variable t for a specific item y which we omit . Intuitively , we split the time interval between two adjacent events into a large set of infinitesimal windows , i.e. , ( tk , tk+1 ] = ∪τ ( τ , τ + dt ] , and insert Poisson likelihood with label n ( τ ) = 0 to all previous windows and n ( τ∗ ) = 1 to the last window . Plugging in equation 3 we get∑ τ logP ( dN ( τ ) = n ( τ ) |λ ( τ ; s ( τ ) ) dt ) = log ( λ ( τ∗ ; s ( τ∗ ) ) dt ) − ∑ τ λ ( τ ; s ( τ ) ) dτ dτ→0⇒ log p ( Tnext = tk+1|tk ; λ ) = log λ ( tk+1 ; s ( tk+1 ) ) − ∫ tk+1 tk λ ( τ ; s ( τ ) ) dτ . ( 4 ) Note that the dt inside the logarithmic term is moved to the left side when we convert the probability mass function to probability density function in the continuous variable Tnext . Reinhart ( 2018 ) ; Chen ( 2016 ) discuss rigorous derivations as well as additional properties . Equation 4 is known as the inter-arrival time distribution , which directly formulates a loss function to learn the intensity parameters , without the need to explicit bucket time in equation 2 . The intensity functions can further be time-varying based on user states . The numerical integration may be solved by importance sampling ( Mei & Eisner , 2017 ) , ODE solvers ( Chen et al. , 2018 ) , or analytical forms in the case of Hawkes processes . Marked-TPP Decomposition . While TPPs can be built for each event type ( i.e. , item choice ) with cross-channel influences from other event types , this may lead to drastically different models than existing sequential ItemRec models . Instead , we utilize a Marked-TPP decomposition , λ ( x , y ) = λ ( x ) p ( y|x ) , ( 5 ) where p ( y|x ) comes from a regular sequence model in equation 1 . MTPP resembles Bayes rule and considers the user sequence x = s ( t ) as a point process . It is the framework of our RIM proposals .

Summary: This paper considers the problem of recommendation in a temporal context. In this setting, which the authors call UserRec, the recommender system must determine whether to recommend an item to a user at a specific point in time given their prior interaction history, or whether to delay the interaction. They draw connections to classical item recommendation, where the recommender must instead recommend some item to a given user, and does not have the option of delaying the recommendation. By modelling the UserRec problem as a temporal point process (TPP) the authors propose a framework to convert algorithms developed for ItemRec into a UserRec model.  They then propose an algorithm for recommendation in matching that leverages their TPP representation to setup a convex optimization problem. Finally, the authors validate their results on three different datasets.