B RIDGING R ECOMMENDATION AND M ARKETING VIA R ECURRENT I NTENSITY M ODELING
Anonymous authors
Paper under double-blind review
A BSTRACT
Personalized item-recommendation (ItemRec) and targeted item-marketing are two sides of the same coin.
While ItemRec chooses the best items immediately upon user requests, marketing considers the strategic item exposition in user-space and time, under budget constraints.
Inspired by recent advances in sequential
ItemRec models, we study the reverse problem we call UserRec, which is to select users based on their real-time states on behalf of the item providers.
UserRec turns out to be nontrivial and our key insights include a novel connection to Marked-Temporal Point Processes (MTPPs), with which we build a family of models we call Recurrent Intensity Models (RIMs).
To further integrate with ItemRec regimes where real-time decisions are required, we discuss and improve an online optimal decision strategy we call CVX-Online, based on the duality theories in convex optimization.
The union of ItemRec and UserRec reveals new dimensions for exploration.
We conduct experiments on three real-world datasets where we show that RIM is highly effective on both UserRec and ItemRec.
With
CVX-Online, we also successfully diversify item exposures with minimal compromises in relevance, thanks to the new ability to explore in the user-state distributions.
We open-source our evaluation framework with a rich set of baselines.
I NTRODUCTION
Many ML applications today involve decision making based on streams of events.
In recommender systems (RecSys), reasoning on the stream of events in a user’s past history has allowed real-time user-based item recommendations (ItemRec), leading to signiﬁcant impacts (Hidasi et al., 2016;
Hidasi & Karatzoglou, 2018; Ma et al., 2020; Chen et al., 2019; Kang & McAuley, 2018).
However, being user-centric, ItemRec has limitations in creating a globally inclusive environment for the item producers.
For example, ItemRec tends to over-expose a few popular items to improve the immediate user satisfaction, yet this causes the selected items to get even more popular, discouraging the other producers from creating seminal items with higher potential values in the future.
This is an important limitation because, as a RecSys involves the participation of both users and item producers, we must consider the success of the item producers for the long-term richness of the RecSys environment.
We consider a UserRec problem which aims to recommend users on behalf of the item providers, based on their likelihood to interact with a given item in the near future.
UserRec is a mathematical generalization of several item-marketing scenarios.
For example, item providers may choose to bid on search keywords or previously browsed items as a strategy to increase the exposure of their items.
These searching or browsing events may be considered as part of the user-history features, with which the likelihood scores can be predicted.
As another example, a user may be in a particular state where a nudging via push notiﬁcations could lead to positive outcomes.
Instead of hand-crafting a ﬁlter on the states, we may learn latent representations based on a likelihood objective for UserRec.
While UserRec is a great way to communicate the needs of the item providers, this channel cannot be abused.
Bidding too quickly may deplete the advertisers’ budgets and sending push notiﬁcations too often could turn the users away.
Thus, two challenges stand out: how to predict the relevance for
UserRec and how to decide with pre-allocated marketing budgets against future users in real time.
First, the prediction problem can be challenging because the user state is evolving over time .
A past event may stimulate future engagements, whereas sustained inactivity leads to churns.
We must
[Figure]
redeﬁne user identity to address Ship of Theseus
Paradox: whether an object that has had all of its components replaced remains fundamentally the same object.
We note that some of these problems are also present in ItemRec in less obvious ways.
For example, the state of an item may change after a price change or a phenomenal review (Wu et al., 2017; Ma et al., 2020).
Nonetheless, state changes items.
To date, most RecSys have adopted a hidden-state user representation to solve online ItemRec with evolving user states.
The user states are based on the user’s past event histories, passed through sequence models such as recurrent neural networks (RNNs) or Transformers, for the prediction of the item distribution for their next choice (Cho et al., 2014; Hidasi & Karatzoglou, 2018; Ma et al., 2020; Kang & McAuley, 2018).
However, the ItemRec formulation may not directly translate to UserRec, because the item choice is formulated as a user-conditional distribution without an identiﬁable user prior .
Speciﬁcally, due to the softmax activation across all items for the given user, an inactive user may get similar ﬁnal scores as an active user of the same type.
Further, imputing the user prior term with an incorrect ”Bayesian” interpretation may lead to a paradox (Figure 2).
The cornerstone in UserRec is to predict the intensity of a user’s activity in a given item type in a comparable future time window, based on the user’s past data.
We observe a clear connection to
Marked Temporal Point Processes (MTPPs) (Du et al., 2016; Reinhart, 2018), where a complete temporal model includes the prediction of both event-times and types (i.e., items).
In the analogy, by placing the TPP-intensity parameters learned from the event time variables, we ﬁnd theoretical consistency in the ﬁnal scores to be considered for both UserRec and ItemRec.
We call our strategy
Recurrent Intensity Modeling (RIM) .
While we discuss some related MTPP work in appendix, most of them stop at the prediction of event time or data likelihood.
We claim novelty in the application of the intensity predictions to the UserRec problem, as a solution to the “Bayesian” paradox.
Our next challenge is online decision under pre-allocated budgets for a distribution of user(-state)s, often over a period of time.
For example, we may aim to expose an item to the Top-1% of all users who may visit the website in the next week and be interested in this item.
The challenge is that some
UserRec decisions must be made in real-time as the user arrives at a suitable state, e.g., when the user visits the website, despite uncertainties in the user’s relevance and/or purchase intents compared with others who may arrive at a later time.
This is analogous to online secretary/marriage problem, where at most one candidate can be selected for the whole time and we aim to do so optimally with minimal regrets in hindsight.
Further, we focus on the interaction between multiple users and items for their combined beneﬁts.
We develop an independent solution that resembles online dual decomposition (Huang et al., 2016; Mehta, 2013; Jenatton et al., 2016), yet we introduce entropic regularization similar to Sinkhorn’s algorithm (Cuturi, 2013) to improve its convergence speed in our experiments.
We call this algorithm Online-CVX and it complements our RIM predictions to complete a full story of online user-item matching (OnlnMtch) with broader impacts.
Additional discussions on related work can be found in the appendix.
Our contributions are:
A novel connection between UserRec and ItemRec via Recurrent Intensity Modeling; 2.
An improved Online-CVX program to handle long-term budget/exposure constraints; 3.
Empirical results showing RIM’s effectiveness in UserRec and ItemRec, and its superiority in managing relevance-diversity trade-offs;
Truncated user history in last week ( x )
P ( y = D | x )
A,B 0.4 C 0.
∅ 0.3 ∅∅ ×   p ( x =
[ A,B ])
= 0 .p
( x = C )
= 0 .p
( x = ∅ ) = 0 .⇒
  p ( x =
[ A,B ] | y = D ) ∝ 0 .
2 × 0 .
4 = 0 .p
( x = C | y = D ) ∝ 0 .
2 × 0 .
8 = 0 .p
( x = ∅| y = D ) ∝ 0 .
6 × 0 .
3 = 0 .×
  λ ( x =
[ A,B ])
=
λ ( x = C )
=
λ ( x = ∅ ) =
µ ⇒   λ ( x = [ A,B ] | y
=
D )
∝ 2 × 0 .
4 = 0 .λ
( x = C | y
= D )
∝ 1 × 0 .
8 = 0 .λ
( x = ∅| y = D )
∝ µ × 0 .
3 = 0 .
3 µ
R ECURRENT I NTENSITY M ODELING (RIM)
We are interested in modeling a temporal process represented by a series of event times and types of a given user.
Let S =
[( t j , y j ) : ∀ t 1 < t 2 < · · · < t j < . .
, ∀ y j ∈ Y ] to be a random process that generates user behaviors with a continuous time variable and a type variable (i.e., the consumed item in the event) in a large-cardinal set Y .
We consider a user-time as a unique instance variable
X = S ( t )
= { ( t j , y j ) ∈ S :
t j < t } , with observable user history until time t .
We use lower-case x = s ( t ) to indicate its realization and x ∈ X to be the set of all possible user-time instances in the recommendation problem.
In the ItemRec construct, the sequence models are called upon the arrival of a user event to predict the item choice by a conditional categorical distribution:
[Equation]
where h ( · ) is a sequence encoder function, w y is a row in the decoder weight, which is also interpreted as the hidden embedding of a speciﬁc item, b y is a global bias in the item choices, and m ( x ) represents any extra terms that cannot be learned because they cancel between the nominator and the denominator.
The m ( x ) term reminds us that the raw ItemRec scores λ ( x, y ) cannot be used to compare across users.
We aim to repurpose λ ( x, y ) for UserRec by associating m ( x ) with a user prior term that reﬂects their activeness in the RecSys, an important factor in UserRec.
2. M ARKED T EMPORAL P OINT P ROCESSES (MTPP)
Given our goal to select( x | y ) , our ﬁrst thought is to frame it by Bayes rule p ( x | y )
∝ p ( y | x ) p ( x ) , i.e., relating m ( x ) to log p ( x ) and other normalization terms.
However, this leads to a paradox shown in
Figure 2.
The paradox is because we represent a user as a single variable x , instead of a TPP, s ( t ) .
TPP predicts when the next event will happen for a user in addition to what type it is.
TPP is different from other temporal models in that it predicts the event time as a moving target, whereas the other models aggregate time as a feature.
The temporal concept is best explained from a counting perspective.
Here, we model the absolute number of user-item events, ∆ N ( x, y ) , in a future time window, ( t, t + ∆ t ] , where s ( t ) coincides with x .
We adopt a Poisson assumption that the rate at which the desired events happen is smooth in time, given by an intensity parameter:
[Equation]
Besides matching the expectations, we may further prescribe ∆ N ( x, y ) ∼ Poisson ( λ ( x, y )∆ t ) as a parametric distribution, especially in short time windows.
The Poisson log-likelihood is:
[Equation]
[Equation]
The deﬁnition of the intensity parameter is visualized in Figure 1.
The intensity parameter connects three variables of our interests: user sequence x , item y and our prediction target ∆ N ( x, y ) in a future time window ( t, t + ∆ t ] .
Slicing ∆ N ( x, y ) in different directions yields different aspects of the problem.
Most importantly, we slice by ∆ t to connect it to the distribution of future event times.
We expect an active user to have shorter intervals between events and higher intensities in unit time.
Temporal Point Processes (TPPs).
We focus on the connection between the intensity parameter λ ( x ) =
λ ( t ; s ( t )) and the event-time variable t for a speciﬁc item y which we omit.
Intuitively, we split the time interval between two adjacent events into a large set of inﬁnitesimal windows, i.e., ( t k , t k +1 ]
= ∪ τ ( τ, τ + d t ] , and insert Poisson likelihood with label n ( τ ) = 0 to all previous windows and n ( τ ∗ )
= 1 to the last window.
Plugging in equation 3 we get (cid:88)
[Equation]
Note that the d t inside the logarithmic term is moved to the left side when we convert the probability mass function to probability density function in the continuous variable T next .
Reinhart (2018);
Chen (2016) discuss rigorous derivations as well as additional properties.
Equation 4 is known as the inter-arrival time distribution, which directly formulates a loss function to learn the intensity parameters, without the need to explicit bucket time in equation 2.
The intensity functions can further be time-varying based on user states.
The numerical integration may be solved by importance sampling (Mei & Eisner, 2017), ODE solvers (Chen et al., 2018), or analytical forms in the case of Hawkes processes.
Marked-TPP Decomposition.
While TPPs can be built for each event type (i.e., item choice) with cross-channel inﬂuences from other event types, this may lead to drastically different models than existing sequential ItemRec models.
Instead, we utilize a Marked-TPP decomposition,
[Equation]
5) where p ( y | x ) comes from a regular sequence model in equation 1.
MTPP resembles Bayes rule and considers the user sequence x = s ( t ) as a point process.
It is the framework of our RIM proposals.
E STIMATION OF THE I NTENSITY P ARAMETERS
While RIM is motivated to estimate the intensity of each user-item pair separately, we ﬁnd it more convenient to utilize existing sequential ItemRec models.
Based on equation 5, we keep the RNN (or Transformer) model for its categorical prediction of the preference direction , p ( y | x ) , while introducing separate estimators for the user-intensity prior , λ ( x ) .
RNN-Pop is our simplest model, where we use the length of user histories from the collected data.
This approach works due to a homogeneous intensity assumption, i.e., the number of past events directly correlates with the future event intensity.
It also assumes that all user activities are collected over a comparable period of time, which often holds when, in practice, RecSys often truncates the training datasets by a ﬁxed time for scalability concerns.
This is the example shown in Figure 2.
RNN-Hawkes.
To extend the naive RNN-Pop models, we may break the homogeneity assumption to consider user state changes that affect future event intensity.
In this regard, Hawkes process assumes a positive stimulation through past user events and a gradual churn-out effect after sustained inactivity.
It models these effects via an exponentially-decaying kernel in the intensity parameters
[Equation]
where φ r ( t )
= 1 s r exp (cid:16) − ts r (cid:17) 1 t> 0 , and µ, α, s > 0 .
Here, we extend it with a mixture of R latent kernels with learned coefﬁcients and learn it with tick software package Bacry et al.
We design the R kernels to have log-spaced half-lives between 10 − 3 and 10 t max where t max is the largest temporal span in user histories.
A large- s r kernel approximates the RNN-Pop model with a step function after each observation, whereas a small- s r kernel suggests a fast-diminishing effect, often within a short browsing session.
RNN-Hawkes-Poisson (RNN-HP)
For long prediction horizons, we may also calibrate Hawkes scores to eliminate the effects of fast-diminishing kernels.
In fact, the accurate long-horizon model,
[Equation]
takes the expectation with respect to stochastic integrals.
To calibrate for the long-terms intensity scores, we notice that the Hawkes states φ ( t ) are already positive.
We thus model the ﬁnal scores as
[Equation]
Our additive formulation is inspired by (Mei & Eisner, 2017) and we slightly improve it by allocating the softplus to each coordinate to improve numerical stability.
Other Baseline Methods Practical item marketing models are often highly customized, e.g., with proﬁtability or seasonality concerns, and they are not always designed to cover all items in the catalog, which requires training labels for all user-item pairs, including user-item pairs with zero activities in a time period.
Despite so, we illustrate the general idea to use explicit recency, frequency, monetization (RFM) features for user segmentation (Fader et al., 2005).
Here, frequency and monetary spending in related items are correlated with a customer’s future life-time value (CLV) with respect to the given item and so is the time elapsed since last purchase with an inverse relation.
We build a model based on graph-convolutional feature aggregation similar to GCMC (Berg et al., 2017), followed by Bayesian-personalized ranking loss (BPR) (Rendle et al., 2009) against sampled negative users and items in the large space of all user-item pairs.
Notice that this GCMC model uses labels from an explicit time period, which is inferior to RIM because (a) it has fewer usable labels and (b) it bucketizes the labels instead of preserving their continuous time and order.
We attempt a mitigation we call GCMC-Ex that creates multiple periods to extract more labels and order information.
However, we do notice that perfecting this remedy leads to a temporal integration similar to our TPP derivation in equation 4.
More details of this model can be found in Section B.
We also consider matrix factorization (MF) methods, such as ALS (Hu et al., 2008) and LogisticMF (Johnson, 2014) from Implicit package 2 and BPR (Rendle et al., 2009) from LightFM package (Kula, 2015).
Notably, LightFM-BPR is limited to sample either users or items, but not both.
We include our own BPR model for completeness.
Unlike RIM and GCMC, MF methods do not consider temporal features.
R ECOMMENDATION AS A M ATCHING P ROBLEM
Now that we have described how we create RIM scores to allow UserRec by comparing the users’ real-time states, with the obvious beneﬁts of simpliﬁcation and generalization thanks to the continuous-time auto-regressive nature in TPP/RNN/Transformer models, we then suggest why this may lead to broader impacts.
Our main proposal is to integrate UserRec into ItemRec to form a matching problem (Mtch).
Mtch caters to the beneﬁts of both users and items, where the item provider’s beneﬁts are shown as exposure guarantees with best user choices on their behalf.
We will frame the problem, address its eminent scalability challenges, and discuss its connection to online optimal decision processes, as time is an irreversible direction in the user-state distribution.
Formulation of the Matching Problem In UserRec, we make marketing decisions in the space of user-time that is appropriate for the application, e.g., to consider push notiﬁcations for all active users at a given time or users in a particular stage of an experience.
We represent the user-time space as X (cid:51) x .
UserRec then picks Top- C users, which can equivalently be represented as the solution of a (multi-)assignment vector π ( x | y ) :
X → { 0 , 1 } :
[Equation]
For online marketing or with large user pools, the user states can be seen as a draw from a distribution
P ( X ) .
We may replace the summation in equation 9 with Lebesgue integral in the probability space.
In fact, we further extend the assignment problem to consider constraints in both UserRec and ItemRec for an Mtch solution π : X × Y → [0 , 1] that maximizes the global objective
[Equation] [Equation]
where α ( x ) and β ( y ) are user and item capacities in relative terms.
Let m = |X| and n = |Y| .
Choosing α ( x ) =
K/n and β ( y ) =
1 (unconstrained) reduces to the traditional Top-K ItemRec problem.
Choosing α ( x ) = 1 and β ( y )
=
C/m reduces to the ofﬂine UserRec problem equation 9.
In its most general form, equation 10 simultaneously solves for both ItemRec and UserRec to control the global exposure rates of different items.
We may also replace the β -constraints with lower bounds to guarantee the minimal exposure of the given items.
This corresponds to a guaranteed delivery contract in display advertising and our diversity pursuits by promoting cold-start items.
4. O FFLINE G REEDY S OLUTION
We simply sort the λ ( x, y ) in both users and items and then match user-item pairs one at a time until the constraints are satisﬁed.
For lower-bound β -constraints, we ﬁrst solve for the common factors by setting α = min( α, β ) , followed by a “top-off” of the remaining user capacities with respect to the original α .
While Ofﬂine-Greedy appears fast with in-memory datasets, costing O ( mn log( mn )) time, it has fundamental limitations in the lack of mini-batch sampling from inﬁnite user distributions or the adaptation to online decision scenarios, which we aim to resolve.
4. O FFLINE CVX S OLUTION
We craft a solution via online dual decomposition in convex optimization (CVX), which is analogous to quantile-thresholding v ( y ) in the marketing of a single item y , i.e. π ( x, y )
= 1 ⇔ λ ( x, y ) > v ( y ) when a new user-state x is considered.
By using online algorithms, we may learn v ( y ) on streams of user mini-batches.
Our CVX solution further resolves the competing conditions of multiple cooccuring item-marketing campaigns.
We propose an improvement over online subgradient methods (Huang et al., 2016; Mehta, 2013; Jenatton et al., 2016) with faster convergence in our experiments.
We ﬁrst describe the subgradient CVX approach as it yields stronger intuitions .
The general idea is to convert the constraints in equation 10 as Lagrangian multiplers:
[Equation]
where r x ( π ) =
E y
[ π ( x, y )] − α
x ) and r y ( π ) =
E x [ π ( x, y )]
− β ( y ) .
The Lagrangian form is parametrized by ( λ, α, β ) and it introduces dual variables v ( y ) to penalize constraint violations with the inner-loop, e.g., setting min v ( y ) ≥
[ − v ( y ) r y
( π )]
=
−∞ , if r y ( π ) >violates the Top-K ItemRec constraint.
When we use minimal-exposure constraints to promote diversity, i.e., bounding r y ( π ) ≥ 0 , the corresponding dual requirement changes to v ( y ) ≤ 0 .
The Lagrangian approach converts the constrained optimization to a minimax saddle point problem.
The next step is to change the order of operations to form its dual problem:
[Equation]
Note that the inner-max loop can be solved by picking the top items by the modiﬁed ranking scores,
[Equation]
while the outer loop is convex.
The dual problem can be solved by stochastic (sub-)gradient descents (SSGD) through all users in mini-batches, with decreasing learning rates such that (cid:80)
k
η k = ∞ , (cid:80) k
η 2 k < ∞ to guarantee its convergence.
In practice, however, we observe slower convergence with oscillatory patterns (Figure 3).
This is because the dual objective is non-differentiable when λ ( x, y ) − v ( y )
= 0 , causing the solver to bounce between the opposing sides in the convex hull.
Besides, setting one learning rate for all dimensions may not handle heterogeneous score distributions.
[Figure]
d u a l o b
j e c t i v e
[Figure]
equality constraint x [ y )]
= plot the dual objective with different entropic-regularizer (cid:15) as well as the solution paths of different methods.
As (cid:15) → 0 , (sub)gradient steps become oscillatory without carefully-designed decreasing learning rates.
To avoid the tuning of step sizes, we develop a novel solution via annealed alternating optimization, which converges better in our experiments.
Entropic Regularization and Alternating Optimization To improve practical convergence of
CVX on the set of observed users, we independently develop an entropic-regularized version with direct inspirations from Sinkhorn’s algorithm in optimal transport (Cuturi, 2013).
The trick is to replace the 0 ≤ π ≤ 1 constraint with an entropic regularizer, H ( π ) = − π log( π ) − (1 − π ) log(1 − π ) .
To smooth out the objective function, we further incorporate the primal constraint r x ( π ) ≤ 0 via another dual variable u ( x ) ≥ 0 .
The new dual objective and Lagrangian form become
[Equation]
(14) where the new Lagrangian, if expanded, is symmetric in u ( x ) and v ( y ) .
As a subroutine in the dual problem, we ﬁnd an explicit solution to the primal variable π ( x, y ) as:
[Equation]
which is similar to equation 13 but with soft assignments.
Plugging π in the dual form leads to functional gradients with respect to the dual variables: ∂d (cid:15) ( u,v ) ∂u ( x )
= − r x
( f ( u ( x ) , v, λ )) and viceversa for v ( y ) .
Particularly, the dual gradients are monotone in every coordinate of u ( x ) and v ( y ) , which allows efﬁcient bisection search for their roots in the range of: inf y ∈Y [ λ ( x, y ) − v ( y )] ≤ (cid:15)σ − 1 ( α ( x ))
+ u ( x ) ≤ sup y ∈Y [ λ ( x, y ) − v ( y )] , and vice-versa for v ( y ) .
The introduction of (cid:15) allows exact solutions for u ( x ) and v ( y ) in O (log n ) time in each step of alternating optimization.
Alternating with exact solutions is more efﬁcient than subgradient descents in our experiments, illustrated in Figure 3.
See Algorithm 1 and further discussions in Appendix C.
4. O NLINE CVX S OLUTION
Similar to online secretary / marriage problem, Online Matching (OnlnMtch) considers the optimal choices of future users for item exposition under a pre-allocated budget constraint for the whole time.
This is made possible by the CVX programs by carrying over the learned v ( y ) variables to future users, assuming that the past and future users share similar state distributions over time.
In essence, we subtract the RIM score λ
( x, y ) by the CVX parameter v ( y ) to make assignment decisions π ( x, y ) .
Notice that the absence of either RIM or CVX disables OnlnMtch.
While we assume that user-state distributions stay constant, Mehta (2013); Huang et al. (2016); Jenatton et al. (2016) further analyze the adversarial robustness against online changes which may lead to unsatisﬁed constraints and/or sub-optimal objectives in hindsight.
We leave this as future work, as we can show signiﬁcant empirical improvements in item diversity despite the said challenges.
E XPERIMENTS
We conduct experiments in three RecSys datasets with unique properties (code available at anonym).
• Movielens (ML) (Harper & Konstan, 2015) is a movie rating website that collects user preferences to study RecSys.
Over time, it released several versions of datasets.
We use a small-scale ML-1M in an 2003 release with one million rating events between six thousand users and three thousand items to easily validate our ideas.
• Netﬂix (NF) 3 is a movie rental service where movie ratings from a user’s past affects their personalized ItemRec in the future.
NF is widely used in rating prediction and ItemRec, but less so in UserRec and OnlnMtch.
• Yoochoose (YC) 4 is a dataset used in RecSys 2015 Challenge with six months of activities for product recommendation such as general tools, toys cloths, electronics, and much more.
We highlight some data preparation steps.
Most of our RIM methods are self-supervised sequence models.
Thus, we only need to hold out test windows with equal horizons for the evaluation of
UserRec, ItemRec, and Mtch.
We use absolute time windows on NF and consider only the observed users/items in the training data as testing targets, because we may not know which new users/items will show up in the testing windows without temporal leakage from the test set.
Besides temporal splits, it is common for sequence models to split train/test sets by users.
We consider user-splits on
ML and YC, because most users are active at different, non-overlapping clock times.
We hold out time windows only on the test users (Group-B in Table S1), with equal starting time relative to their ﬁrst event as well as equal size or horizon.
All training users (Group A) and the observed histories of the testing users (Group B left part) are considered training data.
RNN-HP and GCMC(*) require further splitting of the training set.
On NF, we create a set-back window between [ T (cid:48) , T ) from all users and on ML/YC, we keep the same time [ T, T + ∆ T ) but change the user base to Group A for validation.
Similarly, to simulate OnlnMtch, we must calibrate
Online-CVX on different users or times to test the transferability of the user-state distributions in online scenarios.
We reuse the set-back window as RNN-HP and GCMC(*) for simplicity.
Statistics of our data splits are shown in Table S1.
5. U SER R EC AND I TEM R EC R ESULTS
We ﬁrst compare various models for the separate tasks of ItemRec and UserRec to validate our proposals in Section 3.
On the ItemRec dimension, we observe the usual story that user-conditional models outperform non-personalized models, but they are then topped by sequence models.
However, the comparison becomes interesting in the UserRec direction, where the adoption of raw probability scores in RNN and Transformers could lead to inferior results than the simple heuristic that recommends top users unconditionally of the given item, i.e., UserPopularity or the more advanced
TPP models for the marginal prediction, λ ( x ) .
This is because the user-conditional probability scores in RNN/Transformer fail to identify the user-marginal intensity priors.
On the other hand, with our RIM proposals, the sequence models become top in the UserRec direction as well.
For the baseline BPR methods, we also compare our implementation with open-source LightFM package.
LightFM only allows User/Item sampling, but not both at the same time.
In contrast, our version samples both sides concurrently, allowing a single model to perform well in both tasks.
We also compare GCMC with GCMC*, a variant that includes multiple time periods to extract more labels and relations.
We include similar plots for the precision metrics in the appendix.
5. O FFLINE AND O NLINE M ATCHING E XPERIMENTS
[Figure]
To validate the discussions in Section 4, we integrate UserRec into ItemRec by setting minimalexposure constraints.
We ﬁx α = 1% in equation 10 and vary 0 ≤ β ≤ 1% as item min-exposure constraints.
(Section F covers a more common marketing scenario where the user-to-item capacity ratio is greater than one, which leads to even larger margins in our RIM methods.
Here, we focus on the case with tighter constraints to highlight the subtleties and new possibilities in our Mtch setting.)
Choosing β
=
corresponds to the unconstrained ItemRec problem.
As we increase β , we observe an increase in item-exposure diversity, measured by perplexity
[Equation]
(16) where π ( y ) is proportional to the total number of times that y is recommended in front of a user.
Figure 5 compares the performance of various methods as well as three Mtch strategies: Ofﬂinegreedy, Ofﬂine-CVX and Online-CVX.
Within each Mtch strategy, we see that all methods exhibit trade-offs between ItemRec relevance and item diversity.
Comparatively, Transformer-Pop/HP yields best performance and compromises least relevance to achieve better diversity, which supports our claim that better modeling of user intensities leads to better welfare for both users and items.
Notice that showing diversity effects in ofﬂine experiments is a challenging task due to the vast amount of missing data for alternative treatment effects.
We attribute our success to the consideration of new problem dimensions.
We show similar results in Section F on other datasets and settings.
We see similar results with different Mtch solvers in Figure 5(b)&(c).
Notably, the Online-CVX solver allows some users or user events to be unseen during training and the match-maker has to make real-time decisions for each user without buffering.
The consistent performance across different scenarios demonstrates great practicality in the integration of ItemRec and UserRec in RecSys.
C ONCLUSION
We study UserRec in an online RecSys paradigm, where each user is represented by their history of item consumption events, generalizing a variety of item-marketing scenarios.
This problem is of great values in keeping the diversity and liveliness of RecSys; by addressing the pain points in item marketing, the item creators can focus on their expertise in creation.
Our main challenge is to compare user sequences in variable lengths and duration.
We novelly connect this challenge to the intensity parameter estimation of a temporal point process.
With the insights, we develop a threepart RIM solution combining RNN (Transformer), TPP, and CVX, where all component adhere to the same user-mini-batch learning strategy, allowing RIM to achieve unlimited scalability in the size of the user set.
Our models evaluate well in both UserRec and ItemRec on three RecSys datasets that were previously used mainly for ItemRec.
RecSys is an area of broad social impacts.
The techniques we develop are aimed to improve the diversity and welfare of all participating parties, i.e., the users and items.
We use global optimality as a surrogate in Mtch reformulation of RecSys.
We show promising connections between RIM and better relevance-diversity trade-offs.
On the ﬂip side, bridging recommendation and marketing may give us better tools to understand some unintended consequences of modern RecSys.
A R ELATED W ORK AND D ISCUSSIONS
Non-sequential marketing strategies Marketing is a general area studied from multiple perspectives.
From a system’s perspective, our example of keyword and related-item bidding comes from close examination of large-scale extreme classiﬁer models (Chang et al., 2020).
The push notiﬁcation example considers explicit user states, which are commonly expressed via Recency, Frequency, and Monetary values (RFM)
(Fader et al., 2005).
By far, the most common and straightforward approach is to use Gradient Boosted Decision Trees to regress on the binary user-item engagement labels using these RFM features.
At a simpler level, without the manually constructed temporal features, the loss function is shared with traditional matrix factorization approaches, such as Alternating Least Squares (Hu et al., 2008), Bayesian Personalized Ranking (Rendle et al., 2009), and
Logistic Matrix Factorization (Johnson, 2014), all of which serve as UserRec baselines.
Neural temporal point processes We further receive inspirations from temporal point processes (Reinhart, 2018; Chen, 2016; Du et al., 2016; 2015),
Neural-ODEs (Chen et al., 2018), and their intersections (Jia & Benson, 2019; Chen et al., 2020; Mei & Eisner, 2017; Zuo et al., 2020).
Most of the works are great introduction of various aspects of TPPs and their modern development in GPU environments.
However, most of the works focus on the prediction for better temporal modeling without novel applications of the intensity parameters for different applications.
Along the line of better temporal models, (Jia & Benson, 2019) shows how Neural-ODE can be applied to a MTPP construction and (Boyd et al., 2020) focuses on the particular challenge that personalized user histories “exhibit large predictive uncertainty at the beginning of the sequences”.
We do observe that many users churn out shortly after initial engagement.
To model this temporal heterogeneity near the users’ beginning, we use a simple trick, where we add another related time series with exactly one event for every user to mark their start time.
Convex optimization and online regrets On the long-term constraints, we reference from
Sinkhorn’s algorithm from optimal transport (Cuturi, 2013; Genevay et al., 2018) and the literature in long-term marketing via online campaigns (Huang et al., 2016; Jenatton et al., 2016;
Ding et al., 2019), with convergence analysis dating back to (Mehta, 2013).
Online convergence analysis further leads to novel applications such as online classiﬁcation with speciﬁcity constraints (Bernstein et al., 2010).
Diversity and Exploration
In our discussion of the CVX solution for OnlnMtch, we mention a dual solution in the form:
[Equation]
This solution is intrinsically connected to the idea of bandit exploration with upper-conﬁdence bound (UCB) (Li et al., 2010; Chu et al., 2011), where our adoption of the the minimal-exposure constraint leads to a positive boost to the raw prediction scores, i.e., − v ( y ) ≥ 0 .
Further exploration in this direction may lead to additional beneﬁts in the OnlnMtch paradigm.
Causality In our work, we consider item exposure as the primary factor and assume positive downstream effects afterwards.
We could also analyze the temporal correlation between an intervention action and the change in hazard rate in a user’s temporal state.
While (Granger) causality is a useful consideration in general, e.g., to prevent sending bad emails with large negative effects, it appears more relevant when the marketed item is a discount coupon and the goal is to attract cold or inactive users.
Some interesting work on temporal causality include (Xu et al., 2016; Li et al., 2018; Jiang &
Li, 2016; Lundberg & Lee, 2017).
B GCMC M ODEL D ETAILS
The basic idea in GCMC-CLV is to split user behavioral data in time to create a temporal-prediction scenario.
We hold out a targeting window from the observed user behaviors as CLV labels and build RFM features for every user at the start of the window.
Here, every user is represented by the aggregation of the items found in the user’s own history, which reﬂects the frequency of interactions in these items.
Due to the sparsity of the user-item pairs in the observed user histories, we use a graph convolution layer for the aggregation, which also extracts the semantic item embeddings similarly to graph-convolutional matrix factorization (GCMC).
As for recency, we construct a single user bias term based on the recency of the last purchase regardless of the item choices.
We omit the peritem recency terms due to their high (inverse-)correlation with the per-item frequency features.
We neither have monetary information in our experiments.
The main challenge in the scalable CLV approach is the vast amount of implicit-negative labels.
While we observe positive CLVs in the target window, we should also include every unobserved user-item pair as a zero CLV label.
This leads to an imbalanced label distribution, i.e., a signiﬁcant amount of zeros which we call negatives, and a small portion of non-zeros which we treat as positive (binary) labels.
To address the imbalance, we adopt a strategy of negative sampling to augment every positive user-item pair with negative users (˜ u ) and items (˜ ı ) , followed by sigmoid-triplet loss:
[Equation]
Details of our negative sampling strategy is similar to Bayesian Pairwise-Ranking (BPR).
However, open-source BPR implementations are often limited to sampling only negative items, leaving negative users unconsidered and causing biases in the CLV predictions.
We thus include our implementation, with the additional beneﬁt of better integration with GCMC framework.
We sample the negative users (and items similarly) according to
[Equation]
where C ( u ) is the total number of appearances of the user (or item) in the training data.
Finally, the BPR scores are translation-invariant, because BPR only focuses on the relative comparisons instead of the absolute values.
This would be a problem when we port the scores for CVX, where we expect positive intensity scores.
We use softplus activation to preserve the larger values in the raw scores.
Our earlier attempt with sigmoid activation leads to saturation in the higher score values, which causes issues due to over-exploration.
C CVX A LGORITHM AND D ISCUSSIONS
Algorithm 1 CVX for ofﬂine or online matching
Require: λ ( x, y ) , α ( x ) , β ( y ) , batch-to-full ratio γ
Ensure: Dual variable { v ( y ) : ∀ y ∈ Y}
Init v 0 ( y ) , ∀ y randomly within sign-constraints for k in [0 , 1 , . .
, 100] do
Receive user-minibatch X ; compute { λ ( x, y ) :
∀ x ∈ X, y ∈ Y} ; set (cid:15) = 0 .
9 k
Find root ∂d (cid:15) ( u,v ) ∂u ( x ) = 0 for every x ∈ X given v k ( y ) ; project to u ( x ) ≥
Find root ∂d (cid:15) ( u,v ) ∂v ( y )
=
for every y ∈ Y given u ( x ) ; project if needed
Take step v k ( y ) =
(1 − γ ) v k − 1 ( y )
+ γv ( y ) , ∀ y end for
Compared with the original subgradient solutions, our (cid:15) -regularized approach yields similar or better computational efﬁciency.
Both approaches are trained by iterating through the user pool via minibatches just like a sequence model.
They also cost similar amounts of computation for each user,
O ( |Y| ) , with additional constants in log( K ) or b -bits of desired numerical precision, respectively.
The true beneﬁt of our proposed (cid:15) -method is in its alternating-direction minimization of v ( y ) based on mini-batches of users.
This automatically tunes for the optimal learning rate which we ﬁnd tricky to design in the practical application of subgradient methods.
On the limitation side, the alternating solution can take overly small steps if we directly input a small (cid:15) → 0 .
We mitigate this issue by annealing (cid:15)
[Table]
D D ATA S TATISTICS AND O FFLINE P RECISION M ETRICS
Figure/Table S1 shows our data splitting strategies and resulting statistics in the three datasets we consider in Section 5.
With temporal split, we carve out a prediction time window between ( T, T + ∆ T ] and a calibration/validation window between ( T (cid:48) , T ] for time-bucketed models like
Hawkes-Poisson and GCMC and the observation of user-state distribution for Online-CVX simulation.
With user split, we pick Group B user to construct a test window between ( T, T + ∆ T ] .
For calibration/validation, we use Group A users with the same temporal splits.
Our deﬁnition of user-state distribution allows sampling by either users or time or both users and time at the same time.
Figure S1 shows the ofﬂine precision benchmark results which resemble our Figure 4 on recall metrics in Section 5.1.
The trends are similar.
We show recall in the main text because it caters more to user-welfare, whereas precision connects to the global objective in Mtch experiments.
Conversely, the precision numbers agree with the unconstrained optimization results in Figure 5,S2,S and Figure S4,S5,S6 for different combinations of tasks and datasets, respectively.
E I TEM R EC WITH M IN -E XPOSURE C ONSTRAINTS (NF AND YC)
Similar to the Netﬂix results in the main text, we show two more Mtch experiments on NF (Figure S2) and YC (Figure S3).
Comparing the Pareto fronts, we observe RNN-TPP variants > RNN > MF > Non-personalized models, which is consistent with our results on ML dataset.
F U SER R EC WITH M AX -U SER -C APACITY C ONSTRAINTS
We consider 1% ≤ α ≤ 100% and β = 1% , both as upper-bound constraints in equation 10.
In the limit α = 100% , the problem becomes UserRec without the consideration of user capacities.
This yields best UserRec relevance, with a risk to overwhelm the users.
As α decreases, the users are expected to take a more balanced load of incoming items.
We use a similar perplexity metric to measure the user-load balances.
[Figure] [Figure]
Figure S4, S5, S6 show the trade-off between relevance and user diversity, which is an indirect result of capacity limitations per user.
From the curves, we ﬁnd larger advantages in the RNN-TPP variants. This is because the problem is more focused on the advertising perspective, giving larger user-to-item capacity ratio, which highlights our RIM contributions.
The rightmost points in the ﬁrst two plots agree with the unconstrained ofﬂine UserRec precision metrics in Figure S
[Figure]
For online UserRec, we observe that most methods, including all of RIM and GCMC methods follow similar trends as ofﬂine UserRec (Figure S4.c, S5.c, and S6.c).
However, we do notice some artifacts in MF methods on ML and YC datasets.
The artifacts are related to under-serving of recommendations with respect to maximal user capacity constraints, which leads to drops in the precision metrics.
For more details, we show the total number of recommendations relative to the β -constraints in Figure S4.d, S5.d, and S6.d. Notice that constraint dissatisfaction is a possibility in online ItemRec as well, but they do not affect our general observation of improvements in diversityrelevance trade-offs and we thus omit the discussions for simplicity. Our discussions here extend to online ItemRec as well. What is special in online UserRec is that the total number of recommendations is highly related to the satisfaction of the β -constraints.
For example, in the extreme case where α = 100% and β = 1% , the user capacities are entirely unconstrained, yet the total number of item exposures have to be controlled.
The other settings where 1% ≤ α ≤ 100% respect the capacity of each individual user, but still over-provision the overall user capacities for the desired number of item exposures.
In these cases, the β -constraints are observed by transferring the dual parameters, v ( y ) , from the calibration set to the test set through either temporal split (Figure S1.a) or user split (Figure S1.b).
I.e., Online-CVX does not use any user statistics in the test set, which are unavailable ahead of time in true online settings.
Despite so, the β -constraints are largely satisﬁed with RIM and GCMC methods in all datasets as well as MF methods on NF dataset with temporal splits (Figure S4.d, S5.d, and S6.d).
[Figure]
[Figure]
The artifacts are only limited to MF methods on ML and YC datasets, both of which use user-splits.
Upon further inspection
[Figure]
we realize that the artifacts are due to the fact that we reuse a part of training data for calibration purposes,
i [Figure]
e., the ( T, T + ∆ T ] time window for Group-A users in Figure S1.b.
This causes Group-A users to have more training data, leading to higher recommendation scores during the learning of v ( y ) .
As a result, v ( y ) tends to be higher, causing over-penalization when they are transferred to Group-B users, who have less training data and generally lower recommendation scores.
This eventually translates to overall lower item-exposure rates compared with what is calibrated on the validation set, shown in Figure S4.d and Figure S6.d. Notice that these artifacts are much less obvious on NF datasets, where the total training-data time (6 months) overwhelms the difference (15 days) between these two splits. They are neither signiﬁcant in RIM / GCMC methods, where the user training data is averaged against the user time span, improving the stability of the user-state distribution. In fact, the artifacts with MF methods reveal a side beneﬁt of our RIM (and
GCMC) methods in our novel design of online experiments.
[Figure]