Bridging Recommendation and Marketing via Recurrent Intensity Modeling
Authors: Yifei Ma
Ge Liu
Anoop Deoras
Venue: ICLR 2022 Poster (2022)
Keywords: Recommender systems
marketing
push notifications
temporal point processes
sequence models

Abstract: This paper studies some under-explored connections between personalized recommendation and marketing systems. Obviously, these two systems are different, in two main ways. Firstly, personalized item-recommendation (ItemRec) is user-centric, whereas marketing recommends the best user-state segments (UserRec) on behalf of its item providers. (We treat different temporal states of the same user as separate marketing opportunities.) To overcome this difference, we realize a novel connection to Marked-Temporal Point Processes (MTPPs), where we view both problems as different projections from a unified temporal intensity model for all user-item pairs. Correspondingly, we derive Recurrent Intensity Models (RIMs) to extend from recurrent ItemRec models with minimal changes. The second difference between recommendation and marketing is in the temporal domains where they operate. While recommendation demands immediate responses in real-time, marketing campaigns are often long-term, setting goals to cover a given percentage of all opportunities for a given item in a given period of time. We formulate both considerations into a constrained optimization problem we call online match (OnlnMtch) and derive a solution we call Dual algorithm. Simply put, Dual modifies the real-time ItemRec scores such that the marketing constraints can be met with least compromises in user-centric utilities. Finally, our connections between recommendation and marketing may lead to novel applications. We run experiments where we use marketing as an alternative to cold-start item exploration, by setting a minimal-exposure constraint for every item in the audience base. Our experiments are available at \url{https://github.com/awslabs/recurrent-intensity-model-experiments}

Published as a conference paper at ICLR 2022
BRIDGING RECOMMENDATION AND MARKETING VIA RECURRENT INTENSITY MODELING
Yifei Ma, Ge Liu & Anoop Deoras
AWS Al Labs
{yi feim, gliua, adeoras @amazon.com
ABSTRACT
This paper studies some under-explored connections between personalized rec- ommendation and marketing systems. Obviously, these two systems are different, in two main ways. Firstly, personalized rem-recommendattoo (ItemRec) is user- centric, whereas marketing recommends the best user-state segments (UserRec) behalf of its item providers. (We treat different temporal states of the same user as separate marketing opportunities.) To overcome this difference, we re- Marked-Tempont Point Processes (MTPPs), where we view both problems as different projections from unified temporal inten- sity model for all user-item pairs. Correspondingly, derive Recurrent Intensity Models (RIMs) to extend from recurrent ItemRec models with minimal changes. The second difference between recommendation and marketing is in the tempo- ral domains where they operate. While recommendation demands immediate re- sponses in real-time, marketing campaigns are often long-term, setting goals cover a given percentage of all opportunities for given item in given period of time. We formulate both considerations into constrained optimization problem we call online match (OninMtch) and derive solution we call Dual algorithm. Simply put, Dual modifies the real-time ItemRec scores such that the marketing constraints can be met with least compromises in user-centric utilities. Finally, our connections between recommendation and marketing may lead to novel ap- plications. We experiments where start item exploration, by setting a minimal-exposure constraint for every item in the audience base. Our experiments are available at /etttp:://ithtbee com/ aws aabs abbs/recurrent intensiii model -exper ieentriment.s
alize a novel connection
marketing alternative to cold-
INTRODUCTION
Many ML applications today involve decision making based on streams of events. recommender systems (RecSys), reasoning the stream of events in user's past history has allowed real-time user-based item recommendations (ItemRec), leading to significant impacts (Hidasi et al.) 2016
Hidasi & Karatzoglou 2018 Ma et al. 2020 Chen et al. 2019 Kang & McAuley 2018). However, being user-centric, ItemRec has limitations in creating globally inclusive environment for the item providers. For example, ItemRec tends to over-expose few popular items to improve the immediate user satisfaction, yet this causes the selected items to get even more popular, discouraging the other providers from creating seminal items with higher potential values in the future. This an important limitation because, as RecSys involves the participation of both users and item providers, we must consider the success of the item providers for the long-term richness of the RecSys environment (Singh & Joachims, 2018 Mladenov et al. 2020 Su et al. 2021).
To cultivate the seed users for the new items, marketing via sponsored advertisements has been a practical strategy. Note that we do not dismiss the possibility of uncontrolled marketing which hurts the utility of the users, but we argue that a minimal amount of controlled marketing in a trusted envi- ronment may actually yield better trade-off's between recommendation relevance and item-diversity. For example, we may assign each item provider an exploration budget so that their items get exposed to minimal percentage of users in target period of time. (We treat two temporal states of the same user as two user(-state) with separate marketing opportunities.) We further integrate marketing into recommender systems to create online matching (OninMtch) environment. Notice that Onin-

---

Published as a conference paper at ICLR 2022
Observed
Predicted
Scorejuser, (item)
User
Liathhood
Scarefoner (item))
User
I At
Time
ANN
Scareforer (item))
(1) M(z, v)
lim
Ar
2.9
Item 1
(a) Definition of the intensity model A(z, v) between User-State and Item y.
Scamfunerr/iemm litem)
User
Figure I: UserRec for an item. Users are considered positive if they indeed interact with the given item in a future time window. A matching hit requires both relevance and activeness of the user. An active user may have multiple hits with different items, whereas an inactive user may have zero. In addition, the user hits in given item are often correlated with their past histories with similar items.
Mtch implies systemic plan for all user-item recommendations based on the explicit exploration constraints. This is different from implicit uncertainty-drive exploration such as contextual bandits (Li et al. 2010) and Thompson sampling (Agrawal & Goyal 2013 Kawale et al. 2015). Specifi- cally, the effects of OnInMtch can be tested immediately using offline datasets, shown in Section whereas the benchmarks for the bandit algorithms often require online updates or their simulations.
To make it happen, we contribute two algorithms: a RIM solution to the UserRec marketing problem on behalf of the item providers and Dual algorithm for the OnInMtch problem with its planning challenge. Both are realistic problems with their own applications, which we describe now.
We start with marketing predictions, where we aim predict the relevance of each user with respect to the given item. We validate the quality of our prediction by comparing our top-user recommenda- tions (UserRec) with the set of users who truly interacted with the given item in the holdout back-test. period. See Figure [ for illustration. Besides direct ranking outputs, the raw prediction scores are also used in the OnInMtch step.
UserRec can be seen as an extension from targeted advertising, where the advertisers bid on search queries or related items to increase the exposure of their chosen items (Chang et al., 2020). In our work, a search query or related item may be seen as a single event often the last event in a user's history. By extending to the full histories of the actual users, we obviously build better user features, but also create novel problem in the space of information retrieval, which is to compare the users as random time series instead of random variables. For example, the time series data allow us to compare between relevant but less active user and an active but less relevant user. This would not be feasible with probability density estimation based on the users' hidden-state variables learned for next-item recommendation. See "Bayesian paradox" in Figure for more details.
Our key observation here is in the implicit temporal sensitivity in marketing. The true goal for marketing is to find the users who are most likely to interact with the marketed item in the next period of time. This objective can be formally defined by the intensity parameter in temporal point processes (TPPs) (Reinhart 2018 Rizoiu al., 2017). See Figure Ta for more details. We further use a Marked- TPPP decomposition, similar to Bayes" rule, to reuse R RNN/Transformer models that are common in recommender systems (Cho et al. 2014 Hidasi & Karatzogiou 2018 Ma et al. 2020) Wu et al. 2017 Bai et al. (2019). Correspondingly. we call our model Recurrent Intensity Models (RIMs). Note that while TPPs have been widely discussed in the space of user-churn prediction, item trends prediction, and physical event modeling (Jia & Benson 2019
Du et al. 2016 Leskovec
al. 2009 Chen al. 2020; (Hosseini et al. 2017] Wang al.] 2019] Li al. 2021). we believe
that TPPs are relatively novel in the UserRec domain, which opens new venues for their applications.
For completeness, while we focus on user-affinity modeling without interventions, our approach can be extended to consider (Granger) causality effects of marketing interventions. In Appendix A we will discuss another example of UserRec where we aim to send email reminders to users who leave their online-shopping carts abandoned for period of time (Lowe & Craig 2021). Different from affinity-base based UserRec, the focus there is on the temporal correlations between two event types: the reminder event and the checkout event. Our RIM framework extend to this case, but omit the discussion for now, because similar intervention signals may not be easily available in the most general cases. Our current discussion is analogous implicit-feedback recommendation.
2

---

Published as a conference paper at ICLR 2022
Table 1: Three marketing strategies where recurrent intensity models may apply
UserRee
Offline Match
Online Match
Pick top users for one marketed item in one go Balanced promotion of multiple items the same time, via constrained optimization over observed user-states Real-time marketing integrated with recommendation, via Dual algorithm with real-time score modifications
Now that we introduced RIMs for marketing predictions, we continue to discuss some practical chal- lenges in marketing planning. Naively, we may simply send push messages to the top UserRee lists to fulfil the marketing campaigns in one go. However, marketing campaigns are often set over long period of time, which allows more freedom for less intrusive strategies. Table summarizes these new strategies. Particularly,
OnInMtch to achieve the exploration goals in experiments.
To solve OnInMtch, we develop a Dual planner based on the empirical user-state distribution from the past. Our main challenge to decouple the one whom need to answer immediately from all users whom the marketing campaign targets. We achieve this by finding globally-calibrate dual variables to modify the original recommendation scores, based the theories of optimization
duality. Our Dual algorithm is conceptually similar to (Huang et al. 2016 Mchta 2013 Ding et al.) 2019) but we use different approach via entropic regularization similar Sinkhorn's algorithm (Cuturi 2013). We found our variant to be easier to scale and converge in our experiments.
The rest of the paper will show details of our RIM model, Dual planner, and experiments. Additional discussions on related work can be found in the appendix.
2 RECURRENT INTENSITY MODELING (RIM)
We are interested in modeling a temporal process represented by series of event times and types of given user. Let S [[t,, 9;) Vt, t2 <.... < <t, < ..., Vy, Y] to be a random process that generates user behaviors with a continuous time variable f and a type variable (i.e., the consumed item in the event) in a large-cardingg cardinal set y E V. We consider a user-time as a unique instance variable
X = S(t {(ty. :t < t}, with observable user history until time f. We use lower-case I = s() to indicate its realization and x E X to be the set of all possible user-time instances in the recommendation problem. In the ItemRec construct, the sequence models are called upon the arrival of a user event to predict the item choice by a conditional categorical distribution:
X(x.y) P(Y y|æ) Eyey Ala.y") where M(x, y) exp (u,, M(x) + by + m(x)),
(1)
where la(-) is a sequence encoder function, wy is a row in the decoder weight, which is also inter- preted as the hidden embedding of specific item, by is global bias in the item choices, and m(x) represents any extra terms that cannot be learned because they cancel between the nominator and the denominator. The m(x) term reminds that the raw ItemRec scores A(x, y) cannot be used to compare across users. We aim to repurpose X(z, y) for UserRec by associating m(x) with user prior term that reflects their activeness in the RecSys, an important factor UserRec.
Timeless Bayes rule versus temporal intensity Given goal to select(xly), our first thought is to frame it by Bayes rule p(rly) x p(y]=)p(z), i..., relating mxxr) to log p(z) and other normaliza- tion terms. However, this leads to paradox shown in Figure 2 The paradox is because we represent a user as a timeless variable x, instead of a temporal point process (TPP) through the lens of st)).
TPP predicts when the next event will happen for user in addition what type it is. TPP is different from other temporal models in that it predicts the event time as moving target, whereas the other models aggregate time as feature. The temporal concept best explained from counting perspective. Here, we model the absolute number of user-item interactions, AN y), in a future time window, [t. + At), where s(t) coincides with z. We adopt Poisson assumption that the rate at which the desired events happen is smooth in time, given by an intensity parameter:
X(s(t), v) ((z, y) lim AN
x. y]
E T&N
xx, v]
(2)
At-+
A
dr
3

---

Published as a conference paper at ICLR 2022
Truncated user history in last week (x]
p/= = [4, BJ) =02
p/= = [4, Blly = D) x 0.2 0.4 0.08
P(y = P(r)
(Bayesian)
Mx - C) 0.2
p(r Cly D) x 0.2 0.8
p(x 0) 0.6
p(r - Øly D) 0.6
0.3 0.18
A. B
0.8
((x== [A, B)) =2
A(2 [A, BJyy D) x
(MTPP)
Mir - C) A(=
Mr - Cly D) x 0.8 0.8
x 0ly D) x x 0.3 0.3g
Figure 2: Bayesian paradox. We start with an ItemRec model p(y(#) that assigns nonzero scores all users, including cold or inactive users, who are abundant in RecSys due power-law effects. The Bayes rule incorrectly associates the concentration in quantity with the engagement prior of each user, leading to non-intuitive predictions. Instead, our RIM proposal treats each user as draw from MTPP and we use the intensity parameter (defined in Figure Ta to predict their future engagements, p(> 0) a background intensity assigned all processes, including inactive users.
The definition of the intensity parameter is visualized in Figure Besides matching the expec- tations, we may further preseribe AN y) ~ Poisson (A(z, y) At) as a parametric distribution, especially in short time windows. The Poisson log- likelihod is:
log P ((AN(a,)) y) = n X(z, At) = n log(A(x, y) - A(x, y) log(n!).
(3)
The intensity parameter connects three variables of our interests: user sequence item y and our prediction target A (x, y) in a future time window [t, + At). Slicing AN (zz, y) in different directions yields different aspects of the problem. Most importantly, we slice by At to connect it to the distribution of future event times. We expect an active user to have higher intensities in unit time or, equivalently, shorter intervals between events.
Temporal Point Processes (TPPs) We focus on the connection between the intensity parameter Xax, y) = >(s(t), y) and the time t., for the next interaction with item y. Intuitively, split the time interval until the next event into a large set of infinitesimal windows, i.e., (t,tt) = UT[T, + dt), and insert Poisson likelihood with label n(T) to all previous windows and n(T.) to the last window. Based on (3). we have
Σ Eisret. LITTEL log ((dN(r) = maT) | ((((T).)d)) = log (A((()()))) - Eisres, A(s(T),)ddr
dr
logp(T. = t, t: t(A = log log(A(((tt)..u)
J.¹ x(s(T),8)dr dr.
(4)
Note that the dl inside the logarithmic term disappears when we convert the probability mass func- tion to probability density function with continuous variable T.. Reinhart (2018); Chen (2016)
discuss rigorous derivations well as additional properties.
Equation 4 is known as the inter-arrival time distribution, which directly formulates a loss function to learn the intensity parameters, without the need to explicitly bucketize time in (2). Equation also implies that the intensity function can be time-varying based on relative history s(t). The numerical integration may be solved by importance sampling (Mei & Eisner 2017). ODE solvers (Chen al. 2018). or analytical forms in the case Hawkes processes, which we use this paper for simplicity.
Marked-TPP Decomposition While TPPs can be built for each event type (i.c., item choice) with cross-channel influences from other event types, this may lead to drastically different models than existing sequential ItemRec models. Instead, we utilize Marked TPP decomposition,
(5) where p(y|z) comes from regular sequence model { and X(z) corresponds the user-glu uler-global intensity for all item types. MTPP is the framework of our RIM proposals. It resembles Bayes rule and also considers the user sequence st) as point process.
(œ,y)= = {x))p(y(z).
3 ESTIMATION OF THE INTENSITY PARAMETERS
While RIM is motivated to estimate the intensity of each user-item pair separately, we find it more convenient utilize existing sequential ItemRec models. Based on (5). we keep the RNN (or Trans-
4

---

Published as a conference paper at ICLR 2022
former) model for its categorical prediction of the preference direction, p(y|z), while introducing separate estimators for the user-intensity prior, X(x).
RNN-Pop is our simplest model, where use the length of the user histories in the training sets. This approach works due to a homogeneous intensity assumption, where the number of past events directly correlates with the future event intensity. It also assumes that all user activities over a comparable period of time, which often holds in practice, because RecSys often truncates the training sets by fixed time for scalability reasons. RNN-Pop is used in the example in Figure 2
recorded
RNN-Hawkes To extend the naive RNN-Pop models, we may break the homogeneity assumption to consider user state changes that affect future event intensity. In this regard, Hawkes process assumes a positive stimulation through past user events and a gradual churn-out effect after period of inactivity. models these effects via an exponentially-decayin kernel in the intensity parameters
x(s(1) 11
Σ Σ j.yytr=
1. exp ar (1 - 13), where or (A/)) 3r
(- Ar 9. 1 Tarso, and 14, ar 0.
Here, we extend it with a mixture of R latent kernels with learned coefficients and learn it with tick software package package Bacry et al. (2017). We design the kernels to have log-spaced half-lives between 10-3 and 10 where (max is the largest temporal span in user histories. A large-s, kernel approximates the RNN-Pop model with a step function after each observation, whereas a small-s, kernel suggests fast-diminishing effect, often within a short browsing session.
RNN-Hawkes Moe--aa (RNN-HP) For long prediction horizons, we may also calibrate Hawkes scores to eliminate the effects of fast-diminishing kernels. In fact, the accurate long morizor model,
((r, v) E(AN v)] E [ At X(S(T), v) dr( m)
takes the expectation with respect to stochastic integrals. To calibrate for the long-terms intensity scores, we notice that the Hawkes states étt) are already positive. We thus model the final scores as
A(x) = (t) O softplus(w) = <(t) @ (og(11 + exp(w)).
Our additive formulation is inspired by (Mei & Eisner 2017) and modified by allocating the softplus each coordinate to improve numerical stability. Another approach is to replace (6) with over- dispersed distributions like negative binomial (Salinas et al. 2020), which we leave as future work.
Other Baseline Methods Practical personalized marketing models are often highly customized, e.g., with profitability or seasonality concerns, and they are not always designed to cover all items in the catalog, which requires training labels for all user-item pairs, including user-item pairs with zero activities in a time period. Despite so, we illustrate the general idea to use explicit recency, frequency, monetization (RFM) features for user segmentation (Fader et al.) (0005). Here, frequency and monetary spending in related items are correlated with customer's 's future life-time value (CLV) with respect to the given item and so is the time elapsed since last purchase with an inverse relation.
We build a model based on graph-convolutions feature aggregation similar to GCMC (Berg et al., 2017), followed by Bayesian-personallze ranking loss (BPR) (Rendle et al. 2009) against sampled negative users and items in the large space of all user-item pairs. Notice that this GCMC model uses labels from an explicit time period, which inferior RIM because (a) it has fewer usable labels and (b) it bucketizes the labels instead of preserving their continuous time and order. We attempt a mitigation we call GCMC-Ex that creates multiple periods to extract more labels and order information. However, we do notice that perfecting this remedy leads to a temporal integration similar to our TPP derivation (4. More details of this model can be found in Scction B
We also consider matrix factorization (MF) methods, such ALS (Hu et al. 2008) and Logis- ticMF (Johnson 2014) from Implicit package" and BPR (Rendle et al., 2009) from LightFM pack-
age (Kula 2015). Notably, LightFM-BPP is limited to sample either users items, but not both. We include our own BPR model for completeness. Unlike RIM and GCMC, MF methods do not consider temporal features.
ht http:///ggthhbb.cc/m//eeee////// islt
5

---

Published as a conference paper at ICLR 2022
Algorithm Dual planner for online (and offline) matching
Require: Azy A(z. y). ideally scaled to À I user-capacity K//11; item-constraint 33;
user-state distribution P(t) from a past period of time; step-size 7 Ensure: by realtimme modified recommendation for any user-state by TopK(Aay By Uy: E
init iy Unif(-1,0)) Vy E E
for [0, 1, ,.........1. 100] do
set = 0.8* sample batch of user-states X' ~ P(X); compute (Auy) Vx E EX', y y}
find root r= (fte(()::: uzz, i)) = 0 for every x X' given é; project to 0
find root ( (f (Aey. ui (ly)) 0 for every y E y given u'; project to 2 I 0
take step Ey = 2)0y + my, Vy E y
4 ONLINE MATCHING AND DUAL PLANNING
So far, we have learned RIM models as extensions from ENN/Tranforrmers using MTPP intuitions. They often yield better accuracy than the tine-bucketized RFM baselines and timeless MF models. In this section, we continue to focus on their greater impacts when we consider different marketing scenarios in Table ! Our main goal is to derive the marketing dual variables, Eg. which directly integrate with the recommendation score outputs on the "Ensure" Line of Algorithm
OnInMich as Constrained Optimization We start by reformulating offline UserRec by con- strained optimization. Say we want to send marketing emails to Top-C users at a given time to promote Item y. We represent the space of all user statestates at the given time as X' x and rewrite Awy X(I, u) for convenience. We express the top-user recommendation as binary assignments Tzy {0, 1,, Ve E X. easy and important to verify that Tay can equivalently solved by:
ExeX s.t. Enex Tzy S C, assuming Xzy 2 0.
max (0,1),)
For online marketing matching, extend the user-state A' draw from a distribution P(2'), which unions all user-states in period of time by sampling every user at regular time intervals or based on their actual times of interactions, whichever is appropriate for the application. Each sample point is counted as an independent user-state instance. We also replace the summations over X' in with expectations over the probability space P(X). We then solve for the OninMtch assignments Tzy E 1), Vx E X, Vy E y with a linearly-relaxxe optimization problem: max Euy (Ary s.I. 0S#51
By (Tay) S a, Vr E X
- cspr-capacity constraint with dual variable uy; item minimal-exposure constraint with dual variable Ugi
Ez (TTy) 2 Vy y
(10)
Ex (Tzy)= S Vy
item maximal-exposure constraint that rewrites (8)
where we express the thresholds in relative terms, i.e., K/111 for Top-K ItemRec given user- state x and 3' =C//x| for Top-C UserRec on behalf of an item y. Unique to OnInMtch, we can also insert minimal-exposure constraint for each item with threshold B, Vy y. Without loss of generality, we keep /3 and omit 3 to keep the discussions simple.
For the OnInMtch problem, our challenge that the item constraints are measured over a distribu- tion of users in a period of time, while OnInMtch requires real-time recommendation / marketing decisions. This challenge can be met with optimization duality, which we describe next.
Dual algorithm We take ideas from optimization duality to rewrite the constraints in a Lagrangian form. Le., equation 9 (leaving out the omitted 3 term) is equivalent to
max min Le [TT u, v) Eny ( ) E., (u,T2 (T)) Ey (U.Tg( (m)) + eLEay (Tay))
w20,050
(TT))) Eyy(Tyy)
residual in the the user-capacity constraint;
(T( (TTy) - B - residual the item minimal-exposure constraint;
where
H(T) = - -ogg( (T) -(1- π) log(1 π)
- entropic regularization for 0 S S 1 with annealed > 0.

---

Published as a conference paper at ICLR 2022
Next, we apply the standard trick to define dual objective by switching the order of operations:
min w20,050
(u, u)
max Le u,
(11)
After some derivations in Lemma
ST
Section C.I we find analytical solution the inner-loop:
Tay = sigmoid
Auy
define as fhe (Ayyy Ugs Un).
(12)
The solution intuitively suggests that the assignment of a user-item pair not only depends on their predicted intensity score, but also on the global dual variables uh and Vy. For example, if Azn > Azya for the majority of users while we intend to keep the global exposure rates for 1/2 and to be equal, then we may choose Byy uy2, causing the final aeem-ssignmmens be reverted for some users to achieve the desired exposure outcomes. Notice that the item- changes (-2,) are nonnegative, which agrees with our intuition to boost the item exposure rates at least B.
Finally, we discuss the solution to the outer-loop, which also yields an intuitive algorithm. To do so, we plug (12) to [T]. The derivations in Lemma S2 in Section C.T lead us a clean result:
Od.(u, v) = -r (#te (ifte(ass Uz, e)); Bry
Ode(u, v)
= -ry (ft (Rey) u, Uyy))
(13)
Ou,
Setting both values to zero yields an iterative algorithm that alternates between solving and v. We use simulated annealing from a larger initial c > to produce more stable results. See Algorithm for more details. In the extreme case where c - O, the two root- root-finding steps degenerate as
b, y E y}) for every x E X;; project to u2 2 Find Bly = Quantile( - Ug x E x. (3) for every E ); project to by I 0
Find u, TopKThrrshold((1>r
(14)
bisection search to solve for in each step, which is possible because the gradient functions are monotonic with respect to the variables uy and Vy. respectively. The com- putational complexity is similar to finding quantiles by full sorts See Appendix for further discussions on the computational and storage complexity.
For larger
Connections to Online Bidding and Pacing. Real-time bidding is a common practice in the on- line advertising industry. The bidding price from advertiser depends the expected reward due to the placement, the estimated competitor-pricin the campaign budget over a period of time. In our Dual algorithm, we may see flavors of all the components: the dual variable fy affects the overall ranking of the item in each marketing oppor- tunity, receives influences from all other items in each iteration of the alternating algorithm (14), and is calibrated against the empirical user-state distribution to achieve smooth pacing over time. Additionally, Dual planner may extend to future time periods we assume stationary user-state distribution, which we empirically observe and discuss further in Section F
distribution, as well as its pacing plan to manage
Dual planner has direct applications guaranteed display advertising, where advertisers buy con- tracts to show their ads to guaranteed size of audience in period of time (Bharadwaj et al., 2010). Further, Huang et al. (2016); Mchta (2013): Ding et al. (2019) analyzed the online regrets in such guarantees with slightly different algorithms. These algorithms are designed to minimize online-regret in adversarial settings. Comparatively, our work under stochastic and stationary as- sumptions. but we may consider adjustments after short periods of time for more practicality. Álso, we found some difficulties in the convergence of their algorithms, potentially due to the skewed intensity- distributions in experiments. We leave the detailed comparisons for future work.
5 EXPERIMENTS
We conduct experiments in three RecSys datasets with unique properties.
Movielens (ML) Harper & Konstan 2015) is movie rating website that collects user preferences to study RecSys. We use a small-scale ML- 1M an 2003 release with one million rating events between six thousand users and three thousand items easily validate our ideas.
7

---

Published as a conference paper at ICLR 2022
dataset MovieLens
detaset Netilix
dataset YC
intennily modeling
40.4
UsorRaz Recalli@tont 10
LMF
25.3
30
BPR
30
hanvaa
12.3
GC
itan
3.4
8PR
Gdl
ALS
GC
6AR,
Nome 58M basei
Turm
ROLD Random
8PA
BPR
6PR
PND
ALS
pepu
ALS ALS
GOMC_cn
Logaticee
ItemRec Recall@@top1(xx100
Itemiile Recall@ (eccll@@top11xxx100))
90000000
ItomRcc Rerall@top111/x00
8P% BPR
Figure 3: Suitability of different methods for ItemRec and UserRee tasks, plotted on different axes.
is a movie rental service where movie ratings from a user's past affects their per- sonalized RtemRec in the future. NF is widely used in rating prediction and ItemRec, but less so in UserRec and OnInMtch.
Netflix ( 2
Yoochoose (YC is a dataset used in ReeSys 2015 Challenge with six months of activities for product recommendation such as general tools, toys cloths, electronics, and much more.
We highlight some data preparation steps. Most of our RIM methods are self supervised sequence models. Thus, we only need to hold out test windows with equal horizons for the evaluation of UserRec, ItemRec, and Mtch. We use absolute time windows NF and consider only the observed users/items in the training data as testing targets, because we may not know which new users/items will show up in the testing windows without temporal leakage from the test set. Besides temporal splits, it is common for sequence models to split train/test sets by users. We consider user-splits on ML and YC, because most users are active different, non-overlapping clock times. We hold out time windows only on the test users (Group-B -B in Table with equal starting time relative to their first event as well as equal size or horizon. All training users (Group A) and the observed histories of the testing users (Group B left part) are considered training data.
RNN-HP and GCMC(") require further splitting of the training set. On NE, we create a set-back window between [T', T) from all users and on ML/YC, we keep the same time [T,T + AT) but change the user base to Group A for validation. Similarly, to simulate OnInMtch, we must calibrate Online-Dual Dual on different users or times to test the transferability of the user-state distributions in online scenarios. We reuse the set-back window as RNN-HP and GCMC(*) for simplicity. Statistics of our data splits are shown in Table ST
5.1 USERREC AND ITEMREC RESULTS
We first compare various models for the separate tasks of ItemRec and UserRec to validate our pro- posals in Section 3 On the ItemRec dimension, we observe the usual story that user-conditional models outperform pon-personalized models, but they are then topped by sequence models. How- ever, the comparison becomes interesting in the UserRec direction, where the adoption of raw prob- ability scores in RNN and Transformers could lead to inferior results than the simple heuristic that recommends top users unconditionally of the given item, i.e., UserPopularity or the more advanced TPP models for the marginal prediction, \(x)). This is because the user-conditional probability scores in RNN/Transformer fail to identify the active users with higher intensity priors. On the other hand, with our RIM proposals, the sequence models become top in the UserRec direction as well.
For the baseline BPR methods, we also compare our implementation with open-source LightFM package. LightFM only allows Usen/Item sampling, but not both the same time. In contrast, our version samples both sides concurrently, allowing a single model to perform well in both tasks. We also compare GCMC with GCMC*, a variant that includes multiple time periods to extract more labels and relations. We include similar plots for the precision metrics in the appendix.
https:/ ///ee// kengle com www.oo//hhhhheeee wwwps//hhhssttooo syyocchosee
8

---

Published as a conference paper at ICLR 2022
Rand
.D- HP
Transtorme Transfommer-opp
Transforer--H diversity baseline
Pop
ALS
GC
Item diversity (perplexity)
20*
102
0.01
0.02
0.01
0.02
0.01
0.02
Offline Greedy
Offline Dual
Online Dual
RemRec Prec@366
Figure 4: Matching experiments where set minimal-exposure constraints for every item in addi- tional to regular ItemRec settings. Showing a subset of representative methods on MI. dataset.
5.2 OFFLINE AND ONLINE MATCHING EXPERIMENTS
To validate the discussions in Section 4 we integrate UserRec into ItemRec by setting minimal- exposure constraints, We fix a = 1% in equation 9 and vary S B S 1% as item min-exposure constraints. (Section F covers ratio is greater than one, which leads to even larger margins in our RIM methods. Here, we focus on the case with tighter constraints to highlight the subtleties and new possibilities in our Mtch setting.) Choosing 3 = 0 corresponds to the unconstrained ItemRec problem. As we increase 3, we observe an increase in item- exposure diversity, measured by
common marketing scenario where the user-to-item capacity
perplexity exp (-E, y
log
(15)
where (yy) is proportional to the total number of times that y recommended front of a user.
Figure 4 compares the performance of various methods in three matching environments: Offline- greedy. Offline-Dual and Online-Dual. Within each environment, we see that all methods ex- hibit trade-offs between ItemRec relevance and item diversity. Comparatively, Transformer-Poplee yields best performance and compromises least relevance to achieve better diversity, which supports claim that better modeling of user intensities leads to better welfare for both users and items. No- tice that showing diversity effects in offline experiments is challenging task due to the vast amount of missing data for alternative treatment effects. We attribute our success to the consideration of new problem dimensions. We show similar results in Section on other datasets and settings.
We similar results across different environments in Figure 4 (())). Notably, the Online-Dual solver supports both real-time decisions and long-term exposure constraints over the distribution of all user-states in future period of time. The consistent performance across different scenarios demonstrates great practicality in the integration of ItemRec and UserRec in RecSys.
CONCLUSION
RecSys that purely focuses on the immediate user satisfaction may hurt the overall diversity of the items being recommended. Instead, we study an exploratory marketing mechanism where each item will be guaranteed minimal exposure rate a suitable audience in a period of time. In this way, we address the pain points in marketing and allow the item creators to focus their expertise on creation. We believe this improves the long-term diversity and liveliness of the RecSys.
We make two contributions. Firstly, we turn a sequential ItemRec model around to recommend users on behalf of given item. Our challenge to treat the users as time series instead of random variables. We introduce novel connections to the intensity parameter of a temporal point process and develop a RIM solution that combines RNN (Transformer) and TPP. Our second contribution is to allow RecSys to guarantee minimal (maximal) exposure rate for every item that we intend to nourish (downsize) in a future time period. We call this scenario OnInMtch and introduce Dual planner solution. All of our methods learn by user mini-batches to achieve unlimited scalability in the size of the user-state set. Also, compared with other exploration methods that require online feedback, our methods have unique advantages in the ease of benchmarks using offline datasets.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
We thank Vaibhav Sethi and Balakrishnan (Murali) Narayanaswamy for some early discussions on the idea to repurpose ItemRec models for UserRec applications. We thank Danielle Robinson and Yuyang (Bernie) Wang for their discussions on the topic of Neural-ODEs, which inspired us to look further into the literature around temporal point processes. We appreciate the discussions with Lihong Li and Hsiang-Fu Yu on the general topic of targeted advertising. We finally appreciate the anonymous reviewers for this and our earlier versions for their constructive comments and additional references.
REFERENCES
Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning. PMLR, 2013.
E. Bacry, M. Bompaire, S. Gaiffas, and S. Poulsen. tick: Python library for statistical learning, with a particular emphasis on time-dependent modeling. ArXiv e-prints, July 2017.
Ting Bai, Lixin Zou, Wayne Xin Zhao, Pan Du, Weidong Liu, Jian- Yun Nie, and Ji-Rong Wen. Ctrec: long-short demands evolution model for continuous-time recommendation. In Pro- ceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR' 19, 2019.
Rianne van den Berg, Thomas N Kipf, and Max Welling. Graph convolutional matrix completion. arXiv preprint arXiv: 701..7066.02222 2017.
Andrey Bernstein, Shie Mannor, and Nahum Shimkin. Online classification with specificity con- straints. Advances in Neural Information Processing Systems, 23:190-198, 2010.
Vijay Bharadwaj. Wenjing Ma, Michael Schwarz, Jayavel Shanmugasundaram, Erik Vee, Jack Xie, and Jian Yang. Pricing guaranteed contracts in online display advertising. In Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM '10, 2010.
Alex Boyd, Robert Bamler, Stephan Mandt, and Padhraic Smyth. User-dependent neural sequence models for continuous-time event data. Advances Neural Information Processing Systems, 33: 2 11488-21499, 2020.
Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, and Inderjit S Dhillon. Taming pre- trained transformers for extreme multi-label text classification. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.
Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, and Ed H. Chi. Top-k policy correction for a reinforce recommender system. In Proceedings of the Twelfth ACM International Conference Web Search and Data Mining, 2019.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary dif- ferential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018.
Ricky TQ Chen, Brandon Amos, and Maximilian Nickel. Neural In International Conference on Learning Representations, 2020.
point processes.
Yuanda Chen. Thinning algorithms for simulating point processes. Florida State University, Talla- hassee, FL, 2016.
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre. Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using mn encoder-decoder for statistical machine translation. In EMNLP. 2014.
Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff func- tions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statisties, 2011.
10

---

Published as a conference paper at ICLR 2022
Marco Cuturi. Sinkhom distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 01-22222-3000 2013.
Weicong Ding, Dinesh Govindaraj, and S V N Vishwanathan. Whole page optimization with global constraints. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '19, 2019.
Nan Du, Mehrdad Farajtabar, Amr Ahmed, Alexander J. Smola, and Le Song. Dirichlet-hawkes processes with applications to clustering continuous-time document streams. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD '15 2015.
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1555-1564, 2016.
Peter S. Fader, Bruce G. S. Hardie, and Ka Lok Lee. Rfm and clv: Using iso-value curves for customer base analysis. Journal of Marketing Research, 42:415 430, 2005.
Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn di- vergences. In International Conference on Artificial Intelligence and Statistics, pp. 1608-1617. PMLR, 2018.
F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiss, 5(4):1-19, 2015.
Balázs Hidasi and Alexandros Karatzoglou. Recurrent neural networks with top-k gains for session- based recommendations. In Proceedings of the 27th ACM international conference on information and knowledge management, pp. 443-852, 2018.
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based rec- ommendations with recurrent neural networks, 2016.
Seyed Abbas Hosseini, Keivan Alizadeh, Ali Khodadadi, Ali Arabzadeh, Mehrdad Farajtabar, Hongyuan Zha, and Hamid Rabiee. Recurrent poisson factorization for temporal recommen- dation. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, KDD '17 2017.
Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE International Conference on Data Mining, pp. 263-272, 2008
Jim C. Huang, Rodolphe Jenatton, and Cedric Archambeau. Online dual decomposition for perfor- mance and delivery-based distributed ad allocation. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp- 117-126, 2016.
Rodolphe Jenatton, Jim Huang, and Cédric Archambeau. Adaptive algorithms for online convex optimization with long-term constraints. In International Conference on Machine Learning, pp- 402-411. PMLR, 2016.
Junteng Jia and Austin R Benson. Neural jump stochastic differential equations. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buc, Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Nan Jiang and Lihong Li. Doubly robust off-policy value evaluation for reinforcement learning. In International Conference on Machine Learning, pp. 652-661. PMLR, 2016.
Christopher C Johnson. Logistic matrix factorization for implicit feedback data. NeurIPS 2014 Workshop on Distributed Machine Learning and Matrix Computations, 2014.
Wang-Choe Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference Data Mining (ICDM), 2018.

---

Published as a conference paper at ICLR 2022
Jaya Kawale, Hung H Bui, Branislav Kveton, Long Tran-Thanh, and Sanjay Chawla. Efficient thompson sampling for online matrix- faatiiiiactoiizatiion recommendation. In Advances in Neural In- formation Processing Systems, 2015.
Maciej Kula, Metadata embeddings for user and item cold-start recommendations. In Proceedings of the 2nd Workshop New Trends on Content-Based Recommender Systems co-located with 9th ACM Conference on Recommender Systems (RecSys 2015), Vienna, Austria, September 16-20, 2015., CEUR Workshop Proceedings, 2015.
Jure Leskovec, Lars Backstrom, and Jon Kleinberg. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, KDD '09, 2009.
Lihong Li, Wei Chu, John Langford, and Robert Schapire. A contextual- bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pp. 661-670, 2010.
Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du Yao Xie, and Le Song. Learning temporal point processes via reinforcement learning. In NeurlPS, 2018.
Yunzhe Li, Yue Ding, Bo Chen, Xin Xin, Yule Wang. Yuxiang Shi, Ruiming Tang, and Dong Wang. Extracting attentive social temporal excitation for sequential recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 998- 1007, 2021.
Ryan Lowe and Andy Craig. Solving abandoned cart scenarios using amazon pinpoint event- triggered journeys. AWS Messaging & Targeting Blog, 2021.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. In Advances in Neural Information Processing Systems 30, 2017.
Yifei Ma, Balakrishnan (Murali) Narayanaswamy, Haibin Lin, and Hao Ding. Temporal-contextual recommendation in real-time. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '20, 2020.
Aranyak Mehta. Online matching and ad allocation. Foundations and Trends@ in Theoretical Computer Science, 8(4):265-368, 2013.
Hongyuan Mei and Jason Eisner. The neural hawkes process: neurally self-modulating multivari- ate point process. In Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
Martin Mladenov, Elliot Creager, Omer Ben-Porat, Kevin Swersky, Richard Zemel, and Craig Boutilier. Optimizing long-term social welfare in recommender systems: A constrained matching approach. In International Conference on Machine Learning, pp. 6987-6998. PMLR, 2020.
Alex Reinhart. A Review of Self-Exciting Statistical Science, 33(3):299 - 318,, 2018.
Point Processes and Their Applications.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and I Lars Schmidt-Thiemee Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, pp. 452-461, 2009.
MA Rizoiu, Y Lee, S Mishra, and I. Xie. A tutorial on hawkes processes for events in social media. Frontiers of Multimedia Research, 2017.
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic fore- casting with autoregressive recurrent networks. International Journal of Forecasting. 36(3):1181- 1191, 2020.
Ashudeep Singh and Thorsten Joachims. Faimess of exposure in rankings. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD 18, 2018
12

---

Published as a conference paper at ICLR 2022
Yi Su, Magd Bayoumi, and Thorsten Joachims. Optimizing rankings for recommendation in match- ing markets. arXiv preprint arXiv:2106.01941, 2021.
Chenyang Wang, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. Modeling item-specific temporal dynamics of repeat consumption for recommender systems. In The World Wide Web Conference, WWW 19, 2019.
Chao-Yual Wu, Amr Ahmed, Alex Beutel, Alexander J. Smola, and How Jing. Recurrent recom- mender networks. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM '7, 2017.
Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning granger causality for hawkes processes. In International Conference on Machine Learning, pp. 1717-1726. PMLR, 2016.
Simiao Zuo. Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes process. International Conference on Machine Learning, pp. 11692-11702. PMLR, 2020.
13

---

Published as a conference paper at ICLR 2022
A RELATED WORK AND DISCUSSIONS
Non-sequential marketing strategies Marketing is a general area studied from multiple perspec- tives. From system's 's perspective, example of keyword and related-item bidding comes from close examination of large-scale extreme classifier models (Chang et al. 2020). The push notifica- tion example considers explicit user states, which are commonly expressed via Recency, Frequency, and Monetary values (RFM) (Fader et al., 2005). By far, the most common and straightforward approach is use Gradient Boosted Decision Trees to regress on the binary user-item engagement labels using these RFM features. At a simpler level, without the manually constructed temporal features, the loss function is shared with traditional matrix factorization approaches, such as Alter- nating Least Squares (Hu et al.) 22008) Bayesian Personalized Ranking (Rendle et al. 2009). and Logistic Matrix Factorization (Johnson 2014), all of which serve as UserRec baselines.
Neural temporal point processes We further receive inspirations from temporal point processes
(Reinhart 2018 Chen 2016 Du et al. 2016 2015), Neural-ODEs (Chen et al. 2018). and their
intersections (Jia & Benson 2019 Chen et al. 2020 Mei & Eisner 2017: Zuo al. 22220) Most of
the works are great introduction of various aspects of TPPs and their modern development in GPU environments. However, most of the works focus on the prediction for better temporal modeling without novel applications of the intensity parameters for different applications. Along the line of better temporal models, (Jia & Benson 2019) shows how Neural-ODe can be applied to a MTPP construction and (Boyd et al., (020) focuses the particular challenge that personalized user his- tories "exhibit large predictive uncertainty at the beginning of the sequences". We do observe that many users churn out shortly after initial engagement. To model this temporal heterogeneity near the users' beginning, we use a simple trick, where we add another related time series with exactly one event for every user to mark their start time.
Causality In our work, we consider item exposure as the primary factor and assume positive down- stream effects afterwards. We could also analyze the temporal correlation between an intervention event and the change in the user's temporal intensity in another event type. In the nbandoned-carl example, let ý be the email-reminder event at time and y, be the desired user-checkout event at time t.. We then define the intervention state of the user as ã = x U {(t, 8)} and model the temporal intensity of the subsequent checkout event by
p(T. L, A) log( (PT(A(, (1. (C.-D) 1) v.):
(16)
or its time-varying integral form similar to equation 4 Another difference to the likelihood-bart based intensity models in our main text is the inference routine. To choose which users send the email reminders, we must also consider the users in the hypothetical intervention states. I.e., we augment the user time series as ã U {(,, w)} before computing their intensity scores in the effect items.
Besides the (Granger) causality research, temporal intervention effects can also help prevent bad consequences in marketing. For example, we may combine the positive effects of a marketing email through likelihood- based intensity modeling as well as the risks of undesirable effects like user churn-out through causality models. Some interesting work on temporal causality include (Xu al.) 2016
Li et al. 2018 Jiang & Li 2016; Lundberg & Lee 2017
Convex optimization and online regrets On the long-term constraints, we reference from Sinkhorn's 's algorithm from optimal transport (Cuturi 2013 Genevay et al.] (018)) and the litera- ture in long-term marketing via online campaigns (Huang et al. 2016)] Jenatton et al. 2016 Ding ct al. 2019), with convergence analysis dating back to (Mchta (013). Online convergence analysis
further leads to novel applications such as online classification with specificity constraints (Bernstein et al. 2010).
Diversity and Exploration In our discussion of the Dual solution for OninMtch, we mention a dual solution in the form:
max (=))
[A v) (T)) (z. v)] = TopK - -(v): A(xx - -(w))-
This solution is intrinsically connected to the idea of bandit exploration with upper-confidence bound (UCB) (Li et al. 2010 Chu et al. 2011), where our adoption of the the minimal-exposure constraint
14

---

Published as a conference paper at ICLR 2022
leads to positive boost to the raw prediction scores, i.e., -r(y) 2 0. Further exploration in this direction may lead to additional benefits in the OnInMtch paradigm.
B GCMC MODEL DETAILS
The basic idea in GCMC-CLV is to split user behavioral data time to create a temporal-prediction scenario. We hold out targeting window from the observed user behaviors as CLV labels and build RFM features for every user at the start of the window. Here, every user is represented by the aggregation of the items found in the user's own history, which reflects the frequency of interactions in these items. Due to the sparsity of the user-item pairs in the observed user histories, we use a graph convolution layer for the aggregation, which also extracts the semantic item embeddings similarly to graph-convolutiona matrix factorization (GCMC). As for recency, we construct a single user bias term based the recency of the last purchase regardless of the item choices. We omit the per- item recency terms due to their high (uverse-)correllae with the per-item frequency features. We neither have monetary information in our experiments.
The main challenge in the scalable CLV approach is the vast amount of implicit-negativy labels. While we observe positive CLVs in the target window, we should also include every unobserved user-item pair zero CLV label. This leads to imbalanced label distribution, i.e., significant amount of zeros which we call negatives, and small portion of non-zeros which we treat as positive (binary) labels. To address the imbalance, we adopt strategy of negative sampling to augment every positive user-item pair with negative users (ū) and items (7), followed by sigmoid-tripler loss:
loss = 0.5 E, log(signooid(ccore( a) score(u, 2))
+ Eg log(sigmoid(score(( score(ū, ü, e))
Details of our negative sampling strategy is similar to Bayesian Sharwis-RRanking (BPR). How- ever, open-source BPR implementations are often limited to sampling only negative items, leaving negative users unconsidered and causing biases in the CLV predictions. We thus include our imple- mentation, with the additional benefit of better integration with GCMC framework. We sample the negative users (and items similarly) according to
p(u) x (C(u) + 1)0.5
where C(u) is the total number of appearances of the user (or item) in the training data.
Finally, the BPR scores are mnnslation-invariann because BPR only focuses on the relative compar- isons instead of the absolute values. This would be a problem when we port the scores for Dual, where we expect positive intensity scores. We use softplus activation to preserve the larger values in the raw scores. Our earlier attempt with sigmoid activation leads to saturation in the higher score values, which causes issues due to over-exploration.
C DUAL ALGORITHM: ADDITIONAL DETAILSS
C.1 DETAILED PROOFS OF PRIMAL. AND DUAL SOLUTIONS
Lemma SI. Show that the solution to max, Le(T, u, v) is केz = sigmoid (by=-up-y Vz e X, Vy V.
Proof. The proof is done by taking derivatives with respect to Tay: Var E X, Vy E V. We first expand out the Lagrangian form
(TT u, u) = (Ary Eg (ur (Ey - a)) Ey (Ex (Tx() - 33)
+ cEuy (- Tay log( (ay) - (1 Tzy) log(1 Tzy)).
(17)
15

---

Published as a conference paper at ICLR 2022
Then, take derivative with respect to Tzy
(T, u. = Aay - Uy log(TTy) + elog(1 -Tay) +
Oay
Try
Aay
By c log
Tay
Setting the derivative to zero yields the desired solution
log
Tay
Aay
Uy
Way sigmoid
Azz
Up Uy
Way
Lemma S2. Show that the solution min.20 de (u,v) in the roots of
0 = -r(fte( Un, v)) a (t - us v.y))
projected O, Va X. The other part of min,5o de (u, v) can be similarly derived.
Proof. The proof is done by plugging in the primal solution from Lemma ST to the Lagrangian form and then taking derivatives with respect Uz Vo E X. We first rearrange the Lagrangian form:
LT v) Eay ((Neyy - Us Uy)Tay) + O'E (u²) ) + (1,)
Twy
Euy
xT log
e log(1 T2y)
Way
This simplifies the substitution the solutions from Lemma
ST Specifically, the underlined terms
may cancel. Define Zzy = - "x such that मez Ite
The rest of the terms are
vu, = (uz) + 9E (uy) + (uz) + (.R. (uy) Pay (c (1 + +
- #ny))
Taking the derivative with respect to leads to Od, (u, a E.y C
Ozay
Exy
By 1 e:x
Ou
Ouz
Setting the gradients to zero leads solution the unconstrained problem min, de (u, v). Addi- tionally, since the gradients are monotone and independent with respect to uz Vx E X, we may also obtain a solution to the constrained problem in the space of by direct projections.
C.2 COMPLEXITY AND STORAGE COMPLEXITY
With batch-size of B and an catalog size of [)] n. this leads to temporary storage complexity of (Bn) and a computational complexity O ((n(llg())) log( for each mini-batch. Notice that to enable fast computations on streams of users, we must also avoid the storage of all user-item scores, use a combination sparse and low-rank matrices. For example, we may delay the final dot-product layer between the and item representations in RNN/Transformet models, but instead just store the D-dimensional representations using as little as O(B + n) storage. The computational complexity for the dot-product layer is O(BnD), which is comparable with sorting/ / bisection costs the Dual algorithm. In our experiments, we use 100m mini-batches based on total user states in empirical data.
D DATA STATISTICS AND OFFLINE PRECISION METRICS
Tigure/Table SI shows data splitting strategies and resulting statistics in the three datasets we consider in Section 5 With temporal split, we carve out a prediction time window between (T.T + AT] and Lawkes-PPissso and GCMC and the observation user-state distribution for Online-Dus simu- lation. With user split, pick Group B user construct test window between (T, T AT].
window between (T" T for time-bucketed models like
16

---

Published as a conference paper at ICLR 2022
Time
Time
TRAIN
TEST
TRAIN (Group
NF
ML
YC
warm users
32238
3020
71784
User
warm items
16217
3555
11431
User
training
2437151
376736
1087267
test events
187096
120191
49500
TPP callbration
TRAIN
TEST
Test start
2005/6/15
1520s
458s
Orline window
B
(Group
Test window
14 days
1520s
458s
K=1 items
163
C=1
323
718
User
Temporal split
User
Table S1: Data split and statisties
dataset MaviaLans
dataset NetFlix
dataset YC
RMM (ours)
interiiity modeling
UserRaz seclsinnggopssseec
ALS
0.55
Rance model
BPR
GC
GC
Nono RSM4 bosolines
LMF
ALS
Roa Randon
89A
BPR
GCC pPac
1710
ALS
5PR)
Itemeee. Weccsion@too110c110000
Itemfee Precision@to199001000
Itemitec Precision@toop 961x100)
6P9, EPk, your
Figure S1: Suitability of different methods for ItemRec and UserRee tasks, plotted on different axes.
For calibration/valiiationn, we use Group users with the same temporal splits. Our definition of user-state distribution allows sampling by either users or time or both users and time at the same time.
recall metries in Section 5.1 The trends are similar. We show recall in the main text because it caters more to user welfare,, whereas precision connects to the global objective in Mtch experiments. Con- versely, the precision numbers agree with the unconstrained optimization results in Figure S2 and Figure $4 S5 for different combinations of tasks and datasets, respectively.
Figure SI shows the offline precision benchmark results which resemble our Figure
E ITEMREC WITH MIN-EXPOSURE CONSTRAINTS (NF AND YC)
Similar to the Netflix results in the main text, we show two more Mtch experiments on NF (Fig- ure S2
and YC (Figure S3p. Comparing the Pareto fronts, we observe RNN-TPP variants RNN
MF
Non-personalized models, which is consistent with our results on ML dataset.
..--
Rand
O HP B
BPR
Transformer Transtomer-Poo
Transtormer-Hi
....
Pop
-A- ALS
GC
diversity baseline
diversity (perplexity)
10a
10a
0.000
0.005
0.010 0.000
0,005
0.010
0.000
0.005
0.010
(a] Offiine Greedy
(b) Offline Dual
[c) Online Dual
ItemRec Prec@163
Figure S2: ItemRec with min- exposure constraints in Netflix dataset.
17

---

Published as a conference paper at ICLR 2022
.-o-
Rand -
HP
-B-
BPR
Transformer
Transformer-HP diversity baseline
.P
Pop
ALS
-*-
GC
Transformer earioormmrrPoo Pap
diversity (perplexity)
10
10a
10a
0.000
0.001
0.002
0.003
0.000
0.001
0.002
0.003
0.000
0.001 0.002 0,003
(a) Offline Greedy
(b) Offliie Dual
(c) Online Dual
ItemRec Prec@ 115
Figure S3: ItemRec with min-exposure constraints Yoochoose dataset.
F USERREC WITH
CONSTRAINTS
We consider 1% < a < 100% and B = 1%, both as upper-bound constraints in equation 9 In the limit 100%, the problem becomes UserRec without the consideration of user capacities. This yields best UserRec relevance, with a risk to overwhelm the users. As a decreases, the users expected to take more balanced load of incoming items. We use similar perplexity metric to measure the user-load balances.
Rand
BPR
Transformer
Transformer-HP diversity baseline
.
Pop
-A-
ALS
GC
Transformer-Po Pr-P
diversity (perplexity)
10a
10a
0.02
0.00
0.04
(a) Offline Greedy
(b) Offline Dual
(c) Online Dual
UserRec Prec@31
Figure S4: UserRec with user max-capacity constraints in ML-1M dataset.
....
Rand
-
HP
-B-
BPR
Transformer Transformer-Pop
Transformer-HP div diversity baseline
Pop
-A- ALS
-
GC
diversity (perplexi) plexity)
10*
103
0.002
0.004
0.002 (b)
0.004
0.002
0.004
(a) Offline Greedy
Online Dual
UserRec Prec@323
Figure S5; UserRec with user max-capacity constraints in Netflix dataset.
18

---

Published as a conference paper at ICLR 2022
.-..
Rand
- HP
-B-
BPR
Transformer
Transformer-HP diversity baseline
.
Pop
-A- ALS
-*-
GC'
Transformert eeriormme--Po Pap
diversity (perplexity)
104
10a
0.000
0.001
0.002
0.000
0.001
0.002
0.000
0.001
0.002
(a) Offline Greedy
(b) Offliine Dual
(c) Online Dual
UserRec Prece 718
Figure S6: UserRec with user max-capacity constraints Yoochoose dataset.
show the trade-off between relevance and user diversity, which is an indirect result of capacity limitations per user. From the curves, we find larger advantages in the RNN-TPP variants. This is because the problem is more focused on the advertising perspective, giving larger user-to-item capacity ratio, which highlights our RIM contributions. The rightmost points in the first two plots agree with the unconstrained offline UserRec precision metrics in Figure ST
Figure S4
55
S6
For online UserRec, we observe that most methods. including all of RIM and GCMC methods follow similar trends as offline UserRee (Figure [SeuueeS44e,S5eeee1 and SG)c). However, we do notice some artifacts in MF methods on MI. and YC datasets. The artifacts are related to the under-delivery of recommendations with respect to maximal user-capacity constraints, which leads to drops in the precision metrics. For more details, we show the average percentage of users delivered per marketed item in Figure S7 The ideal percentage is 1% to agree with the 8-constraint. Notice that constraint dissatisfaction is possibility in online ItemRec as well, but they do not affect our general observation of improvements in rreleace- trade-offs and thus omit the discussions for simplicity. Our discussions here extend to online ItemRec as well.
Rand
D HP
-B- BPR
Transformer
Transformer-Ht
Pap
ALS
GC
Transformer-Pop
1.2%
1.0%
0.8%
0.4%
0.2%
%
1.0%
10.0%
100.0%
1.0%
10.0%
100.0%
1.0%
10.0%
100.0%
(a) MovieLens
(b) Netflix
(c) Yoochoose
alpha; capacity measured by of total items
Figure S7: Average percentage of users actually delivered per item. We set , = 1% and a 2 1%, the ideal outcome should be 1%. However, when simulate OninMtch in future period of time, any biases in the empirical user-state distribution could cause the outcomes to be different. We observe the bias to be more obvious in ALS and BPR methods MovieLens and Yoochoose datasets and less so with our RIM and GC methods in all datasets, showing another benefit of stability with sequential models.
What special in online UserRec is that the total number recommendations is highly related to the satisfaction of the 8-constraints. For example, in the extreme where 100% and = 1%, the user capacities are entirely unconstrained, yet the total number of item exposures have to be controlled. The other settings where 1% & a 100% respect the capacity of each individual
19

---

Published as a conference paper at ICLR 2022
but still over-provision the overall user capacities for the desired number of item exposures. In these cases, the 3-constraints are observed by transferring the dual parameters, v(y). from the calibration set to the test set through either temporal split (Figure [S] user split (Figure [S] Le.. Online-Dual does not use any user statistics the test set, which are unavailable ahead of time in true online settings. Despite so, the 8-constraints methods in all datasets as well as MF methods on NF dataset with temporal splits.
largely satisfied with RIM and GCMC
The artifacts are only limited MF methods on ML and YC datasets, both of which user-splits. Upon further inspection, we realize that the artifacts are due to the fact that we reuse a part of training data for calibration purposes, i.e. the [T, T + AT) time window for Group-A users Figure SI This causes Group-A users to have more training data, leading to higher recommendation scores during the learning of o(y)- they are transferred to Group-B users, who have less training data and generally lower recommen- dation scores. This eventually translates to overall lower item-exposure rates compared with what calibrated on the validation set, shown in Figure S7 Notice that these artifacts are much less obvious on NF datasets, where the total training-data time (6 months) overwhelms the difference (15 days) between these two splits. They neither significant in RIM / GCMC methods, where the user training data is averaged against the user time span, improving the stability of the user-state distribution. In fact, the artifacts with MF methods reveal side benefit of our RIM (and GCMC) methods in our novel design of online experiments.
a result, "(y) tends to higher, causing over-penalization when
20


Reviews:
[{"review_id": "zQ7AZTjyJGg", "reviewer": "Reviewer_CMef", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "***Strong aspects***\n1. The research problem is well motivated. Item recommendation and item marketing are indeed two important and close-related problems.\n2. Modeling users' behaviors by a temporal process makes sense, and it has been demonstrated effective in many works.\n3. This paper evaluates the approach on three benchmark datasets.\n\n***Weak aspects***\n1. Baselines are weak and old. Competitive sequential models in recommender systems such as the Transformer-based one, SASRec [1], should be considered. Besides, there is no baseline for item marketing.\n2. Experimental setup in Table 1 is strange. It is obvious that RNN will have better performance since all of the other methods cannot leverage temporal information. \n3. The evaluation is not well explained. Why is diversity needed in the ItemRec task?\n4. Paper writing requires improvement. The figures are not so clear, such as Figure 3. Even some basic explanations, such as the difference between UserRec and ItemRec, are not well presented.\n\n[1] Kang, W. C., & McAuley, J.  Self-attentive sequential recommendation. In IEEE International Conference on Data Mining, 2018.\n\n**Update after rebuttal**\nThe authors have actively provided feedback, and I partly agree with their opinions about baselines. Therefore I have raised my score.", "summary_of_the_paper": "This paper studies item recommendation (ItemRec) and item-marketing (UserRec) as two variants of the same problem. Specifically, the authors convert ItemRec to UserRec based on the temporal point process with the proposed RIM. The authors evaluate the approach on three widely-used datasets.", "summary_of_the_review": "Overall the paper is not written well. The studied problem is important, but the proposed method is not well evaluated and cannot be claimed as a good solution to the research problem. In the current review stage, I would like to recommend rejecting this paper, waiting for author feedback to check whether my concerns can be addressed or not.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "i1XzDYCyaj", "reviewer": "Reviewer_9NaL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a strong paper with many interesting novel contributions. The problem of UserRec is very naturally modeled by the author's proposed framework and their empirical results are impressive.\n\nMy main criticism is that the paper was quite terse and hard to read on a first pass. The paper uses tools that are likely to be unfamiliar to many in the ML community with relatively little introduction. I think this could likely be remedied by moving some of the details that aren't core to the paper to the appendix, and spending more time on exposition during the main body of the paper.\n\nFurthermore, in section 5.1 the authors compare against classic matrix factorization techniques such as ALS. However, they do not compare against matrix factorization methods that take into account the temporal nature of the task, which have been shown to work well in these settings. e.g. timesvd++: https://www.cc.gatech.edu/~zha/CSE8801/CF/kdd-fp074-koren.pdf it would be interesting to see a comparison with such methods.", "summary_of_the_paper": "This paper considers the problem of recommendation in a temporal context. In this setting, which the authors call UserRec, the recommender system must determine whether to recommend an item to a user at a specific point in time given their prior interaction history, or whether to delay the interaction. They draw connections to classical item recommendation, where the recommender must instead recommend some item to a given user, and does not have the option of delaying the recommendation. By modelling the UserRec problem as a temporal point process (TPP) the authors propose a framework to convert algorithms developed for ItemRec into a UserRec model.\n\nThey then propose an algorithm for recommendation in matching that leverages their TPP representation to setup a convex optimization problem. Finally, the authors validate their results on three different datasets.", "summary_of_the_review": "The paper in its current form is already a strong contribution, however I would appreciate it if the authors focused on exposition more in the final version of their paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Y6uDn20QZD", "reviewer": "Reviewer_cWJc", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n- The paper attempts to address the user recommendation problem for item marketing, which is relatively underexplored in the RecSys research community but critical in real-world applications.\n\n- The intensity perspective introduced in this paper is technically well-motivated, fresh and interesting.\n\n- The proposed methods were empirically evaluated on three real-world datasets against standard baselines, where the takeaway message is relatively clear. Apart from the user recommendation task, the trajectory perspective also naturally enables user-inductive generalization (i.e., without explicitly learning identifier-based user representation). This property is consistent with the user-inductive experimentation setup in the paper, also close to the real-world online scenarios.\n\nWeakness:\n\n- It is acceptable to me but the position of this paper might be a little bit awkward - it's apparently different from traditional item recommendation scenarios, attempts to address the marketing problem, but doesn't touch the causal aspect or deep dive into the applications such as online advertising or push notifications.\n\n- Another concern is the technical part of this work can be a bit difficult to digest for general RecSys audience. Including running examples might be helpful for readers to grasp the technical ideas at high-level.\n", "summary_of_the_paper": "This paper proposes a framework to address the so-called user recommendation (UserRec) problem, tasked to match users (as item consumption sequences) to item products for marketing purposes. The framework extends the Temporal Point Process (TPP) and is built on top of a recurrent intensity model (RIM). The proposed methods were tested on three real-world recommender system datasets against standard baselines on both item and user recommendation tasks, in both offline and online matching experiments.", "summary_of_the_review": "Although the paper has some caveats and the presentation of the work could use additional improvements, I still find the work interesting and enables a new perspective for the recommender system community. Notable contributions deserve to be called out are 1) it attempts to address the warfare of all parties of multi-sided marketplaces, and 2) it connects user and item utilities in a time-sensitive space. As a researcher working in industry, I'd like to emphasize that addressing the producer/marketer utility is crucial not only for the business success, but also for the market fairness as well as the long-term health of the ecosystem. In this regard, I personally appreciate the contributions of this work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Between a reject, an accept, and a borderline-accept, this is truly a borderline paper, though I'd lean slightly on the side of accepting it. The most negative review raises issues of weak baselines, along with several more minor issues. The authors rebut this reasonably well, arguing several differences from the setting used in the suggested baseline papers. It is a little hard to follow who is correct between the review and the rebuttal, though the rebuttal makes reasonably convincing arguments. Other than the baselines most of the issues raised by the negative review are more minor and can be easily fixed in a revision. The specific issues mentioned are mostly not raised in either of the more positive reviews.

The borderline (positive) review is by far the most detailed, but overall praises the paper and mostly suggests fixes in terms of better positioning the paper. Overall both the positive and borderline-positive reviews make persuasive arguments as to the paper's conceptual merits, which outweigh some more minor issues.
Decision: Accept (Poster)