Self-Adaptable Point Processes with Nonparametric Time Decays
Authors: Zhimeng Pan
Zheng Wang
Jeff Phillips
Shandian Zhe
Venue: NeurIPS 2021 Poster (2021)
Keywords: Nonparametric
Point Process
Gaussian process

Abstract: Many applications involve multi-type event data. Understanding the complex influences of the events on each other is critical to discover useful knowledge and to predict future events and their types. Existing methods either ignore or partially account for these influences. Recent works use recurrent neural networks to model the event rate. While being highly expressive, they couple all the temporal dependencies in a black-box and can hardly extract meaningful knowledge. More important, most methods assume an exponential time decay of the influence strength, which is over-simplified and can miss many important strength varying patterns.  To overcome these limitations, we propose SPRITE, a $\underline{S}$elf-adaptable  $\underline{P}$oint p$\underline{R}$ocess w$\underline{I}$th nonparametric $\underline{T}$ime d$\underline{E}$cays, which can decouple the influences between every pair of the events and capture various time decays of the influence strengths. Specifically, we use an embedding to represent each event type and model the event influence as an unknown function of the embeddings and time span. We derive a general construction that can cover all possible time decaying functions. By placing Gaussian process (GP) priors over the latent functions and using Gauss-Legendre quadrature to obtain the integral in the construction, we can flexibly estimate all kinds of time-decaying influences, without restricting to any specific form or imposing derivative constraints that bring learning difficulties.  We then use weight space augmentation of GPs to develop an efficient stochastic variational learning algorithm. We show the advantages of our approach in both the ablation study and real-world applications. 

Self-Adaptable Point Processes with Nonparametric Time Decays
Zhimeng Pan, Zheng Wang, Jeff M. Phillips, and Shandian Zhe
School of Computing. University of Utah Salt Lake City, UT 84112
z zzpan, wzhut testah.edt, jeffpécs .ut.ttthhaao edu, zhe@cs. cateess..ahh..e edu
Abstract
Many applications involve multi-type event data. Understanding the complex influences of the events on each other is critical to discover useful knowledge and to predict future events and their types. Existing methods either ignore or partially account for these influences. Recent works use recurrent neural networks to model the event rate. While being highly expressive, they couple all the tem- poral dependencies in a black-bor and can hardly extract meaningful knowledge. More important, most methods assume an exponential time decay of the influence strength, which is over-simplified and can miss many important strength varying patterns. To overcome these limitations, we propose SPRITE, Self-adaptable Point pRocess wIth nonparametric Time dEcays, which can decouple the influences between every pair of the events and capture various time decays of the influence strengths. Specifically, we use an embedding to represent each event type and model the event influence span. We derive general construction that can cover all possible time-decaying functions. By placing Gaussian process (GP) priors over the latent functions and using Gauss-Legendre quadrature to obtain the integral the construction, we flexibly estimate all kinds of decaying influences, without restricting any specific form or imposing derivative constraints that bring learning difficulties. We then weight space augmentation of GPs develop efficient stochastic variational learning algorithm. We show the advantages of our approach in both the ablation study and
unknown function of the embeddings and time
applications.
Introduction
Events of multiple types are ubiquitous, such as in online shopping, social networking and biological signal transduction. Understanding the complex influences of those events on each other, including excitation, inhibition, and how the strength of these influences varies with time, is crucial to discover useful knowledge and to predict future events and their types, which can benefit many applications, e.g., marketing analysis, biological study, and early warnings of catastrophes.
While many excellent works have been proposed for event modeling and analysis B lundell et al. 2012 Linderman and Adams 2014 Xu al.) 2016 Tan et al. 2016) Xu and Zha 2017 Zhang et
2020a most of them are inadequate to capture complex influences between the events. The classical
Poisson processes simply ignore these influences and assume independent increments. Although
Hawkes processes (HP) can estimate the mutual excitation between the events, they ignore the inhibition effect, which is common in real world. More important, existing methods (Xu et al.) 2016)) Zhang et al., (2020a), even for those considering both the excitation and inhibition, mostly assume an exponential time decay the influence strength, which is over-simplified and can miss many other decay patterns. While recent works have also used recurrent neural networks (RNNs) (Du et al.) 2016 Mei and Eisner 2017) to model the event rate and hence are highly expressive, they couple
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

all the temporal dependencies in a black box, and are difficult to distill meaningful and important knowledge, e.g., influence types and strengths.
To address these limitations, we propose SPRITE, a multi-variate self-adaptable point process with nonparametric time decays. Our method not only can discover the type of the influence (ie., excitation or inhibition) between every pair of the events, but also is flexible enough to estimate all kinds of influence strength decaying patterns, not limited to the exponential decay or other particular forms. Specifically, we first introduce an embedding to represent each event type, and model the influence between any two events as an unknown function of the event type embeddings and time span. In so doing, we avoid separately estimating the influence between every pair of the events, and largely reduce the parameters. Next, we propose a general construction that covers all possible time decaying functions. We assign Gaussian process (GP) priors over the free latent functions and use Gauss-L Legendd quadrature to calculate the integral in the construction. In this way, we can flexibly estimate variety of time decaying influences while not needing to impose any derivative constraint, which can bring learning difficulties. For efficient inference, we use weight-space augmentation of GPs to prevent computing huge covariances matrices, and neural network feature mapping to enhance the learning capacity. We then use the re-reparamettrrizatoon trick to develop a scalable stochastic variational learning algorithm.
For evaluation, we first examined SPRITE in an ablation study. We tested the case of single event type with excitation effects only and three time decays, and a bi-type event case with mixed excitation and inhibition effects. In both cases, our method accurately recovered the rate function of each type of events and the influence between the events, performing better than state-of-the-art methods based on or extending Hawkes processes and RNNs. Next, we examined SPRITE in three real-world and one synthetic benchmark datasets. We examined the accuracy in predicting the occurrence time and type of future events. In both tasks, our method nearly always outperforms the competing approaches, often by large margin. Finally, by looking into the embeddings and influence functions learned by our model, we found interesting patterns within the event types and their interactions.
2 Background
Temporal Point Processes. We denote a sequence of events and their types by r = ((1,8))...., (L,,,8M) sM)], where each I; is the time point when the i-th event occurred (11 < ... S (as), 8; is the type of the i-th event. Suppose we have K types of events in total; so each S; E (1...., K}.
Temporal point processes (TPPs) (Daley and Vere-Jones, 2007) provide a fundamental framework to characterize and model the events. A TPP is defined via the rate (or conditional intensity) function of the events, (t). For example, homogeneous Poisson process assumes each X, (t) is constant Xo and the past events {(t,, It) < t) do not have any influence on generating new event at f. While being simple and convenient, Poisson processes completely ignore the interactions between the events. Hawkes processes (HPs) (Hawkes 1971) overcome this issue by modeling the triggering effects between the events, (4) A Ltyett 7a5-1 (((()))))))) (2) where 70, > 0 is called "triggering kernel", and characterizes how largely the previous event at ty encourages
event of type to occur at f. Naturally, this effect is supposed to decay with time A. The most commonly used choice is the exponential decay,
+s (A) Cg'' exp (-Te
where 0''' > 0 and Te's > 0. Given the definition of (.).). the probability of the event sequence is
p(T) = k= exp(- /''' (a(()) II 1=1 As, (4:),
where T is the total span of the events in and so tas S T.
RNN based Point Processes. HPs only consider the triggering effects and hence can be quite restricted. In order to estimate arbitrarily varying rate functions, recent works et al.) (2016); Mei] and Eisner (2017) use recurrent neural networks (RNNs) to model the event dependencies and rates. The key idea is as follows. For each event i, we introduce a hidden state vector B(4.). The event dependency is modeled through the updating rule of the hidden states,
((ta+)) = (B(tz), 1
(3)
2

---

where pl-) is complicated mapping, fulfilled by combination of neural network operations, such as linear transformation, nonlinear activation, and gating. Note that the event type s; can also be incorporated into p(-). The rate function is then defined as transformation of (3()). We can therefore evaluate and maximize the point process likelihood (2) for model estimation.
3 Model
While RNN based methods are highly expressive and can capture arbitrarily complex dependencies, due to the black-box updating rule (see (3), they can hardly distill meaningful knowledge, such as the influence type and strength between the events, which are critical for data analysis and knowledge discovery. On the other hand, while HPs enjoy excellent interpretability due to the additive structure in the intensity, they overlook the inhibition effect among the events, which is common in real world. More important, nearly all the HP based methods adopt exponential time-decaying kernels (or a combination of them) (see D including the recent works (Mei and Eisner 2017]
al. 2020a) that extend HPs to allow negative triggering kernel (asr' < O in (I)) However, an exponential decay can be overly simplistic for various, complex real-world applications. These methods can therefore miss other important decaying patterns (e.g., polynomial decays), resulting in inferior influence estimation and predictions.
To overcome these issues, we propose SPRITE, a multi-variate self-adaptable point process model that can explicitly estimate the influence type and strength between every pair of the events, and flexibly capture all kinds of time decays of the influence strength, not limited to a particular form. Specifically, we first define a raw rate function for each event type s(1 S K) to characterize both the excitation and inhibition effects from the previous events,
(x) = 20 Σ 1=1 [tt < (t t;),
(4)
where 1()) is indicator function, Xº is the background (or base) rate, and h-.- is influence term that characterizes the effect of a previous event of type 8 happened at 12 on the occurrence of a new event of type at f. When hs > 0, it indicates triggering effect while hs,- 0 means inhibition. The absolute value of heg- is the strength of the influence. Since the raw rate can be negative, we then apply a scaled soft-plus transformation (Mei and Eisner 2017) to ensure we obtain positive rate function, (t) Blog (1 (x)().())/3) where ,3 > 0.
Next, we introduce a set of embeddings 4- = {u,) to represent the event types. We sample these embeddings from the standard Gaussian distribution, p(4)) (u, I). Then we model the influence term as a function of the event type embeddings and time span,
(A) = h h(u,, u₂ A).
(5)
The advantage of doing is that only need to estimate one function h to capture the influence among all the events (including those within the same type). The embeddings can further enable us discover the hidden structures of the event types (e.g., cluster and outliers). Otherwise, if we estimate one function for each pair of event types, we have to learn K K(K- - 1)/2 functions, which can be much more challenging and costly, especially when the event types are many (i.e., large K).
We want our model to flexibly learn the influence function h(-) and its strength decaying patterns with time, rather than restrict to any pre-specified, parametric form, e.g., the exponential decay in ( In other words, we only require when h > 0, the time derivative ah.
03 < O, and when h < 0, éh > O.
03
i.e., h 8h < 0, ensure [b] decreases over time. While it is standard to assign Gaussian process (GP) prior (Rasmussen and Williams| (006) over l to enable a flexible, nonparametric function estimation. incorporating the time derivative constraints difficult. The derivative of a GP is still a GP (provided the kernel is differentiable). Since the support of the Gaussian distribution (GP finite projection) is the entire real space, imposing a monotonic constraint is infeasible (there are always nonzero probabilities for positive/negative derivatives). Existing work (Riihiākki and Vehtari 2010) introduces a set of virtual points and encourage the derivative at these points to be as positive (or negative) as possible. This essentially is soft regularization, not an actual constraint.
73
To address this issue, we observe that the time-decaying influence functions have general structure. ah Lemma 3.1. h- 0A < 0 if and only if h has the form
h( Ug A) = g(u,, 41 - f(u,, u, A)),
(6)
3

---

of 05 where ( uz) = h(u,, u, 0), 0 f() 1
> 0, and f(u,, u. 0) 0.
We leave the proof in the Appendix. Next, to construct f. we represent f(() tanh (z(Us;: Ux, 4)) where z(-) is another function. To satisfy the requirements for f in (6). we need to ensure 2 2 0, 03 > 0. According to these constraints, we can represent 2 as an integral r))rr)) dr where 77() ) a free function. Combining with (6. we now derive a general construction of our influence function,
z( (Ux; Us, 0) = O, and de
function, = u( Us A) = I exp (n(u,, us r):
Corollary 3.1.1. h-
ah 03
0 if and only if
h(u,. Ug A) = g(u,, u.) (1- tanh (%/² exp (n(u.. us ug, r)) dr )
where both g(-) and of are free latent functions.
Note that our construction 7 also covers the special case that hn-) = 0 (when 0), i.e. no influence is from the previous event. To flexibly estimate h(.), we can place GP priors over and ry. However, no longer need to incorporate any derivative constraint and hence the learning is much casier. Note that the prior (()) is not GP anymore.
One might be concerned that the integral in is analytically intractable and will bring troubles in model estimation. However, since the integration one dimensional, we can use Gauss- Legende quadrature to calculate the integral quite accurately and evaluate h(-). We will discuss the details in Sec. 4.2
4
Algorithm
The model estimation challenging. The GP prior over the latent function g(-) 7 requires us to compute a multivariate Gaussian distribution of (u,,, us))ss (the finite projection of g). and over v7()) multivariate Gaussian distribution of (n(,, Us, A)J where A are the time differences between every pair of events. Hence, even with moderate number of events and event types (e.g., K = 100), the corresponding covariance (kernel) matrices will be huge and infeasible to calculate. To address this problem, we use the weight space view of GPs (Rasmussen and Williams/ 2006). That is, a GP model is equivalent to Bayesian linear regression after a (nonlinear) feature mapping. We neural networks to construct finite yet highly expressive mapping, and explicitly estimate the posterior of the (augmented) weight vector. In so doing, we can ease the inference and avoid computing the full covariance matrices. We then use Gauss-Legende quadrature and the reparameterization trick to develop an efficient mini-batch stochastic variational learning algorithm.
4.1 GP Weight-Space Augmentation with Neural Network Feature Mapping
Specifically, introduce R dimensional weight vectors wy and w., for the latent functions g(-) and n(-) in 77. We then sample and ry in the following way:
g ~ (,, W,, ~W(w,, I).
Iu, Ugg = w,J (u,,, uk), n7(U,, ug, A) w,, (,,, Uk, A),
where d, and Φ,, are nonlinear feature mappings fulfilled by neural networks (NNs), known that if we marginalize out the weight vectors, we recover the GP model, where the function values at an arbitrary finite set of inputs follow a multivariate Gaussian distribution. The covariance (or kernel) function is inner- product of the mapped feature vector. For example, the kernel for 9(-) will be
([u ug], [u,, u = (Øy (u,, ) ) dy(u,, ue);; the NN parameters dy can be viewed as kernel parameters. Although the kernel does not correspond to an infinite feature mapping in traditional kernels (e.g., RBF), it still can be highly expressive due the capacity of neural networks. We will keep the weight vectors Wy and w,, in our model inference. In so doing, we never need to operate the huge full covariance matrices. The dimension of the weight vectors I is often set to be small, e.g. 16 or 32. Hence the computation will be much more efficient, and the inference are easier and convenient.
Rigorouuly speaking. should be exponential integrable, i.e., exp(n(-)) is integrable in [0, AJ. This is pretty mild condition, which can be satisfied by, egg. continuity of 17())

---

Suppose we have observed a collection of N independent event sequences, D = {I") hIeeH where each l = [tt si),,.., (t"r,, SMM, )] and M, is the number of events in sequence n. According to
and g the joint probability of our model given by (W,,,,
plu, Wg. Wyy D) = II. (u, O, (w
exp (-1,- (x)d)
II
K
II (t,))
(9)
where Tn the total time span across the events in sequence n.
4.2 Stochastic Variational Learning
Given the joint probability (9) we aim to estimate the posterior distribution of the weight vectors Wg and Wg. the event type embeddings U, the parameters in the feature mappings Φn and Φ,, and the other parameters. Due to the intractable model evidence (i.e., normalizer), exact inference is infeasible. Therefore, we use variational inference (Wainwright and Jordan 20088. Specifically, we introduce a Gaussian variational posterior for the weight vectors,
gW w,,) ((w,)q(w,,) = NW My. Ey) E,).
(10)
We use the Cholesky decomposition to parameterize the posterior covariance matrices ensure their positive definiteness, Σg = L and E, = L,,L, where Lg and L, are lower triangular matrices. We then derive a variational model evidence lower bound (ELBO),
L KI. (g(w))))))l)(w))
KL (1(w,)) lp(()))))) + log(p((())
Th Σ [
Xe(t)at
+ + ==1 +
Mr E.g [log (Noy ((2)
+
k=1 Eg
n=1
where p(w,) and p(w,) are the standard Gaussian priors in (.. and p(4) II. W(,, 0, I). To handle large N, K (i.e., the number of sequences and event types) and the intractable integration ITh (x(t)dt, we resort to stochastic optimization. tion all the event sequences of into mini-batches of size B: 1B,,...,BNy types into mini-batches of size C: [CI,...CCk/ch, We observe that the integration of the rate function is piece-wise, fT >x(l)dt = Eyzo f Ae(()dt where o 0 and Ma+ Tn. The rate (() in each interval (3,1 is smooth (see (4). and hence use Gauss-Legendre quadrature to compute each fr dx(())dt. We now talk about how to compute sample of Aa(t) for stochastic optimization, which uses the Gauss-Legende quadrature again. Specifically, we arrange the ELBO as L = -KL Em
Specifically, we parti-
and event
KL ((W,)))l((,))) + log(p((()) N
N EnEB N E E I (x(t)dt)
Em B EnEB..
E.Mn F. [log (Aop (17)]], which can be further viewed as an expectation,
B
C = ==pp(m).ree (0))))000nna (mmih L = KI. (1(w,))lll(w - KI. ((,,,, (((w,))ll(w),, + log(p((())
NKW Σ
( (t)d)t
N
M
Eg
Σ
Σ
E
[log (t?7)),
C
B
nEBm REC.
nEB...
1=1
R where p(m) N m E (I,..., N. p(1) = optimization, each step we first draw mini-batch Bm, and Ct calculate the 2.m. We then the reparameterization trick to draw from g(wg. w,)): w, Bg + L, and w,, - Mb + Lop where Cy ~ WV I) and E, O- I). We substitute these samples into the corresponding terms in the expectations and obtain an unbiased stochastic estimate of Cmi- We then calculate the gradient to obtain an unbiased estimate of V Vim. which is also an unbiased estimate of V.. Accordingly, we can perform stochastic optimization to maximize so estimate q and the other parameters.
f L.LE
k) (I....,,K
To conduct efficient stochastic
However, a critical issue is that after we use the parameterized samples of Wg and W,, to obtain the latent functions g(.) and n7(-), cannot compute the influence function h(-) in the rate function
5

---

(+), due to the intractable integral in 7. To address this issue, we observe that the integration only one dimensional over time. Hence, we can use Gauss Legendre quadrature to calculate the integral with an analytical form,
Jo exp (n(u,,, Us r)) dr 2 Σ W 2
A
exp
(uu,, Us 2 E +
(II)
where {wy} and { are quadrature weights and nodes respectively. Note that this is based on the integral transform of the standard Gauss Legendre quadrature, which requires that the integration interval is [-1,1], 1]. We provide more details in the Appendix.
4.3 Algorithm Complexity
The time complexity of our inference algorithm O(CCM + It²) where B and C' are the mini- batch sizes of the partitions of the event sequences and types, M is the maximum length of the sequences, and R is the dimension of weight vectors w and W,. Therefore, the computational cost is proportional to the mini- batch sizes, rather than determined by the total number of sequences N and events types K. The space complexity is O(Kd + 2R²), which is to store the covariances of q( ((g)) and q( (W,,)) and the event type embeddings U (d is the dimension of the embeddings).
5 Related Work
Many works use Poisson processes to analyze event data for its elegance and convenience, e.g., (Lloyd et al. 2015 Schein et al., 2015 2016 2019). However, Poisson processes assume independent increments in their counting processes and hence overlook the interactions between the events.
Hawkes processes (HPs) (Hawkes) 197I) therefore become popular, due to their capability of
capturing the mutual excitations between the events. Many works propose HP based models to
estimate the temporal relationships from data, e.g., (Blundell et al. [2012] Tan et al.] 2016 Linderman
and Adams 2014 Du al., 2015 He et al., 20155 Wang et al. 2017 Yang et al.) [20 Xu and
Zha 2017]] Xu et al. 2018). Several methods have also been proposed to improve the inference of
HPs. such as nonparametric triggering kernel (and base rate) estimation (Zhou et al. 2013) Zhang ct al. 20206 Zhou et al. 2020). Granger causality (Xu et al. (0166). short doubly-censored event sequences (Xu al.] 2017) and online estimation (Yang et al. 2017b). Another recent line of
research (Zhe and Du 2018 Pan et al. 2020, Wang et al. 2020 uses or extends HPs to learn the representations of event participants.
To overcome the limitation of HPs in only estimating the triggering effects, recent research efforts have been made capture more complex temporal dependencies, egg., inhibition. In (Mei and Eisner 2017). a simple extension of HPs was proposed to allow a negative coefficient in the exponential triggering kernel (i.e., ag' in (T). and the intensity was transformed through soft-plus function to ensure positiveness. Zhou et al. (202I) uses a similar extension, but it models the kernel by a mixture of shifted Beta densities, and then uses sigmoid transform scaled by an upper-bound to obtain a positive rate. The second method in (Mei and Eisner 2017) develops a continues- time Long Thort-Term Memory (LSTM) (Hochreiter and Schmidhuber 1997) model to capture the event dependencies through the nonlinear, black-box updates of the hidden states. The rate is transform of the hidden state. Du al. (2016) developed a discrete RNN to model each event sequence. The rate at any time point / is obtained by taking the exponential of a linear combination of the hidden state of the closest event, and the time span from that event. addition, Omi et al. (2019) used forward NNs to model the cumulative intensity function given the RNN states, so to avoid computing the integration in the point process likelihood (see 2h), which is usually intractable. [Shchur et al. (2019) modeled the time span between the events with mixture of log lormnormal distribution parameterized by RNN states and other parameters.
Despite the expressive power of RNN based models, they encode all the temporal dependencies into the hidden state vectors and blackbox updating mechanisms. We can hardly extract important knowledge including the influence type and strength between pair of events, how they vary with time, etc. The very recent work (Zhang et al., 2020a) uses the self-attention mechanism (Vaswani et al. 2017 Bahdanau et al. 201 4) to capture the correlation between the events. The similar idea appears in the concurrent work (Zuo al. 2020). Zhang et al. (2020a) modeled the rate as the soft-plus transformation of base rate plus an exponential time-decaying kernel, where the base

---

rate, the amplitude and decaying rate are computed via the attention mechanism, which essentially calculates a weighted similarity with the representation the past events. Although the model is more interpretable and highly expressive, like most HP models, it adopts an exponential time-decay for the influence strength, and hence can miss other decaying patterns data. This (partly) motivates us to develop an interpretable yet flexible point process model to capture various time decays from data. There are other temporal models based on ODEs, e.g., (Rubanova et al. 2019).
6 Experiment
6.1 Synthetic Data
To confirm the effectiveness of SPRITE, we first conducted an ablation study with synthetic data. Specifically, we considered two cases: single-type type events and bi-type events. For single-type events, we assumed that they only have excitation effects on each other. We examined three -enednn excitations,
h (A) max 0..4 max(0.4 0.112,00 0), (Quadratic)
h2( () = max (0. 0.2A, 0), (Linear)
ha(2 () = 0.3 (xp(().84)) (Exponential)
(12)
We applied the scaled soft-plus transformation to ensure a positive rate. We set the scale 3 = 0.8. We used Thinning algorithm Lewis and Shedler (1979) to sample event sequences. For each case, we generated 10K" sequences for training and 1K for validation. The length of each sequence is 32.
For bi-type events, we assume type 0 events trigger type 1 events while type inhibits type 0, and no effects are between events of the same type:
ho (A) (1.0 0.05 ,),, h1-o(A) 0.5 exp
(13)
We set B = 0.4 and used scaled soft-plus to obtain the positive rate function. Again, we sampled 10K sequences for training and IK for validation. Each sequence includes 64 events.
We compared with the following state-of-the-art methods: (1) Hawkes processes (HP) with the standard exponential decaying triggering kernel, (2) Recurrent Marked Temporal Point Processes (RMTPP) (Du et al.] 2016) that use a discrete RNN to model the event sequences, and use hidden RNN states and other parameters estimate an overall rate function (for all types of events), (3) Neural Hawkes Processes (NeuralHP) (Mei and Eisner 2017) that use a continuous LSTM to model the event sequences, and estimate the rate function for each event type, and (4) Self-Attentive Hawkes Process (SAHP) (Zhang et al. 2020a) that uses the self-attention mechanism to calculate the aggregated influence from the previous events on the current one. We implemented SPRITE and HP with TensorFlow (Abadi et al. 2016). We used the original implementation of NeuralHP com/
(https:// www.s/aww.oomcccaaaaaaaaaee HME.att.tt.......... JHU/neurawkess and SAHP (attps://qita)) repo), and a high-quality open- open-source implementation of RMTPP (https://github cow wttp://wwwwwwwww/eeeeeeeoo /eRpyya//ePPPa.. RMTPP). For SPRITE and HP, we set the mini- batch size of the event sequences to 16 and learning rate 10 We set the dimension of the embeddings to for SPRITE. The nonlinear feature mappings Φ and Φ,, in (8) were both chosen as single-layer feed-forward NN, with 16 neurons and leaky RELU as the activation function. We used the default settings of all the other methods. We ran each method with 50 epochs (enough for convergence). To avoid an unfair comparison caused by overfitting (especially for RNN based methods), we evaluated the likelihood of a validation dataset after each epoch and stopped training there is no improvement (early stopping).
QlangA_Reseercchrr//aaae
Fig. I a.c.e (left) show the rate function estimations of all the methods for single-type events, As we can see, the estimations of SPRITE nearly always overlap with the ground-truth rate function, demonstrating that our method can perfectly adapt to all the cases and capture different time decays. By contrast, SAHP and RMTPP's estimations significantly deviate from the ground-trute truth. SAHP often severely ever-estimates and under-estimates the rate (see Fig. [ and e). RMTPP correctly captures the trends, but often mistakenly estimates the local details. Although this test favors HP since only the excitation effects were considered, HP still resulted in apparent deviations in the case of quadratic and linear decays (see Fig. D and c), showing the limitation of the rigid exponential- decay assumption. Note that the estimation of HP overlaps with SPRITE and the ground-truth for exponential decays (see Fig. [].. While NeuralHP also gives excellent rate estimations, in a few
7

---

Type Events
Ground Sralh: Type Ground Sralls Type
Estanttdd: Type
Type: Events
Estimatodi Type:
3.0
Fvents
Ground Truth
HP SAMP
RMIPP
NeuralltP
2.0
1.0
(a) Quadratic
(b) HP
2.0
1.0
0.5
10.0
17.5
(c) Linear
(d) NeuralHP
2.0
1.3
1.0
0.5
(c) Exponential
(f) SAHP
1.0
-01
0.6
-033
0.4
0.0
Groand Intaenee 1
-0.
GGeesd Trih intomeee
(g) Recovered Influenceffunction by SPRITE
(b) SPRITE
Figure I: Rate function estimation for single-type events (a.c.e) and for bi-type events (b, d. h), and event influence function estimated by SPRITE for bi-type events (g). The estimated influences for single-type events are given in Fig. I of the Appendix. In all figures, the x-axis is time and y-axis the rate or influence value.
local regions, it still fails to capture the detail and/or gives a worse estimation than SPRITE, e.g., t e [16, 17] in Fig. I1, t e [4.0, 4.5) and [10.5, 11.5] in Fig. D and over-estimations for t around 4.0 and 19.0 in Fig. T..
Fig. b.d.f.h (right) show the rate estimations for bi-type events. We did not compare with RMTPP, since it only produces an overall rate function. As shown in Fig. Dp. HP's estimation for event type 0 is completely wrong, because HP cannot capture the inhibition effects from type I events. In Fig. DF. while SAHP indeed captures both the triggering and inhibition effects, its rate estimations severely deviate from the ground-truth, and miss capturing many local details (see the flat curves in many intervals). Finally, NeuralHP and SPRITE can accurately estimate the rates for both events. The estimation of SPRITE for event type 0 is slightly better, because it completely overlaps with the ground-trume -trthh Note that, however, SPRITE can further tell the influence type and strength between every pair of the events while NeuralHP, based on blackbox LSTM, cannot. All these results have demonstrated the advantage our method in flexibly capturing various time decays from data.
We also examined the estimated influence function by SPRITE. Fig. [ shows the results for bi-type events. We can see that our learned decay very accurate, nearly overlapping with the ground-truthe While the two influence functions (ho-+1 and h1-+o) are very different one is convex and the other concave, they both can be accurately captured by SPRITE. The estimations for the single-type events are given in Fig. 1 of the Appendix, showing SPRITE accurately recovered all the three types of influences. The results demonstrate the power of our nonparametric model. Note that due to the blackbox nature, cannot discover event influence functions from NeuralHP, SAHP and RMTPP.
8

---

244
144
PORTTM
RMSE
0.4
0.0
(a) Retweet
(b) MIMC
(c) SO
(d) SIM
0.7
1.0
0.5
0.9
6.6
11.03
11.4
Accuracy
0.3
0.2
0.1
0.5
0.0
(e) Retweet
(f) MIMC
(g)
(h) SIM
Figure 2: Prediction accuracy of the time (top row) and type (bottom row) of the future events.
Staill
Medium
18
4.8
0.7
Chate
12
1o
18.8
17.4
5.1
(a) Initial strength (A = 0) (b) Influence functions (c) Event-type clusters(d) Initial strength (A 0)
Figure 3: The learned time-decaying influences hetween every types of events ("small", "medium", "large") from Retweet dataset (a,b). The clustering structure of event type embeddings and the initial strength of the influence functions organized in the event type clusters on MIMIC dataset (c, d).
6.2 Predictive Performance
Next, examined the performance of SPRITE in predicting both the occurrence and type of the future events. To this end, we used the following real-world benchmark datasets. (1) Retweets (Zhao) jtt al. 2015), 24, 000 retweet event sequences from twitter com. The events were generated by three types of users, "small" retweeters with less than 120 followers, "medium" retweeters having number of followers between 120 and 1,363, and "large" retweeters with more than 1, 363 followers. The time of the first event in each sequence is labeled O, and the time of the subsequent events are the time spans from the first event. (2) MIMIC (Du et al., 2016), clinical visit sequences of 650 anonymous patients in a seven-year period. Each visit considered as event, and the event type the diagnosis result. We have total 75 event types. (3) SO (Du al. 2016), 6, 633 awarding event sequences in the question and answer site Stack Overflow. The users can awarded based on both their questions and answers. There are 22 types of events, corresponding to 22 awards, including "Nice Question". "Good Answer", "Guru", "Great answer". etc. We downloaded the preprocessed datasets from https:ttp/// / goog le. com/ dr ve fol de rs OOwgm//EccoUs IR1 BKV25YR10 In addition, tested the prediction accuracy on (4) SIM - synthetic data of bi-type events in Sec. 6.1
In addition to the methods mentioned in Sec. 6. we compared with two extra baselines: (1) Simple Statistics (SSTA) that use the average span between consecutive events to predict the future event time, and the most frequent event type to predict the future event type, and (2) homogencous Poisson process (PP) that assumes the event rate a time-invariant constant.
For Retweets, MIMIC and SO, we randomly split the dataset into 70% for training, 10% for validation, 20% for testing. We used each method to predict when the last event in a test sequence occurred and what the type is. We calculated the moot-mean-squar--erne classification accuracy of the type predictions. We chose the dimension of embeddings as the 4th root of the number of event types (the rule of thumb suggested by Google (TensorFlow Team 2017)).
(RMSE) of the time predictions, and

---

We repeated the experiments for 5 times with the 5-fold cross validation splitting scheme, and report the mean and standard deviation of RMSE and aceuracy. For SIM. we randomly generated 10K sequences for training. 1K for validation, and 2K for testing in each experiment. The results of all the methods are shown in Fig. 2 As we can see, SPRITE outperforms all the competing approaches in all the cases, except that in SIM, the prediction accuracy of event types is slightly worse than NeuralHP. In many cases, SPRITE outperforms SHAP, RMTPP and/or NeuralHP by a large margin, e.g. Fig. 2a, d, and f. The results show the advantage of SPRITE in predictive performance.
6.3 Pattern Discovery
Finally, we looked into the patterns discovered by our method. We first examined the learned influence functions from Retweet dataset (see ( In Fig. 3 we show the initial strength (A = 0) and the curve of the influence function for every pair of event types. We found interesting patterns. First, among retwecting events are only excitation effects, and there are no inhibition effects. Second, "small" users retwecting can hardly cause excite "large" users to retweet (the same message); the initial strength of "small - large" is 0.7 (Fig. 3d. By contrast, "large" users' retweeting can strongly incur "small" users to retweet (the initial strength of "large small" is 18.8). This is reasonable, because "large" users have a great many of followers, and most of these followers are "small" users. The result also consistent with the fact that "large" users, especially twitter celebrities, often act as hub nodes, and play a key role in information or opinion dissemination. It is interesting to see, though, "large" users have much less influence other "large" users. Third, we can see across different pairs of event types, the influence varies quite much (Fig. 3)) implying the need of flexible models for their estimation. While approximating them some exponential decays in standard HPs seems feasible, leads to much worse prediction accuracy as compared with SPRITE (see Fig. 2 and e).
Next, we examined if our method can discover hidden structures. To this end, we investigated the learned event type embeddings from MIMIC dataset. We first performed Principled Component Analysis (PCA) to project the embeddings onto plane, and then ran the k- k-means algorithm to find potential grouping structures. We used the elbow method (Ketchen and Shook 1996) to determine the cluster number. As we can within the event types. This is reasonable, because the event types are represented by the diagnosis results of the medical visits and there can be strong associations within those diagnosis results. Note that since the dataset is completely anonymized, we cannot verify the meaning of these clusters. Furthermore, we investigate the (initial) influence strengths between the event types within the same cluster and across different clusters. From Fig. B3,, we can see clear patterns. For examples, the types of events in Cluster 4 mostly excite each other, but inhibit those of Cluster O; the types of events in Cluster O nearly inhibit those across all the clusters, including themselves. We also found interesting structures of the embeddings and influence functions estimated from SO dataset. We leave the results in the Appendix.
from Fig. 3.. the (projected) embeddings reflect clear structure
7 Conclusion
We have developed SPRITE, self-adaptable point process that not only can explicitly estimate the influence type and strength between every pair of events, but also is flexible enough to capture a variety of influence time decays from data.
Acknowledgments
This work has been supported by NSF CAREER Award IIS-2046295 and NSF IIS- 1619287.
References
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016). Tensorflow: system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ((OSDI) 16), pages 265-283.
Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint M.Ki::1409.047.
10

---

Blundell, C., Beck, J., and Heller, K. A. (2012). Modelling reciprocating relationships with hawkes processes. In Advances in Neural Information Processing Systems, pages 2600-2608.
Daley, D. J. and Vere-Jones, D. (2007). An introduction to the theory of point processes: volume II: general theory and structure. Springer Science & Business Media.
Du, N., Dai, H., Trivedi, R., Upadhyay, U., RodrzzRod M., and Song, L. (2016). Recurrent marked temporal point processes: Embedding event history vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564.
Du, N., Farajtabar, M., Ahmed, A., Smola, A. J., and Song. L. (2015). Dirichlet-hawkes processes with applications to clustering continuous-time document streams. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 219-228. ACM.
Hawkes, A. G. (1971). Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90)
He, X., Rekatsinas, T., Foulds, J., Getoor, L., and Liu, (2015). Hawkestopic: A joint model for network inference and topic modeling from text-based cascades. In International conference on machine learning, pages 871 170-
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 988:17335 1780.
Ketchen, D. and Shook, C. (1996). The application of cluster analysis in strategic management research: analysis and critique. Strategic management journal, 17(6):441-458
Lewis, W. and Shedler, G. S. (1979). Simulation of nonhomogeneous poisson processes by thinning. Naval research logistics quarterly, 26(3):403-413.
Linderman, S. and Adams, R. (2014). Discovering latent network structure in point process data. In International Conference on Machine Learning, pages 1413-1421.
Lloyd, C., Gunter, T., Osborne, M., and Roberts, S. (2015). Variational inference for gaussian process modulated poisson processes. In International Conference on Machine Learning, pages 1814-1822. PMLR.
Mei. and Eisner, M. (2017). The neural hawkes process: neurally self-modulating multivariate point process, In Advances in Neural Information Processing Systems, pages 6754-6764.
Omi, T., Aihara, K., al. (2019). Fully neural network based model for general temporal point processes. In Advances in Neural Information Processing Systems, pages 2122-2132.
Pan, 7., Wang, 7., and Zhe, S. (2020). Scalable nonparametric factorization for high-order interaction events. In International Conference on Artificial Intelligence and Statistics, pages 4325-4333 PMLR.
Rasmussen, E. and Williams, C. K. (2006). Gaussian Processes for Machine Learning, MIT Press.
Rithimäki, J. and Vehtari, A. (2010). Gaussian processes with monotonicity information. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 645-652.
Rubanova, Y., Chen, R. T. Q.. and Duvenaud, D. K. (2019). Latent ordinary differential equations for reegularly-sample. time series. In Wallach, H. Larochelle, H.. Beygelzimer, A., d' d'Alché-Bal F., Fox, E., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
Schein, A., Linderman, S., Zhou, M., Blei, D., and Wallach, H. (2019). Poisson-randomized gamma dynamical systems. In Advances in Neural Information Processing Systems, pages 782-793.
11

---

Schein, A., Paisley, J., Blei, D. M., and Wallach, H. (2015). Bayesian poisson tensor factorization for inferring multilateral relations from sparse dyadic event counts. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1045-1054. ACM.
Schein, A., Zhou, M., Blei, D. M., and Wallach, H. (2016). Bayesian poisson tucker decomposition for learning the structure of international relations. In Proceedings of the 33rd International Conference on International Conference on Machine Learning Volume 48, ICML' 16, pages 2810-2819. JMLR.org.
Shehur, O., Bilovs, M., and Günnemann, S. (2019). Intensity- learning of temporal point processes. In International Conference Learning Representations.
Tan, X., Naqvi, S. A., Qi, Y., Heller, A., and Rao, (2016). Content-based modeling of reciprocal relationships using hawkes and gaussian processes. In UAL
TensorFlowTeam (2017).
Introducing tensorflow feature columns. fewtoo---faauuure-uuusss columns.htm..
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. (2017). Attention is all you need. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnetr, R., editors, Advances in Neural Information Processing Systems, volume 30, pages 5998-6008. Curran Associates, Inc.
Wainwright, M. J. and Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Now Publishers Inc.
Wang, Y., Ye, X., Zha, H., and Song, L. (2017). Predicting user activity level in point processes with mass transport equation. In Advances Neural Information Processing Systems, pages 1644-1654.
Wang, Z., Chu, X., and Zhe, S. (2020). Self-modulating nonparametric event-tensor factorization. In International Conference on Machine Learning, pages 9857-9867. PMLR.
Xu, H., Farajtabar, M., and Zha, H. (2016). Learning granger causality for hawkes processes. In International Conference on Machine Learning, pages 1717-1726.
Xu, H., Luo, D., Chen, X., and Carin, L. (2018). Benefits from superposed hawkes processes. In International Conference on Artificial Intelligence and Statistics, pages 623-631. PMLR.
Xu, H., Luo, D., and Zha, H. (2017). Learning hawkes processes from short doubly-censored event sequences. In International Conference on Machine Learning, pages 3831-3840.
Xu, H. and Zha, H. (2017). A dirichlet mixture model of hawkes processes for event sequence clustering. In Advances Neural Information Processing Systems, pages 1354-1363.
Yang, J., Rao, V. A., and Neville, J. (2017a). Decoupling homophily and reciprocity with latent space network models. In UAI.
Yang. Y., Etesami. J., He, N., and Kiyavash, N. (2017b). Online learning for multivariate hawkes processes. In Advances in Neural Information Processing Systems, pages 4937-4946.
Zhang, Q., Lipani, A., Kimap, O., and Yilmaz, E. (2020a). Self-attentive hawkes process. In International Conference on Machine Learning, pages 1 11183-11193 PMIR.
Zhang, R., Walder, C., and Rizoiu, M.-A. (2020b). Variational inference for sparse gaussian process modulated hawkes process. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 34, pages 6803-6810.
Zhao, Q., Erdogdu, M. A., He, H. Y., Rajaraman, A., and Leskovec, (2015). Seismic: A self- exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1513-1522. ACM.
12

---

Zhe, S. and Du, Y. (2018). Stochastic nonparametric event-tensor decomposition. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 6857-6867,
Zhou, F. Li, Z., Fan, X., Wang, Y., Sowmya, A., and Chen, F. (2020). Efficient inference for nonparametric hawkes processes using auxiliary latent variables. Journal of Machine Learning Research, 21(241):1-31.
Zhou, F., Zhang, Y., and Zhu, J. (2021). Efficient inference flexible interaction in spiking- neuron networks. In Proceedings of the International Conference Learning Representations (ICLR).
Zhou, K., Zha, H., and Song, L. (2013). Learning triggering kernels for multi-dimensional hawkes processes. In International Conference on Machine Learning, pages 1301-1309.
Zuo, S., Jiang, H.. Li, Z., Zhao, T., and Zha, H. (2020). Transformer hawkes process. In International Conference on Machine Learning, pages 1 11692-11702 PMLR.
13


Reviews:
[{"review_id": "e-ToKSjWTLM", "reviewer": "Reviewer_T9br", "summary": "This paper proposes a multi-variate self-adaptable point process with nonparametric time decays. First, to flexibly represent arbitrary types of influences between events, they introduce an embedding to represent each event type, and model the influence between any two events as an unknown function of the event type embeddings and time span. Second, they also propose a general construction that covers all possible time decaying functions by assigning Gaussian process (GP) priors over the free latent functions and using Gauss Legendra quadrature to derive the integral in the construction. This gives their model more representative power while being computationally efficient. For efficient inference, they also use weight-space augmentation of GPs to prevent computing huge covariances matrices, and neural network feature mapping to enhance the learning capacity. Lastly, a scalable stochastic variational learning algorithm is developed by taking advantage of the re-reparameterization trick.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper is overall well-written. The problem motivation is clearly described and an efficient approach is proposed accordingly.\n- Results on both synthetic and real world data are promising. The interpretable results are interesting.\n- The proposed learning algorithm is efficient to carry out, which is important for point process models.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- The novelty of the method itself is limited and introducing Gaussian prior to the hidden embedding seems incremental. \n- The representative power of this method is still limited comparing to other neural network based methods, where only excitation or inhibition effects are considered. However, the influences between events can be very complicated in some real applications. For example, an event can be inhibited in some time period, whereas it can also be excited in another time period. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "am5ERF7w16H", "reviewer": "Reviewer_VtoS", "summary": "The paper proposes a semi-parametric class of point processes which is intensity-based while using kernel-based,non-parametric functions to model the effects between types of events in the intensity function. This reduces the number of the learnable parameters hence making the model applicable to datasets with a large number of event types.  In particular, it \n\n(1) formulates in Lemma 3.1 the property of such kernel functions that guarantee a time-decaying effect of the influence functions when there is present either an excitatory effect or an inhibitory effect \n\n(2) proposes a class of kernel functions in corollary 3.1.1 . that respects the property in (1)\n\n(3) suggests a training algorithm that leverages a bayesian linear regression model for constructing the kernel functions and stochastic variational inference on batches of types and event sequences for accelerating inference.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, I think the paper is well-written while it addresses an important problem currently not very -well addressed by current research. Its novelty lies in the design of kernel functions that exhibit time-decaying effect. The experimental results are compelling. However, some points in the model/ training algorithm are not very clear while the experiments should be extended to further demonstrate the model. In particular,\n\n(1) the model uses a soft-plus transformation to guarantee the positivity of the intensity function while maintaining the inhibitory effects imposed by the term in Eq 4. My impression is that this transformation renders the integral in the objective function of Eq 9 not computable in closed form. How do the authors address this (monte carlo etc?)\n\n(2) it is not very clear to me why the authors do not consider a posterior for the type embdeddings u in the final objective L after Eq (10)\n\n(3) I haven't seen partition on event-based datasets with respect to the types of events when performing variational inference? are there convergence guarantees? i think it would help a lot, if convergence, was at least demonstrated with learning curves for some parameters of the model.\n\n(4) It would be interesting if an ablation study was provided which shows predictive performance for different dimensionality of type embeddings R, given that it exhibits cubic time complexity?\n\n(5) I believe that the synthetic experiments should be extended so that synthetic event sequences are generated by a Hawkes process, neural point process etc and show how well the proposed model can recover existing point process models.\n\n(6) the real world dataset experiments would become stronger if the authors also compare with:\n\n[1]Shchur, O., Biloš, M. and Günnemann, S., 2019. Intensity-free learning of temporal point processes. arXiv preprint arXiv:1909.12127.\n\n[2] Mehrasa, N., Jyothi, A.A., Durand, T., He, J., Sigal, L. and Mori, G., 2019. A variational auto-encoder model for stochastic point processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3165-3174).", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "WjguR7KZdlE", "reviewer": "Reviewer_jsqH", "summary": "This work proposes a new marked temporal point process model that manages to leverage neural networks in such a way to recover and identify the influence that certain events have on future ones. This is accomplished by modeling the overall intensity function of the model as a summation of individual decaying effects that prior events have on the rate of occurrence for future events.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "There is a definite lack of interpretability in the methods proposed by the neural point process literature. Point process models are very powerful predictive tools; however, many of the desired use cases for them require interpreting why certain events are more likely to occur given a particular history (e.g., within healthcare). From my perspective, this work makes a solid step towards addressing this issue. The proposal is well founded, has a clear motivation, and adequately demonstrates the effectiveness via empirical results. \n\n As for clarity, the main thrust of the paper is very clear. I do think that from about lines 180 to 200 and the entire synthetic data experiment section is a little dense and could stand to be edited for readability (I do understand that this is most likely a byproduct of the submission page limit).\n\nI am curious about a few things that weren't directly answered in the paper:\n1. What is the added effect of treating the embeddings probabilistically? Do the results differ significantly if, for instance, all of the parameters are learned deterministically without any priors? I would imagine that the results of at least 6.3 would be affected, but I wonder if this is actually the case empirically. Note that I do think the proposal is significant due to the formulation of the intensity function alone. \n2. What sort of structure do $\\phi_g$ and $\\phi_\\eta$ take on? I understand that they are neural networks, but I cannot find any details about their hyper parameters for the experiments (i.e., the number of layers they have, activation function, initial values, etc.).\n3. For the experiments using real data, were the sequences truncated at all or were the entire sequences modeled during training and inference?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think that there could be a bit more discussion surrounding the societal impact of this model. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "Ky7wBB9DFuO", "reviewer": "Reviewer_k7zZ", "summary": "The paper proposes a new algorithm to learn point process models. The proposed algorithm employs non-parametric time decay functions, which are more flexible than exciting approaches. Efficient estimation procedures are drafted. Empirical results backed up the authors claims.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nThe paper proposes a new solution to an important question.\n\n\nQuality:\nMy main concern is that the proposed method is very complicated. The authors claim that existing methods make over-simplified assumptions, and propose a very complicated formulation of the time decay function (Eq. 7). The authors further place a GP prior on the formulation and propose to optimize the derived ELBO. I am not sure whether it’s worth the effort than the relatively simple approaches (e.g., RMTPP), given that the performance gain is not quite significant in some cases (Figure 2, prediction accuracy of Retweet and SO).\n\nClarity:\nPresentation of the paper needs to improve.\n\n1. I suggest to move line 146-147 directly after Eq. 4. It took me a while to realize that $\\tilde{\\lambda}$ is not the intensity function, which is confusing.\n\n2. I suggest to move line 204-211 right after Eq. 7. Section 4.2 has nothing to do with integral estimation.\n\n3. Some notations are not defined, e.g., $\\Delta$ in Eq. 6.\n\n4. In Lemma 3.1, any particular reasons that you choose $f(\\cdot)=\\mathrm{tanh}(\\cdot)$ and $z(\\cdot)$ to be the integral function? Are there other possible choices? The current choices seem arbitrary to me, so please provide more intuition. \n\nSignificance:\nThe paper studies an important problem and provides insights that could benefit future works.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new marked temporal point process model for continuous-time event sequences. While the standard neural stochastic point process models (e.g., Neural Hawkes Processes) parameterized the time-varying intensity functions using recurrent neural networks,  these models are often black boxes and don't allow to estimate the influence (inhibition or excitation) that event types have on future ones. In contrast, the proposed work uses interpretable Gaussian processes and achieves better performance while maintaining interpretability and scalability. In this sense, the approach achieves to both flexibly and transparently estimate various decaying influences between every pair of events. 

With few exceptions on the model/ training algorithm that could be presented more clearly, the paper is overall well-written and mathematically sound. Despite its sophisticated inference scheme, it provides a valuable contribution to machine learning and statistics. 

Please run a grammar check over the paper for the final version since many small mistakes are present. 
Decision: Accept (Poster)