Neural Spectral Marked Point Processes
Authors: Shixiang Zhu
Haoyun Wang
Zheng Dong
Xiuyuan Cheng
Yao Xie
Venue: ICLR 2022 Poster (2022)

Abstract: Self- and mutually-exciting point processes are popular models in machine learning and statistics for dependent discrete event data. To date, most existing models assume stationary kernels (including the classical Hawkes processes) and simple parametric models. Modern applications with complex event data require more general point process models that can incorporate contextual information of the events, called marks, besides the temporal and location information. Moreover, such applications often require non-stationary models to capture more complex spatio-temporal dependence. To tackle these challenges, a key question is to devise a versatile influence kernel in the point process model. In this paper, we introduce a novel and general neural network-based non-stationary influence kernel with high expressiveness for handling complex discrete events data while providing theoretical performance guarantees. We demonstrate the superior performance of our proposed method compared with the state-of-the-art on synthetic and real data.

Published as a conference paper at ICLR 2022
NEURAL SPECTRAL MARKED POINT PROCESSES
Shixiang Zhu', Haoyun Wang', Zheng Dong', Xiuyuan Cheng", Yao Xie
i
Georgia Institute of Technology
* Duke University
yao. xieeisye. wwt.eliiyeggaeehhh.oogom gatech. edu
ABSTRACT
Self- and mutually-exciting point processes are popular models in machine learning and statistics for dependent discrete event data. To date, most existing models assume stationary kernels (including the classical Hawkes processes) and simple parametric models. Modern applications with complex event data require more general point process models that can incorporate contextual information of the events, called marks, besides the temporal and location information. Moreover, such applications often require non-stationary models to capture more complex spatio-temporal dependence, To tackle these challenges, key question is to devise versatile influence kernel in the point process model. In this paper, we introduce a novel and general neural network- high expressiveness for handling complex discrete events data while providing theoretical performance guarantees. We demonstrate the superior performance of our proposed method compared with the sate--oftthearr synthetic and real data.
-ta-saationary influence kernel with
INTRODUCTION
Event sequence data are ubiquitous in our daily life, ranging from traffic incidents, 911 calls, social media posts, earthquake catalog data, and COVID-19 data (see, e.g., egg./Bertozzz/ et al. (2020)) Such data consist of sequence of events indicating when and where each event occurred, with additional descriptive information (called marks) about the event (such as category, volume, free-text). The distribution of events is of scientific and practical interest, both for prediction purposes and for inferring events' underlying generative mechanism.
A popular framework for modeling events is point processes (Daley & Vere-Jones 2008), which can be continuous over time and the space of marks. An important aspect of this model capturing the event's 's triggering effect on its subsequent events. Since the distribution of point processes is completely specified by the conditional intensity function (the occurrence rate of events conditioning on the history), such triggering effect has been captured by an influence kernel function embedded in the conditional intensity. In statistical literature, the kernel function usually assumes a parametric form. For example, the original work by Hawkes (Hawkes 1971) considers an exponential decaying influence function over time, and the seminal work (Ogatal 1998) introduces epidemic-tyy aftershock sequence (ETAS) model, which considers influence function that exponentially decays over space and time. With the increasing complexity of modern applications, there has been much recent effort in developing recurrent neural network (RNN)-based point processes, leveraging the rich representation power of RNNs (Du et al., 2016 Mei & Eisner 2017 Xiao et al. 2017b).
However, there are several limitations of existing RNN-based models. First, such models typically do not consider the kernel function (Du et al. 2016 Li al., 2018) Mei & Eisner 2017| Upadhyay al. 2018 Xiao et al.] [2017a b): thus, the RNN approach does not enjoy the interpretability of the kernel function based models. Second, the popular RNN models such as Long Short-Term Memory (LSTM) (Hochreiter Schmidhuber 1997) still implicitly discounts the influence of events over time (due to their recursive structure) (Vaswani et al.) 2017]] Zhu et al. 2021d). Such assumptions may not hold in many real-world applications. Take the earthquake catalog as an example, which is typical type of discrete event data; most aftershocks occur along the fault plane other faults within the volume affected by the mainshock's 's strain (Zhu et al.) 2021b). This means that different regions may be correlated to their surrounding area differently according their geological structure, which creates a complex non-stationary spatial profile that we would like to capture through the model. Third, majority of the existing works mainly foeus me-dimensional temporal point

---

Published as a conference paper at ICLR 2022
(-9)
200
001
002
0.05
-0.05
400
-0.1
-0.1
500
1000
000
1000
500
2-220
1-100
2.08
800
2.08
-0.05
0.06
1200
800
1000
Figure 1: An example of non-stationary influence kernel k(t', () of event time and future time t >> t'.
processes. Although there are works on marked point processes (uu et al. 2016 Mei & Fisner 2017;
Reinhart 2018), they are primarily based simplifying assumptions that the marks are conditionally independent of the event's 's time and location, which is equivalent to assuming the kernel is separable; these assumptions may fail to capture some complex non-stationary, time- and location-dependent triggering effects for various types of events, as observed for many real-world applications (see, e.g..
(Bertozzi et al. 2020
Contribution. In this paper, we present a novel general non-stationary point process model based on neural networks, referred to the neural spectral marked point process (NSMPP). The key component is a new powerful representation of the kernel function using neural networks, which enables to go beyond stationarity (and thus go beyond standard Hawkes processes) and has the capacity to model high- high-dimensional marks. Vigure[givee an example of non-stationary influence kernel that measures the influence of the past events to the future time t. The premise of the model design is that the conditional intensity function uniquely specifies the distribution of the point process, and the most important component in the intensity function the influence kernel. In summary, the novelty of our approach includes the following:
The kernel function is represented by a spectral decomposition of the influence kernel with a finite-rank truncation in practice. Such a kernel representation will enable us to capture the most general non-stationary process as well as high-dimensional marks. The model also allows the distribution of marks depend time, which is drastically different from the separable kernels considered in the existing literature (Reinhart (2018).
The spectral decomposition of asymmetric influence kernel consists of sum of the product of feature maps, which can be parameterized by neural networks. This enable us to harvest the powerful expressiveness and scalability to high-dimensional input of neural networks for complicated tasks involving discrete events data.
We establish theoretical guarantees of the maximum likelihood estimate for the true kernel function based on functional variational analysis and mnite-dimensional asymptotic analysis, which shed light on theoretical understanding of neural network- kernel functions.
Using synthetic and real data (seismic and police data), we demonstrate the superior performance of our proposed method complex situations; the performance gain is particularly outstanding for cases involving non-stationary point processes.
1998) assume parametric forms of the intensity functions. Such methods enjoy good interpretability and are efficient to estimate. However, classical parametric models are not expressive enough capture the events' dynamics in modern applications.
Related work. Seminal works in point processes modeling (Ogata 1988
Recent research interests aim improve the expressive power of point process models, where recurrent neural networks (RNNs)- based structure is introduced to represent the conditional intensity function (Du et al., 2016 Mei & Eisner, 2017: Xiao al.) 20176). However, most of these works either explicitly or implicitly specify the inter-event dependence in limited form with restrained representative power. For example, [Du et al. (2016) expresses the influence two consecutive events in form of exp ( }. which is exponential function with respect to the length of the time interval f - t; with weights w; Mei & Eisner (2017) enhances the expressiveness of the model and represents the entire history using the hidden state of an LSTM, which still implicitly assumes the influence of the history decays over time due to the recurrent structure of LSTM.
Another line of research uses neural networks to directly model dependence of sequential events without specifying the conditional intensity function explicitly (Li et al.) 20188] Xiao et al. (017a). Some studies consider non-stationary influence kernel using neural networks in spatio-temporal
2

---

Published as a conference paper at ICLR 2022
point processes (huu al. 2021ba a.. Recent work (Omi et al. 2019) also uses a neural network to parameterize the hazard function, the derivative of which gives the conditional intensity function. However, the above approaches either capture the temporal dependence assume the temporal and mark dependence are separable rather than jointly accounting for marked-temporal dependence.
Recently, attention models have become popular in computer vision and sequential data modeling
(Britz et al., 2017 Luong et al.] 2013 Vaswani et al. 2017). This motivates works including
Zhang et (2019): Zhu al. (202lc de Zuo et (2020) model the conditional intensity of point processes using the attention mechanism and characterize the inter-event dependence by
score function. The attention mechanism has proven be more flexible in capturing long- gaagge dependence regardless of how far upart two events are separated and greatly enhances the performance
in practice. However, the main limitation of Zhang et al. (2019): /Zuo et al. al.] (2020) is that they rely on conventional score function dot-product between linear mappings of events, which is still limited representing non-linear dependence between events for some applications. /Zhu et al. (2019) 2021e d) used more flexible and general Fourier kernel substitution for the dot-product score; however, the expressive power the proposed Fourier kernel still limited, and the spectrum the Fourier basis is represented by generative neural network, which is difficult to learn in some cases (Arjovsky & Bottou| 2017).
There also works considering point processes with non-stationary intensities, Chen & Hall (2013) proposed tire-na background intensity for point process, while we focus on the non-stationary triggering kernel depicting complex events dependency. |Remes al. (2017 2018) studied non- stationary kernels combined with Gaussian processes, assuming specific structures of the kernels in the Fourier domain. Such kernels are more restricted than ours since the nature of Gaussian processes requires that the kernel is positive semidefinite.
METHOD
2.1 BACKGROUND: MARKED TEMPORAL POINT PROCESS
Marked temporal point processes (MTPPs) |Reinhart 2018) consist of a sequence of events over time. Each event is associated with a (possibly multi-dimensiona) mark that contains detailed information of the event, such as location, nodal information (if the observations are over networks, such as sensor or social networks) categorical data, and contextual information (such as token, image, and text descriptions). Let T > 0 be a fixed time-horizon, and .M C IR be the space of marks. We denote the space of observation as = [0, T) x M and data point in the discrete event sequence as
(L, I LE (0,
m E M,
(1)
where is the event time and m represents the mark. Let N₂ be the number of events up to time t < T
(which is random), and H4 == (2 IN,) denote historical events. Let N be the counting measure on X' i.e., for any measurable S C X, N(S) |Hr n S). For any function f X + R, the integral with respect to the counting measure is defined
[(o))N((a) = Σ f(x)).
Z.EHTnS
The events' distribution in MTPPs can be characterized via the conditional intensity function X(x), which is defined to be the conditional probability of observing an event in the marked temporal space X' given the events' history He(z), that is,
E (I)())))) A(x)dr.
Above, t(z) extracts the occurrence time of event I, and we omit the dependence on Hi(s)) in the notation of A(x) for simplicity.
As self- and mutual-exciting point processes, Hawkes processes (awwes 1971 have been widely used capture the mutual excitation dynamics among temporal events. The model assumes that influences from past events are linearly additive towards the current event. The conditional intensity function for self-exciting point process takes the form of
[zk] = + Σ k(0',z),
(3)
x'E
3

---

Published as a conference paper at ICLR 2022
where 0 stands for the background intensity, and the so-called "influence kernel" k X R crucial in capturing the influence of past events on the likelihood event occurrence at the current time. Here we use the notation [K] to stress the dependence of the conditional intensity function the kernel function k(r', x). Written in the form of the integral over counting measure, we have that
{[K[(<) - 1
x))((())
where X₂ is the subset of X' with the first component smaller than 4.
The most commonly made assumption in the literature that the process is stationary, where the influence of the past events is shift-invariaa such that k(z', x)) f(x z²) for influence function f Idd - : a common influence function in one-dimensional cases f(t) = a exp( -Bt). where 3 controls the decay rate and current work aims at going beyond stationary point processes, which enables us to better capture the heterogeneity in the events' influence across the spatial-temporal space, which naturally arises in many applications.
controls the magnitude of the influence of event. The
2.2 NEURAL SPECTRAL REPRESENTATION FOR INFLUENCE KERNEL.
We propose represent more general non-stationary influence kernels in the conditional intensity function (xx) specified
Kernel representation. The main idea of the proposed model is represent the influence kernel using a general finite-rank decomposition
k(x' = Σ v (r
1s 22
(5)
where
; R. 6 r=1....,
two sets of feature functions in some smooth functional space (x),,, and v, is the corresponding weight- -or or "spectrum". This representation is motivated by the spectral decomposition of general kernel function. While functional spectral decomposition is usually infinitely dimensional, for practical considerations, we truncate the "spectrum" and only consider a finite rank representation. Note that while we view (5) similar to a spectral decomposition, can be better understood as feature maps in kernel representation, and, particularly, we do not need the feature functions v and o, to orthogonal.
The decomposition represents the kemeel function using three parts: two sets of (normalized) feature functions and the energy spectrum-the spectrum v, plays the role of weights to combine the feature maps. In the learning process, we can train the feature functions (typically neural networks) and the weights separately; since learning the normalized feature maps tend to more numerically stable. The proposed form of kemee is not necessarily positive nemi-definite, and even not symmetrie.
Lcar rable Spectrum
Kamnel Vale
Spuce
Data Space
X
My (X)
(x
Point
Hiddon Erboadiig
Enbeedding
xEX
9 or
(a) Kernel architecture
(b) Network structure
Figure 2: The architecture of the proposed non-stationary neural spectral kernel. hidden emhedding will first summarize data point; then, the emhedding will mapped different features via multi-branch neural networks. The kernel value is calculated by summing up the products between two sets of learned features, { and { weighted by the spectrum [v+). The spectrum, feature functions, and hidden embeddings are jointly learned from data. (b) The structure of the multi-branch neural network. There
shared layers with nodes per layer that generate the hidden embedding; denotes the dimension the input data point; p denotes the dimension of the middle layer that generates the feature. Further specifications of neural networks will be provided in the experiments in Section
4

---

Published as a conference paper at ICLR 2022
Note that the spectral representation allow for completely general kemel that can be used for non-stationary processes (since our influence function does not impose a shift-invariant structure). Moreover, the spectral representation allows us to go beyond monotone decay or parametric form as commonly assumed in the prior literature.
Neural network feature function. As one of the most salient features of our method, feature functions (ør, 1/, are represented using neural networks, leveraging their known universal approxi- mation power. First, the input data point X will be projected to a hidden embedding space via a multi-layer shared network, aiming to extract key information of the input data. Here we have adopted Softplus non-linear activation in this feature extraction sub-network, while other options may be possible. Next, the hidden embedding will be mapped to the different features {x)( through R branched sub-networks. To ensure the output feature is constrained the scaled sigmoidal function as the activation of the output layer, i.e., f(x) s/(1 + exp(-2)), where s a constant to enable rescaling of the output to proper range (in our setting, we set to be 100). The overall architecture of our kernel formulation and the structure of the multi-branch neural network are summarized in Figure 2 (a) and (b), respectively. Further specifications of neural networks will be provided in the experiments in Section 4
bounded space, we choose
2.3 MAXIMUM LIKELIHOOD MODEL RECOVERY
An essential task in learning the neural point process model is to estimate the influence kernel function for the point process. The reason is two-fold: First, the kernel function is the most important component in representing the point process. Second, in practice, the influence kernel offers clear interpretations, such as "how events at a particular time and location will influence future events at a given time and location." Such interpretation is essential for predicting using event data- one of the main applications for point process models.
To estimate the influence kernel function for point process models, we consider popular approach through maximum likelihood estimation (MLE). Formally, the optimal kernel can be found by solving the following optimization problem given M sequences of training event sequences over the time horizon [0, T): (rish 1,..., N,. 1,...,,, M:
( log log (I()),) - J > A11((())dx))
max REK
M Σ
(6)
where A and N, denote the conditional intensity and counting measure associated with the j-th trajectory, respectively, and K. C (X' x X)) represents the family of regular kernel functions induced by the feature function family F and the finite-rank decomposition
Learning algorithm. To solve MLE 6 when the kernel function k is parameterized by neural networks, we use a stochastic gradient as summarized in Algorithm| [ (Appendix (A). Note that to calculate the ligelikel function e in (6). need to evaluate integral (the second term), which does not have a closed-form expression. We approximate the integral numerically by Monte Carlo integration drawing samples and take the average, described in Algorithm 2 (Appendix(A)
THEORETICAL GUARANTEES OF MLE
We first consider the MLE as a functional optimization problem, since when using neural networks to approximate the kernel function, we are interested in such functional approximation results. We show that the expected log-likelihood reaches its maximum at the true kernel with the second-order derivative bounded away from zero, and thus the true kernel function is identifiable by solving the MLE problem under some regularity conditions.
Consider the log- ligeelkkioood of point processes over a family of kernel functions KC which contains K., the family induced by feature functions in FF and non-negative spectrum (v, ] Note that X may go beyond the finite-rank decomposition in (5) Later throughout the theoretical analysis, omit the spectrum as they can be absorbed into the feature functions. The details are discussed in Remark 3.2
Assumption 3.1. (AI) The kernel function family K C C x X) which uniformly bounded, and the true kernel K* E K: (A2) There exist C1, O2 positive constants, such that for any k E K, a.s. for event data trajectory, e1 S (x) & C22 Vx E X.
5

---

Published as a conference paper at ICLR 2022
Note that, apart from (A2), we only need kernel functions to be measurable for theoretical analysis which is guaranteed by (A1). In practice, the proposed neural network parametrization leads to a continuous and smooth (low-rank) kernel function, which induces non-singular X.
We have the following lemma, which shows that when the log- log-likelihood function has a local perturbation around the true kernel function, there is going be decrease in the log-like likeliihoood function value in expectation. Note we assume that k" lies in (or can be well-approximated by) the family of function class K. thanks to the well-known universal approximation power of neural networks.
Lemma 3.1 (Local perturbation of likelihood function around the true kernel function). Under Assumption 3.1 for any & K. and 8k: = k k², we have
2
Σ
M (x)
/
dd, dx (2)
+
M
(oN)(t))
(7)
M
20
j=1
x
j=1
where
822(2) = / 8k(4', 22)N)((z').
(8)
Xe(e)
The implication of Lemma 3.1 is the following. Note that in difference in the likelihood function caused by a small perturbation around the true kernel function, two terms: the first term in ( is martingale integral (since the conditional expectation of (dI) (z))\>,k<*
have nicely decomposed the
dx) is zero, Vj), and the second term is the integral of a quadratic term against the counting measure. The expectation of the first term is zero due to the property of the martingale process. For the second term, per j.
J (6A,(1))) ze
(( (OAs())) 2 2a (8))(x))
and thus perturbation of the intensity function which corresponds to the second (quadratic) term in (7) will be reflected in the decrease of the log-likelihood Furthermore, we have the identifiability of the kernel itself. stated in the following theorem.
Theorem 3.2 (Kernel identifiability using maximum likelihood). Under Assumption 3.1 the true kernel function k- is locally identifiable in that ke" is a local minimum solution of maximum likelihood (6) in expectation.
Remark 3.1 (Function identification and neural network parametrization). Theorem 3.2 derives a result which holds for variational perturbation of the kernel function k in the possibly parametric family I. induced by the feature function family F. When F is the class of functions that a neural network can represent, the perturbation 5k is induced by the change of network parameters. It is common that in neural network models, parameter identification is difficult to search for (e.g., due to the symmetry of permuting hidden neurons); however, the neural network function identification may still hold, e.g., under the mean-field approximation (Mei et al., (2018). Thus the kernel function identification results in Theorem 3.2 important when learning kernel functions by neural networks. Remark 3.2 (Finite rank kernel representation). When assuming the parameter representation (5).
we represent the true kernel as (x',z) = Σn -r= uy (x)) For theoretical analysis purposes, without loss of generality, feature functions. Consider perturbed kernel where the feature functions are v 1h, + duú, and 6, o, K6,,, and remains to satisfy Assumption 3.1 The kernel function variation 8k: in (8) can With
assume 2/, 1, 1,... R, since they can absorbed into the
then be written as 8k(x', .²) = Σm= (ōrg) (z') (x) (1"),,, (z) + (='))
R
rank-R R representation of the kernel and smooth v/r, ør represented by neural networks, we potentially prevent overfitting that leads singular kernels, which may be a problem for the over-complete kernel family IC as in (A1). Later we show experimentally that the learned kernel is smooth and close
to the true kernel in Figure and Figure 4
We also studied the MLE for parameterized kernel Appendix D when the target feature function belongs certain parametric function class with finite number of parameters. This includes, for instance, spline functions, functions represented by Fourier basis, and neural networks, which proves that our proposed method acts a fundamental framework of finite-rank and neural network kernel representation.

---

Published as a conference paper at ICLR 2022
4 NUMERICAL EXPERIMENTS
This section presents experimental results on both synthetic and real data and compares them with several state-of- of-the-arss, including (i) standard Hawkes process with an exponentially decaying kernel
function (Hawkes) (Hawkes 197T: ; (ii) recurrent marked temporal point processes (RMTPP) (Du et al., 2016); and (iii) Neural Hawkes process (NH) (Mei & Fisner 2017). See Appendix B for a detailed review of those existing methods. To perform the experiments, we use the following
procedure: Given training data, we first estimate the kernel function to obtain k((,,) by solving the maximum likelihood problem (6) using stochastic gradient descent, as described in Section 2.3 Note that according to 22, the conditional intensity function can be treated as a prediction of the chance of having an event at a given time after observing the past events. Thus, to evaluate the prediction performance on test data, given a test trajectory, we perform online prediction by feeding the past events (in the test trajectory) into evaluating the conditional intensity function according (3), which gives the conditional probability of a future event given the past observations.
Performance metries. We consider two performance metries: (i) The performance for synthetic data is evaluated by measuring the out-of-sample grean-vverag-erron (MAE) for conditional intensity function. For synthetic data, we know the true kernel, which denoted as k" Given sequence of test data, let A[k*] be the "true" conditional intensity function defined by (3) using the true kernel
k", and let A[R] be estimated conditional intensity function for the same sequence using the estimated kernel h. Thus, {[R] can be viewed as probabilistic prediction of the events (since specifies the likelihood of an event happening given the test trajectory's 's history) when the kernel k is estimated separately from training data. The out-of-sample MAE for one test trajectory is defined fx [z)) - {]]](=) [dx. Then we average this over all test trajectories to obtain our performance measure. (ii) For real data, since we do not know the true intensity function, we report the average out-of-sample predictive log-likelihood. Given test trajectory (information contained in counting measure N(x)), the predictive likelihood for a trajectory using an estimated kernel k is given as fxx log X (ar (x)) - Sx X (z))))) The average sutple predictive log-likelihoon is obtained by average over test trajectories. This has been widely adopted as metric for real-data (Mei & Eisner 2017 Omi et al. 2019 Zhang et al. 2019; Zhu et al. 202 bld): the higher the log- oogliiel,hood, the better the model predicts. See all experiment configurations in Appendix B
Synthetic data. In the experiment, we assume the background rate is a constant (p = 1) and focus on recovering both stationary and non-stationary kernel structures. We generate four one-dimensional and three two-dimensional synthetic data sets, which are simulated by a vanilla Hawkes process and our model described in Section 2.2ifonnn2wwaaa with randomly initialized parameters. The simulated data is generated by the thinning algorithm described in AlgorithmB3 3 ( (Appendix (A), which is an accept-reject method for simulating a point process (Daley & Vere-Jones 2008 Gabriel et al. 2013). For ease of comparison, we normalize the time and mark spaces for all data sets to the range from 0 to 100. Each data set is composed of 1,000 event sequences with an averaged length of 121. We fit the models using 80% of the synthetic data set and the remaining 20% as the testing set.
Kernel recovery. One of the most important ability of our approach is to recover the true kernel. We report the kernel recovery results for one- and two-dimensional synthetic data sets, shown in Figure3Jand Figure 4 respectively. In Figure 3 we observe that our proposed model can accurately recover the true kernel for mieddmmesioonal data. In contrast, the vanilla Hawkes process with stationary kernel failed to capture such non-stationarity. We also present results of two-dimensional data in Figure 4 where a continuous one-dimensional mark is introduced to the events. The first three columns demonstrate wwo-dimensional "slices" of the -our-dimensional kernel evaluation, indicating our model is also capable of recovering complex -igh-dimensional kernel structure. We note that event points are sparsely scattered in the two-dimensional marked-temporal space, where the events' correlation is measured in high-dimensional (in our example, four-dimensional) kernel space.
Intensity prediction. We evaluate the predictive accuracy for conditional intensity {1x using estimated kernel i: from the training data. Note that (f) (x) is a probabilistic prediction since it can be viewed as the instantaneous probability of an event given the historical events. The last panel in Figure showss the predicted one-dimensional conditional intensity functions given a sequence randomly selected from the testing set. We compare our predicted x[]](z) with the one estimated by vanilla Hawkes model with stationary kernel. The result shows that the predicted conditional
7

---

Published as a conference paper at ICLR 2022
True Kernel A)
Proposed Kernel kit,
Sta tionary Kernel
Cond itinnal intenuily
3.0
Santhete Deca
4:0e-00
-20002
Time
-20e-0
6:0el 100
8:3e 30
-40e-02
Proposed
1.5
Stationary
6.0
Time 7
Figure 3: Kernel recovery results on a ine-dimensional synthetic data set. The first three panels show the true kernel that generates the data, kernel learned by our model, and kernel learned by vanilla Hawkes process, respectively. The fourth panel shows the true and predicted conditional intensity functions of test sequence.
Kemdl 2. yo. sel) (t
Kernel ki), or) =.
Kemet e/
Ceeditional aneeslyy
20w
Trus Model
10002
20w130
2de-00
46642
Kerzel 66), Con, on's -
Kereel 6.) L. nr) (
Kamal 60 au (m
Conditioal inmedyy
Leuned Nodd1
220/22
20o02
18pt
00
1.0e-0
Figure 4: Kernel recovery results two-dimensional synthetic data Two rows presents the results of
the true model and our learned model, respectively. The three columns show different snapshots kernel evaluation for each model; the last column shows their corresponding conditional intensity over marked-temporal space given test sequence. where the black dots indicate the location of the events.
True Kernel k2d,
Proposed Kernel 4:t,
Stelionary Ketrel
Conditional intensity
7540
Se47
2e500
Tiee
Synbetic Data Time f
2.
2.6 02
Prepere Seatishary
15e002
Time
15e 02
Time E
1.5xC2
5:00
10w02
16p42
10k02
50s03
5.0.3
50x03
077
Time?
Figure 5; Kernel recovery results on a one- oie-dimensional synthetic data set generated by vanilla Hawkes process with stationary exponentially decaying kernel. This experiment acts sanity check.
intensity (x) suggested by model well matches that using the true kernel (x)* In contrast, the Hawkes model only provides limited flexibility for modeling the non-stationary process. Two panels in the last column of Figure give another comparison between the true and predicted two-dimensional conditional intensities over marked-temporal space. This result confirms that our model can accurately predict the two-dimensional conditional intensity function. To validate the robustness of our proposed method, we also test our model using another one-dimensional data set generated by a vanilla Hawkes process with a stationary parametric kernel, shown in Figure S Though our model is evidently overparametrized for this data, our method is still able to recover the kernel structure, as well as predict the conditional intensity accurately, which demonstrate the adaptiveness of our model and confirm that our model can approximate the stationary kernel without incurring overfitring. More results of kernel recovery experiments can be found Appendix C
Additionally, we compare our method with three other baselines on the synthetic data sets. Figure shows the conditional intensities based on the true kernel A[A=*] (solid black lines) and predicted intensity using each method. We observe that our method obtained much better predictive perfor- mances compared to other baseline approaches. Table summarizes two performance metrics for these methods: the predictive log- ligeelold and the MAE. The result shows that method greatly outperforms other baseline approaches in both metrics. In particular, the MAE of our method is at least 90% lower than the other methods for both one-dimensional and two-dimensional data sets.
8

---

Published as a conference paper at ICLR 2022
1D Synthe:ic Data
1D Synthesic Data
2D Synthetic Data 3
True RSMPP
True HSMPO
Tue NENPP
10
1.5
Hossee RMTPP
Air, mi/IM
RMTPP
WH
1.0
1,505
0.0
Time (
Time
Time 2
Figure 6: Predicted conditional intensity using our method and other baselines for a sequence selected from test data. We aggregate the conditional intensity in mark space for case of presentation and visualize the average conditional intensity over time for two-dimenee cooddimensoonl synthetic data on the third panel.
Table 1: Performance comparison of our method and other existing methods: Hawkes, RMTPP, and NH.
CYMAD!
ID Symbhis
Symbhee
AD Syntheac
Symbenic
Sentheti'
Cantarake
Kothery (1U)
Tnantee/
20.54/2 (3.2)
-3 10 10.33
-80.5 (0.54
225.32 H8.12
33100 (209)
-55992
2015
-IK 30 NA
-132.55 NA
216 282-55
0,245 73.29
MII
-45.34 096.4/33.0 /3:42
-2 97 F0.A
-ML /0/0.44
-299.07 48.1
1M1.4/12 12.45
-220001 26.9
-80. NA
26. /N.10/NA 199.4//AA
0.244/6.1
Havikea -24.12/2.65
70.32
-61
NA
NA/NA
NA
NA.NA
00217 -001
wata's earr the en dhe avarzg podkens beg-taillbadd 10 and "Toaning repooces de fortsliiing hacc.
MAL.
prolicting conditona Limusrty Sunaaas for
Thee Arsaaandd secoads
Real-data results. Finally, we test our method on two large-scale real data sets: (1) Atlanta 911 calls-for-service data. The data set contains the 911 calls-for-service data in Atlanta from 2015 to 2017. We extract 7,831 reported robberies from the data set since robbers usually follow a particular modus operandi (M.O.). Each robbery report is associated with timestamp indicating when the robbery occurred. We consider each series of robberies as a sequence. (2) Northern California seismic data. The Northern California Earthquake Data Center (NCEDC) provides public time series data Northern California Earthquake Data Center. UC Berkeley Seismological Laboratory. Dataset (2014) that comes from broadband, short period, strong motion seismic sensors, GPS, and other geophysical sensors. We extract 16,401 seismic records with a magnitude larger than 3.0 from 1978 to 2018 in Northern California and partition the data into multiple sequences every quarter. We fit the models using 80% of the data set and the remaining 20% as the testing set.
we we report the out-of-sample predictive log- likeliieeood of each method since the ground truth is not available for real data. We can see that our model attains the highest predictive oi-liikldhood for both synthetic and real data sets. In particular, the better performance on real data and significantly higher out-of-sample predictive log-likelihood achieved by our approach than other existing methods show that the non-stationary kernel seemingly captures the nature of the real data better in these cases. This shows the merit of our approach for real data where we do not know the ground truth.
In Table I
Training/ / testing time. T we compare the running time of our model and other baselines on one-dimensional synthetic data set. The size and performance of each model are presented in Table [ The running time refers to wall clock time. We can observe that the training time of our model is similar with other neural point process models (RMTPP and NH). Hawkes process model has the minimum training time because it has only two parameters to be estimated. The testing (inference) time of our model significantly lower than the other two recurrent neural point process models.
5 DISCUSSIONS
We have presented a new non-stationary marked point process model with a general kernel represented by neural network feature maps, motivated by spectral decomposition of the kernel. The flexible kernel with expressive power enabled by neural networks can capture complex dependence across temporal, spatial, and mark spaces, which can be valuable for real-data modeling in various applications. The benefits of our approach are demonstrated by superior out-of-sample predictive log-likelihood on real data. The model can be learned efficiently with stochastic gradient descent. We also develop theoretical guarantee for maximum likelihood identifiability. Generally, model architectures may well depend on specific tasks. For example, neural networks can be based on CNN if the high dimensional markers in image- like shapes and can also be LSTM even BERT if the markers text-based. Thus the choice of the deep model architecture and optimization algorithm can be more systematically explored, especially the comparison of non-stationary neural kernels stationary ones. Understanding and characterizing what kernels can be learned through such an approach is left for future study. Also, when extending to nigh-dimensional mark space, more efficient algorithm is needed for the computation of the log-likelihood.
9

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENT
The work is supported by National Science Foundation (NSF) DMS-2134037, The works of S.Z., H.W., and Y.X. are supported by NSF CAREER Award, CCF CCF- DMS-1830210, DMS- 1938106. The work of X.C. is partially supported by the Alfred P. Sloan Foundation.
REFERENCES
Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint 171.0:/701.04662, 2017.
Andrea L Bertozzi, Elisa Franco, George Mohler, Martin B Short, and Daniel Sledge. The challenges of modeling and forecasting the spread of covid-19. Proceedings of the National Academy of Sciences, 11772)):677221667353 2020.
Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoe Le. Massive exploration of neural machine translation architectures. arXiv preprint arXiv: 1703.03966 2017.
Feng Chen and Peter Hall. Inference for nonstationary self-exciting point process with an application in ultra-high frequency financial data modeling. Journal Applied Probabiliiy, 5064):000-100444 December 2013. doi: 10. 10.
URL https::tps...m........ http://ooiooooo http///..rrg..11111/////////
1389370096
J. Daley and D. Vere-Jones, An introduction the theory of point processes. Vol. II. Probability and its Applications (New York). Springer, New York, second edition, 2008. ISBN 978-0-387-21337-8, General theory and structure.
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD '16, pp. 5555-1564, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 978 1784442342
doi: 10. 144//293967..2999877
Edith Gabriel, Barry Rowlingson, and Peter Diggle. stpp: An package for plotting, simulating and analyzing spatio-temporal point patterns. Journal of Statistical Software, 53:1-29, 04 2013. doi: 10. 10.18637/jss.v053
Alan G. Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1) 90, 04 1971. ISSN 0006-3444. doi: 10.1093/biomet//5..11 htl.https::/dd
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neurul Comput., 9(8): doi: 10.1 (w9.://c..999.......... URL
1735-1780, November 1997. ISSN 0899-7667.
https://don org. www.11166cccooo 1997. 11.99.9.8.111 9978.1735
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du, Yao Xie, and Le Song. Learning temporal point processes via reinforcement learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS '18, pp- 10804-10814, Red Hook, NY, USA, 2018. Curran Associates Inc
Minh-Thang Luong, Hicu Pham, and Christopher D Manning. Effective approaches to attention-based neural machine translation. arXiv preprint / 2015.
Hongyuan Mei and Jason M Eisner. The neural hawkes process: A neurally self self-modulating multivariate point process. In Advances in Neural Information Processing Systems 30, pp. 6754- 6764. Curran Associates, Inc., 2017.
Song Mei, Andrea Montanari, and Phan-Minh Nguyen. A mean field view of the landscape of two- layer neural networks. Proceedings of the National Academy of Sciences, E15633) 2018.
10

---

Published as a conference paper at ICLR 2022
Northern California Earthquake Data Center. UC Berkeley Seismological Laboratory. Dataset.
NCEDC, 2014. BRR//tttss//cceee..
Yosihiko Ogata. Statistical models for earthquake occurrences and residual analysis for point
processes. Journal of the American Statistical association, 83(401):9-27, 1988.
Yosihiko Ogata. Space-time point-process models for earthquake occurrences. Annals of the Institute of Statistical Mathematics, 50(2):379-40)
1998.
Takahiro Omi, naonori ueda, and Kazuyuki Aihara. Fully neural network based model for general temporal point processes. In Advances in Neural Information Processing Systems 32, pp. 2120- 2129. Curran Associates, Inc., 2019.
Alex Reinhart. A review of self-exciting spatio-temporal point processes and their applications. Statistical Science, 33(3):299-318, 2018.
Sami Remes, Markus Heinonen, and Samuel Kaski. Non-stationary spectral kemels. arXiv preprint arXiv:1705.088: 2017.
Sami Remes, Markus Heinonen, and Samuel Kaski. Neural non-stationary spectral kemnl.. arXiv preprint arXiv: 2018
Utkarsh Upadhyay, Abir De, and Manuel Gomez Rodriguez. Deep reinforcement learning of marked temporal point processes. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gareett (eds.), Advances in Neural Information Processing Systems 31, pp. 3168-3178. Curran Associates, Inc., 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems 30, pp. 5998 6008. Curran Associates, Inc., 2017.
Halbert White. Maximum likelihood estimation of misspecified models. Econometrica: Journal of the econometric society, pp. 1-25, 1982.
Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Le Song, and Hongyuan Zha. Wasserstein learning of deep generative point process models. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS '17, pp. 3250-3259, Red Hook, NY,
USA, 2017a. Curran Associates Inc. ISBN 18860964.
Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, and Stephen M. Chu. Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the Thirty- AAAI Conference Artificial Intelligence, AAAI 17, pp. 1597-1603, AAAI Press, 2017b.
Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive hawkes processes, 2019.
Shixiang Zhu, Henry Shaowu Yuchi, Minghe Zhang, and Yao Xie. Sequential adversarial anomaly detection for one-class event data. arXiv preprint arXiv: 1rXiv:1910.09161 2019.
Shixiang Zhu, Ruyi Ding, Minghe Zhang, Pascal Van Hentenryck, and Yao Xie. Spatio-temporal point processes with attention for traffic congestion event modeling. IEEE Transactions on Intelligent Transportation Systems, 2021a.
Shixiang Zhu, Shuang Li, Zhigang Peng, and Yao Xie. Imitation learning of neural spatio-tempont point processes. IEEE Transactions on Knowledge and Data Engineering, pp. 1-1, 2021b. doi: 10.1 2..1109/KKEE.0221..00 300547877
Shixiang Zhu, Henry Shaowu Yuchi, Minghe Zhang, and Yao Xie. Sequential adversarial anomaly detection with deep fourier kernel. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3345-3349. IEEF, 2021c
Shixiang Zhu, Minghe Zhang, Ruyi Ding, and Yao Xie. Deep fourier kernel for self-attentive point processes. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference Artificial Intelligence and Statistics, volume 130 of Proceedings Machine Learning Research, pp. 856 864. PMLR, 13-15 Apr 2021d. URL http: httt:::/ppocceeiinin m 11r.press/v110/mme /00300//22hu112 htmll
11

---

Published as a conference paper at ICLR 2022
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes process. In International Conference on Machine Learning, pp. 11692- 11702. PMLR, 2020.
12

---

Published as a conference paper at ICLR 2022
A ALGORITHMS
Algorithm 1: Stochastic gradient- craseent-bbsed learning algorithm
Input: X { j= is the training set with N sequences; is the number of learning iterations; 7 is the learning rate; M is the size mini-batch; Initialization: model parameters 0u:
1 <- 0;
while < 1) do
Randomly draw M sequences from denoted as X, = {z}]}= C X;
0141 < 0 + 70/000 given X;;
1 1=;
end
Algorithm 2:
Monte Carlo estimation for the integral of conditional intensity in (6)
= is an event sequence, where N₁ is the number of events in the sequence; N is the number of samples uniformly drawn
Input: À denotes the model that we need to evaluate; æ = {ta, m)))
from X:; A is the integral of the conditional intensity over the data space;
A 0;
for n = 1, I.....N.. do
Draw In (fn, mn) X: Hi(p.) +- mt) E 2j : t4 < ta
A + A + (Zn) given He(z,))
end
A XIAAN;
Algorithm 3: Efficient thinning algorithm for simulating point process
input 0, T,T; output set of events H4 ordered by time.;
Initialize H,, 0. 0, m uniform(M):
while t < T do
Sample uni i form(0, 1); fniform(MM; D uni 1); x' + (t, X + H():
t ビ t- u/x;
x - (t; X + {(z|H4);
DĀ > À then
H. < H. U (t, m' + m;
end
end
B EXPERIMENTAL SETTING AND BASELINE METHODS
Now we deseribe the experiment configurations: We set the rank R = 5 in our setting. We consider the shared network that summarizes input data into hidden embedding to be a fully connected three-layer network and the sub-network to be a fully connected network with two hidden layers. The width of the hidden layers in the shared network is n 128, and the width of the input layers in sub-networks (or the output layer in the shared network) is p = 10. We adopt the SoftPlus f()) = 1/ 1/log(1 exp(x)) the activation function of each layer in the network. To learn the model's 's parameters, we adopt the Adam optimizer KanimizerKKngmaa Ba (2014) with a constant learning rate of 10-2 and the batch size is 32. All experiments are performed on Google Colaboratory (Pro
13

---

Published as a conference paper at ICLR 2022
version) with 12GB RAM and dual-core Intel processors, which speed up to 2.3 GHz (without GPU). Codes to reproduce the experimental results are publicly availabld"
This study considers three bascline methods: (1) Standard Hawkes process with exponentially decaying kernel function (Hawkes): it specifies the conditional intensity function as A(t) = μ B exp( -tt t,)), where parameters jr, at can be estimated via maximizing likelihood Hawwes 1971); (2) Recurrent marked temporal point processes (RMTPF): assumes the conditional intensity function X(t) = exp (v h, + w(t - t2) + +), where the j-th hidden state hj in the RNN represents the history influence up to the nearest happened event j. and wot - (t)) represents the current influence; the v, w. à are trainable parameters (Du et al.] 2016); and (3) Neural Hawkes
process (NH): it specifies the conditional intensity function as X
f(Th,,, where h. is the hidden state of a continuous-time LSTM up time representing the history influence, and the f())
(t)= (t)
is a SoftPlus function which ensures the positive output given any input (Mei & Eisner 2017).
We note that, unlike the standard Hawkes process and our NSMPP, RMTPP and NH do not parameterize the kernel function directly; instead, they aim to model the conditional intensity using an LSTM-based structure. Particularly, these models pass the history information sequentially via hidden state, where the recent memory will override the long-term memory. This has led RMTPP and NH to "overemphasize" the recent events and therefore assume the temporal correlation would monotonically decrease over time. In addition, RMTPP and NH can only deal with one-dimensional categorical marks, while our model can be extended to high-dimensional continuous mark space, To ensure comparability, only consider one- and two-dimensional event consisting of time and mark in our experiment (d E (1,2]). For RMTPP and NH, the event's s mark will be discretized and treated as categorical input.
C ADDITIONAL EXPERIMENTAL RESULTS
In this section, we present additional experiments demonstrate the robustness of our approach by considering fitting to stationary exponentially decaying kernel and other non-stationary kernels and by varying the neural network architecture and training sample sizes.
Figure and Figure&show the results on one- and two-dimensional data sets with non-stationary kernely generated by model described in Section 2.2 Note that compared the Hawkes models with a stationary kernel, our model can capture the non-stationarity of the kernel and predict the conditional intensity function À more accurately. Besides, our proposed method also recovers the kernel structure and predicts À well in nigh-dimensional space. Note that the event points are more densely scattered in the areas with high intensity than those with much lower ones.
Figure D and Figure 10 study the effect of increased model size and decreased training sample sizes to the learning of our model. The results of these two ablation studies show that our proposed model works consistently well without overfitting after increasing the model size or decreasing the training sample size both the stationary and non-stationary cases.
D MLE GUARANTEE UNDER FINITE DIMENSIONAL FUNCTIONAL REPRESENTATION
We notice that in practice, the kernel function is usually restricted to some function family space for the problem to be well-defined (given finite number of events) ). In this section, we study such a set-up, where the target feature function belongs to certain parametric function class with a finite number of parameters. This includes, for instance, spline functions, functions represented by Fourier basis, and by neural networks the main interest of this paper. We are particularly interested in neural networks due to its strong expressive power of functional representation.
We start from general framework of feature function basis representation. Assume that the feature functions of the kernel can be well-approximmted by a linear combination of basis functions: (,())
httpss//gittbbb.co//eeooooo -
Spectraetrrrreee wwtp-spptttaaaaaeeeeaeaeee- Processss
14

---

Published as a conference paper at ICLR 2022
A(L.
Proposed Kernel kit,
Stationary Kennel
Conditional intansity
5.De
5.0e03
Three Phopooed
Syneicks Data
00.-00
0.0e-00
5.fe 00
1.30
Stationary
5.x
Time
5it-
11ee 02
Ste-00
5it-(0
True Kernel A(A.
Proposed Kernel 6i1,
Stationary Kernel
Conditional intersity
1.5e-02
-4fe (0
2:25
Sonthettic Deaa
1.3602
1.9.02
Time
1/
Tine
1e002
Time
7.5-03
75w03
5.5e00
Tree Proposed
1.9
5.6e
Time 9.0e
Stationary
Time
Time
Time I
Figure 7: Additional kernel recovery results for two other one-dimensional synthetic data sets. The first three columns show the true kernel that generates the data, kernel learned by our model, and kernel learned by Hawkes process, respectively. The fourth column shows the true and predicted conditional intensity functions for test sequence.
Kerd Mi..am of (t.
Kernel f.l. or) ( =
Kond RlL aur (n
Coreitional inemiity
200002
True Medel
Towe
1:30zt
ECu-00
te
- Ax
L3v32
46+ 00
Kereel M... Kanuue' [c-3]
Kenecl 6.I.t.m. mr)
Kanel moune ('
Conditional intunsity
10002
Lzaned Modd
00x0
to
1.0-000
Katk
00e
60w33
12c 00
-1/-0
10 80
22--00
Kend
Kond kir.e
Conditinal
23
True Model
100-48
25-11
2 65
42c21
90k-9
Karrel Eat.E.mn
Conditionl intusiss
56-3
Leaned Moed
10v40
22 00
No0.00
006-00
22.
6-10
40649
Mark
Liw
Sivet
Figure 8: Additional kernel recovery results for two other two-dimensional synthetic data sets. The first and third rows show the true models and the second and fourth rows show the learned models. The first three columns show different snapshots of kernel evaluation for each model; the last column shows their corresponding conditional intensity over marked-temporal space given test sequence, where the black dots indicate the location of the events.
X' R-i S:
(x) Σ (x),
(,(x)
Σ B,b((1), 1,..., R.
15

---

"Proposed kernel with Published as a conference paper at ICLR 2022
True Kernel ACt, P')
Koned incroeed
Stelionary Ketrel
Conditional
Synbelli Data The
0Ce+00
4.0e-00
Prepeir Seatiorary
-20w 02
Tim
-200-42
Time
-80e-0)
4.2e 02
4.3642
-10c 01
Time
Time
Time Conditional irtensity A
True Kennel Art, P')
Prasses Kond ko.. win vall traniin simple sie
Stalionaty Ketrel
Synbettc Dat
60ee00
3.e+43
4.0e-00
Prepder
Sestirarry
Time
-22e-0
Time
-236-00
Time
-80e-10)
6.0e
6:0x 02
Time
Time
Figure 9: Ablation studies on one-dimensional synthctic data sets used Figure 3
increased network size" refers to the model with one more hidden layer and doubled layer width in the sub- networks (p 20); "Proposed kernel with half training sample size" refers to the model with default architecture but trained with only half of the training samples.
True Kemel A:t. P))
Prepssed Karad Ètt. with inteaae netoor aiz
Stationary Kerre
Conditional irtensity A
28s-22
23e-22
21e
Spatveik Data Time
20ce
2.G022
20 02
10
Phepeiat Statichary
1.e 02
Tiee
me
1.e
0.0
1 02
10402
50c 05
03
3.00.3
Time
Time
mc
Time Conditional irtensity A
True Kernel Art, 8*)
Preaase Kand 66t. 19 with vall tranking sonple siec
Stationary Kerrel
11
Tue
Systrele Data
2.0w02
shepoaat Stanthary
o
02
Time
Time
3.0o.3
Tine
Time
T me
Figure Ablation studies on one-dimensional synthetic data set used Figure
Then the kernel function in 5 can be written as k ( (bi (bb()=)))) and the (p. entry of the matrix A is given by
(aa))TA((>)) where b(x)
Apg Σ Ur Urarprgg
Here we assume each matrix A corresponds to a unique model, i.e. for any A' + A, there exists x', a X' with ((x)) [(x), k (x', + (x at). Under such parametrization, we can write the intensity function in 4 as
:= = ((aa(2)) = 1 +
b(''"" Ab())(((2)) + ( A)
(10)
X(w
which linear in the vectorized A. Here (:.-) is the Frobenius inner product of matrices, and ((x)) SXS is conditioned on He(z) with the (p, -h entry
Upq (x) = f (x(=)),(()))N())))
Xaton
Now the influence kernel estimation problem has been reduced to the problem of estimating the S-by-S matrix A. We will estimate the coefficient matrix A by the Maximum Likelihood Estimator
16

---

Published as a conference paper at ICLR 2022
(MLE) in the set of low-rank matrices A = e R KA XC) (recall that K. is the family of kernels which admits the finite-rank decomposition as in (5). By saying A consists low- low-rank matrices, implicitly assume that R < S). Also denote the family of kernel functions that can be represented by the chosen basis functions as
Kinite {AA A A}.
The MLE is defined as
AMLE arg max La AcA
where the aigllikelihod la is the corresponding variant of (
l M Σ / log (n (1)(1)) A)) (N2 (x)
M
Σ (m (15(1),, A)) dx
With orthonormal bases, the classical theory has that the recovery of A will ensure the recovery of the original kernel function. With set of over-complete bases, R is less than S (the number of bases function), and then A is a low-rank matrix, for which case we provide a theory for the recovery of A via MLE (Theorem D.2).
More generally, the above framework contains other constructions of by (x)'s, For example, if b, (1) are random features, then O,; and Brz can be viewed as weights to combine features. Random feature model can be naturally viewed as neural network with one hidden layer, that is, 6, (x) = a (w x) is the activation the i-th hidden nodes, and L, a;,b;(3) gives the second layer output function. Here, W, are weights in the first layer, and linear combination coefficients a, are weights in the second layer. Thus, the random feature model corresponds to only training the second layer weights, leaving the 1st layer as randomly initialized. In this case, S the number of hidden neurons, and R can be interpreted as the number of heads in attention model.
few consequences. We derive a few necessary basic results based on the model parametrization for presenting the results. Recall that n2x) E R depends only data and basis but not the coefficient matrix A.
For each trajectory j, the intensity under parameter A at æ has partial derivative
0)
1pp.5 (x).
OAp pq
The score function, i.e., the partial gradient of the oogllieeidood function with respect to the coefficient matrix A, is given by
OlA 1
M
)a)(2)) (mm (m. (m) - (z)(r)dz).
0Apm
M
2=1
The Hessian matrix of the log-likelihood function is given by
S2l
M Σ
Xx3 (x) (x))
Amp AAr
M j=1 x
Next, we provide some analysis for the MLE with possible model misspecification, as the true kernel ke may not fall into Kaunte
Theorem D.I (Distance between the true kernel and the optimal fit). Let Ā E A be the one which maximizes the expected log-likeli function, i.e.
Ā= arg max (lA).
(11)
AEA
Under Assumption 3.1 let the fz-norm of a kernel be
11A122
M(x'xx) da' dx.
(12)
Then we have
E - kills M + of of
(()) (Kimttttt
where D(k", Kanite) is the Eg diittanee between the true kernel and the set Kanite- D(k", Kanite)
min kEKanite
||A* k|]22
17

---

Published as a conference paper at ICLR 2022
Remark D.1. This theorem holds true for any k* E Σ. without the rank R assumption kernels. In this case, both feature function approximation by basis and low-rank approximation of the kernel contribute the estimation enror associated with model misspecification.
Next, similar the classic asymptotic normality of the MLE under model misspecification WWite 1982), we have the following result for the low-rank MLE.
Theorem D.2 (Asymptotic normality of low-rank MLE), Under Assumption 3.7
Assumption E.
let the singular value decomposition of Ā be = UAVT. Let T be the expected Hessian matrix of the log-likelihood at A,
82
= -F L
( ( (NN
T=
Öveol A))vcc(A)7)
A=Ā
where the matrices are vectorized by concatenating their columns, and let J be the covariance matrix of single trajectory's score function A,
A Ovec(A)
DlA Evec( (A)2
J Cov
A=A
G E Sx8 be the expected score at A,
G = E
0A
A=A
Let F (1s Is) E R x29R where & is the Kronecker product, Is is the identity matrix of size S,
&
Ō= (Āt G)Qs.s + ((A) G)Qs.s)*
where represents pseudo-inverss -iverse and Qa,b gabxab the permutation matrix such that vec(P")) Qapvec(P P) for any a-by-b matrix P. If FT + C)F shares the same null-space with F, then the low-rank estimator AMLE, solved from the constrained maximum likelihood problem, satisfies
(Vve((AMME)) vec(A)) (O, F(FT (I+Ō) JF(FFFFFCC (I+))((++)))))))
when M + 0C.
Remark D.2 (Parameter recovery guarantee). The result shows that the maximum likelihood estimate for the kernel matrix A converges to the optimal fit A, and the matrices I, J, E captures the residual variance; F' projects the variance caused by I, J. C onto the tangent space of the low-rank manifold at Á. In practice, the variance term can be estimated empirically; J can be estimated from the empirical covariance matrix the score function of each trajectory, while the other matrices can be approximated by the properties of the log- likeeiiddhood function at AMLE. If the true kernel falls into the kernel family Kanite both T, J will equal to the Fisher Information, G, C' will vanish, which greatly simplify the process of estimating the variance. But with the presence of model misspecification, this is generally not the case.
E
PROOFS
E. PROOF OF LEMMA 3.1
Proof of Lemma 3.
For the j-th trajectory, define Ez [k] as
/ log (x) f X A11k((x)de,
( Jx
(13)
then {(k] = M Σ, [k). Let X; = >y(k*), X5 = >1(h), where the conditional intensity A[k] is defined as k- and To are the true kernel and the perturbed one respectively. Then we have
1g( 1,(k*] f (log 2z(x) - log (3(x)) / (5)(()) df(x))dr
(14)
18

---

Published as a conference paper at ICLR 2022
By 4 Ay[k] is linear with respect to perturbation in k. that is,
(x) = - S SxC. x)d))))dN
We then have that
-x)) - -fx (k(2') - 10 (x', (1(,)))dN((xr) - 62)(2)),
(15)
where the last equality is by definition of ő'z.
Back to 14 the second term
J - xj(r))do== / o2,(x))d;;
(16)
The first term under Taylor expansion of log has that, for each x E Æ
log 23 - log 2z(x) = 1z(z) (2z(z) (z)(2)2 (02))))))))))) (0))(z))²
where (x) takes value between N (x) and (x). By Assumption
3.1 both {z(xx) and (z
strictly positive and hopper-buunee by 02, and thus (x)) cz. This means that
2 (8),
(,),(x))² Vo e X.
Thus, the 1st term in [4 satisfies
/ (log)y(2) log (x)) / 812(1) dN,(x)
Fx 2E, (x) (o) dN
15(x)
/ (x) (x)
(8A,(z)) (17)
17
203 x
Putting together (16 and 17).
(x) (6))(x))² (dN
/ 822(z)dr
2,1,] - 1y[k"] S (11(z)
Xz(x)
20 x
/ x
dN,(x)
(2)(2)
dx
(2))()))ddN,(3
\j(x)
20 x
The above holds for each j. and taking the average over the M trajectories proves the lemma.
E.2 PROOF THEOREM 3.2
Lemma E.1 (lower bound for KL-divergence). Under Assumption 3.7
[((* {(5) z of 20
exp - M |
holds true for any &e E K and
k* (the Eg-norm is defined as
12
Proof. By Lemma 3.1 since all trajectories are i.i.d., and for each trajectory the conditional expecta-
tion
dN(x)
E
He(s)) = dx,
{ar=)(xr)
there is
E [[K*]
e(7al) 2 E
( 6X(x))
dN(xx)
dx
1 (BA(z)))dN(())) 20 /
/
(1k=
B.
(6X(x:))
20
19

---

Published as a conference paper at ICLR 2022
Following Assumption 3.1
1 2cB
I ((A()))
1 (AONG)))MMTT)))
(
E
2c3
203
Next we lower bound the term above by taking the integral over the event space = Ui E, of trajectories, where for each i,
E, (2 2i) E X t(x1(())))))))))) <.. < t(xi)) C x
consists of all the trajectories with exactly i events. For each Hr E, let N be the associated counting measure, the probability density of HI
p(HT)) = exp ( log Alk [N[()) - -1 X(k'(())dz exp([Hr]|og c - [M[Tc2) =: g(Hr))
-
E (0x((x))) can be lower bounded by taking the integral over the lower bound of probability density P.
(((A))))) 1=U
E
Σ/
p(Hr)ddadky) dx.,,
202
20g
and by the equivalence ordering of 21,-.. I...
Σ1
(((oN(E))aa))
p(Hrr)xxddz dx,
203 1= &
a Σ /
(((2(()))aa)) g(Hoy ... .xd
= 20
Then substitute 81(x) with Sxice x))((())) and change the order of integrals, @ equals to
20
1
(A J 5k(a', x)8k(a" a 2)dN(a' DaN("" ye p(Hr dd dr, f (
20
it
Xx
6k(z', z))k(2) x)) (ddd))d dry dr,dæ.
20 1=0
Xeco) Xxcn
For x' + x" dN((') (N()')NN(a")
when two of ... ,, equal to x', x'.=". For x' x"
dN(x') (dN("") = 1 when one of æ1. equals to x' = x". Again by the equivalence over ordering X1," ... Iy we assume x' Ii, = x2 when x' + x", a' x" In. (w) equals to
ili 1) Je I Xun
5k(a', x)0k(>" p)p(Hr da dx" dog dr
20
+ 20
8k(2', x)kk(a' x))((r))ds dz .d.dr
Xite
For the first row, we take the integral over æ3, ... and for the second row, take the integral over U2, ... Fza equals to
1)
J.J Mics) JAIC 6k(z', =))k(a" x)) - 27r Rdf exp(-c2)M[T)) dx" dx
20
8k(z'x2)0k() 2)0k(d' a3) Molld
22
i! x (e(=)
C1
ili 11)
M 1- 22 exp( - cg
(1 8k(x', 5k((',z))dt 22)0o do
20
i!
x
20

---

Published as a conference paper at ICLR 2022
8k(2', (x)
Σ
MM expl-ca2|Mp M
202 x X((z)
Note that the first term non-negative.
C1
8k(d', 2)8k(r' x))
1 1)!
| "1 ' exp(-oglMi[T)ds da
202
X' Xe(e)
c7
ōk(1' 2))k((/ x) exp(-(es cc)llM()ddr'd dr
26
X((a)
exp( exp(-(cc2- c1) M
20
Proof of Theorem 3.2
This follows immediately from Lemma E.1
E.3
PROOF THEOREM D.I
Lemma E.2. Under Assumption 3.1 for any 1s E K.
M c2
E (E[k*) - e(k)) <
exp((e2 M))MM] - k=112
20
Proof. Similar to the proof of Lemma 3.1 and Theorem 3.2 for a single trajectory and any k E K. let
8k(d'xx) k(x', ,x) (x', ((xl,x)) x), Val X, xx,(x')) t(x),
8A(x) = S 8k(a', x))((z))
Xi(e)
There is
[ 8M(=)do / log
{TK-] f[k]
d(((x)
f (\(x)de S log [1 oA(x))
dN(()
\[k=](x))
/ 5X(x) de - dN(x)
(a((x)²
+ 2 {(x)² dN(c),
for some X(x) determined by He(a) such that A(x) in between (xx) and \[k](x)) for all X.
Then since the expectation of dx dN(2)/ (0((=///[ A [(x)
(e[k*) {(k] E (Jx
1 1x)(x)²
dN(x))
x 2 Xax)²
( x 2 Max)
1 (A(x))
Alk (x=)(x)dz
C2 2c
(J 6xxx)
For each trajectory Hr E, the probability density of Hr
p(HT)) -exp (f log ([k*)(z)
/ Alk ([+=)(x)dd)) & exp(1?6r| log 12 (M)) : ((HT)
21

---

Published as a conference paper at ICLR 2022
Following similar arguments the proof of Lemma E.I
(Jx (x(x))dz)
E (e[K*)
((Al) -
22 E
2c
{
(Je (x(x)do))
- 20 ==
p(Hr)dd1 ...
E.
(1 Mic
02
2 exp(-c)|MM())
8k(2', 2t)da' da
201
2)
2
I MM cg exep(-epp Booiil///T/ Mi)))da' dx & expl((e2 Mi) (1 ōk(a', 22)ded de
ok(
26
20 x
+
C2
8k(l', x)) ex)c2 - Mi)
2cf
By Souchy-Schwarze inequaliiy,
ōk(u', so)ded ) dr (1
o (exp(((2-c))lM))
20
C2
& expp((e2 - Mc))
/ 8k(e', 2/) do
26
xx
5k(x', ( dx
20
X
Xics)
2 MTT
exp((e2 [1)
20
So together we have
03
M cg
E (e[k*] <
exp( cT) k* k-1l2
207
Then we get back to the proof of Theorem D.I
Proof of Theorem D.I Let Ao E An be the one which minimizes l|k- k2ol12, i.e..
KAo l22 D(k", Kiiiite).
By Lemma E.1
and Lemma E.2
there is
M
T cg
exp((e2 ()) Kanite) E(EK" - E(kAo])
20
oi
2 E3 [e(k** E(k=] 2 exp ( c))MM/>> k
20
E.4 PROOF OF THFOREM D.2
Let A = (A e R [[A] E K). We prove the theorem under the following assumption:
22

---

Published as a conference paper at ICLR 2022
Assumption E.1. Assume À is the unique minimizer in 11 is of rank exactly R, and vec(A) is the interior of vec(.A) = (vec(A) A E A) C RRS
Proof. Consider the local parametrization of An in the neighborhood of À based on the singular value decomposition Ä = UAVT, U, V E R A is a R-by-R R diagonal matrix,
A A+R P1
P PA(APP))
A = (UT)
P3
Here (U U) (V V) are fixed orthogonal matrices where the first It columns are U, respectively, Pi RRXRR R, P2; P3 (RS--)xR are parameters in the neighborhood of 0 such that A Pi is non-singular. By Theorem 3.2 of White (1982), let Pi,MLE, P2MLEE P3,MLE be the parameters which correspond to AMLE, i.c.
AMLE = (UT) A + P1,LLE Pa.MLE
PT 2.MEE PAMLEE + PI,MI.E)
VT (
PT 2,ML
For simplicity, let = (vec" (P1), vec (P2), vec (Pa))² Then
vec( P1,LE vec( P2.MLE) vee( P3.ML..)
VMPMLE VM
2 N(001~1111
where
8PLA
T= E
Öp0pT
Pi P1=0,Pa-0.PP
DlA dEA Op Op"
J E
P==0,P2=0,P==0
For simplicity, from on, write all partial derivatives taken at P. 0, 0, 0, A = = without specifying the location. Let
Ovec(A) OpT
7 = vec(A Ā)
since || = ((1p))))) there
VMy L. 0.
V V.vec(AMLE A) = VM
y
vvec(A)
PMLE D L.N(
@vec(A) 1-
Ovec T (A) -|
Op"
Op"
Öp
Next we look at the covariance matrix of the multivariate Gaussian distribution,
dl ÖvecT(A)
af Op Ōvec(A)
Op
Since Á maximizes E((x) and is the interior of A and hence the interior of the manifold parametrized by p. we have
de Op
Ovec T
(A)
(A) DE OvecT((A) vec(G) = 0.
Ovec"
=
Öp
Tvve((A)
Op
(18)
J = E
Op 3pT
Ovec? (A)
ae aeA
Ovec(A) ÖpT
TE
Op
Övec(A) Ovec (A)
@vecF(A)
de
Ae Övec(A)
Ie Ovec(A))
)))
Ovec(A) A) OpT
Cov Ovec(A) OvecT
+ E
E
Op
(A) (vocT(A)
Ovec? (A) j @vec(A)
0p
OpT
23

---

Published as a conference paper at ICLR 2022
For any i,j E [(25 plla
R)
Ovec" (A)
de Ovec" (A) dla
Op.0pj
Opi @vec(A)0p) (A)ōpj Ōp,0ps Ovec(A)
OvecT (A)
eA @vec(A)@veds (A) Ops
Ovec(A) Ovec" 0lA
+
Öp:
Op,0p, Övec(A)
the (i.j)-th component of
8ea
Ovec" T [vec((A)
+ Ivecc (A)
15 = E
vec(G)
Op:0p;
Opi
Opj
Op,0pj
We observe that some blocks of
8 vec= (A)
vec (r)()
are O.
Op,0ps
(Pec"(A))ece (vec(Pi) Pi (P1)
vved (UPR +U + +UPV +UP P )ee((G) 0,
(1) the second order derivative of UP,VT, UP/V", UPPV over Pi is 0 since they are either linear in or irrelevant with P1, (2) the second order derivative of UP.VT + UPs(A + P over P1 is 0 since it taken at P2 = P3 = 0. Also we have v'vec" (A)
Ovec" (A)vec(() Ōvec (P))dvec (Pz) 0,
@vec( Pi 1veec (P2) O,
vec 7 (A)vec(
Ovec @vveT(AA)vec) Ovec(Pa) (Pa) 0,
O,
dvec P22 Ovvee" (P2)
for similar reasons. The only non-zero block is
Vvec" (A)vec(G) Öveo(Ps P2) (Pz)
Øvec CUPP"(UUAA-PPP v "oogv""vvvci Vve((G)
@vec(P2))vec (Pz)
Ovec (UP3 -1 'PTV" )vce(G)
Ovec
Ovec! (P3)
Ovec @vecT(Ps)(A -1 (PP.'A & U )vcc(G)
(vec((2)
Ovec? (Pa)
((A (A-11P2 V X U (v))())))T
(P2) Ovec((2)
θ
vec (U T GVP2A-
pvec((2)
θ
Ovec P3) vec (Pa)(A -1 88 V'G'U)
---- 1 CTU.
So
0
I Ovect (A) 1 @vec(A)
(0000
0
A 2 V
GTU
Op
Op"
AA UU
GV
0
At last, compute @vec(A))
Övec(A)
Ovec(A) Ovec(A) dvec(A)
Op"
Ovect (PI) Ovecf (Pz)
Ovect (Pa)
(ioc((U,VV))
Ovec(UPFTV)) @veo(UP)33))
Ovect OvecT(P:)
@v2) OvecT
(VOU (V @U)Qs=RRRR V@U).
24

---

Published as a conference paper at ICLR 2022
Let
& IRUUT
Lge
Z
QR,S-R
v & ll
IR(S-R) R))
In & T
Z has full row-rank, and
V LRU
@vec(A), 0p1
2 (V
V IR
In OT)
= =(VVT + VV vu (UU + TUT)
=
Then the covariance matrix of the asymptotic distribution vec( (AMLE)
@vec(A)
-- 1
Ovec (A)) @vec(A)
OpT Z(Z" z" JZ (Z" IZ) z Ovec"
OpT Op
0p
= F" JF 12)
Next we check that
= F.F.
By
18
we know
(vec² (UFT) vee? vec)() vee vee" vec"(ā)
(G) Ovec(((
Z 0.
Op"
So
Ō- -V") -VV
1Z F"
0
=ZT
0
0
A
V
z
0
A-
TV
VT
@R
IR
0
V"
I
0
0
Qs QS- v TTU)
I R
0
(A s0 :T
@VCVVQn.s- nnn-n
VT II
In UT
V @ LR
ll &T
(In 0) & UU RGV))R.s-Rll Q @IR))?)
(Ix (A-1 @ T GV &IR)
((TAA-'Ta & गh ōvV (Qns)"))
(IRA 'I TUT TUTTVV )QRS ) ((A & &))R)))
(A-1 G]QRS
0
25

---

Published as a conference paper at ICLR 2022
1 & U
(VAA- @ G]Qss
(VA- @.ss @U l
VT s0 Ts)
VA- 20)
Qs.s (Tss @ V@
A A- 8
(Is V@l.)
Qnsōo @ A-UUT)
CORs(G@A-1) 0
0
(A- @ G)QRS
0
0) 0).
FTCF= = (1~1 uus
0
9) ((~- Q)Rs
0-
=
T
7F -FFFF.
26


Reviews:
[{"review_id": "zdIqAfSnvl9", "reviewer": "Reviewer_q7Hf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is correctly written and claims are well supported by references. However, mathematical proofs are not correctly written, the best example si page 17 where authors stack equations with several unnecessary informations leaving the reader to a rough experience. \n\n\nExperimental part is based on mostly synthetic data set in low dimension and two real-world data set is not enough to show advantages of your approach. Further, it is clearly incomplete: \n\n- What about the possibility of modeling $\\phi$ and $\\psi$ ? You should study several architectures, parameters ... in order to explore your approach.\n- What about computation time ? it seems that NMPP have computational cost much higher than classical Hawkes process where kernel have a closed-form that does not rely on NNs. \n- Why using SGD in order to optimize the MLE rather than widely used adam or rmsprop or others...?\n", "summary_of_the_paper": "The paper propose to replace the kernel involving in Marked Point Processes (MPP) by the product of two neural networks.", "summary_of_the_review": "Theoretical results appears to have very few links with empirical results and the behavior of the proposed approach in practice. Furthermore, the experimental part is not properly studied and does not support the advantages of their approach. Limitations of the proposed approach are not investigated. The paper could be a good paper with more complete work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Pks9M1jH-Zn", "reviewer": "Reviewer_HJXe", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- Novel method to model complex dependencies in temporal and spatio-temporal event data using the proposed non-stationary kernel, which experimental results shows the superior performance of the method.\n- Theoretical guarantees to find the kernels using MLE (although not fully check the proof).\n\nWeaknesses:\n- In contrast to synthetic data the real data experiment is very brief. Authors compared the proposed method using only likelihood, but it is better to also include some methods to measure the predictive power of various methods to generate events.\n- Unfortunately, there is no details about the structure and size of the neural networks which used to model the kernel functions. Also how the performance of the method is related to the number of parameters of NNs and number of events data that used for training.\n- It is better to include the comparison of the proposed method in terms on training and test/inference time.\n", "summary_of_the_paper": "This paper propose a point process with a non-stationary kernel to model complex event data. The kernel represented by its finite rank decomposition and the basis functions (feature functions) are models using a neural network architecture. To learn the model parameters they used stochastic gradient to maximize the resulting likelihood function. Moreover, they give a theoretical guarantee by showing that under some assumptions, true kernel function is identifiable by solving the MLE problem. They also compared their method against state-of-the-art and baseline on synthetic and real datasets.", "summary_of_the_review": "In overall the proposed method for modeling the event data is novel enough and seems to have superior performance in comparison to other state-of-the-arts methods. It also can capture more complex behaviors in events data.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "Ap_l41uMM4", "reviewer": "Reviewer_CU9d", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nI find that this paper has interesting ideas, and it addresses an important question in learning self-exciting processes in the presence of non-stationarity.\n\nHowever, I think the authors missed an extremely important influential paper by Feng Chen and Peter Hall (2013): https://www.cambridge.org/core/journals/journal-of-applied-probability/article/inference-for-a-nonstationary-selfexciting-point-process-with-an-application-in-ultrahigh-frequency-financial-data-modeling/99E7C55841B746DBD6CE20D657B2CC9D\n\nOf course, the modulation of the excitation function are very different, where the authors used a general function for k(\\cdot, \\cdot). In Chen/Hall's paper, they had the remarkable idea that it MLE estimates be accurately inferred when you have many paths but with T (time to horizon fixed) and not otherwise. In their paper they set T = 1. I would like to see how your framework would subsume theirs (in the most vanilla case), as your k(\\cdot,\\cdot) is that of a much general form.\n\n", "summary_of_the_paper": "In this paper, the authors introduce a neural network framework modulating the excitation functions within the self and mutually exciting count processes.", "summary_of_the_review": "This is an important piece of work, but I would like to see how they contrast their work with Chen/Hall, which I think they missed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a self-exciting temporal point process model with a non-stationary triggering kernel to model complex dependencies in temporal and spatio-temporal event data. The kernel is represented by its finite rank decomposition and a set of neural basis functions (feature functions). The proposed model has superior performance in comparison to other state-of-the-arts methods. All the reviewers recognized that the model is interesting and advances the state of the art in a meaningful way. While they were some concerns regarding the experimental evaluation, particularly in terms of real data, and the presentation, the rebuttal/revision by the authors cleared up these concerns.
Decision: Accept (Poster)