S PATIO - TEMPORAL POINT PROCESSES WITH DEEP NONSTATIONARY KERNELS
Anonymous authors
Paper under double-blind review
A BSTRACT
Point process data are becoming ubiquitous in modern applications, such as social networks, health care, and ﬁnance.
Despite the powerful expressiveness of the popular recurrent neural network (RNN) models for point process data, they may not successfully capture sophisticated non-stationary dependencies in the data due to their recurrent structures.
Another popular type of deep model for point process data is based on representing the inﬂuence kernel (rather than the intensity function) by neural networks.
We take the latter approach and develop a new deep non-stationary inﬂuence kernel that can model non-stationary spatio-temporal point processes.
The main idea is to approximate the inﬂuence kernel with a novel and general low-rank decomposition, enabling efﬁcient representation through deep neural networks and computational efﬁciency and better performance.
We also take a new approach to maintain the non-negativity constraint of the conditional intensity by introducing a log-barrier penalty.
We demonstrate our proposed method’s good performance and computational efﬁciency compared with the state-of-the-art on simulated and real data.
I NTRODUCTION
Point process data, consisting of sequential events with timestamps and associated information such as location or category, are ubiquitous in modern scientiﬁc ﬁelds and real-world applications.
The distribution of events is of great scientiﬁc and practical interest, both for predicting new events and understanding the events’ generative dynamics (Reinhart, 2018).
To model such discrete events in continuous time and space, spatio-temporal point processes (STPPs) are widely used in a diverse range of domains, including modeling earthquakes (Ogata, 1988; 1998), the spread of infectious diseases (Schoenberg et al., 2019; Dong et al., 2021), and wildﬁre propagation (Hering et al., 2009).
A modeling challenge is to accurately capture the underlying generative model of event occurrence in general spatio-temporal point processes (STPP) while maintaining the model efﬁciency.
Speciﬁc parametric forms of conditional intensity are proposed in seminal works of Hawkes process (Hawkes, 1971; Ogata, 1988) to tackle the issue of computational complexity in STPPs, which requires evaluating the complex multivariate integral in the likelihood function.
They use an exponentially decaying inﬂuence kernel to measure the inﬂuence of a past event over time and assume the inﬂuence of all past events is positive and linearly additive.
Despite computational simplicity (since the integral of the likelihood function is avoided), such a parametric form limits the model’s practicality in modern applications.
Recent models use neural networks in modeling point processes to capture complicated event occurrences.
RNN (Du et al., 2016) and LSTM (Mei and Eisner, 2017) have been used by taking advantage of their representation power and capability in capturing event temporal dependencies.
However, the recurrent structures of RNN-based models cannot capture long-range dependency (Bengio et al., 1994) and attention-based structure (Zhang et al., 2020; Zuo et al., 2020) is introduced to address such limitations of RNN.
Despite much development, existing models still cannot sufﬁciently capture spatio-temporal non-stationarity, which are common in real-world data (Graham et al., 2013;
Dong et al., 2021).
Moreover, while RNN-type models may produce strong prediction performance, the models consist of general forms of network layers and the modeling power relies on the hidden states, thus often not easily interpretable.
[Figure]
A promising approach to overcome the above model restrictions is point process models that combine statistical models with neural network representation, such as Zhu et al.
and
Chen et al.
(2020), to enjoy both the interpretability and expressive power of neural networks.
In particular, the idea is to represent the (possibly non-stationary) inﬂuence kernel based on a spectral decomposition and represent the basis functions using neural networks.
However, the prior work (Zhu et al., 2022) is not speciﬁcally designed for non-stationary kernel and the low-rank representation can be made signiﬁcantly more efﬁcient, which is the main focus of this paper.
Contribution.
In this paper, we develop a non-stationary kernel (referred to as DNSK ) for (possibly non-stationary) spatio-temporal processes that enjoy efﬁcient low-rank representation, which leads to much improved computational efﬁciency and predictive performance.
The construction is based on an interesting observation that by reparameterize the inﬂuence kernel from the original form of k ( t 0 , t ) , (where t 0 is the historical even time, and t is the current time) to an equivalent form k ( t 0 , t  t 0 ) (which thus is parameterized by the displacement t  t 0 instead), the rank can be reduced signiﬁcantly, as shown in Figure 1.
This observation inspired us to design a much more efﬁcient representation of the non-stationary point processes with much fewer basis functions to represent the same kernel.
In summary, the contributions of our paper include
• We introduce an efﬁcient low-rank representation of the inﬂuence kernel based on a novel “displacement” re-parameterization.
Our representation can well-approximate a large class of general non-stationary inﬂuence kernels and is generalizable to spatio-temporal kernels (also potentially to data with high-dimensional marks).
Efﬁcient representation leads to lower computational cost and better prediction power, as demonstrated in our experiments.
• In model ﬁtting, we introduce a log-barrier penalty term in the objective function to ensure the non-negative conditional intensity function so the model is statistically meaningful, and the problem is numerically stable.
This approach also enables the model to learn general inﬂuence functions (that can have negative values), which is a drastic improvement from existing inﬂuence kernel-based methods that require the kernel functions to be non-negative.
• Using extensive synthetic and real data experiments, we show the competitive performance of our proposed methods in both model recovery and event prediction compared with the state-of-the-art, such as the RNN-based and transformer-based models.
1. R ELATED WORKS
The original work of A. Hawkes (Hawkes, 1971) provides classic self-exciting point processes for temporal events, which express the conditional intensity function with an inﬂuence kernel and a base rate.
Ogata (1988; 1998) propose a parametric form of spatio-temporal inﬂuence kernel which enjoys strong model interpretability and efﬁciency.
However, such simple parametric forms own limited expressiveness in characterizing the complex events’ dynamic in modern applications.
Neural networks have been widely adopted in point processes (Xiao et al., 2017; Chen et al., 2020).
Du et al.
(2016) incorporates recurrent neural networks and Mei and Eisner (2017) use a continuoustime invariant of LSTM to model event inﬂuence with exponential decay over time.
These RNN-based models may be unable to capture complicated event dependencies due to the recurrent structure.
Zhang et al.
; Zuo et al.
(2020) introduce self-attentive structures into point processes for their capability to memorize long-term inﬂuence by dealing with an event sequence as a whole.
The main limitation is that they assume a dot-product-based score function and assume linearly decaying of event inﬂuence.
Omi et al.
(2019) propose a fully-connected neural network to model the cumulative intensity function to go beyond parametric decaying inﬂuence.
However, the event embeddings are still generated by RNN, and ﬁtting cumulative intensity function by neural networks lacks model interpretability.
Note that all the above models tackle temporal events with categorical marks, which are inapplicable in continuous time and location space.
Recent works adopt neural networks in learning the inﬂuence kernel function.
The kernel introduced in Okawa et al.
(2021) uses neural networks to model the latent dynamic of time interval but still assumes an exponentially decaying inﬂuence over time.
Zhu et al.
(2022) proposes a kernel representation using spectral decomposition and represents feature functions using deep neural networks to harvest powerful model expressiveness when dealing with marked event data.
Our method considers an alternative novel kernel representation that allows the general kernel to be expressed further low-rankly.
B ACKGROUND
Spatio-temporal point processes (STPPs)
(Reinhart, 2018; Moller and Waagepetersen, 2003) have been widely used to model sequences of random events that happen in continuous time and space.
Let
H = { ( t i , s i ) } ni =1 denote the event stream with time t i 2 [0 , T ] ⇢ R and location s i 2 S ⇢ R d S of i th event.
The event number n is also random.
Given the observed history
[Equation]
s i )
[Equation]
|
[Equation]
i < t } before time t , an STPP is then fully characterized by the conditional intensity function
[Equation]
[Equation]
where B ( s,  s ) is a ball centered at s 2 R d S with radius  s , and the counting measure N is deﬁned as the number of events occurring in [ t, t +  t ] ⇥ B ( s,  s ) ⇢ R d S +1 .
Naturally  ( t, s |H t ) for any arbitrary t and s .
In the following, we omit the dependency on history H t and use common shorthand  ( t, s ) .
The log-likelihood of observing H on [0 , T ] ⇥ S is given by (Daley et al., 2003)
[Equation]
Neural point processes parameterize the conditional intensity function by taking advantage of recurrent neural networks (RNNs).
In Du et al.
an input vector x i which extracts the information of event t i and the associated event attributes m i (can be event mark or location) is fed into the RNN.
A hidden state vector h
i is updated by h i = ⇢ ( h i  1 , x i ) , where ⇢ ( · ) is a mapping fulﬁlled by recurrent neural network operations.
The conditional intensity function on ( t i , t i +1 ] is then deﬁned as  ( t ) =
( t, h i ) , where  is an exponential transformation that guarantees a positive intensity.
In Mei and Eisner (2017) the RNN is replaced by a continuous-time LSTM module with hidden states h ( t ) deﬁned on [0 , T ] and a Softplus function  .
Attention-based models are introduced in Zuo et al.
; Zhang et al.
(2020) to overcome the inability of RNNs to capture sophisticated event dependencies due to their recurrent structures.
Hawkes process (Hawkes, 1971) is a well-known generalized point process model.
Assuming the inﬂuences from past events are linearly additive, the conditional intensity function takes the form of
[Equation]
where k is an inﬂuence kernel function that captures event interactions.
Commonly the kernel function is assumed to be stationary , that is, k only depends on t  t 0 and s  s 0 , which limits the model expressivity.
In this work, we aim to capture complicated non-stationarity in spatio-temporal event dependencies by leveraging the strong approximation power of neural networks in kernel ﬁtting.
L OW - RANK DEEP NON - STATIONARY KERNEL
Due to the intricate dependencies between events, it is challenging to choose the form of kernel function that achieves great model expressiveness while enjoying high model efﬁciency.
In this section, we introduce a uniﬁed point process model with a low-rank deep non-stationary kernel capable of modeling complex heterogeneity in events’ inﬂuence over spatio-temporal space efﬁciently.
3. K ERNEL WITH HISTORY AND SPATIO - TEMPORAL DISPLACEMENT
For the inﬂuence kernel function k ( t 0 , t, s 0 , s ) , by using the displacements in t and s as variables, we ﬁrst re-parameterize the kernel as k ( t 0 , t  t 0 , s 0 , s  s 0 ) , where the minus in s  s 0 refers to element-wise difference between s and s 0 when d S > 1 .
Then we achieve a ﬁnite-rank decomposed representation based on (truncated) singular value decomposition (SVD) for kernel functions (Mollenhauer et al., 2020) (which can be understood as the kernel version of matrix SVD, where the eigendecomposition is based on Mercer’s Theorem (Mercer, 1909)), and that the decomposed spatial (and temporal) kernel functions can be approximated under shared basis functions (cf.
Assumption A.2).
The resulting approximate ﬁnite-rank representation is written as (details are in Appendix A.1)
[Equation]
Here { l , ' l : [0 , T ] !
R , l = 1 , . . .
, L } are two sets of temporal basis functions that characterize the temporal inﬂuence of event at t 0 and the decaying effect brought by elapsed time t  t 0 .
Similarly, spatial basis functions { u r , v r : S !
R , r = 1 , . . .
, R } capture the spatial inﬂuence of event at s 0 and the decayed inﬂuence after spreading over the displacement of s  s 0 .
The corresponding weight ↵ lr at different spatio-temporal ranks combines each set of basis functions into a weighted summation, leading to the ﬁnal expression of inﬂuence kernel k .
To further enhance the model expressiveness, we use a fully-connected neural network to represent each basis function.
The history or displacement is taken as the input and fed through multiple hidden layers equipped with Softplus non-linear activation function.
To allow for inhibiting inﬂuence from past events (negative value of inﬂuence kernel k ), we use a linear output layer for each neural network.
For an inﬂuence kernel with temporal rank L and spatial rank R , we need 2( L + R ) independent neural networks for modeling.
The beneﬁts of our proposed kernel framework lies in the following: (i)
The kernel parameterization with displacement signiﬁcantly reduces the rank needed when representing the complicated kernel encountered in practice as shown in Figure 1.
(ii)
The non-stationarity of original inﬂuence of historical events over spatio-temporal space can be conveniently captured by in-homogeneous { l } Ll =1 , { u r } Rr =1 , making the model applicable in modeling general STPPs.
(iii)
The propagating patterns of inﬂuence are characterized by { ' l } Ll =1 , { v r } Rr =1 which go beyond simple parametric forms.
In particular, when the events’ inﬂuence has ﬁnite range, i.e. there exist ⌧ max and a max such that the inﬂuence decays to zero if | t  t 0 | > ⌧ max or || s  s 0 || > a max , we can restrict the parameterization of { ' l } Ll =1 and { v r } Rr =1 on a local domain [0 , ⌧ max ] ⇥ B (0 , a max ) instead of [0 , T ] ⇥ S , which fur- ther reduce the model complexity.
Details of choosing kernel and neural network architectures are described in Appendix C.
Remark 1 (the class of inﬂuence kernel expressed) .
The proposed deep kernel representation covers a large class of non-stationary kernels generally used in STPPs.
In particular, the proposed form of the kernel does not need to be positive semi-deﬁnite or even symmetric (Reinhart, 2018).
The low-rank decomposed formulation equation 4 is of SVD-type (cf.
Appendix A.1).
While each ' l (and v r ) can be viewed as stationary (i.e., shift-invariant), the combination with left modes in the summation enables to model spatio-temporal non-stationarity.
The technical assumptions A.1 and A.2 do not require more than the existence of a low-rank decomposition motivated by kernel SVD.
As long as the 2( R + L ) many functions l , ' l , and u r , v r are sufﬁciently regular, they can be approximated and learned by a neural network.
The universal approximation power of neural networks enables our framework to express a broad range of general kernel functions, and the low-rank decomposed form reduces the modeling of a spatio-temporal kernel to ﬁnite many functions on time and space domains (the right modes are on truncated domains), respectively.
E FFICIENT COMPUTATION OF MODEL
We consider model optimization through Maximum likelihood estimation (MLE) (Reinhart, 2018).
The resulting conditional intensity function could now be negative by allowing inhibiting historical inﬂuence.
A common approach to guarantee the non-negativity is to adopt a nonlinear positive activation function in the conditional intensity (Du et al., 2016; Zhu et al., 2022).
However, the integral of such a nonlinear intensity over spatio-temporal space is intractable and computationally expensive.
To tackle that, we ﬁrst introduce a log-barrier to the MLE optimization problem to guarantee the non-negativity of conditional intensity function  and maintain its linearity.
Then we provide a computationally efﬁcient strategy that beneﬁts from the linearity of the conditional intensity.
The extension of the approach to point process data with marks is given in Appendix B.
4. M ODEL OPTIMIZATION WITH LOG - BARRIER
We re-denote ` ( H ) in equation 2 by ` ( ✓ ) in terms of model parameter ✓ .
The constrained MLE optimization problem for model parameter estimation can be formulated as:
[Equation]
We introduce a log-barrier method (Boyd et al., 2004) to ensure the non-negativity of  .
We penalize the values of  on a dense enough grid U bar ,t ⇥ U bar ,s ⇢ [0 , T ] ⇥ S .
The log-barrier is deﬁned as
[Equation]
where c t , c s indicate the index of the gird, and b is a lower bound of conditional intensity function on the grid to guarantee the feasibility of logarithm operation.
The MLE optimization problem can be ﬁnally formulated as
[Equation]
where w is a weight to control the trade-off between log-likelihood and log-barrier.
Both w and b can be set accordingly during the learning procedure.
Details can be found in Appendix A.2.
Note that previous works (Du et al., 2016; Mei and Eisner, 2017; Pan et al., 2021; Zuo et al., 2020; Zhu et al., 2022) use a scaled positive transformation to guarantee non-negativity conditional intensity function.
Compared with them, the log-barrier method preserves the linearity of the conditional intensity function.
As shown in Table 1, such a log-barrier method enables efﬁcient model computation (See more details in Section 4.2) and enhance the model recovery power.
4. M ODEL COMPUTATION
The log-likelihood computation of STPPs is often intractable and time-consuming due to the existence of neural networks and the double integral term.
Given a sequence of events { x i
=
( t i , s i ) } ni
=1 of number n , the complexity of neural network evaluation is of O ( n 2 ) for the term of log-summation and of O ( Kn )
( K  n ) when using numerical integration for the double integral term with K sampled points in a multi-dimensional space.
In the following, we circumvent the calculation difﬁculty by proposing an efﬁcient computation for L ( ✓ ) with complexity O ( n ) of neural network evaluation through a domain discretization strategy.
Computation of log-summation.
The ﬁrst log-summation term in equation 2 can be written as: n
[Equation]
A
[Equation]
Note that each l only needs to be evaluated at event time { t i } ni =1 and each u r is evaluated at all the event location { s i } ni =1 .
To avoid the redundant evaluations of ' l over every pair of ( t i , t j ) , we set up a uniform grid U t over time horizon [0 , ⌧ max ] and evaluate ' l on the grid.
The value of ' l ( t j  t i ) can be obtained by linear interpolation with values on two adjacent grid points of t j  t i .
By doing so, we only need to evaluate ' l for |U t | times on the grids.
Note that ' l can be simply feed withwhen t
j  t i > ⌧ max without any neural network evaluation.
We directly evaluate v r
( s i  s j ) since numerical interpolation is less accurate in location space.
We do not need to evaluate every pair of index ( i, j ) .
Instead, we have I :=
{ ( i, j ) | v r
( s i  s j ) will be computed } =
{ ( i, j )
| t j < t i  t j + ⌧ max } \ { ( i, j ) | k s i
s j k  a max } .
We feed 0 to other pairs of ( i, j ) .
Computation of integral.
We note that it is unnecessary to perform numerical integration for the whole conditional intensity function since the design of the kernel allows us to decompose the desired integral to integrating basis functions.
Speciﬁcally, we have
[Equation]
To compute the integral of ' l , we take the advantage of the previously evaluated value of ' l on the grid U t .
Let F l ( t ) :
=
R t 0 ' l ( ⌧ ) d ⌧ .
Then F l ( T  t i ) can be computed by the linear interpolation of values of F l at two adjacent grid points (in U t ) of T  t i .
In particular, F l evaluated on U t equals to the cumulative sum of ' l divided by the grid width.
The integral of v r can be estimated based on a grid U s in B (0 , a max ) ⇢ R d S since it decays outside the ball.
For each s i ,
R S
[Equation]
s
[Equation]
S
[Equation]
r
[Equation]
s ) ds , where S  s i :=
{ s 0 | s
[Equation]
= s
s i , s 2 S} .
Thus the integral is well estimated with the evaluations of v r on grid set U s \ S  s i .
Note that in practice we only evaluate v r on
U s once and use subsets of the evaluations for different s i .
More details about grid-based computation can be found in Appendix A.3.
Computation of log-barrier.
The barrier term p ( ✓ , b ) is calculated in a similar way as equationby replacing ( t i , s i , µ ) with ( t c t , s c s , µ  b ) , i.e. we use interpolation to calculate ' l ( t c t  t j ) and evaluate v r on a subset of { ( s c s , s j ) } c s =1 ,..., |U bar ,s | ,j =1 ,...,n .
4. C OMPUTATIONAL COMPLEXITY
[Table]
The evaluation of { u r } Rr =1 and { l } Ll =1 over n events costs
O (( R + L ) n ) complexity.
The evaluation of { ' l } Ll =1 is of O
( L |U t | ) complexity since it relies on the grid
U t .
The evaluation of { v r } Rr =1 costs no more than
O ( RC ⌧ max n )+ O ( R |U s | ) com- plexity.
We note that L, R, ⌧ max , |U t | , |U s | are all constant that much less than event number n , thus the overall computation complexity will be O ( n ) .
We compare the model training time per epoch for a baseline equipped with a softplus activation function ( NSMPP ) and our model with log-barrier method ( DNSK+Barrier ) on a 1D synthetic data set and a 3D synthetic data set.
The quantitative results in
Table 1 demonstrates the efﬁciency improvement of our model by using log-barrier technique.
More details about the computation complexity analysis can be found in Appendix A.4.
E XPERIMENT
We use large-scale synthetic and real data sets to demonstrate the superior performance of our model and present the results in this section.
Experimental details and results can be found in Appendix C.
Codes will be released upon publication.
Baselines.
We compare our method ( DNSK+Barrier ) with: (i) RMTPP ( RMTPP ) (Du et al., 2016); (ii) Neural Hawkes ( NH ) (Mei and Eisner, 2017); (iii) Transformer Hawkes process ( THP )
(Zuo et al., 2020); (iv) Parametric Hawkes process ( PHP+exp ) with exponentially decaying spatiotemporal kernel; (v) Neual spectral marked point processes ( NSMPP )
(Zhu et al., 2022); (vi) DNSK without log-barrier but with a non-negative Softplus activation function ( DNSK+Softplus ).
We note that RMTPP , NH and THP directly model conditional intensity function using neural networks while others learn the inﬂuence kernel in the framework of equation 3.
In particular, NSMPP designs
[Table]
[Figure]
the kernel based on singular value decomposition but parameterizes it without displacement.
The model parameters are estimated using the training data via Adam optimization method (Kingma and Ba, 2014).
Details of training can be found in Appendix A.2 and C.
5. S YNTHETIC DATA EXPERIMENTS
Synthetic data sets.
To show the effectiveness of DNSK+Barrier , we conduct all the models on three temporal data sets and three spatio-temporal data sets generated by following true kernels: (i) 1D exponential kernel (ii) 1D non-stationary kernel; (iii) 1D inﬁnite rank kernel; (iv) 2D exponential kernel; (v) 3D non-stationary inhibition kernel; (vi) 3D non-stationary mixture kernel.
Data sets are generated using thinning algorithm in Daley and Vere-Jones (2008).
Each data set is composed of 2000 sequences.
Details of kernel formulas and data generation can be found in Appendix C.
We consider two performance metrics for testing data evaluation: Mean relative error (MRE) of the predicted intensity and log-likelihood.
The true and predicted  ⇤ ( x ) , ˆ  ( x ) can be calculated using equation 4 with true and learned kernel.
The MRE for one test trajectory is deﬁned as
R X |  ⇤ ( x )  ˆ  ( x ) | /  ⇤ ( x ) dx and the averaged MRE over all test trajectories is reported.
The loglikelihood for observing each testing sequence can be computed according to equation 2, and the average predictive log-likelihood per event is reported.
The log-likelihood shows the model’s goodness-of-ﬁt, and the intensity evaluation further reﬂects the model’s ability to recover the underlying mechanism of event occurrence and predict the future.
The heat maps in Figure 2 visualize the results of non-stationary kernel recovery for DNSK+Barrier and NSMPP on 1D Data set 2 and 3 (The true kernel used in 1D Data set 3 is the one in Figure 1).
DNSK+Barrier recovers the true kernel more accurately than NSMPP , indicating the strong representation power of the low-rank kernel parameterization with displacements.
Line charts in Figurepresent the recovered intensities with the true ones (dark grey curves).
It demonstrates that our method can accurately capture the temporal dynamics of events.
In particular, the average conditional intensity  over multiple testing sequences shows the model’s ability to recover data non-stationarity over time.
While DNSK+Barrier successfully captures the non-stationarity among data, both
RMTPP and NH fails to do so by showing a ﬂat curve of the averaged intensity.
Note that THP with
[Table]
positional encoding recovers the data non-stationarity (as shown in two ﬁgures in the last column).
However, our method still outperforms THP which suffers from limited model expressiveness when complicated propagation of event inﬂuence is involved (see two ﬁgures in the penultimate column).
Tabel 2 summarized the quantitative results of testing log-likelihood and MRE.
It shows that
DNSK+Barrier has superior predictive performance against baselines in characterizing the dynamics of data generation in spatio-temporal space.
Speciﬁcally, with evidently over-parameterization for 1D Data set 1 generated by a stationary exponentially decaying kernel, our model can still approximate the kernel and recover the true conditional intensity without overﬁtting, which shows the adaptiveness of our model.
Moreover, DNSK+Barrier enjoys outstanding performance gain when learning a diverse variety of complicated non-stationary kernels.
The comparison between
DNSK+Softplus and DNSK+Barrier proves that the model with log-barrier achieves a better recovery performance by maintaining the linearity of the conditional intensity.
THP outperforms
RMTPP in non-stationary cases but is still limited due to its pre-assumed parametric form of inﬂuence propagation.
More results about kernel and intensity recovery can be found in Appendix C.
R EAL DATA RESULTS
Real data sets.
We provide a comprehensive evaluation of our approach on several real-world data sets: we ﬁrst use two popular data sets containing time-stamped events with categorical marks to demonstrate the robustness of DNSK+Barrier in marked STPPs (refer to Appendix B for detailed deﬁnition and kernel modeling): (i) Financial Transactions.
(Du et al., 2016).
This data set contains transaction records of a stock in one day with time unit milliseconds and the action (mark) of each transaction.
We partition the events into different sequences by time stamps.
(ii) StackOverﬂow
(Leskovec and Krevl, 2014):
The data is collected from the website StackOverﬂow over two years, containing reward records for users who promote engagement in the community.
Each user’s reward history is treated as a sequence.
Next, we demonstrate the practical versatility of the model using the following spatio-temporal data sets: (i) Southern California earthquake data provided by Southern California Earthquake Data
Center (SCEDC) contains time and location information of earthquakes in Southern California.
We collect 19,414 records from 1999 to 2019 with magnitude larger than 2.5 and partition the data into multiple sequences by month with average length of 40.2.
(ii) Atlanta robbery & burglary data .
Atlanta Police Department (APD) provides a proprietary data source for city crime.
We extractreported robberies and 14958 burglaries with time and location from 2013 to 2019.
Two crime types are preprocessed as separate data sets on a 10-day basis with average lengths of 13.7 and 58.7.
Finally, the model’s ability to tackle high-dimensional marks is evaluated with Atlanta textual crime data.
The proprietary data set provided by APD records 4644 crime incidents from 2016 towith time, location, and comprehensive text descriptions.
The text information is preprocessed by
TF-IDF technique, leading to a 5012 -dimensional mark for each event.
Table 3 summarizes the results of models dealing with categorical marks.
Event time and type prediction are evaluated by Root Mean Square Error (RMSE) and accuracy, respectively.
We can see that DNSK+Barrier outperforms the baselines in all prediction tasks by providing less time RMSE and higher type accuracy.
For real-world spatio-temporal data, we report average predictive log-likelihood per event for the testing set since MRE is not applicable.
Besides, we perform online prediction for earthquake data to
[Figure]
[Table]
demonstrate the model predicting ability.
The probability density function f ( t, s ) which represents the conditional probability that the next event will occur at ( t, s ) given history H t can be written as f
[Equation]
⌘ .
The predicted time and location of the next event can be computed as E [ t n +1 |H
t ] =
[Equation]
t, s ) dtds.
We predict the the time and location of the last event in each sequence.
The mean absolute error (MAE) of the predictions is computed.
The quantitative results in Table 4 show that DNSK+Barrier provides more accurate predictions than other alternatives with higher event log-likelihood.
To demonstrate our model’s interpretability and power to capture heterogeneous data characteristics, we visualize the learned inﬂuence kernels and predicted conditional intensity for two crime categories in Figure 3.
The ﬁrst column shows kernel evaluations at ﬁxed geolocation in downtown Atlanta which intuitively reﬂect the spatial inﬂuence of crimes in that neighborhood.
The inﬂuence of a robbery in the downtown area is more intensive but regional, while a burglary which is hard to be responded to by police in time would impact a larger neighborhood along major highways of
Atlanta.
We also provide the predicted conditional intensity over space for two crimes.
As we can observe, DNSK+Barrier captures the occurrence of events in regions with a higher crime rate, and crimes of the same category happening in different regions would inﬂuence their neighborhoods differently.
We note that this example emphasizes the ability of the proposed method to recover data non-stationarity with different sequence lengths, and improve the limited model interpretability of other neural network-based methods ( RMTPP , NH , and THP ) in practice.
For Atlanta textual crime data, we borrow the idea in Zhu and Xie (2022) by encoding the highly sparse TF-IDF representation into a binary mark vector with dimension d = 50 using Restricted
Boltzmann Machine (RBM) (Fischer and Igel, 2012).
The average testing log-likelihoods per event for each model are reported in Table 4.
The results show that DNSK+Barrier outperforms PHP+exp in Zhu and Xie (2022) and NSMPP by achieving a higher testing log-likelihood.
We visualize the basis functions of learned inﬂuence kernel by DNSK+Barrier in Figure A.4 in Appendix.
C ONCLUSION
We propose a deep non-stationary kernel for spatio-temporal point processes using a different parameterization scheme, which enables the model to be further low-rank when learning complicated inﬂuence kernel, thus signiﬁcantly reducing the model complexity.
The non-negativity of the solution is guaranteed by a log-barrier method which maintains the linearity of the conditional intensity function.
Based on that, we propose a computationally efﬁcient strategy for model estimation.
The superior performance of our model is demonstrated using synthetic and real data sets.