Deep Continuous-Time State-Space Models for Marked Event Sequences
Authors: Yuxin Chang
Alex James Boyd
Cao Xiao
Taha Kass-Hout
Parminder Bhatia
Padhraic Smyth
Andrew Warrington
Venue: NeurIPS 2025 spotlight ()
Keywords: Marked temporal point processes
state-space models
Hawkes processes
Primary area: probabilistic_methods
TL;DR: A novel class of MTPP models inspired by linear Hawkes processes and deep state-space models, that brings linear complexity and sublinear scaling while being highly expressive.

Abstract: Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the _state-space point process_ (S2P2) model, a novel and performant model that leverages techniques derived for modern deep state-space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-time event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an architecture that interleaves stochastic jump differential equations with nonlinearities to create a highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with a parallel scan, bringing linear complexity and sublinear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.

Deep Continuous-Time State-Space Models for Marked Event Sequences
Yuxin Chang University of Californa, Irvine
Alex Boyd" GE HealthCare
Cao Xiao GE HealthCare
Taha Kass-Hout GE HealthCare
Parminder Bhatía GE HcalthCare
Padhraic Smyth
Andrew Warrington GE HcalthCare
University of California, Irvine
yuxinc20duci.ede smythhics wwwphhhwiccccc...... udi.edu falex.boye) andrew. warringean/gghhellccccrer..om
Abstract
Marked temporal point processes (MTPPs) model sequences of events occurring irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the state-space point process (S2P2) model, novel and performant model that leverages techniques derived for modern deep state- space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-ting event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an archi- tecture that interleaves stochastic jump differential equations with nonlinearities to create highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with parallel scan, bringing linear complexity and sub- linear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of of- titte-of-th-attt the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.
Introduction
Marked temporal point processes (MTPPs) are used to model irregular sequences of events in continuous time, where each event has associated type, often called mark. MTPPs model the joint distribution of these sequences of event times and marks. They have been successfully applied to modeling purchasing patterns in e-commerce [Türkmen et al., 2019a Vassøy et al. 2019 Yang
et al., 2018), patient-specific medical events (Hua et al. 2022]. disease propagation [aaardoo and Müller 2023]. and event modeling and prediction across multiple other domains (Williams al.
2020 Sharma et al., 2018 WWag et al.; 2024]. An MTPP can be fully characterized by a marked intensily process, specifying the instantaneous rate of occurrence of each mark conditioned history.
State-of-the-arr neural methods compute hidden states to summarize the event history, which are then used to compute marked intensities at any point in time. However, many models are limited by inexpressive temporal dynamics, lack of support for long-range dependencies, and serial computa- tion [Du et al. 2016 Mci and Eisner 2017), Recent advances in transformer-based MTPPs improved performance and gained parallelism, but scale quadratically in sequence length [Zhang et al. 2020
Authors contributed equally
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Inbersiity Ae
Rank (Lower better, (8) best, x second-hest)
Proj &:Sofiplos
Likelibood
Prediction Time RMSE
Calibration
Composite
ColE Nerm
Model
Mark
Time
Mark Acc.
Mark
Time
Rank
LLIT Laves 16
RMIPP
Du al.
2016
2016
6.8
6.1
6.9
5.0
6.1
SAHP
2hany 2r
4020
6.9
4.9
7.1
4.3
6.8
5.4
5.5
THP
5.5
7.0
LLJ Lawe
IFTPP
Shehar al
4.1
3.1
5.0
4.0
[18
2.6
MHP
2024
54
6,4
4.3
5.5
4.8
6.0
5.3
Cayee Norma
NEI
Mei and Eisner
2017
2.4
3.3
L8
244
LAll Lover
AINHH
1ing ct
2024
244
3.3
2.9
6.6
3.4
3.7
3.7
Evert Siquienes
(Ours]
1.0
14
1.9
(b)
Figure 1: (a) A schematic of our proposed state-space point process (S2P2), deep stack of novel latent linear Hawkes (L.H)) layers interleaved with nonlinear and normalization layers creating an expressive MTPP architecture. (b) Summary table of results we present in Section 6 We summarize ranks for six key metrics, ranking the average held-out test set performance across eight real- datasets for five randomly seeded models; well as a holistic composite rank, defined as the average of the ranks. Our S2P2 model outperforms all baselines by almost an entire rank, strongly indicating state-of-ther thert--artt performance and robustness across metrics and datasets.
Zuo al. 2020 Yang et al. 2022], preventing them from being used in practice for long sequences, such modeling clinical events over a patient's entire medical history spanning many years.
Recently, deep state-space models (often abbreviated as SSMs) have emerged challenger to transformer-based models for discrete sequence modeling [Gu et al. 2022b Smith et al. 2022 Gu and Dao 2023]. SSMs interleave stack of linear state-space recurrences with position-wiss nonlincarities [Gu et al. [202]] This architecture not only achieves superior performance on wide range of tasks [Goel et al. [202]] Deng al. 2023), but retains linear scaling, can be parallelized across the length of a sequence, and can gracefully handle irregularly spaced observations.
Despite being defined in continuous-time, applying SSMs MTPPs is straightforward. Tying the conditional intensity to the model's state is the obvious choice, but SSMs expect a continuous- valued signal to integrate input, whereas event sequences from classical parametric linear Hawkes processes (LHPs) [Hawkes] 1971]. we introduce stochastic jump differential equation on the complex plane to serve the SSM recurrence. We refer this as a latent linear Hawkes (LLH) layer. We derive closed-form update rules for any time that allow the diagonalized continuous-time our state-space point process (S2P2) model, a stack of LLH layers interlcaved with position-wise nonlinear functions (Fig. [a). inspired by deep SSMs. This design yields an MTPP that has both highly flexible conditional intensity function and access to efficient, parallelizable computation.
discontinuous by nature. Taking inspiration
SSM system use efficient parallel scans. We then introduce
This paper is organized as follows: In Section 2 we present the necessary preliminaries MTPPs, LHPs, and deep SSMs. In Section 3 we introduce our state-space point process architecture. We first build the connection between LHPs and deep SSMs, and then derive how to use this connection to make a highly expressive, parsimonious and parallelizable MTPP. In Section 5 we then verify this expressivity on small-scale, targeted synthetic explorations comparing key baselines. In Section we empirically evaluate our model at scale on a range of metrics across eight world datasets, finding that S2P2 matches or exceeds the average predictive performance of baselines. achieving either best- or second-best average performance on all six metrics. These results are sum- marized in Table [b] In Sections 7 and & we conclude by discussing the relative advantages and limitations of S2P2 and promising future opportunities. Due to its robustness, performance, and efficiency, our S2P2 model is powerful model out-of of-the-box for wide range of MTPP applications.
2 Preliminaries
2.1 Marked Temporal Point Processes
We define an event sequence, or history, as a sequence of time-mark pairs, H, := {(t,, I < t for i E ]+ where t; E Ezo, Vi f,-1 1 < t,, and k; E M In this paper, we focus on discrete and
Pleasse refer to Tables 3 and Appendix A for list of notation and acronym definitions, respectively.

---

finite mark spaces, i.e., M: {l..... K}: however, M can be more general, such as countable or continuous. We also define H4- similarly to Hr except that excludes events at exactly time
One way of characterizing an MTPP is through a marked intensity process. The intensity A =
[X,,.... X R characterizes MTPP by describing the rate of occurance of events:
dt == E levent of type k occurs in [t, t dt) [H-]]
with the total intensity > : Σ X being the rate that any event occurs. This intensity also defines a marked counting process N, : [N,..... N E z representing the number of occurrences of events of each type of mark the time span [0, t].
Parameterized forms of À are often trained by optimizing the log- log-likelihood over observed data (e.g.. [Mei and Eisner 2017 Zuo al. 2020]). The log-likelihood for single sequence Hy defined as Daley and Vere-Jones 2003 ch. 7.3]:
[(Hr) = Ex log Xh: - -fa A
Nr
t=
Linear Hawkes Processes foundational MTPP the lineur Hawkes process (LHP). The LHP self-exciting process, where event occurrence increases the rate of occurrence of other events, with the influence decaying according to a kernel. The intensity function the summation of influences, and if the kernel is the exponential function, it has the following integral and differential forms:
A -
exp (- 1.)) = dd, - v)dt + adN
3()
where, to ensure non non-negative marked intensities, v E R and a, exp (-3) E R KxK with exp being the matrix exponential, This form (which is the most common) the LHP is incredibly limited, and hence is used primarily for its interpretability, as opposed to outright predictive performance.
20
2.2 Deep State-Space Models
Deep state-space models (SSMs) are class of recurrent models that have excelled in long-range sequence and language modeling tasks, all while having favorable computational properties (Gu et al. 2022b Smith et al., 2022] Gu and Dao 22023. The backbone of deep SSMs are linear state-space models, which define continuous-time dynamical system with inputs and outputs u(t), E R through linear differential equations:
dt x(t = Ax(t) + Bu(t), A E BERP
(4)
y()) Cx(() Du((),
DER
(5)
where x(t) IRP is the (hidden) state of the system, and A, A,B.C,D define the system's dynamics. Deep SSMs interleave these recurrences with nonlinear position sesition-wise functions a as (t) == (t)) (for layer 1). This yields sequence model where each recurrence is conditionally linear in time but is nonlinear overall due to a.
To evaluate SSM, we first discretize the continuous-time system at appropriate times yield a discrete sequence of closed-form state updates (Smith et al.] 2022]. The resulting discrete-time recurrence can be evaluated using parallel scans [Blelloch] 1990]. with linear work scaling and, importantly, sublinear (theoretically logarithmic) scaling of the computation time with respect to sequence length given sufficient parallel compute. This parallel evaluation natively allows varying observation intervals or latent dynamics.
3 State-Space Point Process
We seek to define an MTPP model that is (a) highly expressive and (b) can access efficient and parallelizable compute methods. In this section, we formally introduce our state-space point process (S2P2) model, as outlined below: Section extends and generalizes the continuous-time form of the LHP, creating a layer refer to the latent linear Hawkes (LLH) layer. This generalization
3

---

Ne
X(
Order 1st
Oflag 2nd
Ale
ig
Previous LLH Lavers
Figure 2: Schematic of our state-space point process. We depict the internals of a single LLH layer of the model in their continuous time form (left) and discrete computations (right). Black arrows can be concurrently computed in logarithmic time, and gray arrows in constant time.
decouples the choice of the width of the layer from the mark space, allowing us to make arbitrarily wide layers, and is the basis for the novel connection between the LHP and deep SSMs. Section 3.2 discusses how we make each LLH efficient and tractable to compute with parallel scans, and Section 3.3
extends the model with time-varying dynamics to be more expressive. We conclude in Section 3.4 by describing S2P2, a highly expressive MTPP model built from the composition of multiple LLH layers and nonlinearities that can be computed with novel parallel inference scheme.
3.1
Ciiiiuoou-Timee Latent Linear Hawkes Layer
To develop recurrent layer for an MTPP model, we start by reviewing the equations for an LHP intensity, Eq. (3), and an SSM state, Eq. 4 If we allow in the LHP to vary over time, we obtain:
LHP:
dA, - X dt + Bu,dt + adN,
E ERK
SSM:
dx((t= Ax(t)dt + Bu(t)
E ERP.
(7)
When presented together the parallels between them become apparent: the LHP intensity, X con- trolled by decay rates [3, is analogous to the state in the linear SSM, x(f) controlled by state matrix is analogous to the SSM input signal after being
A. The time-varying background LHP intensity, projected into the state-space, Bu((). Compared to SSMs which allow for recurrence over latent space RR,, Eq. (6) is limited in expressivity due to restricted dynamics B and fixed dimensionality tied to the number of possible marks K. Not present in Eq. 77) is an impulse ad which is crucial for allowing the recurrence to condition on abrupt event occurrences over time.
With this in mind, combine Eqs. O6) and 7 resolve their respective deficiencies. This results in the following set of stochastic jump differential equations that we call the latent linear Hawkes (LLH) layer, which will serve as a continuous time recurrence layer in our proposed model:
LLH:
dx: = - dt + ABu + EodN,
E RR
(8)
y: = Cx, Du,
E R
(9)
where a E R E PPRR and the standard SSM parameters share the same dimensionality as Eqs. (4) and 55) Note that the resulting dynamics within the L.HH layer are much more expressive than the T.P that it was inspired by, due to both operating in separate set of latent dimensions and by having general dynamics matrix A E R PxP, compared to the more restrictive cxp(-3) E IR200 k For parameter efficiency, we allow the new impulse term EadN, to be computed as a product of model-widd wide, low-rank mark embeddings a of rank and layer-specifer projection into state-space E. For simplicity, we set R = H in practice. Realizations of this layer are shown in Fig. 2
This L.H will serve the main form of recurrence in larger continuous-time model that we define in Section 3.4. In deriving this layer as an extension from LHPs, we retain a strong inductive bias towards event sequences. Furthermore, by drawing connections between this layer and traditional state-space equations, we allow the downstream model to benefit from many of the innovations designed for deep SSMs in recent years, such as parallel computation.
3.2 Diagonalizing, Discretizing & Computing the LLH Recurrence
Unlike the LHP intensity, there no analytical solution to the continuous-time L.HH recurrence due to the ciittnuouul--ittgrattee u(t) signal. We must therefore approximate the system at specific

---

timepoints. If we approximate the input signal by treating it as constant over an update interval, also known as zero-order hold (ZOH) assumption [[serles] 2009). then we can achieve a closed-form exact update to the recurrence relation. To avoid a computationally expensive matrix exponential in the update rule, we adopt the same general approach as Smith et al. [022] for deep SSMs by first diagonalizing the system and then imposing the zero-order hold restriction on it. This converts the matrix exponential into an element-wise exponential operation for each LLH layer in S2P2.
Diagonalization: Let -A be diagonalizable with factorization of VAV-1, where V, E CPxP and A is a diagonal matrix of eigenvalues. An equivalent, diagonalized LLH is then
dx; AX, dt + dt + EodN,
(10)
Yt C*2 + Du,
(11)
where X -V-1,, =V -
V-'E, and Č Č= CV. We then directly parameterize B. C. and avoid learning and inverting V. We use the same initialization strategies as S5 Smith al. 2022] for B, C, and E, based off the HiPPO initialization scheme [Gu et al. 2020 The eigenvalues A are also directly parameterized and constrained with negative real-components to enforce stability. While the dynamics are diagonalized, we note this does nol mean that modeling the intensities of different mark types as independent. Marks interact through both the position-wist nonlinearities and the learned projections B, C and E (since the diagonalized dynamics are equivalent the original dynamics in Eq. given that the system can be diagonalized on the complex plane).
We then ZOH discretization to create a closed-form update from the diagonal- ized continuous-time system; please refer to Fig. for an illustration and Appendix [822 for the full derivation. This results in the following update rule in the diagonalized eigenbasis, that transitions from X2 to Xy without a matrix exponential, where, by construction, no events occur in (t, t'):
Discretization:
Ax + (A I)Bu,- Ax + (Ā I)Bu,- + Eag if event of type l: at t'
if no event at
X =
(12)
where A == (Ap( - t)). ZOH is an exact update when u is constant over the window [t,:'); as (²) to hold constant as the input over the integration
such, we choose any value u, for period. We opt to use ur- so that the model can condition on the fact that no events have occurred between and f'. This design decision and its impact on performance
explored in more detail in and D.6. It is important that uy is not used as the ZOH value to avoid data leakage.
Appendices and
Computing LLH Recurrence: The final component is to derive how to parallel scans to efficiently evaluate the closed-form updates for the modified LLH recurrence in parallel. Parallel scans admit efficient inference over linear recurrences of the form 2,+1 R,z, b, [Blelloch] 1990]. Although we have an impulse in the recurrence, Eq. (8) is still intrinsically of this form, where Z; := X1, R := exp(Az (A, (fx+11 - tc)). and b, := (R, B + Ea As a result, we can leverage parallel scans to compute the sequence of right-limits X in parallel across the sequence length. The corresponding left-limits N which will later be used to calculate event intensities, can then be directly and efficiently computed by subtracting the impulse, EOCk from Xt,xx
3.3 Input-Dependent Dynamics
Inspired by recent developments in modern SSMs (e.g., Mamba [Gu and Dao 2023 )) we also consider allowing the dynamics of the system to vary depending on the input and on the history of previous events. This can allow for more expressive intensities. For instance, dynamically adjusting the real components of A to be smaller will result in more influence from history. Alternatively, larger values will result in more quickly "forgetting" the influence of previous events for given hidden state channel. This is formalized with the following recurrence relation for t e (f,, t411]:
dx; = At A, dt + EodN,
(13)
where A; == diag (softplus( W'u, - b')) A with W' E Px and b' E R... This is conditionally linear time, as even though A, changes, it entirely input-dependent based on u and not dependent on previous values of x, and hence we can still use parallel scans as discussed above.
5

---

3.4 State-Space Point Process Architecture
We are now cell-positiond to present the state-space point process (S2P2), a flexible and parallel deep continuous- time model for marked intensities X. This model is shown in Figs. la and 2
We have demonstrated how define efficient, parallelizable and scalable core MTPP layer in the LLH layer. While the diagonalized and discretized LLH layer is more expressive than the LHP, it remains fundamentally linear. To compensate, we take inspiration from deep SSMs and alternate L LLH layers with position-wise nonlinearities. This creates a nonlinear model that can still use parallel computation over the sequence length, but is highly expressive compared each linear recurrence.
We have an input signal in two parts: (i) the continoouslyimmteraaeed signal u, and (ii) the discrete event impulses Ox-- For the first layer, the only inputs available to condition on are the event impulses themselves, so we set u, impulse as well as the continuously integrated signal from the previous layer. In general, a layer's output y(0 : (LH H) is passed into nonlinear activation function (we use o(=) := GELU(2) [Hendrycks and Gimpel 2016)), summed with the residual stream , and normalized with LayerNorm Ba 2016] for the next layer's input. Formally, for t 2 and = I,..., L, then
for all 2 0. At deeper layers, we have a layer-specific
(1+1) LayerNorm 4 (a(y,) (2) + )
(14)
Due to the unrestricted nature of the recurrences and nonlinearities (and unlike the original LHP), we enforce non-negative intensities by applying an affine projection followed by a rectifying trans- formation, similar to Mei and Eisner 2017]. This is referred as the "Proj. & Softplus" layer in ((Wu((+))) (I++)
Fig. and expressed as A := s @ softplus(( ((Wul element- wise product, W E RKK H and b, log(s) E RRK Then S2P2 can be trained by maximizing the log-likelihood of each sequence, Eq. (2). Similar other neural MTPPs, we use Monte Carlo estimation for the integral term S A,d,N, [Mei and Eisner 2017]. Training the model requires computing intensities at both event times IINN and at randomly sampled times (0,
+ b) @ s 1) for 2 and where @ is an
To compute intermediate intensities at these sampled points, we take advantage of the continuous-time nature of S2P2 and partially evolve the latent state through the system dynamics. To do so, we first compute the right limits of the hidden states at event times, x, (((1L)) as described above (we can do this conditioning in logarithmic depth). Then, Eq. (12) is applied, with no impulse, (T..). From there, the evolved hidden
events are occurring at these intermediate points, to find x; states are used to evaluate the model across depth to compute the intermediate intensity X This operation is both efficient and can be done in constant complexity because intermediate evaluations conditionally independent given the right limits at events. Crucially, there is no separate parametric decoding head, unlike, for instance, Mamba Hawkes processes [Gao et al.) [024] or transformer Hawkes processes [Zuo et al. 2020). Instead, this tying of the intensity to the model's 's continuously evolving hidden states, more like the neural Hawkes processes [Me and Eisner 2017]. makes S2P2 a continuous-time model and contributes its enhanced expressivity (see Section 5 and Section 7 for more discussions). In Algorithms T to 3 we explicitly detail how to use a parallel scan to compute the sequence of right limits events, how then evolve those to compute left limits, and then how to subsequently compute the log- likeiikel of the sequence.
Related Work
Neural MTPPs: MTPPs are generative models that jointly model the time and type of continuous-
time sequential events, typically characterized by mark-specific intensity functions (Daley and
Vere-Jones 2003 Early approaches used parametric intensity functions, such self-exciting
Hawkes processes Hawkes 1971 Liniger 2009]. More recently, neural models such as RNNs [Du
al. 2016 Mei and Eisner 2017 and transtormers (Zhang al. 2020 Zuo et al. 2020 Yang et al.
2022
were developed to enable flexible modeling of conditional intensities. Intensity-free MTPPs
include normalizing flows [Shchur et al. 2020al Zagatti et al.) 2024 transformers (Drxxler et al.
2025
neural processes [Bae et al.] 22023, and diffusion models [Zeng et al. [024 Zhang et al.
2024
however, modeling intensities is more common, requiring fewer modeling restrictions.
Efficient MTPPs: Due to their recurrent nature, RNN-based MTPPs incur
O(N) scaling for length-
N sequences events processed sequentially. Attention-based MTPP models can be applied in

---

parallel across the sequence, but with (N22) computational work. Türkmen et al. 2019b] model events as conditionally independent if they occur within the same time bin of a specified size, resulting in parallel computation within bins, but still scaling as ((N) overall. [Shchur et al. [020b) proposed
an intensity-free TPP using triangular maps and the time-change theorem [Daley and Vere-Jones
2003]. This was extended by ZZaatti et al. 2024) to handle marks but losing benefits of the original model and scaling linearly in the mark dimension -wich can rapidly become untenable as ( K)
work. Our S2P2 scales as (log N) and efficiently in marks; more discussion in Appendix B..
SSMs for Sequential Modeling: SSMs have found recent success as alternatives RNNs, CNNs.
and transformers, enjoying reduced training cost and comparable modelling power [Gu et al. 2022b]
range of variants have been developed [Gu et al.) [202] Gupta et al. 2022 Gu et al. 2022a Smith
et al. 2022] and applied in language modeling [Gu and Dao 2023]. speech [Goel et al. 2022] and
vision Wang et al., 2023, Zhu et al., 2024). The linear recurrence enables parallelism, as well as accessible long contexts that are prohibitive for transformers due to quadratic scaling. However, SSMs have not previously been used as continuous-time models for MTPPs, in part due to the intensity functions having different left and right limits, and the input being stochastic counting process.
2024 propose using an SSM as discrete sequence model, en- coding an event sequence into fixed set of static hidden states, then computing intensities with a separate parametric decoder (similar to transformer Hawkes processes (Zuo et al. 2020]. but with the transformer replaced with Mamba [Gu and Dao (023]). This is fundamentally different to our continuous-tin time model, where instead of a separate parametric decoding head, we leverage the continuously evolving latent state at any time i to compute predicted intensities at corresponding times. We empirically compare to their model MHP in Section [6] finding that MHP (i) performs comparably to the THP, and (ii) is comprehensively outperformed by S2P2.
SSMs for TPPs: Gao et al.
5 Synthetic Experiments
We first verify the ability of our S2P2 model represent known intensity functions through series of targeted experiments on synthetic data. Recent results from the deep SSM literature [Muca Cirone al. 2024] show that sufficiently deep SSMs are able to approximate arbitrary continuous functions; their results apply to our S2P2 model, and hence we should be able to (verifiably) recover arbitrary and known intensity functions. To demonstrate this, we showcase recovering ground truth intensities in three different settings: the classical point processes of Hawkes and self-correcting processes; an inhomogeneous Poisson process with a discontinuous intensity function; and finally marked process with long- pong-range dependencies between marks. Our S2P2 recovers the correct intensity in all cases. We include full results for all baselines and reproducibility details for all the following examples and tasks in Appendix D including intensity visualizations, where S2P2 works as intended, corrects recovers intensities, and matches or exceeds baseline methods performance.
We first apply our model and baseline models to two classical point processes: a Hawkes process and a self-correcting process with intensity functions
Classical Point Processes:
Ground Trath
-212
A = 0.5 + Σ, exp(t, - t) and X = exp(t - Ni). re- spectively. Examples of the recovered intensity functions are shown in Fig. 4a We see that our S2P2 recovers the ground truth intensities nearly perfectly. We also explore applying our model and other baseline models to randomly generated syn- thetic multivariate Hawkes processes in Appendix D.3, finding that all methods perform comparably.
X
0
6
7
Time
Figure 3: Intensity estimates
from trained models when con- ditioned on an empty sequence H, = 0 for NHP [Mei and Eis- ner, 2017] and our S2P2. Dotted lines show the ground truth inten- sity for an inhomogencous Pois- son process. S2P2 accurately cap- tures the background intensity.
Our S2P2 does not have a fixed parametric form for the intensity decoder (cf. THP or MHP) or limited recurrent dynamics (cf. NHP). This flexibil- ity should allow the S2P2 to capture intensities where other methods fail. This is showcased in Fig. 3 Models are trained on sequences drawn from an inhomogeneous Poisson process with square wave for an intensity function (except for i > 7). We observe that baseline models fail in predictable ways due to their expressivity limitations (please refer to Appendix Dp.pedix(D.1rrr for full results), whereas our model successfully captures the true background intensity process almost perfectly.
Inhomogeneous Poisson Process:
7

---

Ground Truth
Predicted X
SC
10+
NHP
X
X1.
0
H1
Hawkes
S2P2
X;
X
1
01
744
111
18
111
11
H4
110
10
20
30
40
50
20
40
60
80
100
(a) S2P2 successfully recovers parametric intensi- ties from self-correcting and Hawkes processes.
(b) Long-range dependencies between |-1 are accurately captured by S2P2, whereas NHP struggles to recover As.
Figure Results for synthetic experiments in Section 5
Lastly, we study a task with known long-range dependencies, depicted in Fig. 4b In this task, "trigger" mark (green) is drawn from homogeneous Poisson process (A=0.1); each trigger is then followed by a "target" mark (orange) predictably long time later. Here "long" refers to the fact that (a) there are often many trigger marks before single target mark, and (b) the variance of the distribution over the trigger-target time is comparatively narrow compared to the mean. Gray marks are "distractors" drawn from homogeneous Poisson process (A=1). We see that S2P2 successfully captures the long-range dependencies, whereas NHP struggles. In spite of the long delay between cause and effect within the data, S2P2 was able to recover 98% of the true data likelihood, whereas NHP only achieved 88%. This shows that the state-space point process successfully captures long-range dependencies and is flexible as approximation family.
Long-Range Dependencies:
6 Real-Worles Experiments
We now present the main experiments in this paper, evaluating on real- world datasets Further results and extensive experimental details are included in Appendices C and D
Datasets: We compare models on eight different datasets, including five datasets available from EasyTPP [Xue et al.] 2023) (Amazon, Retweet, Taxi, Taobao and StackOverflow) We also add two commonly used datasets in the literature (Last.fm and MIMIC-II,, as well as new medical events dataset derived from the publicly available EHRSHOT dataset [Wornow et al.) 2023]. EHRSHOT an order of magnitude larger than any dataset EasyTPP both the number marks and the maximum sequence length (see summary statistics in Table 7). representing challenging application.
Metrics: We performed a comprehensive evaluation using three common families of metrics: (i)
total per-event log- ligeliidl [Mei and Eisner 2017 [Zhang et al., 2020 Zuo et al. 2020 Yang et al.)
2022 Gao et al., 2024) (also broken down into mark- and time- specific likelihoods): )); (ii) predictive
accuracy summary metrics of next-mark classification and next event time RMSE (Du et al. 2016
Zhang et al. 2020 Zuo et al. 2020 Yang et al. 2022, Gao et al. 2024]; and (iii) mark and event time calibration metrics [Zhang et al. 2020 [osssr and Taieb 2023] measuring the reliability of the implied uncertainties (see Appendix D.7). In order to characterize overall performance across multiple diverse metrics, we also compute summary composite rank in Table lb defined as the average rank obtained for a given model across all datasct-metric combinations across five random seeds (i.c., is an average of 240 individual evaluations and five differently initialized trained models).
Models: We use the EasyTPP library [Xue al. 2023) for baseline models and S2P2. We perform extensive hyperparameter searches for each individual pair of baseline model and dataset, then select
*Our model is fully integrated into the EasyTPP Xue et al. 2023 library [link]. Other code changes to reproduce results can be found in our forked repository [lnkk] Model checkpoints are available request.
8

---

Table 2: Results for S2P2 and baselines across key metrics. We show the mean on the held-out test set evaluated across five random training seeds (std. dev. in parentheses). OOM indicates insufficient memory. We also report the mean rank of models across datasets as a summary metric (for which lower 4 is better). S2P2 is consistently the best or second best-performing model on average on each dataset and metric. Extended results and discussion are shown in Appendix D.1
(a) Per event total log- likelihood. Higher values indicate better performance.
Model
Per Event Log- Likellbood
Euss (nais) (Higher is beteer, best. second best)
Averuge
Amazon
Retweet
Taxi
Taobao
StuckOverlloe
Lasifun
MIMIC-I
EHRSHOT
Kank (1)
RMIIP
-2.136 (0.00))
-7.0%%
0,346 (0.002)
10033
-2,40 (0.019)
-1.780
40.72 (3.026)
-5.81
(0.025)
-2.074 (2.09)
-6.708 (0.029)
0.298 (0.077)
-2:541 (0.058)
-0.77
-6800
TEPP
6.659
(0.014)
1.712
(0.011)
(0:096)
IFTPP
0,996 (0.0012)
-10.342 (0:016)
0.453 (0.000)
(8.07))
-2.238 (0.009)
E0,392 (0.017)
0.317 (0.052)
-.59% (0.240)
МHP
-2.091 (020
-6. -.564/00 (0.015)
0.370
(0.00)
0.636
(0.004)
-2.366 (0.012)
-1.676 (0.004)
-0.351 (0.012)
-7.206 (0:407)
3.9
NLIP
0.129 (0.012)
F6.388 (0.000)
0,514
(0.000)
1.157
(0.004)
-2241 (0.002)
0..54 (0.011)
0,060 (0.017)
3.966
(0.058)
3.0
AINHP
0.484 (0.077)
-6.499
0.455 (8.0095
1.259
-2.194 (0.016)
-0.592
-0.170 (3.07)
00M
S2P2 (Ouns)
0.751
M.522
E2.16N (0.009)
-0.557
0.919
22.512 363)
(b) RMSE of the next event time prediction. Lower RMSE values indicate better performance.
Model
Next Event Time RMSE (Lower is better,
second best)
Average
Amazon
Retweei
Taxi
Taobao
StackOverloww
Last.in
MIMIC-I
EHRSHOT
Rank (()
RMTTP
16488
0.283
0:126
1.049 (0.003)
15:873
(0:010)
3425
(0.001)
16192 (62.4)
0.290
0:125
(0.0000
(0.011)
15:257
(0.007)
(0.198)
3374 (9:4)
THP
0.352 (0.000)
16258 (18.7)
0.285 00.001)
0.128
(0.000)
1,033 (0.005)
15.871
(0.000)
0.768 (0.005)
3414 (1.0)
4.3
IFTPP
18127 (0.000)
16625 (0.2)
0.362 (0.1785
0.125
(0.000)
1.340 (0.724)
16.308 (0.555)
0.767
(0.029)
3616 (17.6)
5.0
MHP
0,329 (0.000)
(36.9)36.9)
0.284 (0.0033
0,126
(0.000)
1.046 (0.030)
158871
(0.000)
0.758 (0.065)
2418 (5.8)
4,0
NHI
0.339 (0.000)
11591 (4.0)
0.282 (0.001)
0.126
(0.000)
L019 (0.001)
13.733
(0.008)
0.726 (0.001)
3330 (30.9)
24
HP
2,656 (1.950)
16171 (26.2)
L:759 (0.422)
0.130 (0.000)
1,256 (0.033)
15:865
(0.017)
0.860 (0.022)
OUM
0.0
S2P2 (Ours)
0.327 (0.000)
15967 (13.7)
M.281 (0.000)
0.126 (0.000)
[1.04] (0.001)
[15.220]
00.000)
0.894 (0.054)
2368 (14.4)
2.3
(c) Next mark classification accuracy (top- 10 accuracy on EHRSHOT), Higher values indicate better performance.
Model
Next Mark Classification Accuracy (%) (Higher is better, best, second-best)
Average
Amazon
Retweet
Taxi
Thobuo
StuckOverflow
Last.im
MIMIC-11
EHRSHOT
Rank ())
RMIIP
30.8
53.4
91.4 (0.1)
60.9 (0.1)
45.6 (0.3)
(0.1)
(0.3)
36.5 (0.2)
SAHP
32.4 (1.0)
57.5 (2.2)
91.4 (0.7)
60.5 (0.2)
44.7 (2.0)
51.8 (0.7)
86.8 (0.9)
49.6
7.1
THP
(0.1)
60.2
91.4
60.0 (0.0)
46.6 (0.2)
53.3 (0.1)
(0.2)
49.7
IFTPP
35.9 (0.1)
50.4 (2.5)
91.8 (0.0)
(0.1)
45.6 (0.1)
56.4 (0.1)
93.4 (0.1)
(3.8)
4,3
MHP
(0.1)
60.0 (0.2)
91.4 (0.3)
60.7 (0.2)
46.5
(0.1)
(0.3)
54.8 (4.5)
NHP
39.4 (0.1)
61 (0.0)
(2.9
615 (0.2)
47.1 (0.1)
56.5 (0.1)
94.3 (0.0)
76.5 (0.2)
[L]]
AtNHP
38.9 (0.9)
60.7 (0.2)
92.6 (0.1)
61.3 (0.2)
482)
55.8 (0.6)
92.9 (0.6)
00M
2.9
S2P2 (Ours)
(40.7 (0.0)
61.3 (0.0)
9931 (0.1)
61.1 (0.1)
47.5 (0.3)
55.8 (0.4)
19.0 (0:4)
(0.3)
1.9
the configurations that maximizes validation log-likelihood per model/dataset. The range of grid search and other details are elaborated in Appendix C.I results are reported on a fully held-out test set. All models were trained on a single 24GB NVIDIA A5000 GPU.
We report full numerical results in Table 2 and summary of rankings in Table Ib Table 2a shows that S2P2 consistently achieves the best or the second-best held-out log- likeliied results across all datasets, beating other methods by over whole rank and with a (geometric) mean likelihood ratio of 1.33 (corresponding to a 33% higher likelihood of true events). This is calculated by computing the mean log- likeliiokoood ratio across all datasets and then exponentiating. We further investigate the log-likelihood improvement of S2P2 by separating the time- and mark-specific log- likelihood, finding that S2P2 is mainly driven by better temporal modeling and while achieving gains in both time and mark modeling over existing methods (see Table & and Fig. 6 in Appendix [D. for full results). Additionally, we report prediction summary metrics, next event time prediction and next mark classification, in Table 2h and Table 2c We again see that S2P2 performs well, matching or exceeding the best-perf baseline, and far outpacing other baseline methods.
Main Results:
Additional Experiments: We defer several additional experiments and explorations to Appendices:
1. Calibration results: We also include full calibration results, both in raw numerical form and as reliability graphs [Bosser and Taieb 2023). These show that S2P2 is well calibrated, placing highest amongst intensity- based methods, only being beaten by IFTPP (Shchur et al.) 2000aa. Computational scaling: We define the theoretical time complexity of each baseline, where S2P2 is best with logarithmic complexity and linear work. We verify this scaling empirically.
Ablations: We also perform an ablation study on whether input dependent dynamics and the choice of which input to hold constant over the integration interval (introduced in Sections 3.2 and 3.3

---

affect performance, We find input-dependent dynamics nearly always improve performance, and that performance is less sensitive to the specific form of the LLH's input signal u, in ZOH.
7 Discussion
Continuous-Time Hidden States: The S2P2 and NHP [Mei and Eisner 2017] architectures
closely related, in that both model contimuous-ting time latent states. For the NHP, the intensity function is parameterized by hidden state h(t) evolved via continuous-time LSTM variant. Similarly, S2P2 continuously evolves a set of continuous- time latent states h (t), one for each layer {l...., ., L}. Then the hidden state h (t) of the top layer is decoded into intensities any t. In contrast, models such as THP, MHP, IFTPP, only have latent states h(ty) at discrete event times (I:N): Consequently, some parametric shapes need to be defined to "interpolate" the hidden state values between events, thus sacrificing models' flexibility due to these parametric functions.
Our results are grouped according to whether not models have continuous-time hidden states. We empirically verify that continuously evolving hidden states enable more expressive latent dynamics. where the expressiveness in explaining the data is measured through log- likelihood;; Appendix [.1] S2P2 further improves scalability by incorporating SSMs that are naturally in continuous time and by leveraging efficient parallelized scans. S2P2 also introduces stronger inductive bias than most MTPP models (similar to NHP), helping to achieve superior next event predictions.
Limitation and Future Directions: S2P2 forgoes the interpretability of the parameters and latent states of linear Hawkes processes (LHPs), meaning it is not appropriate when the interpretation of the underlying system is crucial. This tradeoff is common with neural MTPP models (e.g., NHP), because the limited expressivity of I.HP may lead to poor predictive performance, as well as misleading interpretations due to underfitting. Therefore, an exciting opportunity for future research is exploiting the connection to recover LHP-like interpretability while retaining the enhanced predictive power of S2P2. For example, any recently developed techniques for interpreting deep SSMs can be extended to S2P2, such as Ali et al. 22025), while fully taking advantage of the continuously varying attention from continuous attention maps (e.g., large language models).
latent dynamics of S2P2 will provide richer insights compared to discrete-time
Beyond this, further directions include leveraging theories and best practices from deep SSMs, such as the enhanced parameterizations presented by Merril et al. [2024). S2P2 can also naturally be extended and accommodate non-categorical marks by tailoring its architecture the structure of the mark space, such as marks having continuous values or containing richer information. A concrete example is teapioltemporal point processes that have applications in seismic and weather forecasting. Other valuable extensions include serving S2P2 as a pre-trained backbone for downstream applications, such EHRSHOT clinical classification tasks. Additionally, censoring adversarial conditions are commonly seen in practice [Boyd et al.) 20233] [Chakraborty et al.) 2025). Extending the evaluation of S2P2 and MTPP models in general remains an open direction for real-world MTPP deployments.
8 Conclusion
In this paper, we present the state-space point process (S2P2) model-a novel fusion of concepts from LHPs and SSMs. S2P2 uses deep stacks of stochastic jump differential equations to create an expressive and parsimonious MTPP without additional and restrictive intensity decoding heads, while simultancously being able to leverage parameterizations and techniques borrowed from deep SSM architectures. We demonstrated that S2P2 outperforms existing methods across range of standard and new benchmark tasks, and over a range of predictive metrics and efficiency evaluations.
As an intensity- based model, S2P2 requires numerical integration for predicting the expected next event times (as with any neural intensity-baat based MTPP model) and is not able to model point masses in time (due to the MTPP assumption of no concurrent events). All other baselines suffer from at least one key additional deficiency (e.g., NHP has linear scaling, THP has quadratic work, IFTPP has parametric decoder, etc.), all of which are resolved by S2P2, while also achieving better performance. We provide a Py implementation via EasyTPP for out-of-the-box usage. We believe the robustness, state-of-tt -tate-of-the-at predictive performance across range of metrics, computational efficiency, and extensibility of S2P2 make it very competitive model for a wide range of MTPP applications.
10

---

Acknowledgements
We thank the reviewers for their invaluable feedback on improving the paper. This work was supported by the Hasso Plattner Institute (HPI) Research Center in Machine Learning and Data Science at the University of California, Irvine, by National Science Foundation awards 2505006 and 2425932, and by the National Institutes of Health under awards R01-LM013344 and R01 and by GE HealthCare.
References
Ameen Ali Ali, Itamar Zimerman, and Lior Wolf. The hidden attention of Mamba models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1516-1534, 2025.
Jimmy Lei Ba. Layer normalization. arXiv preprint arXiv:1607.06455 2016.
Wonho Bae, Mohamed Osama Ahmed, Frederick Tung, and Gabriel L. Oliveira. Meta temporal point processes. In The Eleventh International Conference on Learning Representations, 2023.
Guy Blelloch. Prefix sums and their applications. Technical report, Tech. rept. CMU-CS-90-199. School of Computer Science, Carnegie Mellon, 1990.
Tanguy Bosser and Souhaib Ben Taieb. On the predictive accuracy of neural temporal point process models for continuous-time event data. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
Alex Boyd, Robert Bamler, Stephan Mandt, and Padhraic Smyth. User-dependent neural sequence models for continuous-time event data. Advances in Neural Information Processing Systems, 34: 1148892149,, 2020.
Alex Boyd, Yuxin Chang, Stephan Mandt, and Padhraic Smyth. Inference for mark-censored temporal point processes. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216, pages 226-236. PMLR, 2023.
Oscar Celma. Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space. Springer, 2010.
Pritish Chakraborty, Vinayak Gupta, Srikanta Bedathur, Abir De, et al. Differentiable adversarial attacks for marked temporal point processes. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 15702, 2025.
Yuxin Chang, Alex Boyd, and Padhraic Smyth. Probabilistic modeling for sequences of sets in continuous-time, International Conference on Artificial Intelligence and Statistics, volume 238, pages 4357-4365. PMLR, 2024.
Daryl Daley and David Vere-Jones. An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods, page 258. Springer, 2003.
Fei Deng, Junyeong Park. and Sungjin Ahn. Facing off world model backbones: RNNs, Transformers, and S4. Advances in Neural Information Processing Systems, 37:72904-72930, 2023.
Felix Draxler, Yang Meng, Kai Nelson, Lukas Laskowski, Yibo Yang, Theofanis Karaletsos, and Stephan Mandt. Transformers for mixed-type event sequences. Advances in Neurul Information Processing Systems, volume 39, 2025.
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pages 1555-1564, 2016.
Álvaro Gajardo and Hans-Georg Müller. Point process models for COVID-19 cases and deaths. 2023.
Journal of Applied Statistics,

---

Anningzhe Gao, Shan Dai, and Yan Hu. Mamba Hawkes process. arXiv preprint arXiv:2407.05300 2024.
Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state- space models. In International Conference on Machine Learning, volume 162, pages 7616-7633. PMLR, 2022.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, arXiv preprint arXiv:2312.00752, 2023.
Albert Gu. Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. HiPPO: Recurrent memory with optimal polynomial projections. Advances Neural Information Processing Systems, 34: 1974-14877 2020.
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in Neural Information Processing Systems, 35:572-585, 2021.
Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 36:35971- 1- 35983, 2022a.
Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces, In Proceedings of The Tenth International Conference on Learning Representations, 2022b.
Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are effective as structured state spaces. Advances in Neural Information Processing Systems, 36:22982-22999, 2022.
Alan Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58 (1):83-90, 1971.
Dan Hendryeks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint (rXiv:1666008455 2016.
William Hua, Hongyuan Mei, Sarah Zohar, Magali Giral, and Yanxun Xu. Personalized dynamic treatment regimes in continuous time: a Bayesian approach for optimizing clinical decisions with timing. Bayesian Analysis, 17(3):44-877,, 2022.
Arieh Iserles. First Course in the Numerical Analysis of Differential Equations. Cambridge University Press, 2nd edition, 2009.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of The Third International Conference on Learning Representations, 2015.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1269-1278, 2019.
Thomas Josef Liniger. Multivariate Hawkes processes. PhD thesis, ETH Zurich, 2009.
Brian McFee, Thierry Partin-Mahieux, Daniel PW Ellis, and Gert RG Lanckriet. The million song dataset challenge. In Proceedings of the 21 International Conference on World Wide Web, pages 909-916, 2012.
Hongyuan Mei and Jason M Eisner. The neural Hawkes process: A neurally self melf-modulating multivariate point process. Advances in Neural Information Processing Systems, 31:6757-6767, 2017.
Hongyuan Mei, Guanghui Qin, and Jason Eisner. Imputing missing events in continuous-time event streams. In International Conference on Machine Learning, volume 97, pages 4475-4485. PMLR, 2019.
William Merrill, Jackson Petty, and Ashish Sabharwal. The illusion of state in state-space models. In International Conference Machine Learning, volume 235, pages 35492-35506. PMLR, 2024.
12

---

Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons. Theoret- ical foundations of deep selective state-space models. Advances in Neural Information Processing Systems, 38:127226-127272 2024.
Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of The 2019 Conference on Empirical Methods in Natural Language Processing and The 9th International Joint Conference on Natural Language Processing (EMNIP-IJCNLP)), pages 188-197, 2019.
Mohammed Saeed, Christine Lieu, Greg Raber, and Roger G Mark. MIMIC II: massive temporal ICU patient database to support research intelligent patient monitoring. In Computers in Cardiology, volume 29, pages 641-6444 2002.
Anuj Sharma, Robert Johnson, Florian Engert, and Scott Linderman. Point process latent variable models of larval zebrafish behavior. Advances in Neural Information Processing Systems, 32: 10942 10953, 2018.
Oleksandr Shchur, Marin Biloš, and Stephan Günnemann. Intensity-free learning temporal point processes. In Proceedings of The Eighth International Conference on Learning Representutions, 2020a.
Oleksandr Shehur, Nicholas Gao, Marin Biloš, and Stephan Günnemann. Fast and flexible temporal point processes with triangular maps. Advances in Neural Information Processing Systems, 34: 73 84, 2020b.
Jimmy TH Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. In Proceedings of The Tenth International Conference on Learning Represen- tations, 2022.
Ali Caner Türkmen, Yuyang Wang, and Tim Januschowski. Intermittent demand forecasting with deep renewal processes. arXiv preprint arXiv:I911.100 2019a.
Ali Caner Türkmen, Yuyang Wang, and Alexander J Smola. Fastpoint: Scalable deep point processes. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD, pages 465 480. Springer, 2019b.
Bjørnar Vassøy, Massimiliano Ruocco, Eliezer de Souza da Silva, and Erlend Aune. Time of the essence: a joint hierarchical RNN and point process model for time and item predictions. In Proceedings of The Twelfth ACM International Conference on Web Search and Data Mining, pages 591-599, 2019.
Jianlong Wang, Xiaoqi Duan, Peixiao Wang, A-Gen Qiu, and Zeqiang Chen. Predicting urban signal- controlled intersection congestion events using spatio-temporal neural point process. International Journal of Digital Earth, 17(1):2376270, 2024.
Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, and Raffay Hamid. Se- lective structured state-spaces for long-form video understanding. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 6397, 2023.
Chris Whong. FOILing NYC's taxi trip data. myc_taxii/ 2014. [Online; accessed Oct 15, 2024].
Alex Williams, Anthony Degleris, Yixin Wang, and Scott Linderman. Point process models for sequence detection in high-dimensional neural spike trains. Advances in Neural Information Processing Systems, 11114500-143611 2020.
Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. EHRSHOT: An EHR benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems, 37:67125-67137, 2023.
Siqiao Xue, Xiaoming Shi, James Zhang, and Hongyuan Mei. HYPRO: A hybridly normalized probabilistic model for long-horizon prediction of event sequences. Advances in Neural Information Processing Systems, 36:34641-3466 2022.
13

---

Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan, James Y Zhang, Qingsong Wen, et al. Easy Towards open benchmarking temporal point processes. In Proceedings The Twelfth International Conference on Learning Representations, 2023.
Chenghao Yang, Hongyuan Mei, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of The Tenth International Conference on Learning Representations, 2022.
Guolei Yang, Ying Cai, and Chandan Reddy, Recurrent spatio-temporal point process for check-in time prediction. In Proceedings of the 27th ACM International Conference Information and Knowledge Management, pages 2203-2211 2211, 2018.
Guilherme Augusto Zagatti, See Kiong Ng, and Stéphane Bressan. Learning multivariate temporal point processes via the time-change theorem. In International Conference Artificial Intelligence and Statistics, volume 238, pages 1241-3249. PMLR, 2024.
Mai Zeng,
Florence Regol, and Mark Coates. Interacting diffusion processes for event sequence forecasting. In International Conference on Machine Learning, volume 235, pages 58407-58430. PMLR, 2024.
Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive Hawkes process. In International Conference on Machine Learning, volume 119, pages 11183-11192 PMLR, 2020.
Shuai Zhang, Chuan Zhou, Yang Aron Liu, Peng Zhang. Xixun Lin, and Zhi-Ming Ma. Neural jump-diffung diffusion temporal point processes. In International Conference on Machine Learning. volume 235, pages 60541 60557. PMLR, 2024.
Qingyuan Zhao, Murat Erdogdu, Hera Y He, Anand Rajaraman, and Jure Leskovee. Seismie: self-exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1513-1522, 2015.
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision Mamba: Efficient visual representation learning with bidirectional state space model. In International Conference Machine Learning, volume 235, pages 62429 - -62442. PMLR, 2024.
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process. In International Conference on Machine Learning, volume 119, pages 1 11692-11702
PMLR, 2020.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All of the claims are further expanded upon and justified in either the method- ology section or the two experiment sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should elearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals not attained by the paper.
14

---

2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the last two sections in our concluding thoughts.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: There are no theoretical results in our work.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
15

---

Justification:
Extensive reproducibility details are provided in Appendices C and D
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
Please refer to the footnote on page 8.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
htggli//guuddes/oodeeabbmmmssss Pslicollccyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: wwciiiccguieeeeeeee SoooeeeuumisssioPPoo
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
16

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All details can be found in Appendix
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core paper to a level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All numerical results for experiments are presented with standard deviations describing the spread over five different random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: As mentioned the real-world experiments section, all models were trained single 24GB NVIDIA A5000 GPU.
Guidelines:
17

---

The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:/ppp///aaao...ccoco//////aa
Answer: [Yes]
Justification: We fully comply with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We do not believe this paper has any societal impacts, positive negative. Neural MTPPs have existed for almost decade, and our work is the most recent iteration in this well-established domain.
Guidelines:
The answer NA means that there no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML.)
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
18

---

Answer:
[NA]
All of the models were trained on publicly available datasets without direct risk for misuse.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Credit is given in detail in Appendix C.3
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, www.ssppppsssttteeee.ee thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Data preprocessing scripts and instructions for acquisition for the new dataset (EHRSHOT) we propose are included in Appendix C.3 and our forked repository (see the footnote on page 8). Direct access to the data is not provided due to separate usage agreements and required training from the original source.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the detaset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
19

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Crowdsourcing nor human subjects were used in our work.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: Human subjects were not used in our work.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our method nor paper used LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standardd components.
Please refer to LLM policy (nttps://neur /ateerupppscc///ooeeeeee//////00 for what should or should not be described.
20

---

Supplementary Materials: Deep Continuous- Time State-Space Models for Marked Event Sequences
Table of Contents
Appendix A
Acronyms and Notation
Appendix B
Additional Details on Methods
Appendix
Experimental Configurations and Datasets
Appendix D
Additional Experimental Results
21

---

A Acronyms and Notation
Table Key notation used repeatedly across this paper.
Symbol
Space
Description
Rg0
Time
Rgo
Maximum
given sequence observation window
Rg0
f time
R20
Subscript minus indicutes left-limit
Rg0
Subscript plus indicates right-limit
M {1, [1..... K]
Event mark
H
M 22
Event history N events
N.
Counting process for K marks at time
X
R>0
Intensity of t mark type time
A
B
Vector of K mark intensities time Ground/total intensity (sum mark-specific intensitics)
R20
50
R
Log-likelibaee Log-likelihood the argument under the model
RyD
Background intensity the th mark
a
R.K.
50
(For LHP) Matrix intensity impulses from each type mark
K.K
(For LHP) Dynamics matrix of intensity evolution
20
R
g+
Mark embedding runk
P
z
LLH/SSM hidden dimension
B²
LLH/SSM hidden state time t
R
Leamed H.HH/SMM initial hidden state
H
N
LLH/SSM oulput dimension
Yt
R
LLH/SSM output at time
W
R
LLH/SSM input time
A
RPx
LLH/SSM transition matrix
B
Px
LLH/SSM input matrix
C
LLH/SSM output matrix
D
RIxI
H.SH passthrough matrix
E
R
PxR
LLH mark embedding matrix (P x R low-rank factorization)
L
Number of linear recurrences in S2P2 model; model "depth"
R
(For S2P2) Mark impulses (R x K in low-rank factorization)
Tilde (e.g., B) denotes variuble is
diagonalized eigenbusis
CPXP
Matrix cigenvalues A: diagonalized dynamics matrix
CPXP
Discretized diagonal dynamics matrix
N/A
Superscript index in parenthesis indicates layer (i.c., x for layer ()
Table 4: Key acronyms used throughout this paper.
Acronym
Page number
Definition
CNN
7
Convolutional neural network
LHP
2
Linear Hawkes process
Latent linear Hawkes
MTPP
Marked temporal point process
RNN
Recurrent neural network
SSM
(Deep) State-space model
TPP
7
Temporal point process
ZOH
Zero-order hold
RMTPP
2
Recurrent marked temporal point process
Du at
2016
NHP
2
Neural Hawkes process Mei and Eisner.
2017
SAHP
2
Self Self-attentive Hawkes process Zhang et al 2020
Transformer Hawkes process (uio el al. 2020
AUNHP
Attentive neural kes process Yang
IFTP
Intensity- temporal point
2020a
MHP
2
Manba Hawkes process [ao al. NT24
S2P2
State space point process (ours)
22

---

B Additional Details on Methods
B.1 State-Space Point Process Algorithms
Algorithm 1 State-Space Point Process: Get Right State Limits
Input: S2F2 layer parameters e
{ A44,B0 c,00 C,,,,, DD B0) (00,, 80 Jizi"
(4
event intervals Ar.x. nonlinearity o, shared mark
embeddings o.:N-
Output: Right limits (3:2) 1.
1: "Cy.
Left input limits
2: for I do
A = Discretize (A AAT:N)
b Zero-order hold, see Eq.
4: f1N ParallelScan A 1.x (A L:N I)B
u +E
Compure right a limits
5: x()) x00
Compute left limits
6: B LayerNorm (o (00 p D -
Compute layer's left limits
7: end for
8: (1:6)
Algorithm 2 State-Space Point Process: Get Intensity From Right Limit
Input: S2P2 layer parameters e = { AA ((, (O,00,) D(O, B00, D() 86 Ji=1' Previous state right limits x (144 Integration period öt,
nonlinearity Intensity function Fnt
Outpul:
Intemsity left limit A
1: Ur
Left input limit
do 3: ( Discretine (110, (0,6t)
2:
hold, see Eq.
4: 82451- = A x( (A -I)BB) W:451-
D Evolve state
5: Ue+ = LayerNorm
( 2481 D4tee))) + uutse))
D. Compute event left u limits
6:
A1tzc IntenstyyFn(uussc))
Rectify intensity, see Section
8: Actse
Algorithm 3 State State-Space Point Process: Compute Log-Likelln Likelihood
Input: S2P2 layer parameters @ {A B()), 8 D _()
2(0) I=1' Event times f1N. mark types ki.v. nonlinearity
shared mark embedding function EarkdMarks, number integration points per event M, Intensity function Intensity Fu..
Output: Log-likelihood C
1: aI::N EimbeeMarlss(k.x))
Shared emheddinggs
2: To :m 0
AfiN = fu:N fo:N-1
4: $1:N,1:M U(U,, Afr.)
Sample M integration points interval (non-inclussive)
5: ((-L)) GellRightStateLinittee... Sh.x,a,au.a
Algorithm Olog N) parallel
6: for n 1m:
This embarrussingly parallelizable with vnap. O(1) parallel time
7: Ar.. GellneensiyPPooRRightLnii (e, (1:b) At.. IntensityFu)
Algorithm (1)) parallel time
8: for m in 1: do
D This is embarnasinggy parallelizable with vnap. O(1) parallel time
9: Am.m Peelnteneity-PooRRignttnne (e, (1:L) 8n.m. InteusityFn)
Algorithm O(1) parallel time
10: end for
end
12: C NN log Ahn 2.N
-M N Nh
with Monte-Carlo approximation of integral
13:
23

---

B.2 Discretization and Zero Order Hold
The linear recurrence is defined in continuous-ting time. This mirrors the (M)TPP setting, where event times are not on a fixed intervals. We use the zero-order hold (ZOH) discretization method, to convert the continuous- time linear recurrence into sequence of closed-fome updates, given the integration times, that can also be efficiently computed. We refer the reader to Iserles 2009 for a comprehensive introduction to the ZOH transform.
The main assumption of the ZOH discretization is that the input signal is held constant over the time period being integrated. Under this assumption, is possible to solve for the dynamics and input matrices that yield the correct state at the end of the integration period. For the LLH dynamics in Eq. (8), when no events occur in (t, ('), this becomes
- f. Ax: + du = Ax; + ABu, assuming du;= = E [t,
(15)
where the resulting discretized matrices are
A e AB A (cAs! IJBB, where At = -t
(16)
The ZOH does not affect the output or passthrough matrices C and D. To compute the matrices A and AB however requires computing a matrix exponential and a matrix inverse. Smith et al. [2022] avoid this by diagonalizing the system (also avoiding dense matrix-matrix multiplication in the
parallel scan). The diagonalized dynamics and input matrices are denoted A (a diagonal matrix) and B respectively. In this case, Eq. (16) reduces to
A Asr
(17)
AB = A - I)AB
(18)
= (e IjB (diagonal matrices commute)
(19)
where cA is trivially computable as the exponential of the leading diagonal of AAA. These
operations embarrassingly parallelizable across the sequence length and state dimension given the desired evaluation times.
To contextualize, suppose event occurs at time t, Eq. 19 allows to exactly (under the constant- input assumption) efficiently evaluate the linear recurrence at subsequent times t'. We use this extensively in S2P2 to efficiently evaluate the recurrence (and hence the intensity) at the irregularly- spaced event times and times used to compute the integral term.
It should be noted the discretization was done to compute left-limit x from previous right-limit X₃- Should an event not occur i', then the left- and right-limits agree and x = X Xp. an event does occur at time t' with mark k, then the left-limit X can be incremented by Exg to compute x Xy. This increment is exact and leverages no discretization assumption.
B.3 Interpretation for Input-Dependent Dynamics
Consider the input-dependent recurrence for an LLH layer. defined in Eq. (33):
dx, := A,X,- di + A, B,B dt + EodN, for f. E (4,, (4+1] where A; := diag(A,) A with the input-dependent factor defined as A, == softplus( (W ur, + b') T050- This factor can be thought of as the input-dependent relative-time scale for the dynamics. To see this, we first note that for vectors p, q E IR,, the following holds true: diag(p)q o q q @ p where @ is the Hadamard or element- product. It then follows that
(20)
dx; = A, dt + A, A, di + EadN,
(21)
A, Bu, )dt EadN,
(22)
diag A(x; Bu,-)dr )dl + EadN,
(23)
= [A(*,- + B] ( (A,dt) + EadN,
(24)
As shown, the positive vector A, can be thought of as changing the relative time-scale for each channel the hidden state x. Large values of A, will act as if time is passing quickly, encouraging the state to converge to the steady-state sooner. Conversely, smaller values will make time pass more slowly causing the model to retain the influence that prior events have on future ones (for that specific channel in x at least).
24

---

B.4 Forwards and Backwards Zero Order Hold Discretization
In Section 3.2 we highlighted that the ZOH discretization is exact when is held constant over the integration window. This raises unique design question for S2P2: what constant value should u, take on when evolving x from time / to !'? For the first layer of the model, the input is zero by construction, so there is no choice to be made-in fact, since u is constant for the first layer the updates are exact. However, the input is non-zero at deeper layers, and, crucially, varies over the integration period.
We must therefore decide how to select a u value over the integration period. This should be a value in (or function of) {u, 8 [)), Note this is because the value l', Uy' cannol be incorporated as this would cause data leakage in our model; while values prior to È would discard the most recent event occurrence. For this work, we explore two natural choices: (i) the input value at the beginning of the interval, u, and (ii) the left-limit at the end of the interval, ur- Note the end of the interval need not align with an event (crucial for when computing intermediate intensity values). We refer to these options as forwards and backwards ZOH, respectively. We illustrate the backwards variant in Fig. 2 where in the leftmost panel, we use the u, values at each layer to calculate Xt- and x,, as opposed to u, for t < t < 1.41- All experiments in the main paper utilize backwards ZOH.
It is not obvious a priori which one of these modes is more performant. We therefore conducted an ablation experiment in Table [4] We see that there is little difference between the two methods. We also note that models are learned through this discretization, and so this decision does not mean that a model is "incorrectly discretized" one way or the other, but instead they define subtlety different families of models. Theoretical and empirical investigation of the interpretations of this choice an interesting area of investigation going forwards, extending the ablations we present in Table 14
B.5 Theoretical Complexity
We include in Table 5 brief summary of the theoretical complexity of each of the methods we consider, broadly grouped by their architectures. We analyze complexity by the work, memory complexity and theoretical best parallel application time of the forward pass (used when conditioning on a sequence, the left-hand term of Eq. (2)) and evaluating the integral term in Eq. 2 given that the forward pass has been completed (as this is either required by the method, and is nearly always evaluated in conjunction with the forward pass). We then state the limiting best-case theoretical parallelism of the two components.
The reasoning behind the calculated values are as follows:
The forward pass of RMTPP, NHP and IFTPP use non-linear RNNs, and hence incur memory and work that is linear in the sequence length, and cannot be parallelized. MHP uses an RNN, but that is logarithmically parallelizable. These models re-use the computed hidden states compute the integral term, and hence, while they incur work and memory that scales in the sequence length and number of events, this work can be perfectly parallelized. This results in best-case parallelism of O(N) (dominated by the forward pass; Ollog N) for MHP).
SAHP, THP and AINHH all use self-attention, and hence have a work and memory that scales quadratically in the sequence length, although this work can be parallelized across the sequence length, resulting in logarithmic parallel depth. SAHP and THP re-use embeddings and a parametric decoder, and hence estimating the integral scales like the RNN, and hence the limiting parallelism is still the forward pass. AuNHP is slightly different in that it re- applies the whole independently attention mechanism for each integration point. However, this work is parallelizeable and hence still reduces to best-case depth of Olog N).
S2P2 is an RNN and hence has linear work and memory in the forward pass, but can be parallelized best-case depth of Oloo N) using the parallel scan. We then re-use the states computed in the forward pass for estimating the integral, which, as with the other RNN methods, perfectly parallelizable, resulting in a theoretical parallel depth of Olog N).
Note that these figures do not account for the number of layers required by each model, which must be evaluated in sequence.
25

---

Table Comparison of methods based on memory and compute complexity. We see that our S2P2 matches the best performing baseline in all categories. N denotes to the sequence length, and M denotes to the number of Monte Carlo grid points per-event used in evaluating Eq. (2). IFTPP an intensity-free method, it does not need to estimate Axdl as the other methods do.
Forward Pass
Estimating f Aed
Overall
Method
Theoretical Parallelism
Theoretical Parallelism
Theoretical
Memory
Work
Memory
Work
Parallelism
RMTPP
((N)
((N)
O(N)
((NM)
((NM)
((1)
O(N)
NHP
O(N)
((N)
O(N)
O(NM)
O(NM)
O(1)
O(N)
MHP
((N)
((N)
Ollo N)
((NM)
((N
O(1)
Olog N)
ITPP
O(N)
((N)
O(N)
N/A
N/A
N/A
O(N)
SAHP
((N2)
O(N2)
Olog N)
O(NM)
O(NM)
O(1)
Olog N)
THP
O(N2)
O(N2)
Olog N)
O(NM)
O(NM)
O(1)
(log N)
AUNHP
O(N2)
O(N2)
Ollg N)
O(N*M) O(N2M)
OllggN)
O (log N)
S2P2
O(N)
O(N)
Ollog N)
O(NM)
O(NM)
O(1)
Olog N)
To validate the scaling properties, we measure the wallclock time for a full forward pass and log- likelihood evaluation on random input sequences with lengths ranging from eight events to over half million events. The architectures and mark spaces are the same as the StackOverflow experiments
(see Tables 6a and 7).
We note: our EasyTPP PyTorch S2P2 written in pure PyTorch, and hence is optimized as other methods (compared to, for instance, IFTPP, which uses GPU-optimized implementation of the GRU). We therefore include the runtimes of standalone JAX S2P2 implementation, which allows for comparable levels of optimization through JIT compilation.
We observe the predicted scaling in practice. NHP scales linearly across all sequence lengths, and is far outpaced by all other methods. The THP scales well before reverting to superlinear scaling, and then runs out of memory. The IFTPP very fast at shorter runtimes, but quickly reverts to linear scaling, due to its simple but highly optimized implementation and inherently sequential operation. Both S2P2 implementations scale linearly long sequence lengths, but have near-constant runtime at shorter sequences. At shorter sequence lengths, the more optimized JAX implementation is faster than the unoptimized pure PyTorch implementation. While this indicates that there additional opportunities to accelerate the PyTorch implementation further (e.g., exploiting kernel fusion or writing lower-level Triton implementation), these results still confirm that our S2P2 can exploit parallel scans to scale to long sequences more effectively than other methods while retaining strong performance.
S2P2 (Jax)
S2P2 (Torch)
NHP (Torch) THP (Torch)
IFTPP (Torch)
10
Log-I ikelihood
10a
Conditioning
wallelock (s)
wallelock
10
10
10
10-2
10-3
10-3
10-4
10
10
102
103
104
105
101
10a
103
104
105
Sequence Length
Sequence Length
Figure 5: Median runtime of various models against increasing sequence lengths when conditioning on sequence (Algorithm [ and for likelihood evaluation (Algorithm 3 over random seeds
(variance negligible). S2P2 is faster across a wide range of sequence lengths. Crosses indicate where THP runs out of memory or IFTPP throws an error.
26

---

C Experimental Configurations and Datasets
C.1 Training Details & Hyperparameter Configurations
All baseline models used up-to-date PyTorch implementations, provided by the EasyTPP library [Xue] et al.
2023] as of May 2025.
We apply grid search for all models on all datasets for hyperparameter tuning. We use default batch size of 256 for training. For models/datasets that require more memory (e.g., large mark space or long sequences), we reduce the batch size and keep them as consistent as possible among all the models on each dataset. We use the Adam stochastic gradient optimizer [Kingma and Ba 2015]. with a learning rate of 0.01 and linear warm-up schedule over the first 1 1% iterations, followed by a cosine decay. Initial experiments showed this setting generally worked well across different models and datasets leads to convergence within 300 epochs. We also clip the gradient norm to have a max norm of 1 for training stability. We use Monte- Carlo samples to estimate the integral in log- likelihood, where we use 10 Monte-Carle -Crllo points per event during training.
On the five EasyTPP benchmark datasets and MIMIC-II that are smaller in their scales, we choose an extended grid based on the architecture reported in the EasyTPP paper. Specifically, we search over hidden states size h { [16, 32, 64, 128, 256) for RMTPP, h (32, 64, 128) for NHP, and h h= ( (16, 32, 64) for IFTPP. For SAHP, THP, and AttHH, we searched over all combinations of number of T. [1, 2, 3. hidden state size = 116, 32, 64, 128), and number of heads = {1, 2, 4). For MHP, we followed their paper to fix L = 4, then we search over / (4, 8, 16, 32, 64. 128, 256, 512). Finally, for S2P2, we considered combinations for number of layers = (1. 2, 4) (16, 32, 64, 128) and h = (16, 32, 64, 256). We generally found a range of reasonable hyperparameters to yield similar performance on S2P2, while multiple layers were key for performance (intuitively deep stack allows nonlinear and complex dynamics), but there not critical dependence on depth.
We fixed the activation function as GeL.U (Hendryeks and Gimpel 2016) and apply post norm with layer norm [Ba 2016]_ We fix the dropout as 0.1 for S2P2 on the five core benchmark datasets, and add dropout- O. 0.T) to the grid search for the other three datasets. Due to the scale of Last.fm and EHRSHOT datasets, we perform smaller search over architectures that roughly match the parameter counts for all models at three levels: 25k, 50k, 200k, and choose the model with the best validation results. AttNH has expensive memory requirements that tends to have smaller batch sizes than other models. We were unable to train any AuNHP on EHRSHOT. The final model architectures used are reported in Tables 6a and 6b
Table 6: Model architectures for the five EasyTPP benchmark datasets in Table 2
(a) Model architectures for the five EasyTPP benchmark datascts in Table
Made
Amzzon
Netweot
Tasi
Taoban
StackCherllov
RAIITP
13s
b 198
16
750
SALEP
A a3.1 2 beade u A BE, - hunde -
A 32. boode
Ai 16.1 hoars aid A 155, o Lenola -
= 33, beaadde in
Bit boode
THPP
A 16. Jaba
A 64,1 L.bande
A 16 landa
IFTPP
64
MINHP
66,1 16,1 heads A 61.p 125,1 =
beeds
1 16.4 boads ii 135 16,1
35, 16, hends
to 20, 16, beods A-m 37. 37,
STP2
on 135, 128, I
-n 37. 16, -
(b) Model architectures for the additional three benchmark datasets Table
Model
Last.fm
MIMIC-1
EHRSHOT
RMTPP
A 256
= 128
h 16
SAHP
J - 136, 2, heads -
A 64,6
hends
h- 8,1 2, heads 4
THP
= 18,1 1 headse =n
A 52, benele
J 32, 1 beudse on
IFTTY
48
258
16
NHP
À = 112
128
80
AUNTIP
28, 16, 1
besuds
64, 18.7
beads =
OOM
S2P2
68.p 16,
64,p
128. 32,2
27

---

C.2 Dataset Statisties
We report the statistics of all eight datasets we used in Table 7 of the five EasTTPP datasets. For all datasets, we further ensure the MTPP modeling assumptions are satisfied that no more than two events occur at the same time (i.e., inter-arrival time is strictly positive), and event times do not lie on grid points that are effectively discrete-time events. Dataset descriptions and pre-processing details are provided in Appendix C.3
We used the HuggingFace version
Table 7: Statistics of the eight datasets we experiment with.
Number of Events
Sequence Length
Number of Sequences
Datusel
K
Train
Valid
Test
Min
Max
Mean
Train
Valid
Test
Amazon
16
288,377
40,995
84,048
44.8
6,454
922
1,851
Retweet
3
2,176,116
215,521
218.465
50
264
108.8
20,000
2,000
2,000
Taxi
10
51,584
7,404
14,820
37.0
1,400
Taobao
17
73,483
11,472
28,455
28
64
56.7
1,300
200
500
StackOverflw
22
90,497
25,762
26,518
64.8
1,401
401
Last. fm
120
1,534,738
344,542
336.676
6
501
207.2
7,488
1,604
1,604
MIMIC-II
75
9,619
1,253
1,223
2
33
3.7
2600
325
325
EHRSHOT
668
759,141
165.237
170,147
5 3,955
177.0
4,329
927
927
C.3 Dataset Pre-processing
We use the default train/ caiin/validaiion/eestt splits for EasyTPP benchmark datasets. For MIMIC-II, we copy Du et al. [2066] and keep the 325 test sequences in the test split, and further split the 2,935 training sequences into 2,600 for training and 325 for validation. In our pre-processed datasets, Last.fm and EHRSHOT, we randomly partition into subsets containing 70%, 15%, 15% of all sequences for training/valiinn valiaaiion//t respectively. We provide a high-level description of all the datasets we used, followed by our pre-procsssing procedure of Last.fm and EHRSHOT in more detail. Note that for datasets that contain concurrent events or effectively discrete times (e.g.. StackOverflow, Retweet), we apply small amount of jittering to ensure no modeling assumptions are violated the MTPP framework.
Amazon Ni et al. 2019) contains user product reviews where product categories are considered as marks. Retweet [Zhao et al. 2015) predicts the popularity of retweet cascade, where the event type is decided by if the retweet comes from users with "small", "medium". or "large" influences. measured by number of followers [Mei and Eisner 2017). Taxi data [Whong] 2014 Mei et al., 2019] uses data from the pickups and dropoffs of New York taxi and the marks are defined as the Cartesian product of five discrete locations and two actions (pickup/dropoff). Taobao [Xue et al. 2022] describes the viewing patterns of users on an e-commerce site, where item categories are considered as marks. StackOverflow contains the badges (defined as marks) awarded to users on question-answeriin website. Finally, MIMIC-II (Saeed et al. [002] records different diseases (used as marks) during hospital visits of patients. We add small amount of noise to the MIMIC-II event times so that events do not lie on a fixed grid. Both StackOverflow and MIMIC-II datasets were first pre-processed by Du et al. [016].
Last.fm [Celma 2010 McFee et al., 2012 records 992 users' music listening habits that has been widely used in MTPP literature Kumar et al., 2019 Boyd et al., 2220 Bosser and Taieb 2023). Mark types are defined as the genres of song, and each event a play of a particular genre. Each sequence represents the monthly listening behavior of each user, with sequence lengths between 5 and 500. If the song is associated with multiple genres we select random one of the genres, resulting a total of 120 different marks.
EHRSHOT [Wonoww et al. 2023 a newly proposed large dataset of longitudinal de-identified pa- tient medical records, and has rich information such as hospital visits, procedures, and measurements. We introduce an MTPP dataset derived from EHRSHOT, where medical services and procedures are treated as marks, as identified by Current Procedural Terminology (CPT-4) codes. Each patient defines an event sequence, and we retain only CPT-4 codes with at least 100 occurrences in the dataset. For the < 1% events of events where there are more than 10 codes at a single timestamp, we retain the top 10 codes with the most frequencies and discard the rest. We then add small amount of random noise to the event time to ensure they are not overlapping. This process ensures we still satisfy the MTPP framework, and can reasonably instead compute top-10 accuracy for the next mark prediction; other work has considered extending the MTPP framework to consider simultancous event
28

---

occurrence [Chang et al. 2024]. Then we standardize each sequence to start at = 0 and pad the start and end of sequences with specific padded event token. Note that we do not score these events. Event times are normalized to be in hours. We discard sequences that have less than 5 events or single timestamp. This leads the final version of our dataset having 668 marks, and the sequence lengths range from 5 to 3955 events, reflecting patient histories that can span multiple years.
We include code in our forked repository for preparing the EHRSHOT event sequence dataset from the raw EHRSHOT dataset. Note that we cannot distribute the raw data (or derivative dataset) under the terms of the original EHRSHOT dataset requiring credentialed access through PhysioNet.
29

---

D Additional Experimental Results
D.1 Full Results on Benchmark Datasets
We provide the full log- logllikelihood results and corresponding plots in Table 8 and Fig. 6 respectively, where we decompose the likelihood into time and mark likelihoods. The improvement of our S2P2 model is mainly driven by better modeling of time, though we also often obtain best- or second-best predictive performance on marks from the next event prediction accuracy results conditioned on true event time in Table [c] In aggregate, our model achieves a 1.33 per-event likelihood ratio between itself and the next best method across all datasets (a 33% improvement in likelihood). This is calculated by computing the mean log-likelihood ratio across all datasets and then exponentiating. Doing so is equivalent to taking the geometric mean across likelihood ratios.
Configuration and training details of all models can be found in Appendix C.1 As discussed in Section andd grouped in the results, models with continuous- hidden states can present richer class of intensities and often empirically outperform those with discrete hidden states. Note for the RMSE results in Table 2b we follow Mei and Eisner 2017] and use the expected next event time as next event time predictions to minimize the Bayes risk. Unlike them, however, we estimate these with the trapezoidal rule rather than Corte simulation via the thinning algorithm. In practice, we have found this to produce an estimator with much lower variance and be faster due to being more readily parallelizable. This stands in contrast to the thinning algorithm which has more hyperparameters (e.g., dominating rate, sampling boundary) that can exacerbate bias.
Table 8: Complete per event log-likelihood results on the held-out test for the eight benchmark datasets we consider, averaged over 5 random seeds. In Table Sa we show the full log- loglli.eli We then decompose this log-likelihood into the log-likelihood of the event time in Table [8b and the cime-conditional log-likelihoon of the mark type in Table Sc OOM indicates out of memory; standard deviation in parentheses. We box the pest-performing model and underline the second-best. We also report the average rank of models across datasets as summary metric (lower is better). S2P2 consistently the best or second pertromef model across all datasets.
(a) Full log- likelihood results (equal to the summation of Table Sb and Table &c Extended version of Table 2a
Model
Per Event Log-LEielihhedd,
Arg. Ranking
Amrzzon
Reterzt
Tisi
Tarhan
StackDoeflow
1ast.fm
MIMIC-I1
FHRSHOT
RMTPP
-2.136
(1.013)
-7.0%% (0.217)
0,346 (QU802
1,053 (0.004)
-2.400 (0.019)
(0.005
-0442 (1.26)
-5,051 (BU025
7.1
SAHP
2.074
(IU25)
6.708 (0.029)
0.298 (3.05)
1.168 (0.029)
2.341 (0.055)
-1.646 (0.083)
0.677 (0.02))
6.804 (0.126)
588
THP
2.096 (0U002)
6,669 (0.007)
0,572 (0.002)
0.790 (0.002)
2.338 (0.014)
(0.1100011)
0.577 (0.011)
1.208 (0.266
6.1
I/TFP
0.496 (0,002)
10.344 (0.016)
(0.00)
1L318 (0.017)
2,233 (0.009)
10,992 (0.017)
0.317 (0.02))
6.596 (0.240)
3.0
MHP
2.091 (0.002)
6,564 (0.015)
(0.00)
0,876 00.004)
-2.445
(0.012)
(0.00)
4.351 (0.012)
-3.206 (0.400)
53
-2.941
.3996
AWNNHP
(1.03)
104775
(0.000)
-2.194
-0.592
(OtEs)
0.78T (0:011)
00.00)
D.512 (0.000)
1,304 (3.939)
62
(0.009)
0.557 (0.016)
6919
55312 (0.360)
(b) Per event log likelihood of the event times (higher is better).
Model
Per Event Next Event Time Log-I ikelihood, Cra (nats) (1)
Arg. Ranking (.)
Amuoca
Retwect
Tuxi
Taobso
StacKverslow
Lost Lin.
MIMIC-11
EHRSHOT
RMTFP
0.011
(0.001)
6.191
(0.083)
0.622 (0.002)
2.428 (0.0044
-0.297 (0:005)
0.256 (0.007)
-0.188
(0.015)
(1.13(0.025)
SATP
0.115
-58872
(0.062)
0.645 (0.044)
2.604 (0.008)
-0.200 (0:001)
0.489 (0.075)
-0.244
(0:040)
-1.501 (0.08)
4.9
THP
-0.068
(0.00))
-5.874
(0.007)
0.621 (0.000)
3.242
(0.007)
-0.772 (0.006)
0.220 (0.01))
20.211
(0.004)
-1921 (0.02))
7.0
2.483 (0.001)
18335 (0.002)
2.208 (0.01
-0,622 (0.007)
1227 (0.016)
0.555
(8.090)
-2.640 (0115)
MIIP
0,065 (0.002)
5.774
(0.015)
0.620 (0.000)
2.093
(0.004)
0.261 (0.006)
0,230 (0.003)
0.140 (0.008)
2.119 (0.188)
SHP
2.116
(0.009)
(0.001)
0.727 (0.003)
2,578
(0.006)
-0,699 (0.00.)
1.198 (0.005)
0.225 (0.015)
-0.221 (0.045)
3,3
AGENUP
3.726
0.714 (0.010)
.0.84
1,205 (0.015)
0.001 (0.055)
COM
S2F2 (Ouns)
2.53
(0.009)
3,598
(0.002)
0:333 (0.003)
A719
(0.038)
56661 (.003)
1,257 (0.022)
[008 (0.065)
D. (.262)
X33
(c) Per event likeliooo of mark type conditioned on the arrival time (higher better).
Model
Event Next Mark Fieel.dellhood, Ctass (mate) (1)
Avg Ranking (.)
Amazon
Retweet
T2ob5o
StocKwveflow
MIMIC-II
BHRSHOT
RMTPP
(3.000)
-0.908
40:276 (0.000)
-1,683 (0.01
-2,035 (0.004)
.02244 (0:014)
(1022)
SAHP
(U003))
-0.836 (0.056)
.0.346 (0.024)
-1,68 (0.02))
-2.136
(1.070)
-0443
(1053)
-5003
THP
-2.028
(10022
-0.755 (0.003
00249 (0.001)
-1.451 (0.0000
-1.565 (0.005)
(0.0055
4.306
-5.287
(0.100)
5.5
IFTPP
-1.98 (0.00)
-0.844 (0.00)
-0.82
-1-07 (0.005)
-1.521 (0:003)
-1.00
0.004)
-0299 (0000))
-1,956 (0
MHP
-2.027
(0.00)
-.700 (0000)
46.251
(0.008)
-1.456
-1.585 (0/005)
11906
(0.007)
-0210
.5.087
(0.206)
5.4
NEP
11957 (0.000)
EA204 (0.000)
0,213
-1421 (0.004)
11542 (0.001)
1.272
(0,000)
0.188
(0.002)
3.148
(1016)
2.4
ARNUIP
(..333000030
00/75000000
0:221 (0.002)
-1.995 (0.0155
E3.310 00003
-1795
(0.000))
-0200 (0.025)
OUM
24
SOP3 (Ouno
+1.771
(0.002)
-0967 00.000)
10.2111 (0/0023
-1,415
(0.005)
-1.521
(0:008)
-1.114
(0.025)
|4.15
+2,899
(0.009)
11.99
30

---

Log-Liclllood
Log-Lideibood
Log-Lilllilod
3
Ammozon
Retwoet
0.2
Twohso
-044
StackOwetfon
Last fm
0.5
MIMIC-II
EARSHOT
GOM
Figure 6: Visualization of Liaa decomposed into Lime and LMak for all models and all datasets relative to RMTPP, normalized by number of events, as discussed Section The improvement of S2P2 is mainly driven by better modeling of Lime, while it improves both Lime and Carkk
31

---

D.2 Full Results for Synthetic Poisson Experiments
We present the full results in Fig. for all models regarding the synthetic Poisson experiments discussed in Section 5 All models are trained until convergence using a set of 5,000 generated sequences, where we use 20 Monte Carlo points per event estimate the integral of log log-likelihood during training to accommodate the sparsity of events. We used small models so they do not overfit; model architecture and parameter counts are reported in Table 9 conditioned on empty sequences using 1,000 equidistant grid points between the start and end points. Our model the only one that perfectly recovers the underlying ground truth intensity, while also using the fewest parameters.
We plot the estimated intensity
RMTPP Estimated Intensity
SAHP Estimated Intensity
Intensity
1.0
Intensity
1.0
0.5
0.0
Time
Time
THP Estimated Intensity
IFTPP Estimated Intensity
Intensity
1.0
Intensity
0.5
2
2
Time
Time
MHP Estimated Intensity
NHP Estimated Intensity
Intensity
1.0
Intensity
1.0
0.5
0.5
0.0
Time
Time
AttNHP Estimated Intensity
S2P2 (ours) Estimated Intensity
Intensity
Intensity
1.0-
0.5
0.0
5
6
Time
Time
Figure 7: Results for all baseline models for the synthetie Poisson experiment introduced in Section The estimated intensity (blue lines) conditioned on an empty sequence are plotted against the ground truth (dotted black lines).
Table 9: Model architectures and parameter counts for synthetic Poisson experiments.
Model
Architecture
Parameters
RMTPP
h 16
627
SAHP
h = 16, = 2, heads =
1738
THP
h 16, heads
1684
IFTPP
h: = 16
1899
MHP
h 4
2240
NHP
h = 8
1010
AUNHP
h 8, 2, 2, heads =
1178
S2P2 (Ours)
h = 4,p p 4,
178
32

---

D.3 Additional Synthetic Results on Multivariate Hawkes Processes
We evaluate our model and baseline models against the true model on randomly initiated parametric Hawkes process with three possible marks. Following the notation in Section 2.1 we draw all parameters from the following distributions: vs
sid 1ni 0.5]. Onj sid Unif(0.5, 0.8), and Bag iid
Unif[0.4, 1.2] for j, E (1,
models are trained until convergence using a set of 50,000 generated sequences, where we use 20 Monte Carlo points per event to estimate the integral of log-likelihood during training. Model architecture and parameter counts are reported in Table I0] We plot three example sequences drawn for additional test set for each model in Fig. 8 using 1,000 equidistant grid points for any inter- event interval. Dotted lines refer to the intensities under the true underlying parametric model; solid lines different model estimates from trained models.
As we see inhomogeneous Poisson processes, our model can recover the ground truth intensities with the fewest parameters. Visually, all SAHP, IFTPP, NHP, AttNHP and S2P2 (our model) perform well at recovering the ground truth intensities. is also worth noting that our model 7-9x quicker than NHP and AttNHP regarding wallclock runtime on single A5000 GPU. Our results synthetic experiments validate the model's ability to recover the ground truth intensities. We further evaluate all models quantitatively using 1,000 test sequences generated from the same multivariate Hawkes process and evaluated both log- lig-likeldhood and RMSE for the immediate next event. We see our method competitive again on both metrics.
Table 10: Model architectures and parameter counts for multivariate Hawkes processes experiments.
Model
Architecture
Parameters
RMTPP
h = 16
697
SAHP
h 16, heads
1902
THP
h 16, 2, heads
1756
IFTPP
h 16
1965
МHP
h = 4
2264
NHP
h 8
1046
AttHH
= 1- = 2, 1 = 2, heads =
1230
S2P2 (Ours)
h 8, p 4,
358
Table 11: Performance comparison of models on the multivariate Hawkes processes experiment presented above. Higher per-event log-likelihood indicates better performance, whereas lower root mean squared error (RMSE) indicates better performance.
Model
Total Log-Likelihood C (t) Next-Event Time RMSE (L)
RMTPP
-0,550
0.648
SAHP
-0.537
0,647
THP
-0,543
0.648
IFTPP
-0,534
0,647
MHP
-0.551
0.648
NHP
-0.530
0.647
АtNHР
-0.533
0,652
S2P2 (Ours)
-0.527
0.647
33

---

Intennlly 2
RATTPP Esliuatel Iutendly
PNTPP Esiiaated Intenils
Eat. 2-14
SAHP Fasteasted Triveddtt
SAPIP Fettessted Interndiyy
SARP Reterated Trtenelty
THP Halieattod lntorrity
THP Hstinated Inteadit
THP Haliaatted Intendly
11
IFTPP Eatisakel Intecodlt
IPTPP Edtisation Intenndiy
IFTPP Estisatted Intecoity
True
True
MEIP Extinsated Internity
MIEP Estinaatd Tateewity
MHP Estinnated Iaternity
Texe Book
Est. Audi
Trae dird
Dat. Aurk Bark herd
Tese Bool True dick
Est. Aud
True dric
and
Irue and
and
foid
XHP Edinntol Lation sky
XHP Ee Aiinnttoo Inticnally
XHP Satinntol Intion dky
creee Tea larl
Fit Andl
creeen Trae dnad
Rot. Kual
..eee Tew Bael True 2 Tixe
Judl
Trua 2-2
Ext. 2-12
Irua 2-2
Eor. 2-12
Tite
Audk
AHENHPP Eetirated Tebrrattc
AuENTIP Declested Interdity
AHENEP Detienated Tsteealte
Est. Kack Awd)
Dall Kark Awd
Ext.
True
Teue Time Book
Time
Fot. budk
Vbme
Exemuntd
Ediczutedd
(cuns) Estamord
Tix Aud knd
Aud
The
bod
anii
Jersuty
Figure 8: Our proposed S2P2 model and baseline models trained with 50k training sequences drawn from randomly instantiated multivariate Hawkes process where K = 3. For each model, three example test sequences are plotted; the locations of colored bars III indicate the true event times.
34

---

D.4 Full Results for Hawkes and Self-Correcting Process
We train approach synthetic data generated from known, classical temporal point processes, namely the self-correcting and Hawkes processes. These are characterized by intensity functions x exp (t 0.5.,) and X 0.5 + E 0.5 cxp(: t), respectively. Models were fit using data drawn from these processes, 6,000 sequences for training and 2,000 for validation. The learned intensity functions, evaluated on held-out test sequence, can be seen for the self-correcting process data in Fig- and for the Hawkes process data Fig.
10 for all methods. The hyperparameters for the models were chosen by grid search (see Table 12 We see many of the models, including ours, do well at capturing the ground truth intensity.
Self-Correcting Process
RATTP A.
Ground Predicted A
SAIP A
THIP Av
IFFP A.
MItP A
NIIP Ac
AINNII Ar
S2P2 A+
Evente
30
Figure 9: Synthetic self-correcting process experiment visualization of predicted intensities compared the ground truth intensity for given held-out sequence. The vertical lines present for IFTPP due to the conversion from density to intensity being unstable near A/- = 0.
35

---

Hawkes Process
EMTPP Ar
Ground Trath Prodicted At
SAIIP Ae
THP At
IFFP Ae
MITP Ae
NHP At
AHNEF A
S2P2 Ar
10
20
30
40
50
Figure 10: Synthetic Hawkes process experiment visualization of predicted intensities compared to the ground truth intensity for given held-out sequence. The vertical lines present for IFTPP are due to the conversion from density to intensity being unstable near At 0.
36

---

Table 12: Model architectures and parameter counts for synthetic Hawkes and self-corretting process experiments.
Model
Architecture
# Parameters
RMTPP
h = 16
627
SAHP
= 16, 3. heads
2554
THP
16, heads 4
2500
IFTPP
h 32
6859
MHP
h 16
13556
NHP
h 32
14658
ArNHP
h 16, 16, = 2, heads =4 4
13298
S2P2 (Ours)
h 16, p 16,
7202
D.5 Full Results for Long-Range Dependency Experiment
To measure the ability to capture long-range dependencies by neural MTPPs, we constructed generative process with long-range dependencies. For this, generate sequences the time window of [0, 100), with three possible marks. The first mark is "distractor" mark, meaning has no influence over other events. These events are drawn from homogeneous Poisson process with rate 1. The second mark is a "trigger" mark, which are directly tied to the third "target" mark. Triggers are also drawn from homogeneous Poisson process with rate 0.1. For every trigger event (t,, k = 2) drawn, a corresponding target event (tj, k; = 3) is generated conditionally independent of all other events according to ł it, Nt; 40, 0.1).
All models were trained with the same hyperparameters as in Table 12 The predicted intensity functions for single sequence can be seen in Fig. [1] and the likelihood ratio between the trained models and the ground truth process on held-out test sequences can be found in Table 13 We can that both S2P2 and AttNHP do very well, both qualitatively and quantitatively. This is expected as both architectures are well suited for long-range dependencies while still being continuous-time models, allowing for expressive intensity functions.
Table 13: Likelihood ratios between models and ground truth process for held-out data on long- experiment.
Model
Ground Truth
RMTPP
SAHP
THP
IFTPP
MHP
NHP
AttHH
S2P2
Lik. Ratio
100%
80.4%
81.0%
94.0%
88.00
87.3%
87.9%
99.7 %
97.8%
37

---

Long Range Modeling
(Zoom-In)
10
RMTTY At
.. Govard Truld Predicted
10
SART A
THP A1
10
IFEVT A
10
MEP A
NTP A2
ABNNI A1
S272 A
Events
20
100
45
50
55
00
Figure 11: Synthetic rongrragee experiment visualization of predicted intensities compared to the ground truth intensity for a given held-out sequence. The vertical lines present for IFTPP are due to the conversion from density intensity being unstable near Al 0.
38

---

D.6 Ablation for Different S2P2 Variants
We perform an ablation study of different model variants that we proposed on all datasets and summarize the results in Table 14 We train EHRSHOT using 10% of its training data because larger dataset scale requires more training time (but use the original validation and test sets for model selection and reporting results). Forward and backward discretization are very close in performance, with backwards discretization having a slight edge. Models that are input-dependent achieve better performance on most datasets, although on certain datasets input dependence appears to harm performance. It is an interesting direction for future work to explore theoretically and empirically when each of these variants is best. We select backward discretization with input dependence for the results in the main paper.
Table 14: Ablation for different model variants log-likelihood (LL). ID stands for input-dependent, see Section 3.3 Backward and Forward respectively refer to using Ut and ut (i.e., the previous right limit current left limit), Appendix B.4
Dataset
Model variant
LL
Arival time LL
Mark LL conditioned time
Forward
0.705
2.617
-1.912
Forward ID
0.748
2.634
-1.886
Amazon
Backward
0.740
2,640
-1.899
Backward
0.765
-1.873
Forward
-6.055
-5.625
-0.780
Forward ID
66.70
-5.602
40.767
Retweel
Backward
-6,398
-5.618
-0.780
Backward ID
-6.367
-5.600
-0.767
Forward
0.473
0.697
-0,224
Forward
0.525
0.733
40,208
Taxi
Backward
0.477
0.705
-0.228
Backward ID
0,528
0.738
-0,209
Forward
1.207
2.643
-1.435
Forward ID
1.332
2.742
-1.410
Taobao
Backward
1.215
2,648
-1.432
Backward
1.332
Forward
2.249
-0.676
-1,572
Forward ID
-2.144
-0.644
-1.530
StackOverflow
Backward
-2.225
-0.679
-1.547
Backward ID
-2.165
-0.636
-1.529
Forward
-0.463
1.309
-1.772
Forward
-0.477
1.302
-1.779
Last.fm
Backward
-0.474
1.303
-1.777
Backward ID
-0.49
1.294
-1.790
Forward
0.555
0.847
-0,292
Forward ID
1.319
1.405
-0.086
MIMIC-II
Backward
0.322
0.601
-0,279
Backward
1.231
Forward
-3.885
0.105
-3.900
Forward
-38488
-0.021
-3,827
EHRSHOT (10%)
Backward
-4,571
-0.432
-4.139
Backward ID
-4.684
-0.641
-4.043
39

---

D.7
Model Calibration
To further probe the models, we evaluate the calibration of MTPPs, as proposed by Bosser and Taieb [2023]. Calibration has a different focus than likelihood-baaee or iccuracy-based evaluation. Calibration instead describes how well the uncertainry in the model reflective of the observed data. It is important to note, however, model can achieve perfect calibration simply by predicting the marginal distribution. Better calibration therefore does not necessarily indicate better predictive performance only better calibrated emrors and so should be taken in context with the performance under other metrics. We provide summarized statistics for both probabilistic calibration error (PCE) for time calibration and expected calibration error (ECE) for mark calibration in Table 15 and
visualize the calibration curves in Figs. 12 and 13
We see that, on the whole, MTPP models produce fairly well-calibrated predictions. IFTPP is the best calibrated of the models. this may be as result of having parametric distributions for inter-arrival time (although IFTPP does fail on some datasets such as Retweet). The S2P2 is particularly well calibrated in time (PCE) among intensity- -based methods, suggesting again that our S2P2 capturing time dependencies better than other models. It is also the second-best calibrated on mark prediction (ECE) on average.
Table 15: Calibration results for the models and datasets tests.
(a) Probabilistic calibration error (PCE) for time calibration in percentage.
Model
Probabilisii Calibration Error (PCE) (
Ranking ())
Amazon
Rerwert
Taxi
Stac&Overllo
MIMIC-II
FHRSHOT
(0:02)
(0:01)
(0.00)
12.04
(1.86)
3.18
(0:57)
2,53
2.28 (0:44)
20.20 (1.09)
THP
12.38 (0.05)
5.68 (0.08)
3.34 (0.02)
6.36 (0.04)
2.06 (0.11)
1.02 (0.08)
1.10 (0.06)
13:46 (0.45)
5,4
IFTPD
1159 (0.09)
23.85 (0.26)
0.40 (0.10)
[1.61 (0.74)
10.84 (0.34)
0.46 (0.45)
1.35 (0.33)
16.58 (3.34)
12.6
MLP
12,22
4.89
3.43 (0.05)
8.77
1.58
1,25 (0.05)
6.21
15.24 (0.92)
6.0
NHP
845 (0.28)
0.20 (0.19)
0.87 (0.50)
7 (0.68)
1.51 (0.10)
(0.3))
5.92 (0. (0.14)
2.28 (0.49)
3,6
ARNHH
6.36 (0.63)
2.09 (0.85)
0.84 (0.27)
3,08 (0:16)
1.65 (0.24)
1.43 (0.14)
4.70 (0:33)
ODM
3,7
S2P2 (Ouns)
5.88
0.55
(0.33)
2,07 (0.32)
1,03 (0.15)
1.38 (0.52)
11.70.0.68 (0.68)
12,06 (0.51)
28
(b) Expected calibration error (ECE) for mark calibration in percentage.
Model
Expected Calibration Error (ECE) (1)
Arg, Ranking (.)
Amapon
Rerweet
Taxi
Toohan
StuckOverflos
Last fm
MIMIC-II
EHRSHOT
RMTPP
6.58 (0.15)
3.99 (8.28)
2.42 (0.16)
1.89 (0:24)
2.10 (0.27)
2.47 (0:45)
2.79 (0.43)
3.47 (0.31)
5.8
SAHP
8.17 (2.00)
6.27 (2.23)
6.77 (0.21)
2,68 (0:35)
1.71 (0.77)
6.26 (1.30)
5.41 (0.26)
5.85 (1.95)
6.8
THP
(1)
0.95 (1.12)
033 (0.19)
1.20 (0.20)
(0:54)
(0:05)
1.37
(0.61)
L
MHP
1,65 (0.16)
1.18 (0.12)
1.91 (0.11)
4.15 (0.36)
0.82 (0.18)
233 (0:50)
2.22 (0:34)
10.00 (1.71)
4
(0.21)
1.35 (0.00)
0.79
(0.00)
(0.16)
(0:41)
2.24 (0.33)
4.18
AWNHP
3.13 (0.61)
0.52 (0.16)
0.56 (0.10)
2.47 (0.12)
1.37 (0.42)
[61 (0.16)
2.28 (0.50)
00M
3.4
S2P2 (Ours)
0.88 (0.34)
0.52 (0.13)
0.58 (0.12)
1.96 (0.07)
1.98 (0.19)
1.01 (0.63)
1,82 (0.24)
2.51 (0:44)
3.0
Amazon
Retwoes
Text
Toobao
0.5
01
Empiriad
SucaOwel
Last Imi
MIMI--I
EHRSHOT
0-
0.5
0.5
0.55
0.5
Prodicted CDF
Figure 12: Reliability diagram for predicted inter- time for each model on all datasets. Diagonal dashed lines refer to perfect calibration.
40

---

Amazzon
Retweet
Taxi
Taobso
StarkOverflow
Last.Im
MIMIC-II
EHRSHOT
RMTPP
SAHP
THP
Accuracy
IFTPP
MHP
NHP
AANHP
OOM
S2P2
0.5
0.5
0.5
0.5
0.5
Confidence
Figure 13: Reliability diagram for mark prediction of all models and all datasets. The r-axis specifies the confidence of model estimates grouped into 20 bins, and the y-axis of the bar plot is the model accuracy within that bin. The diagonal lines represent perfect calibration. The solid curves depict the distribution of confidences, and do not share the y-axis. The grey dashed lines indicate the overall prediction accuracy of the model for the next event conditioned on true event time.
41

---

Finally, in Figs. 14 and 15 we plot the log-likelihood of time and mark respectively, versus their corresponding calibration results, to provide an overall view of the performances of different models. Our S2P2 model consistently achieves higher oggliikeldood while maintaining good calibration on both time and mark components on most datasets.
RMTPP
SAHP
THP
IPTPP
MHP
NHP
AUNHP
S2P2
Amazon
Retweet
Taxi
Taobao
2.75
6
2
0.70
2.50
Time Log- Likelihood
-8
0.65
2.25
0.25
0.025
0.05
StackOverflow
Last.fm
MIMIC-11
EHRSHOT
1.0
1.0
-0.7
0.5
#
0.5
0.0
2
-0.8
2000
0.02
0.025
0.1
0.2
PCE
Figure 14: Log-likelihood of time vs. PCE for all models grouped by datasets. Higher log-likelihood and lower PCE better (i.e., top left comner).
ПMTPP
SAHP
THP
IPTPP
MHP
NHP
AHNHP
S2F2
Amazon
Retweet
Taxi
Taobao
1.400
-0 0.8
00.25
-2.0 28
Mark Log- Lige-iikeihod
1.425
-0.30
0.9
1.450
-2.2
0.35
0
0.05
0.05
0.05
0.05
SteckOverflow
Last fm
MIMIC-II
EHRSHOT
-1.55
0.2
-1.60
-2.0
-1.65
0.4
0.02
0.05
0
0.05
0.1
ECE
Figure 15: Log-likelln Log-likelihood mark vs. ECE for all models grouped by datasets, Higher log. likelihood and lower ECE are better (i.e., top left corner).
42


Reviews:
[{"review_id": "pr6UgD9SgS", "reviewer": "Reviewer_cv8v", "summary": "This paper presents a novel state-space model (SSM)-inspired framework, S2P2, for modeling marked temporal point processes (MTTPs). At the core of the framework is a newly proposed Latent Linear Hawkes (LLH) layer, which naturally integrates the linear Hawkes process with SSM principles. The authors further demonstrate how LLH can be effectively combined with techniques developed for modern deep SSMs (e.g., S4, S5, Mamba), enabling scalable, efficient, and tractable model training. In synthetic experiments, S2P2 successfully recovers the underlying signals where existing methods fall short. On real-world datasets, S2P2 achieves overall SOTA performance.", "questions": "Q1: For the \"parametric decoding head\" in THP and MHP, are you referring to following operation?\n$$\\lambda_k(t|\\mathcal{H}_t) = \\text{softplus}_k\\left(\\alpha_k \\frac{t-t_j}{t_j} + \\mathbf{w}_k^T \\mathbf{h}(t_j) + b_k\\right)$$\nWhat is the fundamental difference between this operation and the final 'Projection & Softplus' layer?\n\nQ2: Could S2P2 serve as a pretrained model, where the output of its final hidden layer (i.e., the layer preceding the 'Projection & Softplus' module) is used for downstream tasks such as clinical classification in EHRSHOT? Given its expressiveness, S2P2's representations are expected to outperform the count-based baselines on the EHRSHOT leaderboard.\n\nQ3: I am surprised that NHP seems to be significantly better than THP, as NHP was the baseline in THP. Did you make any modification to the original NHP?", "limitations": "- Althrough you do not have to address the interpretability issue in this paper, it is worthwhile explore some frameworks that make SSMs more explanable in the discussion.\n- Minor Fixes:\n\t- Eqn (3) no subscript in $\\boldsymbol\\alpha$ on the left hand side\n\t- line 211, should use `\\mathcal`", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- I really enjoyed reading this paper. It is well-organized, well-written, and easy to follow.\n- The design of the LLH layer is clearly articulated, with a detailed explanation of how it combines LHP with SSM. The authors provide well-supported insights from both theoretical and engineering perspectives.\n- The authors provide a complet set of experiments for both synthetic data and real-world datasets and suffcient baseline comparisons.\n\nWeaknesses:\n- For transformer-based HP models, analyzing their attention maps provides critical insights into how historical events influence the future. SSMs in general have weaker interpretability. However, given that S2P2 captures input-dependent dynamics, can these be visualized similarly to attention maps? Maybe there exist ways to calculate some implicit attention scores in a SSM (e.g. [1]) (You don't have to address this in for the rebuttal).\n\n[1] Ali A, Zimerman I, Wolf L. The hidden attention of mamba models. arXiv preprint arXiv:2403.01590. 2024 Mar 3.", "was_revised": true, "final_justification": "My review was favorable and the authors' responses have further strengthened my assessment.", "rebuttal": "We thank the reviewer for their thoughtful feedback and kind words about our paper. We appreciate the recognition of our theoretical framing, empirical results, and clarity. Below, we address each question and suggestion in turn.\n\n\n**On Model Interpretability**\n\nThis is a great point; incorporating or extracting interpretability from S2P2 is an exciting direction for future work. In relation to the referenced paper: Conditioned on the event times, S2P2 computes update matrices similar in form to Mamba, hence S2P2 is eligible to utilize the method in [1] (and likely other techniques being developed for deep SSMs in general). For [1] specifically, it is worth noting that since S2P2 inherently lives in continuous time, it will exhibit a continuously varying form of this implicit attention. Fully taking advantage of this aspect will require special care, but, excitingly, would provide richer insights when compared to discrete-time attention maps.\n\n[1] Ali A, Zimerman I, Wolf L. The hidden attention of mamba models. arXiv preprint arXiv:2403.01590. 2024 Mar 3.\n\n**Question 1 (THP/MHP Decoding Head)**\n\nYes, you are correct; this is the parametric decoding head that we referred to. We describe it as “parametric” because, after conditioning on a history $t_1, …, t_j$, the model encodes this sequence into a single hidden state $\\mathbf{h}(t_j)$. The intensity function is then computed as\n$$\n\\lambda_k(t|\\mathcal{H}_t) = \\text{softplus}_k\\left(\\alpha_k\\frac{t-t_j}{t_j} + \\mathbf{w}_k^\\top\\mathbf{h}(t_j) + b_k\\right),\n$$\n If we combine terms, conditioned on a particular history, the intensity for each mark simplifies to $\\text{softplus}(a t + b)$, where $a,b$ are scalar values representing combinations of parameters/values from the initial expression. \n\nIn contrast, S2P2 models the intensity as:\n$$\n\\lambda_k(t | \\mathcal{H}_t) = \\text{softplus}_k\\left(\\mathbf{w}_k^\\top \\mathbf{h}(t) + b_k \\right).\n$$\nWe see now that a constituent of the input to the softplus is $\\mathbf{h}(t)$ (instead of $\\mathbf{h}(t_j)$).  As a result, there is no simplification rendering the intensity as the softplus of a linear-in-$t$ function (as before); it is instead the softplus of a _non-linear_ and _history-dependent_ function of $t$, because $h$ is defined by the entire S2P2 stack and event history. This is why we describe THP as being “parametric”; and why S2P2 can represent a much richer class of intensities across $t$. We will add further clarification of this in the methods section and appendix.\n  \n\n**Question 2 (S2P2 as a Pretrained Backbone)**\n\nYes, S2P2 can indeed be used as a pretrained model for downstream tasks. One can use the final hidden state $\\mathbf{h}(t_n)$ after conditioning on $n$ events, or alternatively, leverage the latent state trajectory $\\mathbf{h}(t)$ across time $t \\geq t_n$ for richer representations. Given the expressiveness of S2P2, we agree this direction is promising. Exploring these variants systematically, e.g., via ablations on clinical classification in EHRSHOT, will be a valuable extension for future work. We will add this in the discussion section of any camera-ready version.\n\n**Question 3 (Empirical Performance of NHP vs. THP)**\n\nAll baselines used up-to-date PyTorch implementations provided by the EasyTPP library [2]. The library’s implementations for NHP and THP are adopted from Yang _et al._ [3]. These versions were explicitly confirmed by the original authors as being correct (see footnote on pg. 8 of [3]). As noted in [3] and confirmed in our experiments, NHP often outperforms THP under this setup. We will clarify these implementation details in the final version. \n\n[2] Xue, Siqiao, et al. \"EasyTPP: Towards Open Benchmarking Temporal Point Processes.\" _Proceedings of the Twelfth International Conference on Learning Representations (ICLR)_. 2024.\n\n[3] Yang, Chenghao, Hongyuan Mei, and Jason Eisner. \"Transformer Embeddings of Irregularly Spaced Events and Their Participants.\" _Proceedings of the Tenth International Conference on Learning Representations (ICLR)_. 2022.\n\n**Minor Fixes**\n\nThank you for spotting these typos! We will be sure to correct them in a camera-ready version.\n\n\n**Summary**\n\nWe sincerely thank the reviewer for taking the time to assess our submission, for the encouraging and insightful feedback, and for pointing out areas where our presentation can be refined. We are happy to clarify anything further!\n\n-- Submission 20144 Authors"}, {"review_id": "TisWqG7u2e", "reviewer": "Reviewer_CQ9o", "summary": "the paper introduces S2P2 a MTPP model based on a neat connection between state-space models and linear Hawkes' processes. The key contribution of this paper is to utilize this connection to embed mark-specific dynamics in a marked temporal sequence to a, potentially more expressive, latent space. The paper also offers computationally tractable and efficient training of the resulting in a practical and quite novel approach to sequence modeling.", "questions": "Q1. Can the approach be extended to non-categorical marks? It would be interesting since some of the applications of Hawkes' processes have worked with spatio-temporal setting with non-categorical marks. \n\nQ2. how sensitive is the model for potentially noisy/adversarial observations -- either in observed time or intensity. In many sensor settings this is a common occurance, and there have been some works that have improved the robustness of Hawkes' processes (e.g., Chakraborty et al., AAAI'25)", "limitations": "There is no discussion, but I do not see that as a limitation.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "__Strengths:__\n\n1. The paper makes a novel and interesting connection between linear Hawkes processes and deep state-space models. \n\n2. proposes efficient update handling through diagonalization which is technically quite interesting. \n\n3. experimental results over synthetic and real-world data with small and large mark space clearly demonstrates the superiority of s2p2. \n\n\n__Weaknesses:__\n\n1. The paper is fairly dense technically - particularly the combination of diagonalization and discretization stages. \n\n2. latent mark space is quite non-interpretable in comparison to seen marks.", "was_revised": true, "final_justification": "I am quite satisfied with the submitted version and the resulting discussions. I will retain my rating.", "rebuttal": "We thank the reviewer for their strong support and thoughtful feedback. We are glad to hear the core contributions and empirical results were found novel and impactful. Please find our response to your questions and comments below.\n\n**Weakness 1 (Paper is Dense)** \n\nWe agree that the math can be a little dense in places. As you say, space constraints meant some exposition was deferred to the appendix (see, for instance, B.2). In the camera-ready version, we commit to using the additional space to add more signposting, intuitive explanations, and clarifying examples to improve readability and accessibility for a broader audience.\n\n**Weakness 2 (Latent Space is Non-Interpretable)**\n\nThis is a good observation. Indeed, compared to parametric Hawkes models, which have more immediate interpretability, our latent mark representations trade interpretability for expressiveness. This tradeoff is common between neural MTPP models in general; S2P2 is similar in interpretability to, for instance, neural Hawkes process. While we did initially note this in the conclusion, we will have a more expanded discussion of this and opportunities this presents for future research.  For instance, as also noted by reviewer cv8v, recent work on interpretability in SSMs could be extended to S2P2, which we believe is a promising direction for future work.\n\n**Question 1 (Non-Categorical Marks)**\n\nYes, S2P2 can naturally be extended to non-categorical marks with architectural adjustments tailored to the structure of the mark space. For instance, in a spatial setting with marks $k \\in \\mathbb{R}^2$, one could:\n- Embed the continuous-valued mark into a dense vector at the input, similar to how categorical marks are handled;\n- At output, rather than computing separate intensities $\\lambda_k(t)$ for each mark, model the marked intensity as $\\lambda_k(t) := \\lambda(t) \\cdot p(k | t)$, where $\\lambda(t)$ is the total intensity and $p(k | t)$ is the conditional mark density over $\\mathbb{R}^2$ (for spatial).\n\nThe total intensity $\\lambda(t)$ can be computed as in the categorical case, while the conditional density $p(k | t)$ could be parameterized by a Gaussian, mixture model, or more flexible methods (e.g., normalizing flows), all conditioned on the latent state $\\mathbf{h}(t)$. This allows S2P2’s continuous-time latent dynamics to fully inform both time and mark predictions. These extensions are a fascinating direction to explore and potentially very impactful. We will add this in the discussion section of the camera-ready version.\n\n**Question 2 (Sensitivity to Adversarial Observations)**\n\nFormal evaluation under adversarial or corrupted conditions is an open direction for S2P2. We evaluated calibration as part of our suite of metrics, finding that, in general, models are well-calibrated—but we acknowledge we evaluated this under 'normal' conditions. Extending this evaluation to adversarial or corrupted is therefore a very natural follow up. We thank the reviewer for pointing to Chakraborty _et al._ (AAAI’25) as a starting point, and will include discussion of this in the discussion/conclusion section. \n\n**Summary**\n\nWe again thank the reviewer for taking the time to review our submission, for providing such positive and helpful feedback, and some great suggestions for follow-up works! We are more than happy to answer any further questions you may have!\n\n-- Submission 20144 Authors"}, {"review_id": "oqbyXsl7nF", "reviewer": "Reviewer_kua6", "summary": "The paper adds to a line of work on adding non-linear activation to the classical Hawkes process framework, in order to capture more complex marked point processes. The authors describe the model carefully, and explain the advantages in terms of expressivity and computational efficiency w.r.t the state of the art. Extensive experiments with synthetic and real-world data establish the power of the model.", "questions": "In the experiments, I did not understand how hyperparameters (# layers, layer width in particular) were chosen. How sensitive are results to these?\n\nDetails:\n- what is ZOH? please define", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is very carefully written in terms of defining and motivating the model, and was a pleasure to read - although I wonder whether they actually pack too much into this paper (thanks to unlimited supplementary). \n\nThe model is intuitive and well explained. The relationship with the state of the art is less so: it is difficult to appreciate the difference in assumptions/model expressivity with NHP, AttNHP, IFTPP in particular, and why S2P2 is more expressive. Again, I think the authors are forced to be too cursory given the desire to pack everything into this paper. From what I can judge, the model makes sense and has well-reasoned advantages, essentially combining well-known features of state-space models with specific computational tricks to make it work for (marked) point processes.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback, as well as their positive remarks on the clarity and motivation of our model. We are glad to hear the paper was a pleasure to read and that the core modeling ideas and empirical strengths came through. Below, we address your questions and suggestions in detail.\n\n**S2P2’s Relationship to State of the Art is Unclear**\n\nWe intended to raise comparisons between S2P2 and other baselines at the relevant points throughout the paper.  However, as the reviewer points out, we did therefore forego a single explicit block describing the baseline methods (also partly due to space constraints). We will make the descriptions and distinctions between models explicit in the related works and experiments section in any camera-ready version.\n\nPlease also let us know if there was anything specific on regarding baselines that you would like further clarified here!\n\n\n**On Hyperparameter Selection**\n\nThank you for this question. Due to space, we deferred much of this discussion to the appendix. We will use part of the extra page in a camera ready paper to (a) bring up a high-level summary of our hyperparameter search from Appendix C to the main paper, and (b) make the pointer to Appendix C more visible in the main paper.\n\nTo summarize here: S2P2’s main hyperparameters are: the number of layers $L$, the hidden state dimension $P$, and the residual stream dimension $H$. Intuitively, $P$ controls the model’s memory capacity, while $H$ limits the rank of the final transform from hidden states to marked intensities. For the experiments in Tables 1 and 2, individual to each baseline and dataset pairing, we performed a grid search over all hyperparameters to select configurations maximizing validation log-likelihood (note we report fully held-out test performance in the tables). The search space and best configurations are detailed in Appendix C.1 and Table 6.\n\nRegarding sensitivity, we generally found a range of reasonable hyperparameters to yield similar performance. We found that multiple layers $L$ was key for performance, but that there was not a critical dependence on depth. The dimensions $P$ and $H$ were more sensitive across datasets with a wider range, i.e., 16 to 128, being typically acceptable. A reasonable starting point is $H=P=32$ or $64$, followed by fine-tuning as needed when computational resources allow.\n\n\n**What is ZOH?**\n\nThank you for spotting this! We accidentally cut _any_ mention of the full definition in Appendix B.2 (and Table 3) from the main text. We will add a short definition, discussion, and a reference to the full definition back to the main text.  \n\nFor completeness: ZOH stands for _zero-order hold_, a piecewise-constant interpolation technique applied to discrete-time signals to obtain continuous-time approximations. In our model, exact integration of inputs into the recurrent layers is computationally expensive (to the point of being intractable). Approximating the input with ZOH enables a tractable update rule that closely approximates the true continuous-time dynamics. The ZOH approximation is common in the deep SSM literature (see, e.g., S4D, S5).\n\n**Summary**\n\nWe sincerely thank the reviewer for their careful and constructive feedback, as well as their kind words regarding the clarity and motivation of our work. We hope our detailed responses help clarify the points raised, and we welcome any further questions or requests for elaboration. \n\n-- Submission 20144 Authors"}, {"review_id": "4cIyBuvNqe", "reviewer": "Reviewer_V2CN", "summary": "This paper proposes S2P2, a deep learning architecture for modeling Marked Temporal Point Processes (MTPPs) by blending ideas from Hawkes processes and state space models (SSMs). The authors introduce a new recurrent building block, the Latent Linear Hawkes (LLH) layer, which generalizes the dynamics of linear Hawkes processes using the expressive, continuous-time latent recurrences of SSMs.\nS2P2 is tested on snythetic data as well as eight real-world datasets.", "questions": "In practice, how does S2P2 handle sequences with >10,000 events? Are there any memory or stability concerns in such regimes, especially when evaluating intermediate intensities at random times?", "limitations": "yes", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I have to say, that I am not very familiar with models for temporal point processes.\nStill I found the following strengths in the paper:\n\nRather than just using a black box model, their latent linear Hawkes layer has an inductive bias towards event sequences which extends the classical linear hawkes process.\n\nThe figures, especially figure 2, are highly informative and help the reader to understand the main ideas of the paper.\n\nThe experimental section includes synthetic data as well as real-world datasets and highlights the efficacy of the proposed method.", "was_revised": true, "final_justification": "I appreciate the author’s thoughtful responses to my questions. I keep the original score.", "rebuttal": "We sincerely thank the reviewer for their positive evaluation and for highlighting the strengths in our model design, visualization, and experiments. We are glad that the inductive bias of the latent linear Hawkes layer and the clarity of the figures came through clearly, even for someone unfamiliar with the specific area.\n\n**How does S2P2 handle long sequences?**\n\nThe memory usage for S2P2 scales linearly with the sequence length, enabling it to handle long sequences efficiently in practice. As shown in Section B.5 of the Appendix, we benchmarked inference latency for S2P2 and other models across sequences ranging from 8 to 500,000 events. _In these tests, S2P2 exhibited no memory or stability issues during inference._\n\nWe expect this stability to extend to training on long sequences as well. This is due to the way the latent dynamics in the latent linear Hawkes layer are parameterized: the hidden states are guaranteed to converge to a steady state over time, which helps mitigate instability in long-range computations.\n\n\n**Summary**\n\nWe thank the reviewer again for their positive and thoughtful comments. We’re glad that our explanations and figures helped make the core ideas accessible, and we hope our response clarifies S2P2’s scalability and stability on long sequences. Please don’t hesitate to reach out if we can provide any additional clarifications or details to assist your understanding further.\n\n-- Submission 20144 Authors"}]

Author rebuttal:

Meta-review: The paper introduces **S2P2**, a state-space point process model that integrates linear Hawkes processes with modern deep state-space modeling techniques to effectively capture the dynamics of marked temporal point processes. Its key contributions are the Latent Linear Hawkes layer, efficient parallelizable inference, and strong empirical gains (33% average improvement over baselines across eight datasets). Reviewers highlighted the paper’s technical novelty, careful exposition, and thorough experiments as major strengths. Concerns raised were mainly about interpretability, clarity on comparisons with some baselines, dense mathematical exposition, and extensions to non-categorical marks or robustness to corrupted data. The rebuttal convincingly addressed these points by clarifying distinctions from prior work, elaborating on hyperparameter selection, scalability, and stability, and outlining interpretability and robustness as promising directions for future research. All reviewers maintained positive scores after discussion, with one strong accept. Given the combination of conceptual novelty, rigorous methodology, and state-of-the-art empirical performance, I recommend **acceptance**. This paper makes a significant and timely contribution to probabilistic modeling of event sequences and will likely have strong impact in both theoretical and applied domains.
Decision: Accept (spotlight)