Title: Deep Continuous-Time State-Space Models for Marked Event Sequences
Venue: NeurIPS 2025
Accepted: yes
Keywords: Marked temporal point processes

state-space models

Hawkes processes
Abstract: Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the _state-space point process_ (S2P2) model, a novel and performant model that leverages techniques derived for modern deep state-space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-time event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an architecture that interleaves stochastic jump differential equations with nonlinearities to create a highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with a parallel scan, bringing linear complexity and sublinear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.

Deep Continuous-Time State-Space Models for Marked Event Sequences

Abstract

Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the state-space point process (S2P2) model, a novel and performant model that leverages techniques derived for modern deep state-space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-time event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an architecture that interleaves stochastic jump differential equations with nonlinearities to create a highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with a parallel scan, bringing linear complexity and sublinear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.

1 Introduction

Marked temporal point processes (MTPPs) are used to model irregular sequences of events in continuous time, where each event has an associated type, often called a mark . MTPPs model the joint distribution of these sequences of event times and marks. They have been successfully applied to modeling purchasing patterns in e-commerce [Türkmen et al., 2019a, Vassøy et al., 2019, Yang et al., 2018], patient-specific medical events [Hua et al., 2022], disease propagation [Gajardo and Müller, 2023], and event modeling and prediction across multiple other domains [Williams et al., 2020, Sharma et al., 2018, Wang et al., 2024]. An MTPP can be fully characterized by a marked intensity process , specifying the instantaneous rate of occurrence of each mark conditioned on history.

State-of-the-art neural methods compute hidden states to summarize the event history, which are then used to compute marked intensities at any point in time. However, many models are limited by inexpressive temporal dynamics, lack of support for long-range dependencies, and serial computation [Du et al., 2016, Mei and Eisner, 2017]. Recent advances in transformer-based MTPPs improved performance and gained parallelism, but scale quadratically in sequence length [Zhang et al., 2020,

(a)

Rank

(Lower

↓

is better,

x

= best, x = second-best)

Likelihood

Calibration

Composite

Model

Mark

Time

Time RMSE

Mark

Time

Rank

RMTPP

[Du et al., 2016]

6.8

6.1

6.9

5.0

5.8

6.1

6.1

SAHP

6.9

4.9

7.1

4.3

6.8

5.4

5.5

THP

5.5

7.0

5.6

4.3

5.0

5.4

5.5

IFTPP

4.1

3.1

5.0

4.0

1.8

2.6

3.6

MHP

[Gao et al., 2024]

5.4

6.4

4.3

5.5

4.8

6.0

5.3

NHP

[Mei and Eisner, 2017]

2.4

3.3

1.8

2.4

4.9

3.6

2.9

AttNHP

Yang et al. [2022]

2.4

3.3

2.9

6.6

3.4

3.7

3.7

S2P2

(Ours)

1.9

1.4

1.9

2.3

3.0

2.8

2.1

(b)

Zuo et al., 2020, Yang et al., 2022], preventing them from being used in practice for long sequences, such as modeling clinical events over a patient's entire medical history spanning many years.

Recently, deep state-space models (often abbreviated as SSMs) have emerged as a challenger to transformer-based models for discrete sequence modeling [Gu et al., 2022b, Smith et al., 2022, Gu and Dao, 2023]. SSMs interleave a stack of linear state-space recurrences with position-wise nonlinearities [Gu et al., 2021]. This architecture not only achieves superior performance on a wide range of tasks [Goel et al., 2022, Deng et al., 2023], but retains linear scaling, can be parallelized across the length of a sequence, and can gracefully handle irregularly spaced observations.

Despite being defined in continuous-time, applying SSMs to MTPPs is not straightforward. Tying the conditional intensity to the model's state is the obvious choice, but SSMs expect a continuous-valued signal to integrate as input, whereas event sequences are discontinuous by nature. Taking inspiration from classical parametric linear Hawkes processes (LHPs) [Hawkes, 1971], we introduce a stochastic jump differential equation on the complex plane to serve as the SSM recurrence. We refer to this as a latent linear Hawkes (LLH) layer. We derive closed-form update rules for any time that allow the diagonalized continuous-time SSM system to use efficient parallel scans. We then introduce our state-space point process (S2P2) model, a stack of LLH layers interleaved with position-wise nonlinear functions (Fig. 1a), inspired by deep SSMs. This design yields an MTPP that has both a highly flexible conditional intensity function and access to efficient, parallelizable computation.

This paper is organized as follows: In Section 2 we present the necessary preliminaries on MTPPs, LHPs, and deep SSMs. In Section 3 we introduce our state-space point process architecture. We first build the connection between LHPs and deep SSMs, and then derive how to use this connection to make a highly expressive, parsimonious and parallelizable MTPP. In Section 5 we then verify this expressivity on small-scale, targeted synthetic explorations comparing to key baselines. In Section 6 we empirically evaluate our model at scale on a range of metrics across eight real-world datasets, finding that S2P2 matches or exceeds the average predictive performance of baselines, achieving either best- or second-best average performance on all six metrics. These results are summarized in Table 1b. In Sections 7 and 8 we conclude by discussing the relative advantages and limitations of S2P2 and promising future opportunities. Due to its robustness, performance, and efficiency, our S2P2 model is a powerful model out-of-the-box for a wide range of MTPP applications.

2 Preliminaries

2.1 Marked Temporal Point Processes

We define an event sequence, or history , as a sequence of time-mark pairs, H t := { ( t i , k i ) | t i ≤ t for i ∈ Z + } , where t i ∈ R ≥ 0 , ∀ i : t i -1 < t i , and k i ∈ M . 2 In this paper, we focus on discrete and

finite mark spaces, i.e., M := { 1 , . . . , K } ; however, M can be more general, such as countable or continuous. We also define H t -similarly to H t , except that it excludes events at exactly time t .

One way of characterizing an MTPP is through a marked intensity process . The intensity λ t := [ λ 1 t , . . . , λ K t ] ⊤ ∈ R K ≥ 0 characterizes an MTPP by describing the rate of occurance of events:

with the total intensity λ t := ∑ K k =1 λ k t being the rate that any event occurs. This intensity also defines a marked counting process N t := [ N 1 t , . . . , N t K ] ⊤ ∈ Z K ≥ 0 , representing the number of occurrences of events of each type of mark in the time span [0 , t ] .

Parameterized forms of λ are often trained by optimizing the log-likelihood over observed data (e.g., [Mei and Eisner, 2017, Zuo et al., 2020]). The log-likelihood for a single sequence H T is defined as [Daley and Vere-Jones, 2003, ch. 7.3]:

Linear Hawkes Processes A foundational MTPP is the linear Hawkes process (LHP). The LHP is a self-exciting process , where event occurrence increases the rate of occurrence of other events, with the influence decaying according to a kernel . The intensity function is the summation of influences, and if the kernel is the exponential function, it has the following integral and differential forms:

where, to ensure non-negative marked intensities, ν ∈ R K ≥ 0 and α , exp( -β ) ∈ R K × K ≥ 0 with exp being the matrix exponential. This form (which is the most common) of the LHP is incredibly limited, and hence is used primarily for its interpretability, as opposed to outright predictive performance.

2.2 Deep State-Space Models

Deep state-space models (SSMs) are a class of recurrent models that have excelled in long-range sequence and language modeling tasks, all while having favorable computational properties [Gu et al., 2022b, Smith et al., 2022, Gu and Dao, 2023]. The backbone of deep SSMs are linear state-space models, which define a continuous-time dynamical system with inputs and outputs u ( t ) , y ( t ) ∈ R H through linear differential equations:

where x ( t ) ∈ R P is the (hidden) state of the system, and A , B , C , D define the system's dynamics. Deep SSMs interleave these recurrences with nonlinear position-wise functions σ as u ( l ) ( t ) := σ ( y ( l -1) ( t )) (for layer l ). This yields a sequence model where each recurrence is conditionally linear in time but is nonlinear overall due to σ .

To evaluate an SSM, we first discretize the continuous-time system at appropriate times to yield a discrete sequence of closed-form state updates [Smith et al., 2022]. The resulting discrete-time recurrence can be evaluated using parallel scans [Blelloch, 1990], with linear work scaling and, importantly, sublinear (theoretically logarithmic) scaling of the computation time with respect to sequence length given sufficient parallel compute. This parallel evaluation natively allows varying observation intervals or latent dynamics.

3 State-Space Point Process

We seek to define an MTPP model that is (a) highly expressive and (b) can access efficient and parallelizable compute methods. In this section, we formally introduce our state-space point process (S2P2) model, as outlined below: Section 3.1 extends and generalizes the continuous-time form of the LHP, creating a layer we refer to as the latent linear Hawkes (LLH) layer. This generalization

decouples the choice of the width of the layer from the mark space, allowing us to make arbitrarily wide layers, and is the basis for the novel connection between the LHP and deep SSMs. Section 3.2 discusses how we make each LLH efficient and tractable to compute with parallel scans, and Section 3.3 extends the model with time-varying dynamics to be more expressive. We conclude in Section 3.4 by describing S2P2, a highly expressive MTPP model built from the composition of multiple LLH layers and nonlinearities that can be computed with a novel parallel inference scheme.

3.1 Continuous-Time Latent Linear Hawkes Layer

To develop a recurrent layer for an MTPP model, we start by reviewing the equations for an LHP intensity, Eq. (3), and an SSM state, Eq. (4). If we allow ν in the LHP to vary over time, we obtain:

When presented together the parallels between them become apparent: the LHP intensity, λ t controlled by decay rates β , is analogous to the state in the linear SSM, x ( t ) controlled by state matrix A . The time-varying background LHP intensity, ν t , is analogous to the SSM input signal after being projected into the state-space, Bu ( t ) . Compared to SSMs which allow for recurrence over a latent space R P , Eq. (6) is limited in expressivity due to restricted dynamics β and fixed dimensionality tied to the number of possible marks K . Not present in Eq. (7) is an impulse α d N t , which is crucial for allowing the recurrence to condition on abrupt event occurrences over time.

With this in mind, we combine Eqs. (6) and (7) to resolve their respective deficiencies. This results in the following set of stochastic jump differential equations that we call the latent linear Hawkes (LLH) layer, which will serve as a continuous-time recurrence layer in our proposed model:

where α ∈ R R × K , E ∈ R P × R , and the standard SSM parameters share the same dimensionality as in Eqs. (4) and (5). Note that the resulting dynamics within the LLH layer are much more expressive than the LHP that it was inspired by, due to both operating in a separate set of latent dimensions and by having a general dynamics matrix A ∈ R P × P , compared to the more restrictive exp( -β ) ∈ R K × K ≥ 0 . For parameter efficiency, we allow the new impulse term E α d N t to be computed as a product of model-wide, low-rank mark embeddings α of rank R and a layer-specific projection into state-space E . For simplicity, we set R := H in practice. Realizations of this layer are shown in Fig. 2.

This LLH will serve as the main form of recurrence in a larger continuous-time model that we define in Section 3.4. In deriving this layer as an extension from LHPs, we retain a strong inductive bias towards event sequences. Furthermore, by drawing connections between this layer and traditional state-space equations, we allow the downstream model to benefit from many of the innovations designed for deep SSMs in recent years, such as parallel computation.

3.2 Diagonalizing, Discretizing & Computing the LLH Recurrence

Unlike the LHP intensity, there is no analytical solution to the continuous-time LLH recurrence due to the continuously-integrated u ( t ) signal. We must therefore approximate the system at specific

timepoints. If we approximate the input signal by treating it as constant over an update interval, also known as a zero-order hold (ZOH) assumption [Iserles, 2009], then we can achieve a closed-form exact update to the recurrence relation. To avoid a computationally expensive matrix exponential in the update rule, we adopt the same general approach as Smith et al. [2022] for deep SSMs by first diagonalizing the system and then imposing the zero-order hold restriction on it. This converts the matrix exponential into an element-wise exponential operation for each LLH layer in S2P2.

Diagonalization: Let -A be diagonalizable with a factorization of VΛV -1 , where V , Λ ∈ C P × P and Λ is a diagonal matrix of eigenvalues. An equivalent, diagonalized LLH is then

where ˜ x t = V -1 x t , ˜ B = -V -1 B , ˜ E = V -1 E , and ˜ C = CV . We can then directly parameterize ˜ B , ˜ C , and ˜ E to avoid learning and inverting V . We use the same initialization strategies as S5 [Smith et al., 2022] for ˜ B , ˜ C , and ˜ E , based off the HiPPO initialization scheme [Gu et al., 2020]. The eigenvalues Λ are also directly parameterized and constrained with negative real-components to enforce stability. While the dynamics are diagonalized, we note this does not mean that we are modeling the intensities of different mark types as independent. Marks interact through both the position-wise nonlinearities and the learned projections ˜ B , ˜ C and ˜ E (since the diagonalized dynamics are equivalent to the original dynamics in Eq. (8), given that the system can be diagonalized on the complex plane).

Discretization: We then use a ZOH discretization to create a closed-form update from the diagonalized continuous-time system; please refer to Fig. 2 for an illustration and Appendix B.2 for the full derivation. This results in the following update rule in the diagonalized eigenbasis, that transitions from x t to x t ′ without a matrix exponential, where, by construction, no events occur in ( t, t ′ ) :

where ¯ Λ := exp( Λ ( t ′ -t )) . ZOH is an exact update when u is constant over the window [ t, t ′ ) ; as such, we can choose any value u s for s ∈ [ t, t ′ ) to hold constant as the input over the integration period. We opt to use u t ′ -so that the model can condition on the fact that no events have occurred between t and t ′ . This design decision and its impact on performance are explored in more detail in Appendices B.4 and D.6. It is important that u t ′ is not used as the ZOH value to avoid data leakage.

Computing LLH Recurrence: The final component is to derive how to use parallel scans to efficiently evaluate the closed-form updates for the modified LLH recurrence in parallel. Parallel scans admit efficient inference over linear recurrences of the form z i +1 = R i z i + b i [Blelloch, 1990]. Although we have an impulse in the recurrence, Eq. (8) is still intrinsically of this form, where z i := x t i , R i := exp( Λ i ( t i +1 -t i )) , and b i := ( R i -I ) ˜ Bu t i +1 -+ ˜ E α k i +1 . As a result, we can leverage parallel scans to compute the sequence of right-limits x t 1: N in parallel across the sequence length. The corresponding left-limits x t 1: N -, which will later be used to calculate event intensities, can then be directly and efficiently computed by subtracting the impulse, ˜ E α k 1: N , from x t 1: N .

3.3 Input-Dependent Dynamics

Inspired by recent developments in modern SSMs (e.g., Mamba [Gu and Dao, 2023]), we also consider allowing the dynamics of the system to vary depending on the input and on the history of previous events. This can allow for more expressive intensities. For instance, dynamically adjusting the real components of Λ to be smaller will result in more influence from history. Alternatively, larger values will result in more quickly 'forgetting' the influence of previous events for a given hidden state channel. This is formalized with the following recurrence relation for t ∈ ( t i , t i +1 ] :

where Λ i := diag ( softplus ( W ′ u t i + b ′ )) Λ with W ′ ∈ R P × H and b ′ ∈ R P . This is conditionally linear in time, as even though Λ i changes, it is entirely input-dependent based on u and not dependent on previous values of x , and hence we can still use parallel scans as discussed above.

3.4 State-Space Point Process Architecture

We are now well-positioned to present the state-space point process (S2P2), a flexible and parallel deep continuous-time model for marked intensities λ t . This model is shown in Figs. 1a and 2.

We have demonstrated how to define an efficient, parallelizable and scalable core MTPP layer in the LLH layer. While the diagonalized and discretized LLH layer is more expressive than the LHP, it remains fundamentally linear. To compensate, we take inspiration from deep SSMs and alternate L LLH layers with position-wise nonlinearities. This creates a nonlinear model that can still use parallel computation over the sequence length, but is highly expressive compared to each linear recurrence.

We have an input signal in two parts: (i) the continuously-integrated signal u t and (ii) the discrete event impulses α k . For the first layer, the only inputs available to condition on are the event impulses themselves, so we set u (1) t = 0 for all t ≥ 0 . At deeper layers, we have a layer-specific impulse as well as the continuously integrated signal from the previous layer. In general, a layer's output y ( l ) := LLH ( l ) ( u ( l ) , H ) is passed into a nonlinear activation function σ (we use σ ( z ) := GELU ( z ) [Hendrycks and Gimpel, 2016]), summed with the residual stream u ( l ) , and normalized with LayerNorm [Ba, 2016] for the next layer's input. Formally, for t ≥ 0 and l = 1 , . . . , L , then

Due to the unrestricted nature of the recurrences and nonlinearities (and unlike the original LHP), we enforce non-negative intensities by applying an affine projection followed by a rectifying transformation, similar to Mei and Eisner [2017]. This is referred to as the 'Proj. & Softplus' layer in Fig. 1, and expressed as λ t := s ⊙ softplus (( Wu ( L +1) t -+ b ) ⊙ s -1 ) for t ≥ 0 and where ⊙ is an element-wise product, W ∈ R K × H , and b , log( s ) ∈ R K . Then S2P2 can be trained by maximizing the log-likelihood of each sequence, Eq. (2). Similar to other neural MTPPs, we use Monte Carlo estimation for the integral term ∫ T 0 λ s d N s [Mei and Eisner, 2017]. Training the model requires computing intensities at both event times t 1: N and at randomly sampled times t ∼U (0 , T ) .

To compute intermediate intensities at these sampled points, we take advantage of the continuous-time nature of S2P2 and partially evolve the latent state through the system dynamics. To do so, we first compute the right limits of the hidden states at event times, x (1: L ) t i , as described above (we can do this conditioning in logarithmic depth). Then, Eq. (12) is applied, with no impulse, as no events are occurring at these intermediate points, to find x (1: L ) t . From there, the evolved hidden states are used to evaluate the model across depth to compute the intermediate intensity λ t . This operation is both efficient and can be done in constant complexity because intermediate evaluations are conditionally independent given the right limits at events. Crucially, there is no separate parametric decoding head, unlike, for instance, Mamba Hawkes processes [Gao et al., 2024] or transformer Hawkes processes [Zuo et al., 2020]. Instead, this tying of the intensity to the model's continuously evolving hidden states, more like the neural Hawkes processes [Mei and Eisner, 2017], makes S2P2 a continuous-time model and contributes to its enhanced expressivity (see Section 5 and Section 7 for more discussions). In Algorithms 1 to 3, we explicitly detail how to use a parallel scan to compute the sequence of right limits at events, how to then evolve those to compute left limits, and then how to subsequently compute the log-likelihood of the sequence.

4 Related Work

Neural MTPPs: MTPPs are generative models that jointly model the time and type of continuoustime sequential events, typically characterized by mark-specific intensity functions [Daley and Vere-Jones, 2003]. Early approaches used parametric intensity functions, such as self-exciting Hawkes processes [Hawkes, 1971, Liniger, 2009]. More recently, neural models such as RNNs [Du et al., 2016, Mei and Eisner, 2017] and transformers [Zhang et al., 2020, Zuo et al., 2020, Yang et al., 2022] were developed to enable flexible modeling of conditional intensities. Intensity-free MTPPs include normalizing flows [Shchur et al., 2020a, Zagatti et al., 2024], transformers [Draxler et al., 2025], neural processes [Bae et al., 2023], and diffusion models [Zeng et al., 2024, Zhang et al., 2024]; however, modeling intensities is more common, requiring fewer modeling restrictions.

Efficient MTPPs: Due to their recurrent nature, RNN-based MTPPs incur O ( N ) scaling for lengthN sequences as events are processed sequentially. Attention-based MTPP models can be applied in

parallel across the sequence, but with O ( N 2 ) computational work. Türkmen et al. [2019b] model events as conditionally independent if they occur within the same time bin of a specified size, resulting in parallel computation within bins, but still scaling as O ( N ) overall. Shchur et al. [2020b] proposed an intensity-free TPP using triangular maps and the time-change theorem [Daley and Vere-Jones, 2003]. This was extended by Zagatti et al. [2024] to handle marks but losing benefits of the original model and scaling linearly in the mark dimension-which can rapidly become untenable as O ( NK ) work. Our S2P2 scales as O (log N ) and efficiently in marks; more discussion in Appendix B.5.

SSMs for Sequential Modeling: SSMs have found recent success as alternatives to RNNs, CNNs, and transformers, enjoying reduced training cost and comparable modelling power [Gu et al., 2022b]. A range of variants have been developed [Gu et al., 2021, Gupta et al., 2022, Gu et al., 2022a, Smith et al., 2022] and applied in language modeling [Gu and Dao, 2023], speech [Goel et al., 2022], and vision [Wang et al., 2023, Zhu et al., 2024]. The linear recurrence enables parallelism, as well as accessible long contexts that are prohibitive for transformers due to quadratic scaling. However, SSMs have not previously been used as continuous-time models for MTPPs, in part due to the intensity functions having different left and right limits, and the input being a stochastic counting process.

SSMs for TPPs: Gao et al. [2024] propose using an SSM as a discrete sequence model, encoding an event sequence into a fixed set of static hidden states, then computing intensities with a separate parametric decoder (similar to transformer Hawkes processes [Zuo et al., 2020], but with the transformer replaced with Mamba [Gu and Dao, 2023]). This is fundamentally different to our continuous-time model, where instead of a separate parametric decoding head, we leverage the continuously evolving latent state at any time t to compute predicted intensities at corresponding times. We empirically compare to their model MHP in Section 6, finding that MHP (i) performs comparably to the THP, and (ii) is comprehensively outperformed by S2P2.

5 Synthetic Experiments

We first verify the ability of our S2P2 model to represent known intensity functions through a series of targeted experiments on synthetic data. Recent results from the deep SSM literature [Muca Cirone et al., 2024] show that sufficiently deep SSMs are able to approximate arbitrary continuous functions; their results apply to our S2P2 model, and hence we should be able to (verifiably) recover arbitrary and known intensity functions. To demonstrate this, we showcase recovering ground truth intensities in three different settings: the classical point processes of Hawkes and self-correcting processes; an inhomogeneous Poisson process with a discontinuous intensity function; and finally a marked process with long-range dependencies between marks. Our S2P2 recovers the correct intensity in all cases. We include full results for all baselines and reproducibility details for all the following examples and tasks in Appendix D, including intensity visualizations, where S2P2 works as intended, corrects recovers intensities, and matches or exceeds baseline methods performance.

Classical Point Processes: We first apply our model and baseline models to two classical point processes: a Hawkes process and a self-correcting process with intensity functions λ t = 0 . 5 + ∑ i 0 . 5 exp( t i -t ) and λ t = exp( t -N t ) , respectively. Examples of the recovered intensity functions are shown in Fig. 4a. We see that our S2P2 recovers the ground truth intensities nearly perfectly. We also explore applying our model and other baseline models to randomly generated synthetic multivariate Hawkes processes in Appendix D.3, finding that all methods perform comparably.

Inhomogeneous Poisson Process: Our S2P2 does not have a fixed parametric form for the intensity decoder (cf. THP or MHP) or limited recurrent dynamics (cf. NHP). This flexibility should allow the S2P2 to capture intensities where other methods fail. This is showcased in Fig. 3. Models are trained on sequences drawn from an inhomogeneous Poisson process

with a square wave for an intensity function (except for t > 7 ). We observe that baseline models fail in predictable ways due to their expressivity limitations (please refer to Appendix D.2 for full results), whereas our model successfully captures the true background intensity process almost perfectly.

(a) S2P2 successfully recovers parametric intensities from self-correcting and Hawkes processes.

(b) Long-range dependencies between | | → | | are accurately captured by S2P2, whereas NHP struggles to recover λ t .

Long-Range Dependencies: Lastly, we study a task with known long-range dependencies, depicted in Fig. 4b. In this task, a 'trigger' mark (green) is drawn from a homogeneous Poisson process ( λ =0.1); each trigger is then followed by a 'target' mark (orange) a predictably long time later. Here 'long' refers to the fact that (a) there are often many trigger marks before a single target mark, and (b) the variance of the distribution over the trigger-target time is comparatively narrow compared to the mean. Gray marks are 'distractors' drawn from a homogeneous Poisson process ( λ =1). We see that S2P2 successfully captures the long-range dependencies, whereas NHP struggles. In spite of the long delay between cause and effect within the data, S2P2 was able to recover 98% of the true data likelihood, whereas NHP only achieved 88%. This shows that the state-space point process successfully captures long-range dependencies and is flexible as an approximation family.

6 Real-World Experiments

We now present the main experiments in this paper, evaluating on real-world datasets. 3 Further results and extensive experimental details are included in Appendices C and D.

Datasets: We compare models on eight different datasets, including five datasets available from EasyTPP [Xue et al., 2023] (Amazon, Retweet, Taxi, Taobao and StackOverflow). We also add two commonly used datasets in the literature (Last.fm and MIMIC-II), as well as a new medical events dataset derived from the publicly available EHRSHOT dataset [Wornow et al., 2023]. EHRSHOT is an order of magnitude larger than any dataset in EasyTPP in both the number of marks and the maximum sequence length (see summary statistics in Table 7), representing a challenging application.

Metrics: We performed a comprehensive evaluation using three common families of metrics: (i) total per-event log-likelihood [Mei and Eisner, 2017, Zhang et al., 2020, Zuo et al., 2020, Yang et al., 2022, Gao et al., 2024] (also broken down into mark- and time-specific likelihoods); (ii) predictive accuracy summary metrics of next-mark classification and next event time RMSE [Du et al., 2016, Zhang et al., 2020, Zuo et al., 2020, Yang et al., 2022, Gao et al., 2024]; and (iii) mark and event time calibration metrics [Zhang et al., 2020, Bosser and Taieb, 2023] measuring the reliability of the implied uncertainties (see Appendix D.7). In order to characterize overall performance across multiple diverse metrics, we also compute a summary composite rank in Table 1b, defined as the average rank obtained for a given model across all dataset-metric combinations across five random seeds (i.e., is an average of 240 individual evaluations and five differently initialized trained models).

Models: We use the EasyTPP library [Xue et al., 2023] for baseline models and S2P2. We perform extensive hyperparameter searches for each individual pair of baseline model and dataset, then select

(c) Next mark classification accuracy (top-10 accuracy on EHRSHOT). Higher values indicate better performance.

the configurations that maximizes validation log-likelihood per model/dataset. The range of grid search and other details are elaborated in Appendix C.1; results are reported on a fully held-out test set. All models were trained on a single 24GB NVIDIA A5000 GPU.

Main Results: We report full numerical results in Table 2, and a summary of rankings in Table 1b. Table 2a shows that S2P2 consistently achieves the best or the second-best held-out log-likelihood results across all datasets, beating other methods by over a whole rank and with a (geometric) mean likelihood ratio of 1.33 (corresponding to a 33% higher likelihood of true events). This is calculated by computing the mean log-likelihood ratio across all datasets and then exponentiating. We further investigate the log-likelihood improvement of S2P2 by separating the time- and mark-specific loglikelihood, finding that S2P2 is mainly driven by better temporal modeling and while achieving gains in both time and mark modeling over existing methods (see Table 8 and Fig. 6 in Appendix D.1 for full results). Additionally, we report prediction summary metrics, next event time prediction and next mark classification, in Table 2b and Table 2c. We again see that S2P2 performs well, matching or exceeding the best-performing baseline, and far outpacing other baseline methods.

Additional Experiments: We defer several additional experiments and explorations to Appendices:

Calibration results: We also include full calibration results, both in raw numerical form and as reliability graphs [Bosser and Taieb, 2023]. These show that S2P2 is well calibrated, placing highest amongst intensity-based methods, only being beaten by IFTPP [Shchur et al., 2020a].

Computational scaling: We define the theoretical time complexity of each baseline, where S2P2 is best with logarithmic complexity and linear work. We verify this scaling empirically.

Ablations : Wealso perform an ablation study on whether input dependent dynamics and the choice of which input to hold constant over the integration interval (introduced in Sections 3.2 and 3.3)

affect performance. We find input-dependent dynamics nearly always improve performance, and that performance is less sensitive to the specific form of the LLH's input signal u t in ZOH.

7 Discussion

Continuous-Time Hidden States: The S2P2 and NHP [Mei and Eisner, 2017] architectures are closely related, in that both model continuous-time latent states . For the NHP, the intensity function is parameterized by a hidden state h ( t ) evolved via a continuous-time LSTM variant. Similarly, S2P2 continuously evolves a set of continuous-time latent states h 1: L ( t ) , one for each layer { 1 , . . . , L } . Then the hidden state h L ( t ) of the top layer is decoded into intensities at any t . In contrast, models such as THP, MHP, IFTPP, only have latent states h ( t j ) at discrete event times t j ∈{ 1: N } . Consequently, some parametric shapes need to be defined to 'interpolate' the hidden state values between events, thus sacrificing models' flexibility due to these parametric functions.

Our results are grouped according to whether or not models have continuous-time hidden states. We empirically verify that continuously evolving hidden states enable more expressive latent dynamics, where the expressiveness in explaining the data is measured through log-likelihood; see Appendix D.1. S2P2 further improves scalability by incorporating SSMs that are naturally in continuous-time and by leveraging efficient parallelized scans. S2P2 also introduces a stronger inductive bias than most MTPP models (similar to NHP), helping to achieve superior next event predictions.

Limitation and Future Directions: S2P2 forgoes the interpretability of the parameters and latent states of linear Hawkes processes (LHPs), meaning it is not appropriate when the interpretation of the underlying system is crucial. This tradeoff is common with neural MTPP models (e.g., NHP), because the limited expressivity of LHP may lead to poor predictive performance, as well as misleading interpretations due to underfitting. Therefore, an exciting opportunity for future research is exploiting the connection to recover LHP-like interpretability while retaining the enhanced predictive power of S2P2. For example, any recently developed techniques for interpreting deep SSMs can be extended to S2P2, such as Ali et al. [2025], while fully taking advantage of the continuously varying attention from continuous-time latent dynamics of S2P2 will provide richer insights compared to discrete-time attention maps (e.g., large language models).

Beyond this, further directions include leveraging theories and best practices from deep SSMs, such as the enhanced parameterizations presented by Merrill et al. [2024]. S2P2 can also naturally be extended and accommodate non-categorical marks by tailoring its architecture to the structure of the mark space, such as marks having continuous values or containing richer information. A concrete example is spatio-temporal point processes that have applications in seismic and weather forecasting. Other valuable extensions include serving S2P2 as a pre-trained backbone for downstream applications, such as EHRSHOT clinical classification tasks. Additionally, censoring or adversarial conditions are commonly seen in practice [Boyd et al., 2023, Chakraborty et al., 2025]. Extending the evaluation of S2P2 and MTPP models in general remains an open direction for real-world MTPP deployments.

8 Conclusion

In this paper, we present the state-space point process (S2P2) model-a novel fusion of concepts from LHPs and SSMs. S2P2 uses deep stacks of stochastic jump differential equations to create an expressive and parsimonious MTPP without additional and restrictive intensity decoding heads, while simultaneously being able to leverage parameterizations and techniques borrowed from deep SSM architectures. We demonstrated that S2P2 outperforms existing methods across a range of standard and new benchmark tasks, and over a range of predictive metrics and efficiency evaluations.

As an intensity-based model, S2P2 requires numerical integration for predicting the expected next event times (as with any neural intensity-based MTPP model) and is not able to model point masses in time (due to the MTPP assumption of no concurrent events). All other baselines suffer from at least one key additional deficiency (e.g., NHP has linear scaling, THP has quadratic work, IFTPP has a parametric decoder, etc.), all of which are resolved by S2P2, while also achieving better performance. We provide a PyTorch implementation via EasyTPP for out-of-the-box usage. We believe the robustness, state-of-the-art predictive performance across a range of metrics, computational efficiency, and extensibility of S2P2 make it a very competitive model for a wide range of MTPP applications.

References

Ameen Ali Ali, Itamar Zimerman, and Lior Wolf. The hidden attention of Mamba models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 1516-1534, 2025.

Jimmy Lei Ba. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016.

Wonho Bae, Mohamed Osama Ahmed, Frederick Tung, and Gabriel L. Oliveira. Meta temporal point processes. In The Eleventh International Conference on Learning Representations , 2023.

Guy Blelloch. Prefix sums and their applications. Technical report, Tech. rept. CMU-CS-90-190. School of Computer Science, Carnegie Mellon, 1990.

Tanguy Bosser and Souhaib Ben Taieb. On the predictive accuracy of neural temporal point process models for continuous-time event data. Transactions on Machine Learning Research , 2023. ISSN 2835-8856.

Alex Boyd, Robert Bamler, Stephan Mandt, and Padhraic Smyth. User-dependent neural sequence models for continuous-time event data. Advances in Neural Information Processing Systems , 34: 21488-21499, 2020.

Alex Boyd, Yuxin Chang, Stephan Mandt, and Padhraic Smyth. Inference for mark-censored temporal point processes. In Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence , volume 216, pages 226-236. PMLR, 2023.

Oscar Celma. Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space . Springer, 2010.

Pritish Chakraborty, Vinayak Gupta, Srikanta J Bedathur, Abir De, et al. Differentiable adversarial attacks for marked temporal point processes. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 15704-15712, 2025.

Yuxin Chang, Alex Boyd, and Padhraic Smyth. Probabilistic modeling for sequences of sets in continuous-time. In International Conference on Artificial Intelligence and Statistics , volume 238, pages 4357-4365. PMLR, 2024.

Daryl J Daley and David Vere-Jones. An Introduction to the Theory of Point Processes: Volume I: Elementary Theory and Methods , page 258. Springer, 2003.

Fei Deng, Junyeong Park, and Sungjin Ahn. Facing off world model backbones: RNNs, Transformers, and S4. Advances in Neural Information Processing Systems , 37:72904-72930, 2023.

Felix Draxler, Yang Meng, Kai Nelson, Lukas Laskowski, Yibo Yang, Theofanis Karaletsos, and Stephan Mandt. Transformers for mixed-type event sequences. In Advances in Neural Information Processing Systems , volume 39, 2025.

Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1555-1564, 2016.

Álvaro Gajardo and Hans-Georg Müller. Point process models for COVID-19 cases and deaths. Journal of Applied Statistics , 50(11-12):2294-2309, 2023.

Anningzhe Gao, Shan Dai, and Yan Hu. Mamba Hawkes process. arXiv preprint arXiv:2407.05302 , 2024.

Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with statespace models. In International Conference on Machine Learning , volume 162, pages 7616-7633. PMLR, 2022.

Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023.

Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré. HiPPO: Recurrent memory with optimal polynomial projections. Advances in Neural Information Processing Systems , 34: 1474-1487, 2020.

Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Advances in Neural Information Processing Systems , 35:572-585, 2021.

Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems , 36:3597135983, 2022a.

Albert Gu, Karan Goel, and Christopher Re. Efficiently modeling long sequences with structured state spaces. In Proceedings of The Tenth International Conference on Learning Representations , 2022b.

Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as structured state spaces. Advances in Neural Information Processing Systems , 36:22982-22994, 2022.

Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika , 58 (1):83-90, 1971.

Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415 , 2016.

William Hua, Hongyuan Mei, Sarah Zohar, Magali Giral, and Yanxun Xu. Personalized dynamic treatment regimes in continuous time: a Bayesian approach for optimizing clinical decisions with timing. Bayesian Analysis , 17(3):849-878, 2022.

Arieh Iserles. A First Course in the Numerical Analysis of Differential Equations . Cambridge University Press, 2nd edition, 2009.

Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of The Third International Conference on Learning Representations , 2015.

Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 1269-1278, 2019.

Thomas Josef Liniger. Multivariate Hawkes processes . PhD thesis, ETH Zurich, 2009.

Brian McFee, Thierry Bertin-Mahieux, Daniel PW Ellis, and Gert RG Lanckriet. The million song dataset challenge. In Proceedings of the 21st International Conference on World Wide Web , pages 909-916, 2012.

Hongyuan Mei and Jason M Eisner. The neural Hawkes process: A neurally self-modulating multivariate point process. Advances in Neural Information Processing Systems , 31:6757-6767, 2017.

Hongyuan Mei, Guanghui Qin, and Jason Eisner. Imputing missing events in continuous-time event streams. In International Conference on Machine Learning , volume 97, pages 4475-4485. PMLR, 2019.

William Merrill, Jackson Petty, and Ashish Sabharwal. The illusion of state in state-space models. In International Conference on Machine Learning , volume 235, pages 35492-35506. PMLR, 2024.

Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons. Theoretical foundations of deep selective state-space models. Advances in Neural Information Processing Systems , 38:127226-127272, 2024.

Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of The 2019 Conference on Empirical Methods in Natural Language Processing and The 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 188-197, 2019.

Mohammed Saeed, Christine Lieu, Greg Raber, and Roger G Mark. MIMIC II: a massive temporal ICU patient database to support research in intelligent patient monitoring. In Computers in Cardiology , volume 29, pages 641-644, 2002.

Anuj Sharma, Robert Johnson, Florian Engert, and Scott Linderman. Point process latent variable models of larval zebrafish behavior. Advances in Neural Information Processing Systems , 32: 10942 - 10953, 2018.

Oleksandr Shchur, Marin Biloš, and Stephan Günnemann. Intensity-free learning of temporal point processes. In Proceedings of The Eighth International Conference on Learning Representations , 2020a.

Oleksandr Shchur, Nicholas Gao, Marin Biloš, and Stephan Günnemann. Fast and flexible temporal point processes with triangular maps. Advances in Neural Information Processing Systems , 34: 73-84, 2020b.

Jimmy TH Smith, Andrew Warrington, and Scott Linderman. Simplified state space layers for sequence modeling. In Proceedings of The Tenth International Conference on Learning Representations , 2022.

Ali Caner Türkmen, Yuyang Wang, and Tim Januschowski. Intermittent demand forecasting with deep renewal processes. arXiv preprint arXiv:1911.10416 , 2019a.

Ali Caner Türkmen, Yuyang Wang, and Alexander J Smola. Fastpoint: Scalable deep point processes. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD , pages 465-480. Springer, 2019b.

Bjørnar Vassøy, Massimiliano Ruocco, Eliezer de Souza da Silva, and Erlend Aune. Time is of the essence: a joint hierarchical RNN and point process model for time and item predictions. In Proceedings of The Twelfth ACM International Conference on Web Search and Data Mining , pages 591-599, 2019.

Jianlong Wang, Xiaoqi Duan, Peixiao Wang, A-Gen Qiu, and Zeqiang Chen. Predicting urban signalcontrolled intersection congestion events using spatio-temporal neural point process. International Journal of Digital Earth , 17(1):2376270, 2024.

Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, and Raffay Hamid. Selective structured state-spaces for long-form video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6387-6397, 2023.

Chris Whong. FOILing NYC's taxi trip data. https://chriswhong.com/open-data/foil_ nyc_taxi/ , 2014. [Online; accessed Oct 15, 2024].

Alex Williams, Anthony Degleris, Yixin Wang, and Scott Linderman. Point process models for sequence detection in high-dimensional neural spike trains. Advances in Neural Information Processing Systems , 34:14350-14361, 2020.

Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason Fries, and Nigam Shah. EHRSHOT: An EHR benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems , 37:67125-67137, 2023.

Siqiao Xue, Xiaoming Shi, James Zhang, and Hongyuan Mei. HYPRO: A hybridly normalized probabilistic model for long-horizon prediction of event sequences. Advances in Neural Information Processing Systems , 36:34641-34650, 2022.

Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan, James Y Zhang, Qingsong Wen, et al. EasyTPP: Towards open benchmarking temporal point processes. In Proceedings of The Twelfth International Conference on Learning Representations , 2023.

Chenghao Yang, Hongyuan Mei, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of The Tenth International Conference on Learning Representations , 2022.

Guolei Yang, Ying Cai, and Chandan K Reddy. Recurrent spatio-temporal point process for check-in time prediction. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management , pages 2203-2211, 2018.

Guilherme Augusto Zagatti, See Kiong Ng, and Stéphane Bressan. Learning multivariate temporal point processes via the time-change theorem. In International Conference on Artificial Intelligence and Statistics , volume 238, pages 3241-3249. PMLR, 2024.

Mai Zeng, Florence Regol, and Mark Coates. Interacting diffusion processes for event sequence forecasting. In International Conference on Machine Learning , volume 235, pages 58407-58430. PMLR, 2024.

Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive Hawkes process. In International Conference on Machine Learning , volume 119, pages 11183-11193. PMLR, 2020.

Shuai Zhang, Chuan Zhou, Yang Aron Liu, Peng Zhang, Xixun Lin, and Zhi-Ming Ma. Neural jump-diffusion temporal point processes. In International Conference on Machine Learning , volume 235, pages 60541 - 60557. PMLR, 2024.

Qingyuan Zhao, Murat A Erdogdu, Hera Y He, Anand Rajaraman, and Jure Leskovec. Seismic: A self-exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1513-1522, 2015.

Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision Mamba: Efficient visual representation learning with bidirectional state space model. In International Conference on Machine Learning , volume 235, pages 62429 - 62442. PMLR, 2024.

Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process. In International Conference on Machine Learning , volume 119, pages 11692-11702. PMLR, 2020.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: All of the claims are further expanded upon and justified in either the methodology section or the two experiment sections.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations in the last two sections in our concluding thoughts.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: There are no theoretical results in our work.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Extensive reproducibility details are provided in Appendices C and D.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Please refer to the footnote on page 8.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All details can be found in Appendix C.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: All numerical results for experiments are presented with standard deviations describing the spread over five different random seeds.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: As mentioned in the real-world experiments section, all models were trained on a single 24GB NVIDIA A5000 GPU.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We fully comply with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: We do not believe this paper has any societal impacts, positive or negative. Neural MTPPs have existed for almost a decade, and our work is the most recent iteration in this well-established domain.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: All of the models were trained on publicly available datasets without direct risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Credit is given in detail in Appendix C.3.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: Data preprocessing scripts and instructions for acquisition for the new dataset (EHRSHOT) we propose are included in Appendix C.3 and our forked repository (see the footnote on page 8). Direct access to the data is not provided due to separate usage agreements and required training from the original source.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Crowdsourcing nor human subjects were used in our work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Human subjects were not used in our work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: Our method nor paper used LLMs.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Supplementary Materials: Deep Continuous-Time State-Space Models for Marked Event Sequences

Table of Contents

A Acronyms and Notation

B Additional Details on Methods

B.1 State-Space Point Process Algorithms

Algorithm 1 State-Space Point Process: Get Right State Limits

Algorithm 2 State-Space Point Process: Get Intensity From Right Limit

Input: S2P2 layer parameters Θ = { Λ ( l ) , ˜ B ( l ) , ˜ C ( l ) , D ( l ) , ˜ E ( l ) , ˜ x ( l ) 0 } L l =1 , Previous state right limits x (1: L ) t , Integration period δt , nonlinearity σ , Intensity function IntensityFn .

Output: Intensity left limit λ t + δt

1:

-

=

u

0

t

δt

+

4:

= ¯

+( ¯

˜

x

Λ

x

5:

6:

7:

8:

l

(

)

l

l

(

)

(

)

t

+

δt

u

t

+

end for

λ

t

+

δt return

= IntensityFn(

λ

t

+

-

δt

-

t

= LayerNorm

u

t

+

δt

Algorithm 3 State-Space Point Process: Compute Log-Likelihood

Input: S2P2 layer parameters Θ = { Λ ( l ) , ˜ B ( l ) , ˜ C ( l ) , D ( l ) , ˜ E ( l ) , ˜ x ( l ) 0 } L l =1 , Event times t 1: N , mark types k 1: N , nonlinearity σ , shared mark embedding function EmbedMarks , number of integration points per event M , Intensity function IntensityFn .

Output: Log-likelihood L

13:

return

L

Λ

(

(

l

)

σ

(

δt

-

-

I

C

)

(

l

)

B

(

x

˜

l

(

l

t

)

u

)

+

δt

˜

t

+

-

δt

+

-

D

(

l

)

u

t

+

δt

-

)

+

u

t

+

δt

-

)

▷ Left input limit

▷

Zero-order hold, see Eq. (19)

▷

Evolve state

▷

Compute event left

u

limits

▷ Rectify intensity, see Section 3.4

B.2 Discretization and Zero Order Hold

The linear recurrence is defined in continuous-time. This mirrors the (M)TPP setting, where event times are not on a fixed intervals. We use the zero-order hold (ZOH) discretization method, to convert the continuous-time linear recurrence into a sequence of closed-form updates, given the integration times, that can also be efficiently computed. We refer the reader to Iserles [2009] for a comprehensive introduction to the ZOH transform.

The main assumption of the ZOH discretization is that the input signal is held constant over the time period being integrated. Under this assumption, it is possible to solve for the dynamics and input matrices that yield the correct state at the end of the integration period. For the LLH dynamics in Eq. (8), when no events occur in ( t, t ′ ) , this becomes

where the resulting discretized matrices are

Λ B respectively. In this case, Eq. (16) reduces to

The ZOH does not affect the output or passthrough matrices C and D . To compute the matrices A and AB however requires computing a matrix exponential and a matrix inverse. Smith et al. [2022] avoid this by diagonalizing the system (also avoiding a dense matrix-matrix multiplication in the parallel scan). The diagonalized dynamics and input matrices are denoted Λ (a diagonal matrix) and ˜

where e Λ ∆ t is trivially computable as the exponential of the leading diagonal of Λ ∆ t . These operations are embarrassingly parallelizable across the sequence length and state dimension given the desired evaluation times.

To contextualize, suppose an event occurs at time t , Eq. (19) allows us to exactly (under the constantinput assumption) efficiently evaluate the linear recurrence at subsequent times t ′ . We use this extensively in S2P2 to efficiently evaluate the recurrence (and hence the intensity) at the irregularlyspaced event times and times used to compute the integral term.

It should be noted the discretization was done to compute a left-limit x t ′ -from a previous right-limit x t . Should an event not occur at t ′ , then the left- and right-limits agree and x t ′ -= x t ′ + = x t ′ . If an event does occur at time t ′ with mark k , then the left-limit x t ′ -can be incremented by ˜ E α k to compute x t ′ + = x t ′ . This increment is exact and leverages no discretization assumption.

B.3 Interpretation for Input-Dependent Dynamics

Consider the input-dependent recurrence for an LLH layer, as defined in Eq. (13):

for t ∈ ( t i , t i +1 ] where Λ i := diag (∆ i ) Λ with the input-dependent factor defined as ∆ i := softplus ( W ′ u t i + b ′ ) ∈ R P > 0 . This factor can be thought of as the input-dependent relative-time scale for the dynamics. To see this, we first note that for vectors p , q ∈ R d , the following holds true: diag ( p ) q = p ⊙ q = q ⊙ p where ⊙ is the Hadamard or element-wise product. It then follows that

As shown, the positive vector ∆ i can be thought of as changing the relative time-scale for each channel in the hidden state ˜ x . Large values of ∆ i will act as if time is passing quickly, encouraging the state to converge to the steady-state sooner. Conversely, smaller values will make time pass more slowly causing the model to retain the influence that prior events have on future ones (for that specific channel in ˜ x at least).

B.4 Forwards and Backwards Zero Order Hold Discretization

In Section 3.2 we highlighted that the ZOH discretization is exact when u t is held constant over the integration window. This raises a unique design question for S2P2: what constant value should u t take on when evolving x from time t to t ′ ? For the first layer of the model, the input is zero by construction, so there is no choice to be made-in fact, since u is constant for the first layer the updates are exact. However, the input is non-zero at deeper layers, and, crucially, varies over the integration period.

We must therefore decide how to select a u value over the integration period. This should be a value in (or function of) { u s | s ∈ [ t, t ′ ) } . Note this is because the value at t ′ , u t ′ , cannot be incorporated as this would cause a data leakage in our model; while values prior to t would discard the most recent event occurrence. For this work, we explore two natural choices: (i) the input value at the beginning of the interval, u t , and (ii) the left-limit at the end of the interval, u t ′ -. Note the end of the interval need not align with an event (crucial for when computing intermediate intensity values). We refer to these options as forwards and backwards ZOH, respectively. We illustrate the backwards variant in Fig. 2, where in the leftmost panel, we use the u t -values at each layer to calculate x t -and x t , as opposed to u t i for t i < t < t i +1 . All experiments in the main paper utilize backwards ZOH.

It is not obvious a priori which one of these modes is more performant. We therefore conducted an ablation experiment in Table 14. We see that there is little difference between the two methods. We also note that models are learned through this discretization, and so this decision does not mean that a model is 'incorrectly discretized' one way or the other, but instead they define subtlety different families of models. Theoretical and empirical investigation of the interpretations of this choice is an interesting area of investigation going forwards, extending the ablations we present in Table 14.

B.5 Theoretical Complexity

We include in Table 5 a brief summary of the theoretical complexity of each of the methods we consider, broadly grouped by their architectures. We analyze complexity by the work, memory complexity and theoretical best parallel application time of the forward pass (used when conditioning on a sequence, the left-hand term of Eq. (2)) and evaluating the integral term in Eq. (2) given that the forward pass has been completed (as this is either required by the method, and is nearly always evaluated in conjunction with the forward pass). We then state the limiting best-case theoretical parallelism of the two components.

The reasoning behind the calculated values are as follows:

The forward pass of RMTPP, NHP and IFTPP use non-linear RNNs, and hence incur memory and work that is linear in the sequence length, and cannot be parallelized. MHP uses an RNN, but that is logarithmically parallelizable. These models re-use the computed hidden states to compute the integral term, and hence, while they incur work and memory that scales in the sequence length and number of events, this work can be perfectly parallelized. This results in a best-case parallelism of O ( N ) (dominated by the forward pass; O (log N ) for MHP).

SAHP, THP and AttNHP all use self-attention, and hence have a work and memory that scales quadratically in the sequence length, although this work can be parallelized across the sequence length, resulting in logarithmic parallel depth. SAHP and THP re-use embeddings and a parametric decoder, and hence estimating the integral scales like the RNN, and hence the limiting parallelism is still the forward pass. AttNHP is slightly different in that it reapplies the whole independently attention mechanism for each integration point. However, this work is parallelizeable and hence still reduces to a best-case depth of O (log N ) .

S2P2 is an RNN and hence has linear work and memory in the forward pass, but can be parallelized to a best-case depth of O (log N ) using the parallel scan. We then re-use the states computed in the forward pass for estimating the integral, which, as with the other RNN methods, is perfectly parallelizable, resulting in a theoretical parallel depth of O (log N ) .

Note that these figures do not account for the number of layers required by each model, which must be evaluated in sequence.

O

O

O

O

O

O

O

To validate the scaling properties, we measure the wallclock time for a full forward pass and loglikelihood evaluation on random input sequences with lengths ranging from eight events to over half a million events. The architectures and mark spaces are the same as in the StackOverflow experiments (see Tables 6a and 7).

We note: our EasyTPP PyTorch S2P2 is written in pure PyTorch, and hence is not as optimized as other methods (compared to, for instance, IFTPP, which uses a GPU-optimized implementation of the GRU). We therefore include the runtimes of a standalone JAX S2P2 implementation, which allows for comparable levels of optimization through JIT compilation.

We observe the predicted scaling in practice. NHP scales linearly across all sequence lengths, and is far outpaced by all other methods. The THP scales well before reverting to superlinear scaling, and then runs out of memory. The IFTPP is very fast at shorter runtimes, but quickly reverts to linear scaling, due to its simple but highly optimized implementation and inherently sequential operation. Both S2P2 implementations scale linearly at long sequence lengths, but have near-constant runtime at shorter sequences. At shorter sequence lengths, the more optimized JAX implementation is faster than the unoptimized pure PyTorch implementation. While this indicates that there are additional opportunities to accelerate the PyTorch implementation further (e.g., exploiting kernel fusion or writing a lower-level Triton implementation), these results still confirm that our S2P2 can exploit parallel scans to scale to long sequences more effectively than other methods while retaining strong performance.

10

1

10

2

10

3

10

4

10

5

Sequence Length

10

-

4

10

-

3

10

-

2

10

-

1

10

0

Conditioning

wallclock (s)

10

1

10

2

10

3

10

4

10

5

Sequence Length

10

-

4

10

-

3

10

-

2

10

-

1

10

0

Log-Likelihood

wallclock (s)

S2P2 (Jax)

S2P2 (Torch)

NHP (Torch)

THP (Torch)

IFTPP (Torch)

C Experimental Configurations and Datasets

C.1 Training Details & Hyperparameter Configurations

All baseline models used up-to-date PyTorch implementations, provided by the EasyTPP library [Xue et al., 2023] as of May 2025.

We apply a grid search for all models on all datasets for hyperparameter tuning. We use a default batch size of 256 for training. For models/datasets that require more memory (e.g., large mark space or long sequences), we reduce the batch size and keep them as consistent as possible among all the models on each dataset. We use the Adam stochastic gradient optimizer [Kingma and Ba, 2015], with a learning rate of 0.01 and a linear warm-up schedule over the first 1% iterations, followed by a cosine decay. Initial experiments showed this setting generally worked well across different models and datasets leads to convergence within 300 epochs. We also clip the gradient norm to have a max norm of 1 for training stability. We use Monte-Carlo samples to estimate the integral in log-likelihood, where we use 10 Monte-Carlo points per event during training.

On the five EasyTPP benchmark datasets and MIMIC-II that are smaller in their scales, we choose an extended grid based on the architecture reported in the EasyTPP paper. Specifically, we search over hidden states size h = {16, 32, 64, 128, 256} for RMTPP, h = {32, 64, 128} for NHP, and h = {16, 32, 64} for IFTPP. For SAHP, THP, and AttNHP, we searched over all combinations of number of L = {1, 2, 3}, hidden state size = {16, 32, 64, 128}, and number of heads = {1, 2, 4}. For MHP, we followed their paper to fix L = 4 , then we search over h = {4, 8, 16, 32, 64, 128, 256, 512}. Finally, for S2P2, we considered combinations for number of layers = {1, 2, 3, 4}, p = {16, 32, 64, 128} and h = {16, 32, 64, 256}. We generally found a range of reasonable hyperparameters to yield similar performance on S2P2, while multiple layers were key for performance (intuitively deep stack allows nonlinear and complex dynamics), but there was not a critical dependence on depth.

We fixed the activation function as GeLU [Hendrycks and Gimpel, 2016] and apply post norm with layer norm [Ba, 2016]. We fix the dropout as 0.1 for S2P2 on the five core benchmark datasets, and add dropout = {0, 0.1} to the grid search for the other three datasets. Due to the scale of Last.fm and EHRSHOT datasets, we perform a smaller search over architectures that roughly match the parameter counts for all models at three levels: 25k, 50k, 200k, and choose the model with the best validation results. AttNHP has expensive memory requirements that tends to have smaller batch sizes than other models. We were unable to train any AttNHP on EHRSHOT. The final model architectures used are reported in Tables 6a and 6b.

(a) Model architectures for the five EasyTPP benchmark datasets in Table 2.

C.2 Dataset Statistics

We report the statistics of all eight datasets we used in Table 7. We used the HuggingFace version of the five EasyTPP datasets. For all datasets, we further ensure the MTPP modeling assumptions are satisfied that no more than two events occur at the same time (i.e., inter-arrival time is strictly positive), and event times do not lie on grid points that are effectively discrete-time events. Dataset descriptions and pre-processing details are provided in Appendix C.3.

C.3 Dataset Pre-processing

We use the default train/validation/test splits for EasyTPP benchmark datasets. For MIMIC-II, we copy Du et al. [2016] and keep the 325 test sequences in the test split, and further split the 2,935 training sequences into 2,600 for training and 325 for validation. In our pre-processed datasets, Last.fm and EHRSHOT, we randomly partition into subsets containing 70%, 15%, 15% of all sequences for training/validation/test respectively. We provide a high-level description of all the datasets we used, followed by our pre-processing procedure of Last.fm and EHRSHOT in more detail. Note that for datasets that contain concurrent events or effectively discrete times (e.g., StackOverflow, Retweet), we apply a small amount of jittering to ensure no modeling assumptions are violated in the MTPP framework.

Amazon [Ni et al., 2019] contains user product reviews where product categories are considered as marks. Retweet [Zhao et al., 2015] predicts the popularity of a retweet cascade, where the event type is decided by if the retweet comes from users with 'small', 'medium', or 'large' influences, measured by number of followers [Mei and Eisner, 2017]. Taxi data [Whong, 2014, Mei et al., 2019] uses data from the pickups and dropoffs of New York taxi and the marks are defined as the Cartesian product of five discrete locations and two actions (pickup/dropoff). Taobao [Xue et al., 2022] describes the viewing patterns of users on an e-commerce site, where item categories are considered as marks. StackOverflow contains the badges (defined as marks) awarded to users on a question-answering website. Finally, MIMIC-II [Saeed et al., 2002] records different diseases (used as marks) during hospital visits of patients. We add a small amount of noise to the MIMIC-II event times so that events do not lie on a fixed grid. Both StackOverflow and MIMIC-II datasets were first pre-processed by Du et al. [2016].

Last.fm [Celma, 2010, McFee et al., 2012] records 992 users' music listening habits that has been widely used in MTPP literature [Kumar et al., 2019, Boyd et al., 2020, Bosser and Taieb, 2023]. Mark types are defined as the genres of a song, and each event is a play of a particular genre. Each sequence represents the monthly listening behavior of each user, with sequence lengths between 5 and 500. If the song is associated with multiple genres we select a random one of the genres, resulting in a total of 120 different marks.

EHRSHOT [Wornow et al., 2023] is a newly proposed large dataset of longitudinal de-identified patient medical records, and has rich information such as hospital visits, procedures, and measurements. We introduce an MTPP dataset derived from EHRSHOT, where medical services and procedures are treated as marks, as identified by Current Procedural Terminology (CPT-4) codes. Each patient defines an event sequence, and we retain only CPT-4 codes with at least 100 occurrences in the dataset. For the < 1% events of events where there are more than 10 codes at a single timestamp, we retain the top 10 codes with the most frequencies and discard the rest. We then add a small amount of random noise to the event time to ensure they are not overlapping. This process ensures we still satisfy the MTPP framework, and can reasonably instead compute top-10 accuracy for the next mark prediction; other work has considered extending the MTPP framework to consider simultaneous event

occurrence [Chang et al., 2024]. Then we standardize each sequence to start at t = 0 and pad the start and end of sequences with a specific padded event token. Note that we do not score these events. Event times are normalized to be in hours. We discard sequences that have less than 5 events or a single timestamp. This leads to the final version of our dataset having 668 marks, and the sequence lengths range from 5 to 3955 events, reflecting patient histories that can span multiple years.

We include code in our forked repository for preparing the EHRSHOT event sequence dataset from the raw EHRSHOT dataset. Note that we cannot distribute the raw data (or derivative dataset) under the terms of the original EHRSHOT dataset requiring credentialed access through PhysioNet.

D Additional Experimental Results

D.1 Full Results on Benchmark Datasets

We provide the full log-likelihood results and corresponding plots in Table 8 and Fig. 6 respectively, where we decompose the likelihood into time and mark likelihoods. The improvement of our S2P2 model is mainly driven by better modeling of time, though we also often obtain best- or second-best predictive performance on marks from the next event prediction accuracy results conditioned on true event time in Table 2c. In aggregate, our model achieves a 1.33 per-event likelihood ratio between itself and the next best method across all datasets (a 33% improvement in likelihood). This is calculated by computing the mean log-likelihood ratio across all datasets and then exponentiating. Doing so is equivalent to taking the geometric mean across likelihood ratios.

Configuration and training details of all models can be found in Appendix C.1. As discussed in Section 7 and grouped in the results, models with continuous-time hidden states can present a richer class of intensities and often empirically outperform those with discrete hidden states. Note for the RMSE results in Table 2b, we follow Mei and Eisner [2017] and use the expected next event time as next event time predictions to minimize the Bayes risk. Unlike them, however, we estimate these with the trapezoidal rule rather than Monte-Carlo simulation via the thinning algorithm. In practice, we have found this to produce an estimator with much lower variance and be faster due to being more readily parallelizable. This stands in contrast to the thinning algorithm which has more hyperparameters (e.g., dominating rate, sampling boundary) that can exacerbate bias.

Table 8: Complete per event log-likelihood results on the held-out test for the eight benchmark datasets we consider, averaged over 5 random seeds. In Table 8a we show the full log-likelihood. We then decompose this log-likelihood into the log-likelihood of the event time in Table 8b, and the time-conditional log-likelihood of the mark type in Table 8c. OOM indicates out of memory; standard deviation in parentheses. We box the best-performing model and underline the second-best. We also report the average rank of models across datasets as a summary metric (lower is better). S2P2 is consistently the best or second best-performing model across all datasets.

D.2 Full Results for Synthetic Poisson Experiments

We present the full results in Fig. 7 for all models regarding the synthetic Poisson experiments discussed in Section 5. All models are trained until convergence using a set of 5,000 generated sequences, where we use 20 Monte Carlo points per event to estimate the integral of log-likelihood during training to accommodate the sparsity of events. We used small models so they do not overfit; model architecture and parameter counts are reported in Table 9. We plot the estimated intensity conditioned on empty sequences using 1,000 equidistant grid points between the start and end points. Our model is the only one that perfectly recovers the underlying ground truth intensity, while also using the fewest parameters.

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

RMTPP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

SAHP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

THP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

2

4

6

Intensity

IFTPP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

MHP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

NHP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

AttNHP Estimated Intensity

0

1

2

3

4

5

6

7

Time

0

.

0

0

.

5

1

.

0

Intensity

S2P2 (ours) Estimated Intensity

D.3 Additional Synthetic Results on Multivariate Hawkes Processes

We evaluate our model and baseline models against the true model on a randomly initiated parametric Hawkes process with three possible marks. Following the notation in Section 2.1, we draw all parameters from the following distributions: ν i iid ∼ Unif [0 . 1 , 0 . 5] , α ij iid ∼ Unif [0 . 5 , 0 . 8] , and β ij iid ∼ Unif [0 . 4 , 1 . 2] for i, j ∈ { 1 , 2 , 3 } .

All models are trained until convergence using a set of 50,000 generated sequences, where we use 20 Monte Carlo points per event to estimate the integral of log-likelihood during training. Model architecture and parameter counts are reported in Table 10. We plot three example sequences drawn for an additional test set for each model in Fig. 8, using 1,000 equidistant grid points for any interevent interval. Dotted lines refer to the intensities under the true underlying parametric model; solid lines are different model estimates from trained models.

As we see in inhomogeneous Poisson processes, our model can recover the ground truth intensities with the fewest parameters. Visually, all SAHP, IFTPP, NHP, AttNHP and S2P2 (our model) perform well at recovering the ground truth intensities. It is also worth noting that our model is 7-9 × quicker than NHP and AttNHP regarding wallclock runtime on a single A5000 GPU. Our results on synthetic experiments validate the model's ability to recover the ground truth intensities. We further evaluate all models quantitatively using 1,000 test sequences generated from the same multivariate Hawkes process and evaluated both log-likelihood and RMSE for the immediate next event. We see our method competitive again on both metrics.

0

2

4

6

8

10

Time

0

1

2

3

Intensity

RMTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

RMTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

RMTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

SAHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

SAHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

SAHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

THP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

THP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

THP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

IFTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

IFTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

IFTPP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

MHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

MHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

MHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

NHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

NHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

NHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

AttNHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

AttNHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

AttNHP Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

S2P2 (ours) Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

S2P2 (ours) Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

0

2

4

6

8

10

Time

0

1

2

3

Intensity

S2P2 (ours) Estimated Intensity

True

k

=1

True

k

=2

True

k

=3

Est.

k

=1

Est.

k

=2

Est.

k

=3

D.4 Full Results for Hawkes and Self-Correcting Process

We train our approach on synthetic data generated from known, classical temporal point processes, namely the self-correcting and Hawkes processes. These are characterized by intensity functions λ SC t = exp( t -0 . 5 N t ) and λ H t = 0 . 5 + ∑ N t i =1 0 . 5 exp( t i -t ) , respectively. Models were fit using data drawn from these processes, 6,000 sequences for training and 2,000 for validation. The learned intensity functions, evaluated on a held-out test sequence, can be seen for the self-correcting process data in Fig. 9 and for the Hawkes process data in Fig. 10 for all methods. The hyperparameters for the models were chosen by a grid search (see Table 12). We see many of the models, including ours, do well at capturing the ground truth intensity.

0

1

RMTPP

t

Self-Correcting Process

Ground Truth

Predicted

t

0

1

SAHP

t

0

1

THP

t

0

1

IFTPP

t

0

1

MHP

t

0

1

NHP

t

0

1

AttNHP

t

0

1

S2P2

t

0

10

20

30

40

50

Events

0

2

RMTPP

t

Hawkes Process

Ground Truth

Predicted

t

0

2

SAHP

t

0

2

THP

t

0

2

IFTPP

t

0

2

MHP

t

0

2

NHP

t

0

2

AttNHP

t

0

2

S2P2

t

0

10

20

30

40

50

Events

D.5 Full Results for Long-Range Dependency Experiment

To measure the ability to capture long-range dependencies by neural MTPPs, we constructed a generative process with long-range dependencies. For this, we generate sequences over the time window of [0 , 100] , with three possible marks. The first mark is a 'distractor' mark, meaning it has no influence over other events. These events are drawn from a homogeneous Poisson process with rate 1. The second mark is a 'trigger' mark, which are directly tied to the third 'target' mark. Triggers are also drawn from a homogeneous Poisson process with rate 0.1. For every trigger event ( t i , k i = 2) drawn, a corresponding target event ( t j , k j = 3) is generated conditionally independent of all other events according to t j | t i ∼ N ( t i +40 , 0 . 1) .

All models were trained with the same hyperparameters as in Table 12. The predicted intensity functions for a single sequence can be seen in Fig. 11, and the likelihood ratio between the trained models and the ground truth process on held-out test sequences can be found in Table 13. We can see that both S2P2 and AttNHP do very well, both qualitatively and quantitatively. This is expected as both architectures are well suited for long-range dependencies while still being continuous-time models, allowing for expressive intensity functions.

0

1

10

RMTPP

t

Long-Range Modeling

Ground Truth

Predicted

t

(Zoom-In)

0

1

10

SAHP

t

0

1

10

THP

t

0

1

10

IFTPP

t

0

1

10

MHP

t

0

1

10

NHP

t

0

1

10

AttNHP

t

0

1

10

S2P2

t

0

20

40

60

80

100

Events

45

50

55

60

D.6 Ablation for Different S2P2 Variants

We perform an ablation study of different model variants that we proposed on all datasets and summarize the results in Table 14. We train EHRSHOT using 10% of its training data because larger dataset scale requires more training time (but use the original validation and test sets for model selection and reporting results). Forward and backward discretization are very close in performance, with backwards discretization having a slight edge. Models that are input-dependent achieve better performance on most datasets, although on certain datasets input dependence appears to harm performance. It is an interesting direction for future work to explore theoretically and empirically when each of these variants is best. We select backward discretization with input dependence for the results in the main paper.

D.7 Model Calibration

To further probe the models, we evaluate the calibration of MTPPs, as proposed by Bosser and Taieb [2023]. Calibration has a different focus than log-likelihood-based or accuracy-based evaluation. Calibration instead describes how well the uncertainty in the model is reflective of the observed data. It is important to note, however, a model can achieve perfect calibration simply by predicting the marginal distribution. Better calibration therefore does not necessarily indicate better predictive performance - only better calibrated errors - and so should be taken in context with the performance under other metrics. We provide summarized statistics for both probabilistic calibration error (PCE) for time calibration and expected calibration error (ECE) for mark calibration in Table 15, and visualize the calibration curves in Figs. 12 and 13.

We see that, on the whole, MTPP models produce fairly well-calibrated predictions. IFTPP is the best calibrated of the models, this may be as a result of having parametric distributions for inter-arrival time (although IFTPP does fail on some datasets such as Retweet). The S2P2 is particularly well calibrated in time (PCE) among intensity-based methods, suggesting again that our S2P2 is capturing time dependencies better than other models. It is also the second-best calibrated on mark prediction (ECE) on average.

(a) Probabilistic calibration error (PCE) for time calibration in percentage.

(b) Expected calibration error (ECE) for mark calibration in percentage.

Empirical CDF

Predicted CDF

0

0.5

1

Amazon

RMTPP

SAHP

THP

IFTPP

MHP

NHP

AttNHP

S2P2

Retweet

Taxi

Taobao

0

0.5

1

0

0.5

1

StackOverflow

0

0.5

1

Last.fm

0

0.5

1

MIMIC-II

0

0.5

1

EHRSHOT

Confidence

0

1

RMTPP

Amazon

Retweet

Taxi

Taobao

StackOverflow

Last.fm

MIMIC-II

EHRSHOT

0

1

SAHP

0

1

THP

0

1

IFTPP

0

1

MHP

0

1

NHP

0

1

AttNHP

OOM

0

0.5

1

0

1

S2P2

0

0.5

1

0

0.5

1

0

0.5

1

0

0.5

1

0

0.5

1

0

0.5

1

0

0.5

1

Finally, in Figs. 14 and 15 we plot the log-likelihood of time and mark respectively, versus their corresponding calibration results, to provide an overall view of the performances of different models. Our S2P2 model consistently achieves higher log-likelihood while maintaining good calibration on both time and mark components on most datasets.

Time Log-Likelihood

PCE

0

0.1

0

1

2

Amazon

RMTPP

SAHP

THP

IFTPP

MHP

NHP

AttNHP

S2P2

0

0.25

-

8

-

6

Retweet

0

0.025

0

.

65

0

.

70

Taxi

0

0.05

2

.

25

2

.

50

2

.

75

Taobao

0

0.02

-

0

.

8

-

0

.

7

StackOverflow

0

0.025

0

.

5

1

.

0

Last.fm

0

0.1

0

.

0

0

.

5

1

.

0

MIMIC-II

0

0.2

-

2

0

EHRSHOT

ECE

0

0.05

-

2

.

2

-

2

.

0

Amazon

RMTPP

SAHP

THP

IFTPP

MHP

NHP

AttNHP

S2P2

0

0.05

-

0

.

9

-

0

.

8

Retweet

0

0.05

-

0

.

35

-

0

.

30

-

0

.

25

Taxi

0

0.05

-

1

.

450

-

1

.

425

-

1

.

400

Taobao

0

0.02

-

1

.

65

-

1

.

60

-

1

.

55

StackOverflow

0

0.05

-

2

.

0

-

1

.

8

Last.fm

0

0.05

-

0

.

4

-

0

.

2

MIMIC-II

0

0.1

-

6

-

4

EHRSHOT

Peer reviews:
comments: SUMMARY:
This paper proposes S2P2, a deep learning architecture for modeling Marked Temporal Point Processes (MTPPs) by blending ideas from Hawkes processes and state space models (SSMs). The authors introduce a new recurrent building block, the Latent Linear Hawkes (LLH) layer, which generalizes the dynamics of linear Hawkes processes using the expressive, continuous-time latent recurrences of SSMs.
S2P2 is tested on snythetic data as well as eight real-world datasets.

STRENGTHS AND WEAKNESSES:
I have to say, that I am not very familiar with models for temporal point processes.
Still I found the following strengths in the paper:

Rather than just using a black box model, their latent linear Hawkes layer has an inductive bias towards event sequences which extends the classical linear hawkes process.

The figures, especially figure 2, are highly informative and help the reader to understand the main ideas of the paper.

The experimental section includes synthetic data as well as real-world datasets and highlights the efficacy of the proposed method.

QUESTIONS:
In practice, how does S2P2 handle sequences with >10,000 events? Are there any memory or stability concerns in such regimes, especially when evaluating intermediate intensities at random times?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-17

comments: SUMMARY:
The paper adds to a line of work on adding non-linear activation to the classical Hawkes process framework, in order to capture more complex marked point processes. The authors describe the model carefully, and explain the advantages in terms of expressivity and computational efficiency w.r.t the state of the art. Extensive experiments with synthetic and real-world data establish the power of the model.

STRENGTHS AND WEAKNESSES:
The paper is very carefully written in terms of defining and motivating the model, and was a pleasure to read - although I wonder whether they actually pack too much into this paper (thanks to unlimited supplementary). 

The model is intuitive and well explained. The relationship with the state of the art is less so: it is difficult to appreciate the difference in assumptions/model expressivity with NHP, AttNHP, IFTPP in particular, and why S2P2 is more expressive. Again, I think the authors are forced to be too cursory given the desire to pack everything into this paper. From what I can judge, the model makes sense and has well-reasoned advantages, essentially combining well-known features of state-space models with specific computational tricks to make it work for (marked) point processes.

QUESTIONS:
In the experiments, I did not understand how hyperparameters (# layers, layer width in particular) were chosen. How sensitive are results to these?

Details:
- what is ZOH? please define

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

comments: SUMMARY:
the paper introduces S2P2 a MTPP model based on a neat connection between state-space models and linear Hawkes' processes. The key contribution of this paper is to utilize this connection to embed mark-specific dynamics in a marked temporal sequence to a, potentially more expressive, latent space. The paper also offers computationally tractable and efficient training of the resulting in a practical and quite novel approach to sequence modeling.

STRENGTHS AND WEAKNESSES:
__Strengths:__

1. The paper makes a novel and interesting connection between linear Hawkes processes and deep state-space models. 

2. proposes efficient update handling through diagonalization which is technically quite interesting. 

3. experimental results over synthetic and real-world data with small and large mark space clearly demonstrates the superiority of s2p2. 


__Weaknesses:__

1. The paper is fairly dense technically - particularly the combination of diagonalization and discretization stages. 

2. latent mark space is quite non-interpretable in comparison to seen marks.

QUESTIONS:
Q1. Can the approach be extended to non-categorical marks? It would be interesting since some of the applications of Hawkes' processes have worked with spatio-temporal setting with non-categorical marks. 

Q2. how sensitive is the model for potentially noisy/adversarial observations -- either in observed time or intensity. In many sensor settings this is a common occurance, and there have been some works that have improved the robustness of Hawkes' processes (e.g., Chakraborty et al., AAAI'25)

LIMITATIONS:
There is no discussion, but I do not see that as a limitation.
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel state-space model (SSM)-inspired framework, S2P2, for modeling marked temporal point processes (MTTPs). At the core of the framework is a newly proposed Latent Linear Hawkes (LLH) layer, which naturally integrates the linear Hawkes process with SSM principles. The authors further demonstrate how LLH can be effectively combined with techniques developed for modern deep SSMs (e.g., S4, S5, Mamba), enabling scalable, efficient, and tractable model training. In synthetic experiments, S2P2 successfully recovers the underlying signals where existing methods fall short. On real-world datasets, S2P2 achieves overall SOTA performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- I really enjoyed reading this paper. It is well-organized, well-written, and easy to follow.\n- The design of the LLH layer is clearly articulated, with a detailed explanation of how it combines LHP with SSM. The authors provide well-supported insights from both theoretical and engineering perspectives.\n- The authors provide a complet set of experiments for both synthetic data and real-world datasets and suffcient baseline comparisons.\n\nWeaknesses:\n- For transformer-based HP models, analyzing their attention maps provides critical insights into how historical events influence the future. SSMs in general have weaker interpretability. However, given that S2P2 captures input-dependent dynamics, can these be visualized similarly to attention maps? Maybe there exist ways to calculate some implicit attention scores in a SSM (e.g. [1]) (You don't have to address this in for the rebuttal).\n\n[1] Ali A, Zimerman I, Wolf L. The hidden attention of mamba models. arXiv preprint arXiv:2403.01590. 2024 Mar 3.\n\nQUESTIONS:\nQ1: For the \"parametric decoding head\" in THP and MHP, are you referring to following operation?\n$$\\lambda_k(t|\\mathcal{H}_t) = \\text{softplus}_k\\left(\\alpha_k \\frac{t-t_j}{t_j} + \\mathbf{w}_k^T \\mathbf{h}(t_j) + b_k\\right)$$\nWhat is the fundamental difference between this operation and the final 'Projection & Softplus' layer?\n\nQ2: Could S2P2 serve as a pretrained model, where the output of its final hidden layer (i.e., the layer preceding the 'Projection & Softplus' module) is used for downstream tasks such as clinical classification in EHRSHOT? Given its expressiveness, S2P2's representations are expected to outperform the count-based baselines on the EHRSHOT leaderboard.\n\nQ3: I am surprised that NHP seems to be significantly better than THP, as NHP was the baseline in THP. Did you make any modification to the original NHP?\n\nLIMITATIONS:\n- Althrough you do not have to address the interpretability issue in this paper, it is worthwhile explore some frameworks that make SSMs more explanable in the discussion.\n- Minor Fixes:\n\t- Eqn (3) no subscript in $\\boldsymbol\\alpha$ on the left hand side\n\t- line 211, should use `\\mathcal`", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-20"}