TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
Authors: Shukai Gong
YIYANG FU
Fengyuan Ran
Quyu Kong
Feng Zhou
Venue: NeurIPS 2025 poster ()
Keywords: Transformer point process
sampling
speculative decoding
Primary area: general_machine_learning
TL;DR: TPP-SD is a fast and distributionally consistent sampling framework for Transformer-based temporal point processes, achieving 2–6× speedup over autoregressive methods by adapting speculative decoding with a lightweight draft model.

Abstract: We propose TPP-SD, a novel approach that accelerates Transformer temporal point process (TPP) sampling by adapting speculative decoding (SD) techniques from language models. By identifying the structural similarities between thinning algorithms for TPPs and speculative decoding for language models, we develop an efficient sampling framework that leverages a smaller draft model to generate multiple candidate events, which are then verified by the larger target model. TPP-SD maintains the same output distribution as autoregressive sampling while achieving significant acceleration. Experiments on both synthetic and real datasets demonstrate that our approach produces samples from identical distributions as standard methods, but with 2-6$\times$ speedup. Our ablation studies analyze the impact of hyperparameters such as draft length and draft model size on sampling efficiency. TPP-SD bridges the gap between powerful Transformer TPP models and the practical need for rapid sequence generation.

TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
Shukai Gongl"
Yiyang Fu
Fengyuan Ran
Quyu Kong*
Feng Zhou't
CCenter for Applied Statistics and School of Statistics, Renmin University of China
Schooo of Information, Renmin University of China
3 Schooo of Cyber Science and Engineering, Wuhan University
Alibaba Group
1,2 (shukaii_goon
fuyiyyng2022201555,, feng whoucuuuu...... cn Pryy Ref edu. en
com
Abstract
We propose TPP-SD, novel approach that accelerates Transformer temporal point process (TPP) sampling by adapting speculative decoding (SD) techniques from language models. By identifying the structural similarities between thinning algorithms for TPPs and speculative decoding for language models, we develop efficient sampling framework that leverages smaller draft model generate multiple candidate events, which are then verified by the larger target model parallel. TPP-SD maintains the same output distribution as autoregressive sampling while achieving significant acceleration. Experiments on both synthetic and real datasets demonstrate that our approach produces samples from identical distributions as standard methods, but with 2-6x speedup. Our ablation studies analyze the impact of hyperparameters such as draft length and draft model size sampling efficiency. TPP-SD bridges the gap between powerful Transformer TPP models and the practical need for rapid sequence sampling. Code publicly available at https::t/// // thub. com/ /DNGSSHKAA/ttppad
Introduction
Temporal point processes (TPPs) are essential stochastic models for describing discrete event occurrence patterns in continuous time. Classical models include the Poisson process [12), which is
history-independent, and the Hawkes process [10 (9]3))] which incorporates historical dependencies. Recently, advancements in deep learning have facilitated the integration of deep models with point processes, significantly enhancing their expressive capabilities in capturing history dependence [19]
Notable examples include models based traditional RNNs 77), LSTMs [17), and Transformers (40
36 20 performance. In contrast, Transformer TPPs have gained popularity for their ability capture long-term dependencies and support parallel training.
RNN-type methods often suffer from vanishing exploding gradients (26), limiting their
Most current Transformer TPPs' research focuses enhancing model expressiveness and training methodologies, with limited attention to improving sampling efficiency. Sampling from a learned temporal point process is essential for synthesizing event sequence data, making predictions based on observed history, validating model goodness-of-fit, and gaining insights into complex process dynamics. The standard sampling procedure employs the thinning algorithm [15 23), which simulates
"Equal contribution. *Coresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

events from target process with conditional intensity X (t) by iteratively generating candidates 1 from a simple proposal process (e.g.. homogeneous Poisson process with rate λ) and accepting with the probability )-((+11), repeating until acceptance. This is highly inefficient because evaluating the acceptance probability for each proposed candidate requires a forward pass through the Transformer, potentially requiring many forward passes per single accepted event. Despite KV caching [27] reducing inference complexity from quadratic to linear, the large number of parameters in the Transformer architecture makes each forward pass computationally expensive.
technique designed accelerate token generation large language models (LLMs). It generates multiple tokens in parallel, rather than predicting each token sequentially, thereby improving the efficiency of the sampling process. Drawing inspiration from the analogy between next-event sampling in TPPs and next-token prediction in LLMs, we investigate the potential application of SD from LLMs Transformer TPPs, aiming to enhance sampling efficiency.
Speculative decoding (SD)
Specifically, we make the following key contributions in this work: (1) We identify a strong similarity between the thinning algorithm used in point process sampling and the SD technique in LLMss (2) We propose TPP-SD, which successfully applies SD to Transformer TPPs, resulting in accelerated sampling. (3) Extensive experiments are conducted on both synthetic and real datasets. The results show that TPP-SD and standard autoregressive sampling produce samples from the same distribution, but TPP-SD achieves speedup of approximately 2-6x. Ablation studies analyze the impact of different hyperparameters.
2 Preliminary Knowledge
In this section, we provide background TPPs, the thinning algorithm for TPPs, and SD for LLMs.
2.1 Temporal Point Processes
TPPs model discrete event sequences in continuous time, denoted S = (4i; where (42, ki) indicates an event of type k; E KC = (1,.... .....,K, K} occurring at time I; E (0, Tj. with 0 < l < Iy T where N denotes the random number of events. TPPs can be defined using different parameterizations. One approach is to specify the conditional distribution of events. We denote [t++1, k (H,,) be the conditional density function (CDF) of event (t++1, k++1) given the history of previous events (t1, ..,, (t4, k.). In this work, H, denotes the history of events up to but excluding time f, while HA includes whether an event occurs time Consequently, the joint distribution of all events can be factorized as:
p(S) II (1,,, .) F(T(H
where F(T - f(e dt., f(t = ft, ((,,). As is customary, further decompose [t, k(H,,) [(1HH,,) t). The term (1 F(T)Hey)) the probability that no event occurs in (4N, T). Another approach is to specify the conditional intensity function (CIF):
X (t, k))tdkk = ftt, K|H,, (. = (df x dk)[H4-),
1 1 F(t,H,,
where * indicates that the CIF is dependent the history H,- These different parameterizations are equivalent because it can be easily proven that the CIF (1, k)) and the CDF [(1, k(H,,) are one-to-one 3 It is worth noting that the event timestamps can also be equivalently expressed as
inter-event intervals 7; = 4 4411 i = 1 1..... . with to = 0. Therefore, the CDF of the timestamps can also be equivalently expressed as the CDF of the inter-event intervals g(T)?H,, = f(t H(H).. Correspondingly, TPPs have two equivalent forms of log- likellidddood expression:
N
K
log p(S)
Σ log (t,
Σ/
(tttk))
(1)
1=1
k=
[og (T, H(,,,) + log (k; (kkl)H,....:))) t:)] + log(1 G(T )),
(2)
where G(T
So T-
g( (ew)) dr. Eq. is CIF-based, and Eq. ( is CDF-based.

---

2.2 Thinning Algorithm for TPPs
For illustration purposes, we discuss the sequential simulation of timestamps from point process using the thinning algorithm. Given the event history M4 to sample timestamp from the original point process model (denoted as OriP) parameterized by CIF (t). we use a proposal point process. typically a homogeneous Poisson process (denoted as PoiP) with intensity À supre(t,,c)) X (4), to sample the next candidate timestamp:
[++1 PoiP
Next, the original point process model evaluates the candidate timestamp by calculating the condi- tional intensity at the given position:
X (((++1) OriP(1+41 H4.)
Then, a rejection sampling step applied to verify the candidate timestamp. The b accepted
(T((+11 6~ U~-nnform((, 1].
This rejection sampling ensures that the final sequence adheres to the original point process distribu- tion. If the candidate timestamp is rejected, the process is repeated until candidate is accepted. The accepted timestamp is then added to the event history, forming Hi., and the procedure continues until the sequence is fully generated. The efficiency of the thinning algorithm depends heavily on the alignment between the proposal point process and the original point process. A closer alignment results in higher acceptance rate, improving overall performance.
2.3 Speculative Decoding for LLMs
Autoregressive sampling LLMs is inherently sequential and inefficient. SD addresses this ineffi- ciency by delegating auto-regressive sampling to smaller language model (draft model, denoted as LLMam), which generates multiple candidate tokens sequentially given the context s:
(c1.91).-... (Cy, (Cy,47) 47) L.L
where 7 is the length of candidate tokens, cI...., Cyy are the sampled tokens, and q1.... 9- are their corresponding probability distributions. Then, the original language model (target model, denoted as LLM processes these candidate tokens in parallel, producing probability distributions:
P1. P.+ LLM (L..,.
where the extra probability distribution p-++ corresponds to processing the candidate token Cy,, sequence of token-level rejection sampling is then applied to verify the candidate tokens. A candidate token c; accepted if all previous tokens are accepted and
Pi(C) G (,(c,)
4; Uniform 1].
This rejection sampling ensures that the final sequence adheres to the distribution of the target model. If is the first rejected token, replacement be sampled from adjusted distribution norm(max
4.)). When all candidates are accepted, an additional token is sampled from Py+ The accepted tokens are then added the context s, and the process repeats until the sequence is complete. The efficiency of SD heavily depends on the alignment between the draft and target models. closer alignment leads higher acceptance rate, improving overall performance.
3 Related Works
In this section, we discuss related works on point process sampling. In traditional statistical point processes, the CIF parameterization generally preferred because the CDF is not convenient way to specify history-dependent point processes. In contrast, the CIF provides more convenient way to specify how the present depends on the past. Therefore, sampling methods in the statistical point process field are generally based on the CIF. Inverse method 288 simulates unit-intensity

---

((k)a(b(44))
(ts (6y,k)) pm (22, A2) (ta (41,
h(11)
h(ta)
h(t,)
Draft Model MI
Transformer
3 K
type
(22 (ls (i (1)
a(7++1 In(4))
Target Model Mr
((t))
x(ta)
(f:))
Aijarel Dianboniss
0
(1)
(fas kg) ...
(t:, Kz)
T
time
(1))k)
(22.
(la.ks) (44.44 ...
(a) CDF-based Transformer TPP
(b) TPP-SD
Figure I: (a) Overall architecture of our proposed CDF-based Transformer TPP. (b) The visualization of our proposed TPP-SD sampling method as elaborated in Section 4.3
Poisson process and transforms it into the desired point process using the inverse of the integrated CIF, leveraging the time-rescaling theorem [2], and [30] applies the inverse map parallel to accelerate this process. However, this method is not suitable for complex processes that do not have analytical forms for the inverse of integrated CIF. Another more easily implementable sampling method the thinning algorithm, which was originally proposed for sampling from inhomogeneous Poisson processes [15] and later adapted for bistory-dependent processes [3]. With the development of the deep neural point process, increasing number of works have started using the CDF parameterization. For example, [29] used normalizing flows and mixtures of log-normal densities to model the CDF the inter-event interval. Although this approach was initially proposed for training convenience, another advantage brings is facilitating sampling. This is because we can easily sample the next event directly from the CDF (24). Whether based CIF or CDF, both of the above methods for sampling history-dependent point processes can only be performed sequentially, leading to inefficient sampling. The key difference of this work is the introduction of parallelization in sampling, which accelerates the sampling process.
Methodology
In this section, analyze the similarity between the thinning algorithm and the SD technique. Inspired by this insight, we propose one SD method for Transformer TPP models.
4.1 Thinning Algorithm v.s. Speculative Decoding
As introduced in Sections 2.2 and 2.3 the thinning algorithm and SD share significant structural
similarities despite their different application domains. Both methods adopt a two-stage "propose- verify" framework that uses draft model to generate candidates and uses the target model for verification. Besides, the efficiency both methods critically depends on the alignment between the proposal and target distributions.
Despite these similarities, several key differences distinguish these approaches. The thinning al- gorithm operates strictly in an autoregressive manner, generating and verifying single candidate timestamp per propose-verify iteration. In contrast, SD can propose and validate multiple tokens per propose-vent verpoy-verifyy iteration, resulting in accelerated sampling. Additionally, while the thinning algorithm may fail to produce a valid timestamp per propose-verify iteration, SD guarantees the generation of at least one valid token per propose-verify iteration.
Motivated by these observations- especially the algorithmic similarities and the limitations of the thinning algorithm- we propose TPP-SD: a novel CDF-based sampling method for Transformer TPPs that applies the principles of SD enhance sampling efficiency.
4.2 CDF- Transformer TPPs
To accelerate TPP sampling using SD, we first need design a CDS-d Transformer TPP model, which is composed of two parts. A similar model design has also been adopted by (24).

---

Encoder. We use Transformer backbones to model the historical dependencies in the event sequence. Specifically, given an observed realization of the point process temporal encoding method from 40 to encode each timestamp 4; into fixed-length vector 244:) R.D. In addition to the temporal encoding, we use an embedding matrix W RKXD, where the D-dimensional embedding for event type k; is given by W k, with k; I being the one-hot encoding of kg Thus, the embedding for the observed realization is given by
{t, k,)] we apply the
X=f(KW 2) RND
ZN] RN are the collections of event types and timestamp embeddings of the observed realization, and f(-..) is fusion function where we use a summation operation. A Transformer is then applied obtain the history embedding ReNxDrraaee matrix H = (X) E RR Each row of the history embedding matrix h (4,) H(i.:) E R corresponds to the history information up to timestamp t,. The choice of encoder To can be any Transformer TPP, such as THP [40] SAHP [36], or AttNHP (18]. The necessity of using Transformer encoder lies its ability to enable parallel computation, which is essential for SD.
where [
kN] E RX and 22,,
Decoder. Inspired by 24 29
to enhance the flexibility of the model, we use mixture of log-
normal distributions to model the CDF of the next timestamp g(T++1 [h(t,),, since the inter-event intervals are positive, i.e.,
(log Ti+1 pimm )2
ge(T+4 h(t,))
Wsm
exp
TV 2
20
Here, go [b(tt)) is referred to as the decoder because it takes the history embedding h(t,) as input and decodes the inter-event intervals. The mixture weights w, = [wii,..., wsM] the mixture means μı; [Pa..... MMav]' T and the mixture standard deviations o; [on...... Jam] obtained by mapping h(,)). Specifically, h(4)) RDP is projected e; Eh((.) RR3D with E E R3DxD, and then sliced into three equal-length parts e, = leiT e² e Subsequently, ej. oz oj are mapped to wi, 14, and a; as follows:
w, ssottma(VViie (eeei bw), pe Vuei bus o, o. V ej ba), where Vw, VA, Vo E RRXD and bw, b by E RM are learnable parameters. The CDF of the next event type so(k+ ((:)) is modeled as a categorical distribution:
fe [h(t)) = softmax
(v (2) tanh
k h(t.) + b bbxk)) (2)
Our proposed based Transformer TPP model illustrated in Fig. This model can formally represented as M {,, g(T.), fkk-)),, where & corresponds the encoder and g (A)-) correspond to the decoder. The model is trained by maximizing the log-likelihood in Eq.
Naïve autoregressive sampling. The next timestamp is easy to sample from the log- normal mixture distribution go (T++1 by (see Appendix A. for proof)
2; (ategorical(w:). c~ C~.00,
f+t exp(1) (1, z, + o. zi),
so the next timestamp is f++1 + f++ The next event type sampled by h441 f [h(t:)) The new event (f++1, k(+1) appended the history and the sampling procedure is repeated until we reach the predetermined end time. However, naïve autoregressive sampling requires forward pass of the Transformer encoder for each event, which becomes inefficient as the number of parameters in the Transformer backbone increases.
4.3 TPP-SD
Suppose we have trained target Transformer TPP model MT with large number of parameters, from which we wish to sample events, and a trained draft Transformer TPP model MD with fewer parameters, which we use approximate Mr. The procedure of TPP-SD consists of three steps, as shown in Fig. Gb and Algorithm!
5

---

Drafting. Sample candidate events {(f+11, k441)..... (fitr. kity)} autoregressively from Mp. Meanwhile, record the interval CDF 9D(+++1))) and type CDF fo (Ri4el-) for all candidate events.
Verification. Run Mr in parallel to compute gr (f and fr(kktt for all candidate events.
Then, calculate the acceptance rates (r(+++))) and fr(k44l-)) for all candidate events. A candidate gr inter-event interval it+ is accepted all previous events are accepted and of (D(f+ 5) of Uniform(0 [0, 1]. A candidate event type k4t is accepted if all previous events and fitz are accepted, and of (ke+s))) of Uniform[0, 1].
gD(f441))
fo(ke)))))
Sampling from adjusted distribution. Once candidate event interval 6.+ or type by+1 is rejected, all subsequent candidate events are automatically discarded and a replacement f.+1 or k441 is sampled from an adjusted distribution defined as below:
g' ( norm(max(0, (k(++))))) norm(max(0,, (1a+t)))))) fo (k+++)))))
9D(T++())))
where norm(-) denotes the normalization. The correctness and necessity of sampling from the adjusted distribution have been established in the original speculative decoding work for LLMs (14) In this work, we extend the proof the TPP domain, as presented Appendix A.2
Eq. (4) a discrete distribution, and its normalization relatively easy to implement, but Eq. continuous distribution, and normalizing it is much more difficult because need to compute the normalizing constant f max(0, 9r (+)+ - 9p (e++l-))drr This is also the main difference between TPP-SD and LLM-SD, continuous distributions are not involved in the LLM application. Inspired by (32,, we employ an acceptance-rejection sampling scheme 4 to sample from g (Ti++)-), as outlined in Theorem |
(refer Appendix A..3 for the proof).
Theorem 1. For sample T+41 (r((++))) (-)) define the acceptance threshold as
max (0, (r.gr(((t)))) 9epp0-gppffff (f(+++l))) a
gr (f(+++))))
Accept Fist i where Uniform(O,, (0,1)) 1). This acceplance-rejection procedure generates samples from the adjusted distribution (TT++el)) defined in Eq. 3
5 Experiments
In this section, we compare TPP-SD with autoregressive sampling (abbreviated as AR sampling) on both synthetic and real datasets. We verify that both methods produce samples from the same underlying distribution, and demonstrate that TPP-SD significantly improves sampling efficiency.
5.1 Evaluation of Sampling Quality and Speed
Unlike point process fitting, which focuses on the accuracy of predicted event types timestamp deviations, point process sampling emphasizes whether the samples generated by TPP-SD and AR sampling follow the same distribution (i.e., sampling quality), as well as the efficiency of the sampling process (i.e., sampling speed). Therefore, we define the following evaluation metrics.
Likelihood Discrepancy (Synthetic and Real). For synthetic data, where the ground truth is known, we generate samples using both AR sampling and TPP-SD, and measure the discrepancy between the ground-truth likelihood (Eq. (T) and the model likelihood of the generated samples (Eq. (2)). Specifically, we compute A cesti Sg Caal for AR sampling and A = Se Lad for TPP-SD. For real data, where the ground truth is unknown, we measure the discrepancy between the AR sampling likelihood and the TPP-SD likelihood, A Lar L-al- Ideally, samples generated by TPP-SD and AR sampling should follow the same distribution the ground-truth distribution. A lower likelihood discrepancy indicates higher sample quality.

---

Table 1: Performance of TPP-SD with draft length ? 10 against AR sampling across synthetic datasets and Transformer encoders. We conduct all experiments using three random seeds and report the mean for each metric. For all metries, the best performance highlighted in bold.
Dataset
Poisson
Hawkes
Hawess
Encoder Type
THP
SAHP
AtNHH
THP
SAHP
AttHH
THP
SAHP
ArNHP
AR Sampling
0,542
0.012
1.879
0.753
0.884
0.220
0.022
0.146
0.334
A. (1)
TPP-SD
0.349
0.204
1.952
0.276
0.630
0.722
0.321
0.070
0.199
AR Sampling
0.038
0.033
0.076
0.044
0.031
0.029
0.069
0.055
0.065
Dxs ())
TPP-SD
0.036
0.050
0.068
0.043
0.028
0.027
0.053
0.080
0.045
Wall-time
AR Sampling TPP-SD
3.477
2.680
12.103
5.147
2.747
20.503
4.007
2,490
12,403
T T())
1.647
2.077
4.063
2.547
1.863
3.567
1.893
1.647
2.770
Speedup Ratio SARND (?)
2.110
1.290
2.967
2.113
1.513
5.743
2.117
1.277
4.467
Kolmogorov-Smirno Statistic (Synthetic Only). For synthetic data where the ground truth is known, the time-rescaling theorem [2] states that for event times {4]]== generated by a point process with CIF X² (t), if the model is correctly specified, the transformed inter-event intervals z = = (T) dr are i.i.d. samples from Exponential(1). Therefore, we apply the ground-truth CIF to transform the samples generated by AR sampling and TPP-SD, and then compute the KS statistic Dks to assess the conformity generated samples to the ground-truth distribution. A smaller DKs indicates higher sample quality. The detailed computation of Dks is provided in Appendix A.4
Wasserstein Distance (Real Only). For real data where the ground truth is unknown, the KS statistic cannot be computed. Instead, we assess the sampling consistency between AR sampling and TPP-SD using the Wasserstein distance Dws. Specifically, using the first M events history, perform N independent repetitions of sampling (M + 1)-th event, yielding {(tAR AA ) from AR sampling and {(t5D) ) from TPP-SD. For the temporal distribution, we compute D the 1-Wasserstein distance between the empirical distributions of (tAR) Ji=l and {t5D] Ti=1+ using ot wasserstein from the POT library [9]. For the event type distribution, we compute D the earth mover's distance between the empirical distributions of {KAR IN and {k:D) Ji=1, using ot emd2 from the same library. A smaller Dws indicates higher sample quality.
Speedup Ratio (Synthetic and Real). Sampling efficiency is central to this work. We quantify the acceleration gain by computing the ratio of execution wall times between AR sampling and TPP-SD, defined as SAR/SD Tan TPP-SD, respectively. A larger SARISD indicates faster sampling speed.
TAR where TaR and TsD denote the execution wall times of AR sampling and
5.2 Experimental Results on Synthetic Data
Datasets and Setup. We consider three synthetic datasets: inhomogeneous Poisson, univariate Hawkes, and multivariate Hawkes processes, each with 1000 sequences within the time window [0, 100]. For each dataset, we train an 8-head, 20-layer target model and 1-head, 1-layer draft model. We compare two sampling approaches: AR sampling using only the target model versus TPP-SD) which combines both target and draft models as elaborated in Section 4.3 Details on data simulation procedures, data splitting, and experimental settings are provided in Appendices B.I and C.3.1
As shown in Table I across all three synthetic datasets and three encoder architectures, TPP-SD consistently demonstrates high-fidelity sampling that closely matches the performance of AR sampling. Specifically, both TPP-SD and AR sampling exhibit low likelihood discrepancies and near-zero KS statistics relative to the ground truth, indicating excellent alignment with the target distribution. Furthermore, as shown in Fig. 2 the KS plots show that the samples generated by both methods consistently fall within the 95% confidence bands.
Results.
TPP-SD consistently achieves faster execution times and 1.3-5.7x speedup across all datasets and encoder architectures. These results demonstrate that TPP-SD maintains the same high sampling fidelity as AR sampling while providing significant improvements in sampling efficiency.
7

---

KS Plot
Empirical Cumulatio aatiatteee Dens.iy
Speculative Standard Real
Empirical Curul lative Density
Speculative Real
Empirica Cumulative Density
Speculative Real
...
45" Line
45" Line
45" Line
1.4
Theoretical Cumulative Density
Theoretical Cumulative Density
Theoretical Cumulative Density
(a) Poisson
(b) Hawkes
(c) Multi-Hawkes
KS plots for (a) Poisson, (b) Hawkes, and (c) Multi-Hawkes datasets. We AttNHP as encoder, and blue, green, and orange points represent samples from TPP-SD (y = 10), AR sampling, and ground truth, respectively. Black dotted lines show 95% KS confidence bands.
Figure 2:
Table Performance of TPP-SD with draft length y 10 against AR sampling across real datasets and Transformer encoders. We conduct all experiments using three random seeds and report the mean for each metric. For all metries, the best performance is highlighted bold.
Dutuset
Taohao
Amazon
Taxi
StackOverllow
Encoder Type
THP
SAHP
AINHH
SAHP
AnNHH
THP
SAHP
AnNHP
THP
SAHP
AINHP
Agol
AR Sampling
0.446
0.148
1.629
0.056
0.099
0.118
1.4411
0.563
0,859
0.587
0,340
0.985
TPP-SD
0.197
Dlas (1)
TPP-SD
AR Sampling
0,236
0.493
0.328
0.116
0.187
0.189
0.019
0.464
0.975
0.201
0.236
0.249
0,391
0.470
0.378
0.677
D&s (1)
AR Sampling
0.267
0.414
0.193
0.184
0.459
0.252
0.055
0.778
0.094
0.376
0.381
0.218
TPP-SD
0,751
0.368
0.206
0.418
1.409
0.327
0.655
0.744
0.134
0.375
0.199
0.507
AR Sampling
5,890
2,400
16:256
1.023
0.900
7.657
1.157
1.183
2,573
1.353
1.423
3:217
7 (1)
TPP-SD
3,460
1.643
5.180
0.290
0.317
1.353
0.453
0.347
0.650
0.700
0.663
0.783
Speedup
SARND
3.183
5.849
1,930
In addition, architectural variations are evident: ArNHP achieves the highest acceleration ratios despite having the slowest wall times; SAHP shows the lowest speedup maintains the fastest wall times; and THP strikes balance between acceleration ratio and runtime. These results suggest that the underlying architecture plays significant role in determining the effectiveness of accelcration. detailed discussion of these differences is provided in Appendix D.4
5.3 Experimental Results on Real Data
Datasets and Setup. For real data, we consider four commonly used datasets: Taobao [[, Ama- 22 Taxi 33). and StackOverflow g We similarly train 8-head, 20-layer target model and
a 1-head, l-layer draft model as in Section 5.2 and compare the sampling performance between AR sampling and TPP-SD. Details data statisties, data splitting, and experimental settings provided in Appendix C.3.2 It is worth noting that for real data, the ground-truth distribution is unknown. Therefore, we use AR sampling To establish a meaningful baseline that accounts for the inherent stochasticity of TPP sampling, we compare two independent runs of AR sampling. In theory, the likelihood discrepancy and Wasser- stein distances between two such runs should be zero. However, due to stochastic variation, even independent autoregressive runs may exhibit small differences. This self-comparison serves baseline for evaluating how well TPP-SD aligns with AR sampling.
reference and quantify how closely TPP-SD approximates
shows TPP-SD consistently delivers high- high-fideliyy sampling across all real datasets and encoder architectures. Similar to the observations on synthetic data, A A remains low, and both Diws and D&s approach zero, indicating strong alignment in both temporal and type distributions. Moreover, TPP-SD achieves 1.6-5.9x faster execution across all settings. These results confirm that TPP-SD matches the sampling fidelity of AR sampling while offering substantial gains in efficiency.
Results.

---

Likelhhodd Discrepancy de
KS Statistic Dut
Acceptance Rate a
Speedup Ratio Sages
Draft Length
Draft Length
Draft Length Y
Draft Length y
Malt lawde
hhod Diserepanyy
Wassarstein
Dis
Wasserstein Distance Diys
Acceptance Rate
Speeduu Ratio Sans2
Drift Length Y
Draft Length Y
Draft Length
Draft Length
Draft Length y
Figure 3: The impact draft length y on sampling quality measured by likelihood discrepancy (AL) and distance (Dks Dws), and sampling speed measured by speedup ratio SAR/D- We conduct all experiments using five random seeds and report the mean and the error band for each metric.
Table 3: Performance of TPP-SD with draft length = 10 under different size of draft model. The distance metrics Dxs is used for synthetic datasets, while ws and is are used for real datasets. We conduct all experiments using three random seeds and report the mean for each metric. For all
metrics, the best performance is highlighted in bold, and the second best is highlighted in underline.
Encoder Type
Draft Model
Distance
Wall-tmee
Speedup Ratio
Dataset
a (+)
head
layer
Das (1)
Dius (L)
Dws (1)
TAR (3)
Tso ())
SARIDD (1)
1
0.098
0.011
0.600
12.403
2.650
4.680
Multi Hawkes
ARNHP
12.403
4.130
4
0.227
0.004
0,740
12.403
5.176
2.676
1
0.276
0,080
0.197
0.220
16.256
5.727
2.838
Taobao
ARNHP
2
0.174
0.200
16.256
2.4%6
4
0.371
0,131
0.190
0.35
16.256
8.81
1.845
An interesting observation is that the speedup inversely correlates with event type cardinality. Datasets with higher cardinality (K 17 for Taobao, K 22 for StackOverflow) tend to yield lower speedup compared to those with lower cardinality (K for Amazon,
10 for Taxi). This is because larger number of event types increases the probability of divergence between the draft and target models, leading to more rejections during SD. Additionally, the AttNHP encoder consistently achieves greater speedups than THP and SAHP, trend also observed in the synthetic data experiments.
5.4 Ablation Studies
We analyze the sensitivity of two critical hyperparameters, draft length 7 and draft model size. Throughout this part, adopt AttHH encoder, and record the likelihood discrepancy AL (A for synthetic and A for real datasets), distance metrics D (Dęs for synthetic and Dws for real wall time T and speedup ratio SARSD-
datasets), acceptance rate
à events accepted
events drafted
Draft Length. To identify the optimal draft length y for TPP-SD, evaluate performance across datasets within the time window j0, 100) for 7 E [1, 60). Fig. shows that variations in have negligible impacts the metries AL and D, confirming the distributional equivalence of TPP- SD and AR sampling. However, acceptance rate a decreases as increases, and speedup SAR/SD peaks before Declining-falling below 1x for excessively large draft lengths due the overhead of evaluating rejected drafts. Moderate draft lengths (e.g., 7 x= 5-15) yield the highest speedup across all datasets.
Draft Model Size. We investigate the impact draft model size on performance by fixing the target model to an 8-head, 20-layer Transformer and evaluating three draft configurations: 1 1-head-1 1--11111- lllayer, 2-heed-4-layer, and 4-head-6-layer. Experiments are conducted on the Multi-HHawes and Taobao datasets. Results show that increasing draft model size preserves sampling quality-aa as measured by AL and D-d improves the acceptance rate a, but reduces the speedup ratio SARSD. Notably, the 1 1-head- llayerr configuration achieves the highest speedup without compromising sample quality.

---

Conclusions
In this work, we introduce the original SD framework from the LLM domain into the context of TPP sampling. By identifying structural similarities between the thinning algorithm in TPPs and speculative decoding LLMs, we develop an efficient framework that employs a lightweight draft model to propose candidate events for verification by the target model. TPP-SD significantly improves sampling efficiency by 2-6x while preserving distributional consistency with AR sampling, thus bridging the gap between the expressive power of Transformer TPPs and the need for efficient sequence generation in practical upplications. As future work, we plan to further optimize TPP-SD by incorporating advancements in SD. Specifically, aim to integrate the draft mechanism into the target model (3), or perform speculative decoding at the feature level rather than the event level [16]
Acknowledgments
This work was supported by the NSFC Project (No.62576346), the MOE Project of Key Research Institute of Humanities and Social Sciences (22JJD1 10001), the fundamental research funds for the central universities, and the research funds of Renmin University of China (24XNKJ13), and Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing.
References
[1] Alibaba. User behavior data from taobao for recommendation, 2018.
[2] Emery Brown, Riccardo Barbieri, Valérie Ventura, Robert E Kass, and Loren M Frank. The time-rescaling theorem and its application to neural spike train data analysis. Neural computation, 14(2):325-344 346, 2002.
[3] Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. Medusa: Simple 11m inference acceleration framework with multiple decoding heads. In Proceedings of the 41st International Conference on Machine Learning, ICML 24. JMLR.org. 2024.
[4] George Casella, Christian P Robert, and Martin T Wells. Generalized accept-reject sampling schemes. Lecture notes-monograph series, pages 342 347, 2004.
[5] Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean- -Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiw:2302.01311, 2023.
[6] Daryl J Daley and David Vere-Jones. An introduction to the theory of point processes. vol. i. probability and its applications, 2003.
[7] Nan Du Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez- Komez-Rodriguez and Le Song. Recurrent marked temporal point processes: embedding event history vector. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1555-1564. ACM, 2016.
[8] Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodrigun and Le Song. Recurrent marked temporal point processes: Embedding event history to vector. In Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi, editors, Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1555-1564, ACM. 2016.
[9] Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras, Nemo Fournier, Léo Gautheron, Nathalie T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78): 1-8, 2021.
10

---

[10] Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.
[11]
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
[12] John Frank Charles Kingman. Poisson processes, volume 3. Clarendon Press, 1992.
[13] Donald Knuth. The art of computer programming, volume (3rd ed.): seminumerical algorithms. Addison-Weste Longman Publishing Co., Inc., USA, 1997.
[14] Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274-19286. PMLR, 2023.
[15] PA W Lewis and Gerald S Shedler. Simulation of nonhomogencous poisson processes by thinning. Naval research logistics quarterly, 26(3):403-413, 1979.
[16] Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: speculative sampling requires rethinking feature uncertainty. In Proceedings of the 41st International Conference on Machine Learning, ICML'224. JMLR.org, 2024.
[17] Hongyuan Mei and Jason Eisner. The neural Hawkes process: A neurally self- ceff-modulaiing multivariate point process. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6754-6764, 2017.
[18] Hongyuan Mei, Chenghao Yang, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In International Conference on Learning Representations, 2022.
[19] Zizhuo Meng, Boyu Li, Xuhui Fan, Zhidong Li, Yang Wang, Fang Chen, and Feng Zhou. Transfeat-tpp: An interpretable deep covariate temporal point processes. In ECAI, 2024.
[20] Zizhuo Meng, Ke Wan, Yadong Huang, Zhidong Li, Yang Wang, and Feng Zhou. Interpretable transformer hawkes processes: Unveiling complex interactions in social networks. In Proceed- ings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2200-2211, 2024.
[21] Paul-André Meyer. Démonstration simplifiée d'un théorème de knight. Séminaire de probabilités de Strasbourg, 5:191-195, 1971.
[22] Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly- labeled reviews and fine- mna-igrined aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference natural language processing (EMNLP-JCCNP)) pages 188-197, 2019.
[23] Yosihiko Ogata. On Lewis' simulation method for point processes. IEEE Transactions Information Theory, 27(1:233-1, 1981.
[24] Aristeidis Panos. Decomposable transformer point processes. In Annual Conference on Neural Information Processing Systems, 2024.
[25) Papangelou. Integrability of expected increments of point processes and a related random change of scale. Transactions of the American Mathematical Society, 165:483-506, 1972.
[26] Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recur- rent neural networks. In Proceedings of the 30th International Conference International Conference on Machine Learning- Volume 28, pages III-1310, 2013.
[27] Reiner Pope, Sholto Douglas, Aakanksha Chowdhery. Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scaling transformer inference. Proceedings of Machine Learning and Systems, 5:606-624 2023.

---

[28] Jakob Gulddahl Rasmussen. Lecture notes: Temporal point processes and the conditional intensity function, 2018.
[29] Oleksandr Shchur, Marin Bilos, and Stephan Günnemann. Intensity-free learning of temporal point processes. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
[30] Oleksandr Shchur, Nicholas Gao, Marin Bilos, and Stephan Günnemann. Fast and flexible temporal point processes with triangular maps. Advances in Neural Information Processing Systems, 33:73-84, 2020.
[31] Qingmei Wang, Minjie Cheng, Shen Yuan, and Hongteng Xu. Hierarchical contrastive learning for temporal point processes. Proceedings of the AAAI Conference on Artificial Intelligence, 37(8):10166-10111 10174, Jun. 2023.
[32] Zili Wang. Robert Zhang, Kun Ding, Qi Yang, Fei Li, and Shiming Xiang. Continuous speculative decoding for autoregressive image generation, 2024.
[33] Chris Whong. Foiling nyc's taxi trip data. https://chrisshoong http://hhwo....ocooooooaaaaaaaaa nyc_taxi/ 2014.
[34] Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Hongyan Hao, Fan Zhou, Caigao Jiang, Chen Pan, James Y. Zhang. Qingsong Wen, Jun Zhou, and Hongyuan Mei. Easytpp: To- wards open benchmarking temporal point processes. In International Conference on Learning Representations (ICLR), 2024.
[35] Siqiao Xue, Xiaoming Shi, Y James Zhang, and Hongyuan Mei. Hypro: A hybridly normalized probabilistic model for monghooizonn prediction of event sequences. In Advances in Neural Information Processing Systems, 2022.
[36] Qiang Zhang, Aldo Lipani, Ömer Kirnap, and Emine Yilmaz. Self-attentive Hawkes process. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 11183-111 11193. PMLR, 2020.
[37] Feng Zhou, Quyu Kong, Zhijie Deng, Jichao Kan, Yixuan Zhang, Cheng Feng, and Jun Zhu. Efficient inference for dynamic flexible interactions of neural populations. Journal of Machine Learning Research, 23(211):1-49, 2022.
[38] Feng Zhou, Quyu Kong, and Yixuan Zhang. Advances in temporal point processes: Bayesian, deep, and Ilm approaches. arXiv preprint arXiv:2501.14291, 2025.
[39] Feng Zhou, Zhidong Li, Xuhui Fan, Yang Wang, Arcot Sowmya, and Fang Chen. Efficient inference for nonparametric Hawkes processes using auxiliary latent variables. Journal of Machine Learning Research, 21(241): 2020.
[40] Simiao Zuo, Haoming Jiang. Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer Hawkes process. In Proceedings of the 37th International Conference Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 11692-1 11702. PMLR, 2020.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. See the Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation of the work is discussed in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

For all theoretical results, the paper provides the corresponding proofs in the appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses the information needed to reproduce the main experimental results in the paper. See the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
14

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the data and code in the supplemental material reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
aublic/guuddes/Coeetbbbmsss cosssiiinooyyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
www.ssssiicggcoooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify all the training and test details necessary to understand the results. See the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
We report the statistical significance of the experiments. In the figures present- ing the experimental results, we provide the standard deviations across multiple runs.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: All experiments conducted the server whose details provided in the paper. See the Experiments section.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't i make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttpp/:/eeerrppscccpppll//////p ctw.eeieeeeicnaiceeeeieeniee
Answer: [Yes]
Justification:
The research conducted in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This paper presents work whose goal is to advance the field of machine learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Guidelines:
16

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All code, models, and datasets mentioned in the text are appropriately cited with their original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: New assets introduced in the paper, such as code, are well documented. The documentation is provided alongside the assets the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p//eearrpppcc///ooeeeeeeee/ //t/////ooeeeeeccc////////..1. for what should or should not be described.
19

---

A Proof of Mathematical Theorems
A.1 Sampling from Log-Normal Mixture Model
To sample a random variable 7 from a log-normal mixture distribution where wm are the mixture weights, j'm the location parameters and om the scale parameters, M is the number of mixture components
M
(log T - 1mmm) )²
p(T)
Wm
exp
(5)
20 m
m==
we only need to do the following procedure
Categorical wM), EV(00,) T= exp(M( + 0= .e),
(6)
where is a single integer E mode of the mixture log-normal with location and scale parameter (1z, oz). Then by using the Slange-of-variabee formula
By Categorical WM). we choose the z z-th
de 1 2
(logT 12222
1
1
(log r 122
p(T) p(e)
exp
exp
dr
20
O2T
TV2TO,
20
(7)
which is exactly the PDF of CN(H,:
A.2
Correctness of Sampling from the Adjusted Distribution
Denote By as the acceptance probability of candidate event interval in SD, i.c.
Br = Er-gn min (1.r(( = min (9r(T))) dr.
go((THll)))
(8)
Note that
max(0, ga (T++)))))))))))))))))))))))))))) (D(4))))) g' (((Tx4d))) J max (0, (T(+t))))))))))))))) - (T++)))))))) :) dr+t1
gr((+4))) min (an ), 9T (T((1))))))
1 B
Denote Placcept, r') as the probability that the sample r' drawn by the draft model is accepted by the target model, and P(reject, r') as the probability that 7' is rejected and subsequently resampled from the adjusted distribution defined Eq. (3). By the law of total probability, the distribution of a sample r' drawn by SD can be calculated as
p(T') P(accept, 7') + P(reject,
(10)
1, (r((())
min
gn(T'1+) (1- (-)-)))
(11)
(T(1)))
(r('1-))
min
go (T'1.) 9D(T'1-) (0T(7'1-) min(go(""1))9r("1))
(12)
(r((')))
(13)
which justifies the correctness of sampling from an adjusted distribution. The derivation also shows that sampling r' directly from the target distribution gr(T++1|-) instead the adjusted distribution g' (-l+ after rejection would result in p(r') + gr(''))), leading incorrect sam- pling. Similarly, denote the acceptance probability of candidate event type in SD as 3x
Ek~fo(k(+1l)) min (1, fr((k=el1) [o(k4411)) E. min (fr(())) can also derive the correctness of sampling K' 22 (kiitt))));
p(k') P(accept, k²) + P(reject, k')
(14)
(1, [r(()))
min
(k(.))))) fo(K)+) + +1 -1 (1-8))((K1))
(15)
1, (r-(k))) FD(k'))) fD(K')) +(F((11)) min(fo((1))
min
(16)
fk
(17)
20

---

A.3 Proof of Theorem
We refer to for the proof in this subsection. We begin by recalling the principle and demonstrating sampling 44, which aims to draw samples from
the correctness of the standard acceptance-rejecttnn target distribution p(T) using a proposal distribution ((T) such that p(T) < Mq(T), VT T. where T is the support of p(r) and q(T), and M 2 1 is a finite constant. The procedure is as follows: (1) Sample a candidate ~ ((T). (2) Sample uniform random variable 6 ~ Uniform(0, 1). (3) p(f)
Accept the candidate f if <
Mq(F) otherwise, reject it and return to step (I). To prove that the accepted samples follow the distribution P(T), let A be any measurable set within the support T. The probability that an accepted sample f falls into A is
P( E Alf is accepted) = IP(f E A, f is accepted)
Pff E A, f is accepted)
(18)
P(f is accepted)
P(f T, is accepted)
Note that
= S.L. f T(f EAT (<< MQ((F)
p(f))
P(f E A, 7 is accepted)
p(t)dedt
(19)
M I Ilf E A) 1)(()
a(F)d+=
M / p(F)dt.
1
(20)
g(f)
Therefore,
LAP(F)AF/M S7 p(F)dF/M = p(F)dt
IP( E Alt is accepted)
(21)
which justifies the correctness of acceptance-rejection sampling. Now we apply this acceptance- rejection sampling framework the specific scenario described in the theorem. The adjusted distribution g' (T+++l-) defined in Eq. (3) corresponds to the target distribution p(T), and gr (T++ corresponds the proposal distribution ((T). We need to find the constant M such that for any sampled fs+L. -f(++++l)) I Mgr(f++ul-). Denote Z = max g0, (f((t+ll))))))) df note that
g'(fx4el-) max(0. (f(+e)))))))))))) - 9n (f((+))l))) gr( - -nn((ggf)))) < Z
gr (fc+rl)) (22)
Z
Z
This inequality confirms that we can set M := 1/Z satisfy the requirement of acceptaanc--rejection sampling. With M = z, the acceptance threshold a is computed as
('(+++il) max(0, gr (fr+1))))) - gD (f max(0,, (o (f(4).)) (D(tt+ll)) 0r((x41l))/2
a "Mgr(fuull-)
gr((++1l-)
(23)
which is precisely the acceptance threshold a defined in Theorem
A.4 Computation of the KS Statistic
We employ the Saapogoroo-Smmimon (KS) statistic to assess whether sampled realizations conform to ground truth distributions in synthetic point processes. For i.i.d. ordered observations {X, ) = from a distribution F(x), the KS statistic is defined as Dks = sup, (F(()) F(x)], where F, (x) is the empirical cumulative distribution of (X.) Theorem 2
The leverage of KS statistic is guaranteed by
Theorem (Time Rescaling Theorem 21 25
Given a point process with CIF (((HH. and with
occurrence times 0 < t1 < ....
< t, ST. T. define
21 A(HHH)dd z 1.
X((HH))dt, i i= 2 ....., n.
(24)
Then 1 independent exponential random variables with rate parameter 1.
By transforming sampled timestamps (1,)2_ using the ground truth intensity function, we obtain
rescaled intervals {= whose empirical distribution F, should approximate F(z) =
e-=, x > if sampling is correct. Low DYS values indicate high conformity to the ground truth distribution.
21

---

B Dataset Statisties
B.1
Synthetic Dataset
We consider three sets of synthetic data, where each dataset contains 1000 sequences and is split into 80%/10%/109 for training/validaaiinneessin The data are simulated using the traditional thinning
algorithm 15 2X]
Poisson is simulated from inhomogeneous Poisson process with intensity function X(t) A (b + sin (wTt)) with A = 5, b = 1, 50 and time span T = 100.
Hawkes simulated from single-variate Hawkes process X(t) - Eusi oxpp(-31 t:)) with μ 2 1, = 2 and time span T 100.
Multi-Hawkes is simulated from M - didddmmnnional Hawkes process, where the intensity for the MM j-th dimension is (t) pm Lim1 Ee <t Okij exp(-Bis(t f.)). Here M 2, M1 = μ12 0.4, 011 = 022 = a,2 = 0.5, 021 = 0.1, 311 = B12 = 821 = B22 2.
B.2 Real Dataset
We consider four sets of real data listed below. For all datasets, we maintained the standard train- ing/validatiion/tesiing splits established in prior work.
captures the temporal behavioral patterns of the 2,000 most engaged anonymous users on the Taobao e-commerce platform during a nine-day period from November 25th to December 3rd, 2017. The dataset encompasses K 17 distinct event categories.
Taobao 1
documents the chronological shopping activities of anonymous customers on the Amazon marketplace from January 2008 to October 2018. The dataset features K 16 different event categories that characterize various user interactions with the platform.
Amazon 22
Taxi 33 comprises pickup and dropoff incidents for taxis operating across New York City's five boroughs (Manhattan, Brooklyn, Queens, The Bronx, and Staten Island). Each borough-specific pickup or dropoff action is classified as distinct event type, yielding total of K 10 event categories.
consists of sequential badge-earning events by users on the prominent question- answering platform StackOverflow. Individual users accumulate various badges over approximately a two-year timeframe, with the dataset containing K 22 unique badge types in total.
SverfOve S
C Addtional Experiment Details
C.1 Algorithm for TPP-SD
The pseudo code for the algorithm of TPP-SD shown in Algorithm I
C.2 Training Details
To implement our proposed CDF-based Transformer TPP model, we modified the codebase from The original RNN encoder for history aggregation was replaced with Transformer backbones proposed by THP [40], SAHP (36), and ArNHH [18]. Given that these models are Cas-based, we extracted only their Transformer architectures. utilizing implementations available in the EasyTPP (4] GitHub repository F In line with [29]. we set the history embedding dimension D = 64 and the number of mixture components M 64.
22

---

Algorithm 1: TPP-SD
Inputs:
Initial event (fo: ko), end time number of draft events 7; target model
Mr (Er,gr(r
gr -)); (rl(f))) ). draft model Mr = (En.gn(T)) (
Initialize: Sampled events S = {(lo, ko)). J. i +- 0:
while 4; < do
for 1 1 do
Sample 7:+1 ~ (T record (f and lats t- lite-1 + fere
Sample kitt ~ (k(-) and record fD(k++
Compute gr(ff+1))))..............)....). 9r.ft-1l).....(fffff....... (ft++>l))) and fr fr((k+)1)) in parallel;
cf....c G,, ,,.e,...... Uniform(0, 1);
gn(t+t ot fo(ktt
+ (min(h,, 12)1 41,2 S 7, aplfete 1.) 22 ( }:
10
if < then
11
Sample f.4 o (r(te.)) and litt. 444-11 + fetr: Sample hitL ('(k+tl):);
S +
SU S(fH44 kir2).4...... (f,2,
14
it
4+4 L:
16
+ ((t,,k) (t), 68,4
17
Return: S
By default, we trained an 8-head, 20-layer target model and a 1-head, 1-layer draft model for each dataset. All models were trained using the Adam optimizer [T] for up 1000 epochs with a batch size of 16 one single NVIDIA RTX 4090. Early stopping based on validation log-likelihodd was applied to prevent overfitting.
C.3 Sampling Details
C.3.1 Sampling Synthetic TPPs
For the experiments in Section 5.2 we evaluate sampling performance within the time window [0, 100]. We report the discrepancy metrics A AL and AC the KS statisties DRS, the execution time TaR and TsD, and the resulting speedup ratio SARSD as defined in Section [.11 We conducted experiments across three random seeds and report the mean for each metric to ensure result robustness.
Apart from numerical metrics, we employ Kolmogorov-Sminng quality, which compare the empirical cumulative distribution F.,(x) of the rescaled intervals { ;=1 against the theoretical cumulative distribution F(x) e e- given by [2]. The KS plot visualizes points {( (F((2,, Fx(2,) (2,))) where perfect sampling would yield points along the 45-degree line.
plots (KS plots) to visualize sampling
We conduct formal statistical assessment using the Kolmogorov-Smiing Smiirroo test with the following hypotheses:
Ho Fa(x) = F(x)
versus H Fx()) F(x),
(25)
c(a)
Reject Ho if: DKs
(26)
Vn"
where is the number of sampled events, DKS sup, (x) - F(x)) is the KS statistic, and c(a)
13 Based on this test, we construct a 95% confidence band CB = ((F(2)))) y) y [F(z) v F(z) + _ }} for each KS plot. If the sampled events conform to the ground truth process at the 95% confidence level, all points in the KS plot should fall within this band.
e(a:) 1.36 significance level a 0.05
We construct direct comparison visualizations between F, (a) and F(r) = 1 - er alongside KS plots across Transformer encoders (THP, SAHP, AttNHP) with draft length 7 = 10 on three synthetic datasets. As evidenced in Fig. 4 F, (x) demonstrates strong concordance with F(x) = 1 e= and all points in the KS plots remain within the 95% confidence bands, providing robust statistical evidence for the distributional fidelity of our TPP-SD method.
23

---

MHD
sndeenaciiieeeee JrA Dandby
(a) THP-Poisson
(b) Hawkes
(c) THP-Multi-
(d) SAHP--oosson
(e) SAWP-Hawkes
(I) SAHP- Halti
00.11
(g) HUNHHP-PissoO
(h)
(i) MINHP-Multic Maai--MMut-t-akkee Hawkes
Figure 4: KS plots across THP (top row), SAHP (middle row), and AttNHP (bottom row) encoders with draft length y = 10 on three synthetic datasets: Poisson (left column). Hawkes (middle column), and Multi Hawkes (right column). Blue, green, and orange points represent samples from TPP-SD (7 = 10), AR sampling, and ground truth, respectively. Black dotted lines show 95% KS confidence bands.
C.3.2 Sampling Real-world TPPs
For the experiments in Section 5.3 we conduct evaluations within the same time window of [0, 100]. In accordance with Appendix C.3.1 we report the discrepancy metric Ac the Wasserstein distance Wws and D the execution time TAR and TSD for AR sampling and TPP-SD, and the resulting speedup ratio SARSD defined in Section 5.1 For computation of Diss and D we set M 100, N = 100, which means that we fix the first M = 100 events as history and perform N 100 independent repetitions of sampling the 101-th event, obtaining {(AA AAR li=l Ji=l from TPP-SD, and subsequently computing Dws and D as
1h0 from
AR sampling and {(t,0) k,5D) 100
elaborated in Section 5.1
Beyond numerical metrics, we visualize event type distributions to qualitatively assess the sam- pling fidelity of TPP-SD on real datasets, where accurate event type sampling critical. Figure S demonstrates that event type distributions from TPP-SD ((A,S)) consistently align with those from AR sampling ( ((LAR) )) across all datasets and encoder architectures, confirming TPP-SD's ability to maintain distributional consistency with AR sampling in event type generation.
C.4 Alation Studies
We have investigated the impact of draft length 7 and draft model size on the sampling quality and sampling speed of TPP-SD with AttHHP encoder. Using the same sampling configuration and evaluation metrics as in Section 5.4 we also conducted experiments using THP and SAHP encoders.
Draft Length. As illustrated Figs. 6a and experiments with THP and SAHP encoders reveal patterns consistent with our findings in Section [5.4 changes in draft length minimally affect likelihood discrepancy AL and distance metrics D, while acceptance rate a also diminishes with increasing 7, and speedup ratio SARSSD exhibits characteristic peak followed by decline, eventually dropping below 1 x when draft lengths become excessive. Intermediate draft lengths (y 5-15) optimize computational efficiency across all experimental configurations.
24

---

(a) THP-Taobao
(b) THP-Amazon
(c) THP-Taxi
(d) THP SteckeeaoO
(c) SAHP-Taobao
(f) SAHP-Amazon
(g) SAHP-Taxt
(h) SAHP-StackOverfie
(i) AuNHIP-Tabba Taobao
ii MINIHP-AAazon
(k) AuNIIP-Taxi
(l) LLNHIP-StackOverlin
Figure 5: Event type histogram across THP (top row), SAHP (middle row), and AtNHH (bottom row) encoders with draft length 7 = 10 on four real datasets: Taobao, Amazon, Taxi and StackOverflow. Blue bars and orange bars represent the frequency of sampled event types from AR sampling and TPP-SD.
Likelihood Discrepancy &c
KS Statistic Oes
Acceptance Rate
Speedup Ratio Supes
62
Dos
Saaa
Draft Løngth
Draft Length
Length y
Draft Length
Hawiaa
hood Wasserstein
Wasserstein Distance Dis
Acceptance Rate
Speedup Ratie
Syaop
Draft Length y
Draft Length
Langth
Length
Length y
(a) THP encoder
Likelhaad Discrepancy Ac
KS Statistic Des
Accoptance Rate
Speedup Ratio Sugera
Dxs
Sans
Draft Lēngth
Draft Length
Length y
Draft Length y
Vili Howees
Discrepancy
Wasserstein D
Wasserstein Dis
Acceptance
Speedup Sansp.
Saup
Draft Length y
Draft Length y
Draft Length
Length
Draft Length y
Tastao
Amason
(b) SAHP encoder
Figure 6: The impact of draft length on sampling quality measured by likelihood discrepancy (AC) and distance (Dxs or Dws), and sampling speed measured up speedup ratio SARISD. We conduct experiments across five random seeds with (a) THP encoder and (b) SAHP encoder, and report the average for each metric.
25

---

Draft Model Size. As evidenced by Table increasing the draft model size with either THP or SAHP encoder preserves sampling quality (measured by AC and D) and improves the acceptance rate but reduces the speedup ratio SARSD. The 1-head, 1-layer configuration remains optimal for the draft model across these encoder architectures.
Table 4: Performance of TPP-SD with draft length y 10 under different size of draft model. We use KS statistic (Dxs) for synthetic datasets and Wasserstein Distance (Dws. Dws) for real datasets. We conducted experiments across three random seeds and report the mean for each metric. For all metries, the best result is shown bold, and the second best result is shown underline.
Encoder Type
Model
Distance
Wall-ume
Speedup Ratio
Dataset
head
layer
AC
Das (1)
Dlus (1) Dis
a (t)
TAR (3)
Tsp
SARSD
1
0.226
0.008
0.790
3.990
1.970
2.025
Multi-Hawkes
THP
2
0.202
0.019
0.830
3,900
2.179
1.831
4
0.144
0.006
0.950
3,990
2.208
1.807
1
0.054
0.008
0.640
3.543
2,143
1,653
Multi-Hawkes
SAHP
2
0.143
0.004
0.740
3,543
1.327
4
0.395
0.008
0.800
3.543
2.734
1.296
1
0.033
0.175
0.222
0.18
5.890
3,460
1.702
Taohao
THP
2
0.339
3,900
1,510
0.409 4
0.291
0.301
0.240
5,890
4,920
1.197
1
0.410
0,302
0.630
0.14
2,460
1.621
1,518
Taohao
SAHP
2
0.742
1,850
1,330
4
0.486
0,545
0.745
0.21
2,460
1,990
1.236
D
Additional Discussions
D.1
Why Not Use CIF-based Speculative Decoding?
In theory, CIF--baeed speculative decoding is feasible. For example, we can homogeneous Poisson process with constant intensity À as the draft model to generate candidate timestamps {7.). These candidates are then input into a CIF-based Transformer point process to compute the CIF at each position, {X* (f.)). and the acceptance ratio (E+(t)/A}, A candidate timestamp & is accepted all previous timestamps are accepted and e < A
(i.).) where e ~ Uniform(0, [0, 1]. However, it has
two significant drawbacks: (1) Determining is challenging because we must ensure that À is greater than the CIF at all candidate timestamps, and the CIF values are history-dependent and stochastic. To be safe, we generally have to use a relatively large A, which reduces the acceptance rate and leads low sampling efficiency. (2) The CIF--baeed TPP-SD may fail generate any valid timestamp in each iteration. If the first proposed timestamp is rejected, the CDF-based TPP-SD can sample replacement from (r)-)) but the CIF-based method cannot do this. This can result in cases where single forward pass of the target model fails to generate any valid timestamp, further reducing sampling efficiency.
D.2 Necessity of Sampling from Large Target Models
critical consideration in TPP sampling is the necessity of sampling from large draft models. For TPP data, large Transformer encoders (e.g., multi-layer, multi-head, with high-dimensional representations) are crucial for capturing complex temporal dependencies. Their larger parameter capacity allows them model intricate data distributions and generate high-quality samples-all albeit at the cost of slower autoregressive sampling. contrast, smaller Transformer encoders (e.g., single- layer, single-head, with low-dimensional representations) are much faster sampling due reduced parameter capacity. However, they often struggle to model complex temporal patterns accurately, resulting in lower-quality samples that may not align well with the underlying data distribution.
To empirically demonstrate this performance gap, we evaluate a simple draft model MD (single- layer, single-head THP) both synthetic (Poisson, Hawkes, Multi- Hawkes) and real datasets (Taxi, StackOverflow, Amazon, Taobao), measuring sampling fidelity via AC and D Kss. Dws, and efficiency via wall-time.
As shown in Table 5 the simplistic architecture of MD struggles to capture temporal dependencies effectively even synthetic datasets. Performance degrades further real datasets, with significantly
26

---

Table 5: Evaluation of sampling methods across synthetic and real-world datasets. Autoregressive sampling with the draft model (AR (draft)) consistently yields lower sample fidelity (higher AC,
DKs, and Dws) compared to the target model (AR (target)) and our proposed TPP-SD. For each metric, the best result is bolded, and the second-best is underlined.
Metric
Sampling Type
Poisson
Hawkes
Mulki-Hawkes
Taxi
StackOverflow
Amazon
Taobao
AR (draft)
2.174
1.348
1.305
1.504
1.136
0.614
2.122
AC(I)
AR (target)
0.542
0.753
0.022
0.441
0.587
0.056
0.446
TPP-SD)
0.349
0.276
0.321
0.065
0.231
0.129
0.033
AR (draft)
0.153
0.141
0.126
Dks(1)
AR (turget)
0.038
0.044
0.069
TPP-SD
0.036
0.043
0.053
AR (draft)
0.765
0.994
0.534
0.577
Dlr(()
AR (target)
0.201
0.470
0.189
0.236
TPP-SD
0.082
0.391
0.078
0.076
AR (draft)
0.768
0.821
0.759
1.384
(1)(U)
AR (target)
0.055
0.376
0.184
0.267
TPP-SD
0.655
0.375
0.418
0.751
AR (draft)
0.987
1.240
1.530
0.150
0.120
0.070
0.410
Wall-time (4)
AR (target)
3,477
5.147
4.007
1.157
1.353
1.023
5,890
TPP-SD)
1.647
2,547
1.893
0.453
0.700
0.290
3.460
larger AL and Dws values, as MD cannot model the complex dependencies present in the data. Draft model alone performs poorly on this task, demonstrating the necessity of large target model and the importance of our proposed TPP-SD framework, which leverages the speed of the draft model without sacrificing the fidelity provided by the target model.
D.3 Feasibility of Using Simple Poisson Draft
The standard thinning algorithm generates candidate events using a homogeneous Poisson process (o)P This motivates a compelling question for our TPP-SD framework: is it possible to employ a draft model as simple as PXiP(X) to further accelerate sampling from complex target models without compromising fidelity?
We replace the I-head- I-laer (abbreviated as I-H-I-L) draft model to a simple homogeneous Poisson process draft model, whose parameter is estimated by maximum likelihood estimation on the data, ie.. Å = #, where is the number of events and T is the time span of the dataset. As for the experiment setup, we adopt THP as the backbone of target model for illustration simplicity. We conduct the additional experiments on 3 synthetic datasets (i.e., Poisson, Hawkes, and Multi-Hawker and 2 real-world datasets (i.e., Taobao and Amazon).
Since Poisson process is history-indepeendett, we can sample the candidate inter-event times all at once from (x)( It is worth noting that for TPPs with marks (c.g. efficient drafting process, we first estimate the mark distribution from the training data using MLE (i.e., P...
to ensure
where n. is the number of events with mark and n is the total number of events), and then sample the marks from the estimated mark distribution all at once as well.
We can see from thbat, synthetic datasets, TPP-SD with Poisson draft maintains high fidelity in sampling while achieving even more significant speedup across three datasets. The fast drafting process of Poisson compensates for the relatively lower acceptance rate. However, on real- world datasets, while TPP-SD with Poisson draft retains the sampling fidelity, we witness drop in speedup ratio. On complex real-world TPPs, the acceptance rate of Poisson draft is too low that it overwhelms the speedup from fast drafting. Therefore, in our setting of TPP-SD, we use a more robust drafting strategy, i.c., slightly more complex draft model (I- (1-layer- 1-ead THP) to ensure that TPP-SD can achieve high fidelity sampling across a wide range datasets.
D.4 Architectural Differences across Encoders
We incorporate the history encoders from THP, SAHP, and ArNHP into our CDF-based Transformer TPPs designed for TPP-SD. The architectural and implementation differences among these encoders may explain the varying sampling speed performance observed in Table [ and Table 2
27

---

Table 6: Comparison between a 1-head- 1-layer (1-H-1-L) Transformer draft model and a simple Poisson process draft within the TPP-SD framework. On synthetic data, the Poisson draft achieves a higher speedup the cost of fidelity. On real-world data, its low acceptance rate leads inferior performance in both speed and fidelity. For each metric, the best result between the two TPP-SD) methods is bolded, and the second-best best is underlined.
Sampling
Acceptance Rate a(t)
Speedup SAR/SD(1)
Dataset
AC(4)
Dxs(L) Dis((t) Div
Method
AR Sampling
0.542
0.038
Poisson
TPD-SD PP--SD(1-H-114 (1-H-I-L draft)
0.349
0.036
0.740 0.370
2.110 3,230
TPP-SD (Poisson draft)
0.630
0.050
AR Sampling
0.753
0.044
Hawkes
TPP-SD (1-H-I-L draft)
0.276
0.043
0.720
2.113
TPP-SD (Poisson draft)
0.286
0.036
0.520
5,430
AR Sampling
0.022
0.069
Multi-Hawke
TPP-SD (1-H-I-L draft)
0.321
0.053
0.750
2.117
TPP-SD (Poisson draft)
0.268
0.057
0.390
3.610
AR Sampling
0,446
0.236
0.267
Taobao
TPP-SD (1-H-I-L draft)
0.033
0.076
0.751
0.260
1.597
TPP-SD (Poisson draft)
0,680
0.091
0.623
0.030
1.370
AR Sampling
0.056
0.189
0.184
Amazon
TPP-SD (1-H-1-L draft)
0.129
0.078
0.418
0.270 0.010
3.550
TPP-SD (Poisson draft)
0.066
0.090
0.086
2.230
Following the notation in Section slightly:
4.2
the temporal encoding z((4) RP for each encoder differs
(sin((/11000///D)
j is even
THP: ((4)); =
coss(t/11000(55))))) j is odd
(27)
[snn)//10000// w,tt)
is even j is odd
SAHP: 2(44)];
(oss6/1000006-1100 + wztt)
(28)
AUNHP: (2(44); (in(tt/mm-65//mm)
j is even
(29)
(55
j odd
where (1, is the j-th dimension of the temporal embedding vector z(1,), and M, m are hyperpa- rameters in AtNNH..
The attention mechanisms differ across encoder architectures. For clarity, we present the single-head attention formulation for each model. The layer-l history embedding of event (t,, kt) is computed as:
f(q (t))))
THP/SAHP: (t,) (ta)
(30)
flq (t,) k)))))
AUNHP: (t))t) (t)) tanh Σ
(31)
1+ Σ f(q (t)),
where f(..)) is a Gaussian kernel (( (ty), k (tt)) = exp( t (t+) (ty)) E R. The key distinction lies in how query, key, and value vectors are derived at layer-l. In THP and SAHP, (t)) = hh(h)) (t.) = K ((-1), v(,))))) = ((-1) are obtained through linear
projection, whereas in AUNHP:
q (1-1((1) (1.) = ( concat (1;2(44 01 (G)
(32)
k" 11 (t.)) = K 1 concat (1;z(t,) ;h (t) (t))))))))))))))))))))))))))
(33)
11-1)(())) ( concat((;;((t)) (
(34)
where Q KO Vi Dx AuNHP's complex attention mechanism doubles the dimen-
sion of intermediate vectors, particularly in multi-head configurations where each head requires
28

---

separate query, key, and value transformations, This architectural complexity results in significantly higher AR sampling latency compared other models, making AttNHP an ideal candidate for acceleration via TPP-SD, where it achieves the largest speedup. Despite architectural similarities between THP and SAHP, implementation differences result in SAHP consistently achieving the shortest wall-time for both AR sampling and TPP-SD, leaving less room for additional acceleration.
D.5 Differences between TPP Sampling and Long-horizon Prediction
While both TPP sampling and long- monghooizon prediction 35 24 involve generating sequences of events over continuous time interval given some history, they have fundamental differences. TPP sampling focuses on generating events from the correct distribution. Since sampling is inherently stochastic, comparing the exact events sampled with specific test set events isn't meaningful the same distribution that we sample from can produce different event in each iteration. What matters is whether the sampled sequences follow the correct statistical patterns. On synthetic datasets, we evaluate the distributional conformity of sampled sequences from TPP-SD to the ground truth using 2 and KS statistic. On real datasets, we measure the distributional discrepancy between the sampled sequences from autoregressive sampling and TPP-SD with Wasserstein distance.
In contrast, long-horiae Long-horizon prediction specifically aims to forecast future events that closely match the ground truth on the set. It's evaluated by metrics such as RMSE and optimal transport distance (OTD) that directly measure the discrepancy between predicted sequences and test-set sequences.
D.6 Limitations
Compared the thinning algorithm, our proposed TPP-SD is deep-learning based which requires large amount of event sequence data to learn the model's parameters. Therefore, the model is less suitable for sampling in data-scarce scenarios.
Besides, TPP-SD adopts the original SD framework from the LLM domain and requires deploying two models at the same time for accelerating sampling, which is slightly more computational demanding and less convenient. Recent advances SD techniques [3] 16 can be incorporated to further optimize TPP-SD, and we leave this as future work.
29


Reviews:
[{"review_id": "CH4XOthoje", "reviewer": "Reviewer_HqH3", "summary": "This paper proposes TPP-SD, a novel method that accelerates Transformer-based temporal point-process sampling through speculative decoding. The framework first leverages a smaller draft model to generate multiple candidate events, then uses a larger target model to verify them. Experiments demonstrate a 2 – 6 × speed-up on both synthetic and real datasets.", "questions": "- Can you report the performance of small model, such as the draft model on those datasets?", "limitations": "yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n- The proposed approach achieves 2 – 6× faster sampling with virtually no loss in ΔL or Wasserstein quality.\n\n- The authors conduct extensive experiments span both synthetic and real datasets, include ablations over γ, draft size, and event-type count K, and consistently confirm the speed–quality trade-off.\n\n**Weaknesses**\n\n- Limited baseline: The authors do not report the performance/speed of draft model on those datasets. It is not clear whether the draft model can already work well on this task efficiently. Notably, as show in Table 3, it seems speculative decoding with smaller draft may achieve better speed up. Therefore, it seems smaller model such as draft model size can also work well on this task. \n\n- The proposed approach does not appear practical for real-world applications. Table 2 shows that the speed-up diminishes as K increases, and in real-world settings K is typically much larger. It is therefore unclear whether the method would remain effective in such scenarios.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your valuable comments and insightful questions. We are encouraged by your recognition on the fidelity-preserving ability and the efficiency of TPP-SD. We hope that our following clarifications and new experiments can address your concerns.\n\n> **Q1: Limited baseline: The authors do not report the performance/speed of draft model on those datasets. It is not clear whether the draft model can already work well on this task efficiently. Notably, as show in Table 3, it seems speculative decoding with smaller draft may achieve better speed up. Therefore, it seems smaller model such as draft model size can also work well on this task.**\n\n**A1:** We provide clarification on two key points:\n\n1. For TPP data, large Transformer encoders (e.g., multi-layer, multi-head, with high-dimensional representations) are crucial for capturing complex temporal dependencies. Their larger parameter capacity allows them to model intricate data distributions and generate high-quality samples—albeit at the cost of slower autoregressive sampling.\n\nIn contrast, smaller Transformer encoders (e.g., single-layer, single-head, with low-dimensional representations) are much faster at sampling due to reduced parameter capacity. However, they often struggle to model complex temporal patterns accurately, resulting in lower-quality samples that may not align well with the underlying data distribution [1,2,3]. \n\nTo summarize, while using a draft model for autoregressive sampling is indeed faster than using a target model, this speed advantage does not imply that the draft model alone is sufficient for high-quality modeling. Its limited expressiveness prevents it from capturing the true complexity of the data, which is why it is primarily used to assist rather than replace the target model. \n\n2. Regarding the comment on “Notably, as show in Table 3......” we believe this is a misunderstanding. A smaller draft model indeed speeds up the speculative decoding process due to faster \"draft\" generation, but this can potentially lead to a lower acceptance rate. Nonetheless, our experiments show that, in general, smaller draft models do accelerate speculative decoding. However, as noted above, this does not imply that the draft model alone can achieve high-quality sampling, as its capacity is insufficient to model complex temporal patterns accurately. \n\n> **Q2: The proposed approach does not appear practical for real-world applications. Table 2 shows that the speed-up diminishes as K increases, and in real-world settings K is typically much larger. It is therefore unclear whether the method would remain effective in such scenarios.**\n\n**A2:** We respectfully disagree with this assessment. Our evaluation includes four established real-world datasets with substantial event type cardinalities ($K=16 \\sim 22$), which are standard benchmarks in the TPP literature (Appendix B.2). These datasets represent realistic complexity encountered in practice. While we observe slight negative correlation between dataset cardinality $K$ and speed-up, the speed-up remains significant even for large $K$ (e.g., $5.849 \\times$ speedup on Amazon with $K=17$, and $4.290\\times$ speedup on Stackoverflow with $K=22$).\n\n> **Q3: Can you report the performance of small model, such as the draft model on those datasets?**\n\n**A3:** We evaluate the draft model $\\mathcal{M}\\_D$ (single-layer, single-head THP[1]) on both synthetic (Poisson, Hawkes, Multi-Hawkes) and real datasets (Taxi, StackOverflow, Amazon, Taobao), measuring sampling fidelity via $\\Delta \\mathcal{L}$ and $D\\_{KS}$/$D\\_{WS}$, and efficiency via wall-time.\n\nThe simplistic architecture of $\\mathcal{M}\\_D$ struggles to capture temporal dependencies effectively even on synthetic datasets. Performance degrades further on real datasets, with significantly larger $\\Delta \\mathcal{L}$ and $D\\_{WS}$ values, as $\\mathcal{M}\\_D$ cannot model the complex dependencies present in the data. Draft model alone performs poorly on this task, demonstrating the necessity of a large target model and the importance of our proposed TPP-SD framework, which leverages the speed of the draft model without sacrificing the fidelity provided by the target model. \n\n| Metric               | Sampling Type | Poisson | Hawkes | Multi-Hawkes |\n| -------------------- | ------------- | ------- | ------ | ------------ |\n| $\\Delta \\mathcal{L}$ | AR (draft)    | 2.174   | 1.348  | 1.305        |\n|                      | AR (target)   | 0.542   | 0.753  | 0.022        |\n|                      | TPP-SD        | 0.349   | 0.276  | 0.321        |\n| $D_{KS}$             | AR (draft)   | 0.153   | 0.141  | 0.126        |\n|                      | AR (target)   | 0.038   | 0.044  | 0.069        |\n|                      | TPP-SD        | 0.036   | 0.043  | 0.053        |\n| Wall-time            | AR (draft)    | 0.987   | 1.240  | 1.530        |\n|                      | AR (target)   | 3.477   | 5.147  | 4.007        |\n|                      | TPP-SD        | 1.647   | 2.547  | 1.893        |\n\n| Metric               | Sampling Type | Taxi  | StackOverflow | Amazon | Taobao |\n|:-------------------- |:------------- | -----:| -------------:| ------:| ------:|\n| $\\Delta \\mathcal{L}$ | AR (draft)    | 1.504 | 1.136         | 0.614  | 2.122  |\n|                      | AR (target)   | 0.441 | 0.587         | 0.056  | 0.446  |\n|                      | TPP-SD        | 0.065 | 0.231         | 0.129  | 0.033  |\n| $D^t_{WS}$           | AR (draft)    | 0.765 | 0.994         | 0.534  | 0.577  |\n|                      | AR (target)   | 0.201 | 0.470         | 0.189  | 0.236  |\n|                      | TPP-SD        | 0.082 | 0.391         | 0.078  | 0.076  |\n| $D^k_{WS}$           | AR (draft)    | 0.768 | 0.821         | 0.759  | 1.384  |\n|                      | AR (target)   | 0.055 | 0.376         | 0.184  | 0.267  |\n|                      | TPP-SD        | 0.655 | 0.375         | 0.418  | 0.751  |\n| Wall-time            | AR (draft)    | 0.150 | 0.120         | 0.070  | 0.410  |\n|                      | AR (target)   | 1.157 | 1.353         | 1.023  | 5.890  |\n|                      | TPP-SD        | 0.453 | 0.700         | 0.290  | 3.460  |\n\nReferences:\n\n[1] Transformer Hawkes Process, ICML, 2020.\n\n[2] Self-Attentive Hawkes Process, ICML, 2020.\n\n[3] Transformer Embeddings of Irregularly Spaced Events and Their Participants, ICLR, 2022."}, {"review_id": "HlUjhGOiSs", "reviewer": "Reviewer_RM4F", "summary": "This paper introduces TPP-SD, an approach that accelerates sampling in Transformer-based temporal point processes by adapting speculative decoding techniques originally developed for large language models. The core idea is to use a smaller draft model to generate multiple candidate events, which are then validated in parallel by a larger, more accurate target model, leveraging the similarities between the thinning algorithm for point processes and speculative decoding for autoregressive generative models. The authors design a CDF-based Transformer TPP amenable to the speculative decoding mechanism, propose an efficient algorithm for continuous-time event sampling, and evaluate the approach on both synthetic and real-world datasets. The experimental results indicate that TPP-SD achieves 2–6× speedups compared to standard autoregressive sampling, while producing event sequences virtually indistinguishable in distribution from those produced by the standard method. The authors further investigate the impact of draft length and draft model size on quality and efficiency.", "questions": "Q1: In theory, the output distributions of TTP-SD and AR sampling should be identical, as the author acknowledges in Section 5.3. So why do these two methods result in differences in fidelity? If these differences arise from stochastic fluctuations, then reporting these metrics might lack the necessary motivation.\n\nQ2: Does TTP-SD and AR sampling support parallel inference in batch samples, and in this case, will the speedup be limited?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Summary Of Strengths\n- Technical Soundness and Theoretical Clarity: The paper provides a well-grounded adaptation of speculative decoding to TPP sampling. The mathematical treatment of the adjusted distribution for continuous variables, as well as the connection to LLM speculative decoding, are clearly motivated, and theoretical correctness proofs are outlined in the main text (Section 4.3) and referenced in the Appendix.\n- Clarity and Organization: The paper is mostly well-structured and the technical writing is generally clear, with necessary background provided in Section 2 and related work in Section 3. The pseudo-code and algorithm descriptions, while referenced as being in the appendix, are described concisely in the main paper.\n\nSummary Of Weaknesses\n- Limited Novelty Beyond Application: While the analogy between thinning and speculative decoding is insightful, the core algorithmic advance is a relatively direct adaptation of existing SD techniques from the LLM domain. The paper does not extensively discuss unique challenges or TPP-specific adaptations (beyond continuous CDF normalization) that set it apart as a fundamentally new algorithm.\n- Limited Baseline: The author lacks an explanation of the acceleration-related background in related work (in the AR sampling or TTP fields). This also affects the choice of baseline, as there is a lack of comparison with acceleration techniques related to AR models, such as RNN/LSTM, which can provide acceleration but pose a risk of performance loss.", "was_revised": true, "final_justification": "Strengths\n- The paper provides a well-grounded adaptation of speculative decoding to TPP sampling. The mathematical treatment of the adjusted distribution for continuous variables, as well as the connection to LLM speculative decoding, are clearly motivated, and theoretical correctness proofs are outlined in the main text (Section 4.3) and referenced in the Appendix.\n\nWeaknesses\n- While the analogy between thinning and speculative decoding is insightful, the core algorithmic advance is a relatively direct adaptation of existing SD techniques from the LLM domain. \n\n- The author lacks an explanation of the acceleration-related background in related work (in the AR sampling or TTP fields). This also affects the choice of baseline, as there is a lack of comparison with acceleration techniques related to AR models, such as RNN/LSTM, which can provide acceleration but pose a risk of performance loss.", "rebuttal": "We are genuinely grateful for your valuable feedback. We are encouraged by your positive comments on our technical soundness, overall clarity and organization. We sincerely hope our clarifications below can address your concerns. \n\n> **Q1: Limited Novelty Beyond Application: While the analogy between thinning and speculative decoding is insightful, the core algorithmic advance is a relatively direct adaptation of existing SD techniques from the LLM domain. The paper does not extensively discuss unique challenges or TPP-specific adaptations (beyond continuous CDF normalization) that set it apart as a fundamentally new algorithm.**\n\n**A1:** While we agree that our work is inspired by the framework of SD from the LLM domain, we would like to clarify that this process is not a straightforward adaptation, and requires overcoming fundamental challenges unique to TPPs.\n\n- Unlike LLMs, which only involve sampling from discrete token spaces, TPPs require sampling both continuous timestamps and discrete event types. While the sampling of discrete event types is analogous to token generation in LLMs, the continuous timestamp sampling is a key distinction and a unique challenge specific to TPPs. \n\nIn particular, our method must sample from adjusted continuous distributions after rejection (see Equation 3), which is not addressed in prior SD literature. To handle this, we propose a novel acceptance-rejection mechanism (Theorem 1) that enables efficient sampling without requiring expensive numerical integration. This constitutes a fundamental advancement in applying speculative decoding to continuous variable generation and is essential for enabling high-fidelity, efficient sampling in TPPs. \n\n- Existing Transformer TPPs[2,3,4] use conditional intensity function (CIF) parameterization, fundamentally incompatible with SD (Appendix D.1). CDF-based alternatives are easier to sample from, but the mainstream implementation[1] relies on the RNN architecture, which is unsuitable for parallel verification. We therefore designed a CDF-based Transformer architecture (Section 4.2) enabling both high-quality modeling and SD compatibility—a non-trivial contribution requiring careful design. \n\n> **Q2: Limited Baseline: The author lacks an explanation of the acceleration-related background in related work (in the AR sampling or TPP fields). This also affects the choice of baseline, as there is a lack of comparison with acceleration techniques related to AR models, such as RNN/LSTM, which can provide acceleration but pose a risk of performance loss.**\n\n**A2:** We appreciate the reviewer's comment and **will include acceleration-related work on AR sampling** (such as advances in SD in LLM domain). To the best of our knowledge, our work is the first systematic approach to accelerating Transformer-based TPP sampling while preserving distributional fidelity, as the TPP literature has primarily focused on model expressiveness and fitting quality rather than sampling efficiency.\n\nWhile using a recurrent architecture like LSTM represents one possible approach to the speed-performance trade-off, our work is motivated by a different premise. The community has validated Transformer TPPs' superior performance in capturing complex dependencies [2,3,4]. Rather than re-evaluating architectural trade-offs, we aimed to enable significant acceleration without compromising the high generative fidelity that makes these SOTA models preferable in the first place. \n\n> **Q3: In theory, the output distributions of TPP-SD and AR sampling should be identical, as the author acknowledges in Section 5.3. So why do these two methods result in differences in fidelity? If these differences arise from stochastic fluctuations, then reporting these metrics might lack the necessary motivation.**\n\n**A3:** The minor differences in fidelity metrics are indeed due to stochastic fluctuations inherent in the sampling process. Our motivation for reporting $\\Delta \\mathcal{L}$ and $D_{KS}$ / $D_{WS}$ is to provide a empirical validation that samples generated by TPP-SD are identical in distribution to those generated by AR sampling. As long as these metrics are statistically close to zero, we demonstrate that TPP-SD achieves speedup without distorting the sampling distribution.\n\n> **Q4: Does TPP-SD and AR sampling support parallel inference in batch samples, and in this case, will the speedup be limited?**\n\n**A4:** We have not explored batch inference for TPP-SD in this work, as we follow the standard SD framework introduced in [5], which operates with a batch size of 1.\n\nIn the speculative decoding literature, enabling batch inference (i.e., batch size \\( > 1 \\) remains an active research topic due to the inherent challenge of synchronizing multiple sequences within a batch. This complexity also applies to TPP-SD, where each sequence may have different lengths and rejection patterns, making parallel decoding non-trivial.\n\nWe consider extending TPP-SD to support efficient batch inference a promising direction for future work.\n\nReferences:\n\n[1] Intensity-Free Learning of Temporal Point Processes, ICLR, 2020.\n\n[2] Transformer Hawkes Process, ICML, 2020.\n\n[3] Self-Attentive Hawkes Process, ICML, 2020.\n\n[4] Transformer Embeddings of Irregularly Spaced Events and Their Participants, ICLR, 2022.\n\n[5] Fast Inference from Transformers via Speculative Decoding, ICML, 2023."}, {"review_id": "COyIHW1Mdf", "reviewer": "Reviewer_k4Sa", "summary": "The paper proposes TPP-SD by connecting thinning in Temporal Point Process and Speculative Decoding. By unifying the two processes in the same proposal-verification framework, the paper introduces a draft model for the proposal distribution in the thinning process. The paper also extends the original speculative decoding algorithm to rejection sampling on continuous CDFs. The experiments support the authors' claim that TPP-SD can achieve sampling speedup with small degradation to quality on both synthetic TPP data and real time series data.", "questions": "- One baseline I'm curious about is a thinning baseline: What happens if the proposal distribution is something simple, such as a Poisson process? This is a single-parameter draft model, where you can estimate the parameter with MLE on the data. When evaluation computation is not the bottleneck, this method should also provide a lot of speedup.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Quality\n- The paper is technically sound, and the claims are well supported with theory and experiments.\n\n# Clarity\n- The paper is very clearly written. I especially enjoyed the section that draws the connection between TPP and SD and found it highly educational.\n\n# Significance\nStrengths:\n- While the techniques (thinning and Speculative Decoding) are not new, the paper points out the connection between thinning and SD, which is novel to the audience who might be familiar with individual subfields. This is very significant since it will encourage more cross-pollination between the fields of TPP and LLM sampling. This alone makes me feel inclined to accept this paper.\n- The experiments support the authors' claims very well, and the theory behind the algorithm is sound.\n\nWeaknesses:\n- The experiments lack some important baseline for it to be practical, namely, directly using a simple TPP as the draft model. See the questions section for more details. This causes me to hesitate from giving a strong acceptance, but I'm happy to increase my score if additional experiments are conducted.\n\n# Originality\n- The connection between TPP and SD is very novel, and this is the first time SD is applied to TPP as far as I know (low confidence).", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal and I felt confident to accept this paper.", "rebuttal": "We are sincerely grateful for your enthusiastic and positive feedback on our technical soundness, insightful investigations, extensive experiments, and good writing, especially for recognizing the novelty of TPP-SD in connecting thinning algorithm with speculative decoding. We hope that our following clarifications and new experiments can address your concerns.\n\n> **Q1: What happens if the proposal distribution is something simple, such as a Poisson process? This is a single-parameter draft model, where you can estimate the parameter with MLE on the data. When evaluation computation is not the bottleneck, this method should also provide a lot of speedup.**\n\n**A1:** This is a very interesting suggestion. \n\nWe replace the 1-head-1-layer (abbreviated as 1-H-1-L) draft model to a simple **homogeneous Poisson process** draft model, whose parameter is estimated by MLE on the data, i.e., $\\hat\\lambda = \\frac{n}{T}$, where $n$ is the number of events and $T$ is the time span of the dataset. \nAs for the experiment setup, we adopt THP as the backbone of target model for illustration simplicity. We conduct the additional experiments on 3 synthetic datasets (i.e., Poisson, Hawkes, and Multi-Hawkes) and 2 real-world datasets (i.e., Taobao and Amazon).\n\nSince Poisson process is history-independent, we can sample the candidate inter-event times $\\tau_i$ **all at once** from $\\text{Exp}(\\hat\\lambda)$. It is worth noting that for TPPs with marks (e.g. Multi-Hawkes), to ensure an efficient drafting process, we first estimate the mark distribution from the training data using MLE (i.e., $\\hat{p}_m = \\frac{n_m}{n}$, where $n_m$ is the number of events with mark $m$ and $n$ is the total number of events), and then sample the marks from the estimated mark distribution **all at once** as well. \n\nWe can see from the table that: on synthetic datasets, TPP-SD with Poisson draft maintains high fidelity in sampling while achieving even more significant speedup across three datasets. The fast drafting process of Poisson compensates for the relatively lower acceptance rate. However, on real-world datasets, while TPP-SD with Poisson draft retains the sampling fidelity, we witness drop in speedup ratio. On complex real-world TPPs, the acceptance rate of Poisson draft is too low that it overwhelms the speedup from fast drafting. Therefore, in our original setting of TPP-SD, we use a more robust drafting strategy, i.e., a slightly more complex draft model (1-layer-1-head THP) to ensure that TPP-SD can achieve high fidelity sampling across a wide range of datasets. \n\nThis question is highly insightful and has greatly inspired us. We will include this additional experiment in the camera-ready version.\n\n| Dataset      | Sampling Method               | $\\Delta \\mathcal{L}$ (↓) | $D_{KS}$ (↓) | $D_{WS}^t$ (↓) | $D_{WS}^k$ (↓) | Acceptance Rate $\\alpha$ (↑) | Speedup Ratio $S_{AR/SD}$ (↑) |\n| ------------ | ----------------------------- | ------------------------ | ------------ | -------------- | -------------- | ---------------------------- | ----------------------------- |\n| Poisson      | AR Sampling                   | 0.542                    | 0.038        | -              | -              | -                            | -                             |\n|              | TPP-SD (1-H-1-L draft) | 0.349                    | 0.036        | -              | -              | 0.740                        | 2.110                         |\n|              | TPP-SD (Poisson draft)        | 0.630                    | 0.050        | -              | -              | 0.370                        | **3.230**                     |\n| Hawkes       | AR Sampling                   | 0.753                    | 0.044        | -              | -              | -                            | -                             |\n|              | TPP-SD (1-H-1-L draft) | 0.276                    | 0.043        | -              | -              | 0.720                        | 2.113                         |\n|              | TPP-SD (Poisson draft)        | 0.286                    | 0.036        | -              | -              | 0.520                        | **5.430**                     |\n| Multi-Hawkes | AR Sampling                   | 0.022                    | 0.069        | -              | -              | -                            | -                             |\n|              | TPP-SD (1-H-1-L draft) | 0.321                    | 0.053        | -              | -              | 0.750                        | 2.117                         |\n|              | TPP-SD (Poisson draft)        | 0.268                    | 0.057        | -              | -              | 0.390                        | **3.610**                     |\n| Taobao       | AR Sampling                   | 0.446                    | -            | 0.236          | 0.267          | -                            | -                             |\n|              | TPP-SD (1-H-1-L draft) | 0.033                    | -            | 0.076          | 0.751          | 0.260                        | **1.597**                     |\n|              | TPP-SD (Poisson draft)        | 0.680                    | -            | 0.091          | 0.623          | 0.030                        | 1.370                         |\n| Amazon       | AR Sampling                   | 0.056                    | -            | 0.189          | 0.184          | -                            | -                             |\n|              | TPP-SD (1-H-1-L draft) | 0.129                    | -            | 0.078          | 0.418          | 0.270                        | **3.550**                     |\n|              | TPP-SD (Poisson draft)        | 0.066                    | -            | 0.090          | 0.086          | 0.010                        | 2.230                         |"}, {"review_id": "Brf1NBe5a2", "reviewer": "Reviewer_yB4e", "summary": "The paper introduces TPP-SD, a speculative decoding framework for accelerating Transformer-based temporal point process (TPP) sampling by drawing inspiration from the structural parallels between thinning algorithms and speculative decoding in language models. By employing a lightweight draft model to generate multiple candidate events and leveraging a larger target model for joint verification of event intervals and types through a mixture of log-normal CDFs and categorical distributions, TPP-SD achieves significant speedup—2–6× across synthetic and real datasets—while maintaining distributional fidelity via adaptive rejection sampling and an adjusted distribution mechanism.", "questions": "1.After drafting candidate events autoregressively, how is the target model ensured to verify all candidates in parallel? In the speculative decoding of LLM, a common practice is to concatenate all candidates into a single vector and enable parallel verification of the target model by modifying the attention map. Does TPP-SD adopt a similar approach? I would appreciate it if you could provide a more detailed explanation.\n\n\n2.In Table 1 and Table 2, for metrics such as Likelihood Discrepancy, KS statistic, and Wasserstein Distance, TPP-SD sometimes achieves better values than AR Sampling. Why does TPP-SD sometimes exhibit higher sample quality than AR Sampling? In my understanding, TPP-SD employs a draft model with fewer parameters, which achieves acceleration while minimizing the sacrifice in output accuracy. However, the current observation that TPP-SD sometimes even attains higher output accuracy has left me confused. If there are any underlying issues or nuances I may have overlooked, I would appreciate it if you could clarify or point them out.", "limitations": "The authors have listed the limitations of the current work in Section D.4 Limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.This work applies Speculative Decoding to TTP sampling for the first time, which has the potential to inspire subsequent research.\n\n2.This work cleverly uncovers the structural similarities between thinning algorithms for TPPs and speculative decoding for language models. It further demonstrates the adaptive verification and resampling strategy that maintains the same output distribution as autoregressive sampling when both discrete and continuous distributions coexist.\n\n3.Sufficient experiments on both synthetic and real datasets demonstrate the acceleration effect, while also verifying that the impact on sampling accuracy is minimal.\n\nWeaknesses:\n\n1.In Section 4.2, formulas are listed to describe the structure of CDF-based Transformer TPPs, which is somewhat complex. Inserting a diagram here would enhance readability and clarify the structural details, making it easier for readers to follow.\n\n2.The experimental section does not compare the sample quality and sampling speed of TPP-SD with other related TPP works under similar settings, which fails to intuitively demonstrate the advantages of TPP-SD compared to existing approaches.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal\nStrength:\n\n1.First to integrate Speculative Decoding with TTP sampling, offering a new research direction.\n\n2.Links TPP thinning to speculative decoding, enabling an adaptive verifier that preserves the exact joint discrete/continuous output distribution.\n\n3.Experiments on synthetic and real data confirm speed-ups with negligible loss in accuracy.", "rebuttal": "We sincerely thank you for your precious time and valuable comments. We are encouraged by your positive comments on our research novelty, technical soundness and extensive experiments, especially the recognition of unveiling the structural similarity between thinning algorithm and speculative decoding. We hope that our following clarifications can address your concerns.\n\n> **Q1: In Section 4.2, formulas are listed to describe the structure of CDF-based Transformer TPPs, which is somewhat complex. Inserting a diagram here would enhance readability and clarify the structural details, making it easier for readers to follow.**\n\n**A1:** Sure, we will include a diagram in Section 4.2 to visually represent the structure of CDF-based Transformer TPPs in the camera-ready version.\n\n> **Q2: The experimental section does not compare the sample quality and sampling speed of TPP-SD with other related TPP works under similar settings, which fails to intuitively demonstrate the advantages of TPP-SD compared to existing approaches.**\n\n**A2:** To the best of our knowledge, existing literature in the community of TPP has primarily focused on improving model fitting and parameter estimation, with little emphasis on sampling acceleration. TPP-SD is the first work to accelerate high-performance Transformer-based TPP sampling. As such, there are no existing works that provide a direct comparison with TPP-SD.\n\n> **Q3: After drafting candidate events autoregressively, how is the target model ensured to verify all candidates in parallel? In the speculative decoding of LLM, a common practice is to concatenate all candidates into a single vector and enable parallel verification of the target model by modifying the attention map. Does TPP-SD adopt a similar approach? I would appreciate it if you could provide a more detailed explanation.**\n\n**A3:** Similar to [1], we adopt a similar parallel verification approach. We concatenate candidate events with the prefix (generated events) into a single vector and feed it to the target model. During parallel verification of the target model, we maintain the causal attention map (lower triangular matrix) to ensure that when computing probabilities for candidate events, each event strictly attends to previous events in the draft sequence.\n\n> **Q4: In Table 1 and Table 2, for metrics such as Likelihood Discrepancy, KS statistic, and Wasserstein Distance, TPP-SD sometimes achieves better values than AR Sampling. Why does TPP-SD sometimes exhibit higher sample quality than AR Sampling? In my understanding, TPP-SD employs a draft model with fewer parameters, which achieves acceleration while minimizing the sacrifice in output accuracy. However, the current observation that TPP-SD sometimes even attains higher output accuracy has left me confused. If there are any underlying issues or nuances I may have overlooked, I would appreciate it if you could clarify or point them out.**\n\n**A4:** This is a keen observation. The reviewer is correct that the slight variations in sample quality metrics are indeed due to the stochasticity of the sampling process over a finite number of runs. Though it seems that TPP-SD sometimes achieves better scores than AR sampling (or vice versa) on $\\Delta \\mathcal{L}$, $D_{KS}$, or $D_{WS}$, these differences does not imply that one method is superior to the other in sampling fidelity. As long as these metrics for both methods are statistically close to zero, it empirically demonstrates that sampled events from TPP-SD are distributionally identical to those from AR sampling, i.e., preserving the target model's high fidelity. \n\nWe also want to emphasize that our TPP-SD method does not involve any \"sacrifice\" in generative quality. The verification step is theoretically guaranteed to ensure that the distribution of the final output samples matches that of standard autoregressive sampling. \n\nReferences:\n\n[1] Fast Inference from Transformers via Speculative Decoding, ICML, 2023."}]

Author rebuttal:

Meta-review: This paper integrates speculative decoding techniques into transformer-based temporal point-process sampling, which achieves 2 to 6 times speed-up on both synthetic and real datasets. All the reviewers appreciated the methodological contributions in the paper, although the idea might not be utterly novel.

Several concerns are lacking baselines and insufficient discussion in the related work on AR sampling. During the rebuttal, authors provided additional numerical results to lift these concerns. Although formally incorporating these results require some extra work, it is believed that these results can be added within a short period. Therefore, I recommend acceptance of the paper.
Decision: Accept (poster)