Conformal Anomaly Detection in Event Sequences
Authors: Shuai Zhang
Chuan Zhou
Yang Liu
Peng Zhang
Xixun Lin
Shirui Pan
Venue: ICML 2025 poster (2025)
Keywords: Conformal Inference
Anomaly Detection
Event Sequence
Temporal Point Process
Primary area: general_machine_learning->sequential_network_and_time_series_modeling
TL;DR: We propose a novel conformal anomaly detection method for event sequences, which combines two newly designed non-conformity scores with provably valid p-values for hypothesis testing.

Abstract: Anomaly detection in continuous-time event sequences is a crucial task in safety-critical applications. While existing methods primarily focus on developing a superior test statistic, they fail to provide guarantees regarding the false positive rate (FPR), which undermines their reliability in practical deployments. In this paper, we propose CADES (Conformal Anomaly Detection in Event Sequences), a novel test procedure based on conformal inference for the studied task with finite-sample FPR control. Specifically, by using the time-rescaling theorem, we design two powerful non-conformity scores tailored to event sequences, which exhibit complementary sensitivities to different abnormal patterns. CADES combines these scores with Bonferroni correction to leverage their respective strengths and addresses non-identifiability issues of existing methods. Theoretically, we prove the validity of CADES and further provide strong guarantees on calibration-conditional FPR control. Experimental results on synthetic and real-world datasets, covering various types of anomalies, demonstrate that CADES outperforms state-of-the-art methods while maintaining FPR control.

Conformal Anomaly Detection in Event Sequences
Shuai Zhang' Chuan Zhou
12
Yang Liu
Peng Zhang? Xixun Lin Shirui Pan
Abstract
Detecting anomalies in event data plays a vital role in safety- critical applications, including information security, finance, and healthcare. For example, rapid information spreading may signal rumors (Naumzik & Feuerriegel, 2022), abnor- mal transaction activities may reveal fraud (Zhu et al., 2020), and irregular patient health records may suggest rare medi- cal conditions (Liu & Hauskrecht, 2021).
Anomaly detection in continuous-time event se- quences is a crucial task in safety-critical appli- cations. While existing methods primarily focus on developing a superior test statistic, they fail to provide guarantees regarding the false posi- tive rate (FPR), which undermines their reliability in practical deployments. In this paper, we pro- pose CADES (Conformal Anomaly Detection in Event Sequences), a novel test procedure based on conformal inference for the studied task with finite-sample FPR control. Specifically, by us- ing the time-rescaling theorem, we design two powerful non-conformity scores tailored to event sequences, which exhibit complementary sensi- tivities to different abnormal patterns. CADES combines these scores with Bonferroni correc- tion to leverage their respective strengths and addresses non-identifiability issues of existing methods. Theoretically, we prove the validity of CADES and further provide strong guarantees on calibration-conditinaall FPR control. Experimen- tal results synthetic and real-world datasets, covering various types of anomalies, demonstrate that CADES outperforms state-of-the- -hrtt---tttttttt methods while maintaining FPR control.
Continuous time event sequences are commonly modeled using temporal point processes (TPPs), a class of stochas- tic processes that characterize the random occurrence of events (Zhang et al., 2020). Given access to normal training data, i.e., in-distribution (ID) data, previous works (Shchur et al., 2021: Zhang et al., 2023) frame the task of detecting anomalous event sequences as an uutoo--distribution (OOD) detection problem for TPPs. They perform hypothesis test- ing to determine whether event sequence is drawn from the unknown data-generating TPP. While these studies pri- marily focus on developing superior test statistic, they lack rigorous theoretical guarantees for controlling the false pos- itive rate (FPR, i.e., the probability of misclassifying an ID sample as OOD). Such guarantees, however, are erucial for the deployment OOD detection methods in safety-critical applications (Kaur et al., 2022: Magesh et al., 2023).
Conformal inference (a.k.a. conformal prediction, Vovk et al. (2005)) provides flexible framework for establishing the desired FPR guarantees. Its application in OOD detec- tion is often referred to as conformal OOD detection (Kaur et al., 2022) or conformal anomaly detection (Smith et al., 2015). The detection performance largely depends on the choice of the non-conformity score' (Kaur et al., 2022: An- gelopoulos et al., 2024), which quantifies how different an imput is from the ID samples. Recent advances in this field have received significant attention (Bates et al., 2023; Ma et al., 2024; Marandon et al., 2024), but these approaches have mainly focused on image data, leaving continuous-tin event sequence data unexplored.
1. Introduction
Event sequences consist of the timestamps and marks of discrete events occurring in continuous time. They are abun- dant and ubiquitous in our daily life. Examples include news dissemination on social networks (Trivedi et al., 2019), trading actions in stock markets (Zuo al., 2020), and elec- tronic health records medical systems (Shi al., 2023).
AAcadeyy of Mathematics and Systems Science, Chinese Academy of Sciences "School of Cyber Security, University of Chinese Academy of Sciences 'Cyberspace Institute of Advanced Technology. Guangzhou University "Institute of Information Engi- necring, Chinese Academy of Sciences è *Griffith University. Corre- spondence to: Chuan Zhou <zhouchuan @ @c.c>.
In this paper, we aim to extend conformal inference to anomaly detection in event sequences. Common test statis- tics for event sequences. such as the Kolmogorov-Smiinov (KS) statistic (Barnard, 1953) and the sum-of-squared spacings statistic (Shchur et al., 2021), suffer from severe Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
'In this paper, we use the terms non-conformity score and test statistic interchangeably.

---

Conformal Anomaly Detection in Event Sequences
non-identifiabilty issues (Zhang et al., 2024b), producing similar values for two markedly distinct sequences. This significantly reduces their detection power, motivating to design more expressive non-conformity score. Due to the variable length of event sequences and the intricate dependencies between events, the design of this score is both essential and non-trivial.
2. Preliminaries
Temporal Point Processes. A temporal point process (TPP, Daley et al. (2003)) is a stochastic process whose real- ization is an event sequence X = {(t, m.)] where t [0, T] are the event timestamps with t, < f4+1, and my E . = {1,...,,, M) are the event types (marks). It can be equivalently represented as M counting processes {Nm (t) ( where Nm (t) denotes the number of type-m events occurrig up to time t. The most common way to characterize TPP is through conditional intensity functions (CIFs), defined for each event type m E .M as follows:
To this end, we introduce CADES (Conformal Anomaly De- tection in Event Sequences), novel method based con- formal inference for detecting anomalous event sequences. Specifically, we design two powerful non-conformity scores tailored to event sequences using the time-rescaling theo- rem (Brown et al., 2002). These scores are complementary in their sensitivity to different abnormal patterns and address the limitations mentioned above of existing test statistics. Unlike standard conformal inference methods, which rely on one-sided p-values and a single non-conformity score, CADES utilizes two-sided p-values and applies the Bonfer- roni correction (Shaffer, 1995) to combine both scores. This is because two-sided p- p-values are crucial, as we show that both small and large values of the proposed scores can indi- cate OOD sequences. Moreover, by combining both scores, CADES fully exploits the strengths of each, enabling more accurate anomaly detection. Theoretically, we prove the validity of CADES and provide guarantees on calibration- conditional FPR, which are stronger than marginal FPR guarantees averaged over all possible calibration sets.
X (1) di E [d. (1) H,r],
(I)
where H.M. = [(t,, m.) tt < t, m, E MM represents the historical events of all types before time t, and the symbol indicates that the intensity is conditioned on H,M. This CIF describes the expected instantaneous rate of happening next type-m event given historical events.
Traditional TPP models include Poisson processes (King- man, 1992), Hawkes processes (Hawkes, 1971), and self- correcting processes (Isham & Westcott, 1979). Recently, neural TPPs, which parameterize the CIFs using neural networks such as RNNs or Transformers, have gained sig- nificant attention (Mei & Eisner, 2017; Zuo et al., 2020).
Conformal Anomaly Detection. We focus on the com- putationally efficient inductive (split) conformal anomaly detection method (Laxhammar & Falkman, 2015), which divides the clean training dataset D into two disjoint subsets: proper training set Darain and a calibration set Dest- First, a model is trained on Drain to define a non-conformity score X + R. The calibration scores {s(Xaa)) : Xeat Deat and the test score s(X1es) are then calculated. Next, the classical conformal p-value (Vovk et al., 2005) for the test sample Xles is given by:
We summarize our contributions as follows:
We establish the connection between conformal inference and anomaly detection in event sequences. Building upon this, we propose CADES, a novel test procedure that com- bines two proposed scores with Bonferroni correction to perform hypothesis testing, offering statistically sound anomaly detection method.
1[Xeal E Deal : :((eea))))) S s(Xaa) +1 +1
We design two new powerful non-conformity scores tai- lored to continuous-time event sequences. We demon- strate that these scores complement each other in terms of sensitivity to different anomalies and address non- identifiability issues of existing test statistics.
p(Xust)
(2)
[Deal +1
If the elements of Deal U {X are exchangeable, i.e., their joint distribution is invariant under any permutation, then the conformal p-value valid, ensuring that the FPR con- trolled (Papadopoulos et al., 2002). In this case, if p( ((est) is smaller than a pre-specified significance level a E (0. 1), X is classified as an anomaly.
We prove the validity of the p-values used in CADES. This guarantees controlling the marginal FPR pre- specified level. We further provide theoretical guarantees on dalibrattio-condiiionn FPR, which are stronger than marginal FPR guarantees.
Problem Statement. Assuming that normal training data is available, detecting anomalous event sequences can be viewed as an OOD detection problem. Concretely, given a set of event sequences D = {X+)==_ that are i.i.d. samples from unknown distribution Px, the objective is to iden- tify whether new independent observation Xtest is OOD, in the sense that it was not drawn from Px. This problem can be framed as a hypothesis testing for TPPs:
We conduct extensive experiments on synthetic and real- world datasets, covering various types of anomalies, to validate the effectiveness of CADES". The results show that CADES outperforms state-of of-the-art art methods with respect to both detection performance and FPR control.
"hh code is publicly available https: //tp::////ttbbbb//pp com/
2h- Shuai / /CADES.
Ho : X ~ Px vs.
H1 Xies 2 Px.
(3)
2

---

Conformal Anomaly Detection in Event Sequences
Here we consider Px to be some unknown data-generating TPP. If the null hypothesis Ho is rejected, we declare that Xiest is OOD or abnormal; otherwise. it is considered ID or normal. In this paper, our goal is to develop a test procedure that can identify true anomalies as accurately as possible, while controlling the FPR at a pre-specified level.
Kolmogorov-Smimov (KS) statistics, which are popular in the goodness-of-fit (GOF) test for TPPs (Barnard, 1953; Gerhard & Gerstner, 2010; Li et al., 2018). Nevertheless, KS statistics lack sensitivity to the event count and suffer from severe non-identifiability issues, producing similar val- ues for two markedly distinct event sequences (Shchur et al., 2021: Zhang et al., 2024b). Similarly, the 3S statistic, along with the Q+ and Q- statistics (Zhang et al., 2023), are insensitive to relatively uniform spacings (i.e. inter-event times), which significantly reduces their detection power.
3. Proposed Method: CADES
In this section, we present CADES, novel test procedure based on conformal inference for detecting anomalous event sequences. We begin by introducing the concept of the valid p-value (Casella & Berger, 2024), which ensures control over the FPR as specified by (4).
The above limitations highlight the urgent need for a more expressive non-conformity score that can accurately quan- tify the discrepancy between an input sequence and ID se- quences. Different from traditional settings, where each sub- ject is characterized by fixed number of features, an event sequence X {(t,, m,) on [o, varies in length. Moreover, events are associated with diverse marks and intricate dependencies exist among them. Therefore, design- ing a reasonable and expressive score s(X) that accounts for these factors is a crucial yet non-triviaa task. We draw inspi- ration from the following time-rescaling theorem (adapted from Daley et al. (2003)) to transform the event sequence and lay the foundation for a novel scoring mechanism.
Definition 3.1. A p-value p(X) is a statistic satisfying 0 I p(X) < 1 for every sample X. Small values of p(X) give evidence against the null hypothesis Ho. A p-value is valid if, for any a E (0, 1),
Pno S sa) S.
(4)
Typically, the p-value is the probability under the null hy- pothesis of obtaining real-valued test statistic least as extreme as the one observed (Haroush et al., 2022). How- ever, in the OOD detection problem (3), the null distribution Px is unknown, making it infeasible to compute the exact p- values. Inspired by conformal inference (Vovk et al., 2005), we estimate p-values using conformal p-values. Before this, it is necessary to design a non-conformity score (i.e. test statistic) quantify how different an observed sequence is from the training (ID) sequences. The design of this score is key factor for detection performance (Angelopoulos et al., 2024): different designs can lead to very different results, and a poorly designed score can lead to completely ineffective detection, e.g., with extremely low detection power. Although many expressive non-conformity scores have been developed for image data (Bates et al., 2023; Ma et al., 2024; Marandon et al., 2024), their exploration for event sequences remains limited.
Theorem 3.2. Let X = {(ts. m,) N. be a sequence of random event points on the interval [0, T] corresponding to a TPP {Nm [Nm(1)]m M with CIFs (Ni (1) For each m E M. denote the events of type-m from Nm(() as fm))
X X(m) = (41m)) (mm.. N..ty)) where the number of events satisfies EM (m N. If each Xm Xm((.) is positive on [0, and Nm (T) = So (5) ds < co almost surely, then for each m E M, the transformed sequence
2(m) (A% (14")) Am ((mm))))) (m))
(5)
forms a Poisson process with unit rate on [0, Ar (T)). More- over, the sequences {Z(m) I m=1 are independent.
In what follows, we discuss the limitations of existing test statistics for event sequences and elaborate on the definition of our two non-conformity scores in Section 1. Section 3.2 details our test procedure, CADES, which combines the two proposed scores for OOD detection. We prove the valid- ity of the p- value used in CADES and provide theoretical guarantees on calibration-conditional FPR in Section 3.3.
This theorem states that by using the integrated CIFs, an event sequence X can be transformed into M independent realizations {2(m)) Jm= of the unit-rate Poisson process (a.k.a. the standard Poisson process, SPP). Following prior work (Shchur et al., 2021), we concatenate the rescaled sequences (Z (m)) |m=1 (MM)) for each mark into a single SPP re- alization Z on the interval [0. MMM Ar, (T)). Specifically, Z is expressed as:
3.1. Non-Conformity Scores for Event Sequences
A natural way to score an event sequence is to compute its negative log-likelihood (NLL) under a trained neural TPP model. However, Shchur et al. (2021) demonstrated that the NLL statistic is less effective than their proposed sum-of of cum-oof-quareee sqquared-spacing (3S) statistic for detecting anoma- lous event sequences. Other widely used statistics include
Z z= Z(L)
7mmm ||
|| (2(M)
(6)
where the symbol || denotes vector concatenation, [(m)
Vm + (mm) Vin + N# (1("))... Vn. + Nin (N))TT)
(m)
and V., = -EET AT))), for each m E M.
3

---

Conformal Anomaly Detection in Event Sequences
For simplicity, we rewrite -= (71: -......... (N and denote its observation length as V - =Em=1 (T)) We adopt the inductive conformal inference method (Vovk et al., 2005), where the dataset D of size n is randomly divided into two disjoint subsets: proper training set Durin of size Parain < n and calibration set Deal of size Thaal = Remin- To design a non-conformity score for an event sequence X, we first rescale X to obtain Z using a neural TPP model trained on D Duain and then quantify how different the transformed sequence is from the SPP on the interval [o, Vj. Due to the powerful capability of neural networks, the neural TPP model provides a more accurate approximation of the true CIFs of training sequences.
based on the fact that the inter-event times we = T in the SPP follow Exponential(1) distribution (Cox & Lewis, 1966). Specifically, Sint is defined as:
M
Sim (X)== (iI(fimllfim)) = =1 fim(()) log
fin(()) fm(())
dx,
(8)
where (x) = =c 10,0
(x) is the exponential PDF, and fi((x) h2(N+1] Eist of " is the KDE of the inter- event times wi; Ti TE- with TO 0 and TN+1 V. h2 > is another bandwidth parameter.
We find that Smm is more sensitive to relatively uniform ar- rival times compared to 8am- For example, when all arrival times т; and the observed length V lie near integer grid points, the inter-event times w; are close to 1. This causes the KDE Jun to concentrate around 1, making it clearly dis- tinguishable from the exponential PDF. In contrast, Sim is less sensitive than Sarr to reductions in event count. As result, Sur and Sunt offer complementary sensitivities to dif- ferent types of anomalies. To fully exploit their respective strengths, we propose combining these two scores for OOD detection, as explained in the next subsection.
One property of the SPP is that, conditionally on the event count in [0, V] being equal to N, the normalized arrival times /V.V.... ..N...NN/ TV/V are independently and uniformly distributed on [o. 1] (Lewis, 1965; Cox & Lewis, 1966). Building on this property, we propose a non-conformity score using the Kullback-Leibler (KL) divergence to quan- tify the discrepancy between the underlying distribution of the normalized arrival times and the Uniform([0, [0, 1]) distri- bution. In the literature, KL divergence is natural measure of the difference between two probability distributions, mak- ing an appropriate choice for our non-conformity score. Since the continuous version of KL divergence is defined on probability density functions (PDFs), we apply kernel density estimation (KDE, Scott (2015)) with the Gaussian kernel to estimate the underlying PDF of the normalized arrival times. Consequently, the proposed score for an event sequence X is formally defined as follows:
3.2. Test Procedure with Bonferroni Correction
With the proposed score sam the classical conformal p-value for the test sequence Xtest is calculated as follows:
Par (Xies)= = ((xeal E Deal : &aar (Xies) I 8am Paa((eaa)) ++
neal + 1
(9)
This p-value is right-sided because larger score 8ar typ- ically suggests that a sequence is OOD, as a large KL di- vergence provides evidence of the difference between two distributions. However, in certain alternative scenarios, such as arrival times being near integer grid points or the self- correcting process generating more evenly-spaced events than the SPP, the KDE far becomes closer to the uniform PDF In these cases, small values of 8an may also indicate OOD sequences. Thus, we utilize the two-sided p-value:
Sar (X): == DKI. (Jurll [ Jan( (orl)log log
Fair (x)
cla,
far((r)
where far(x)) 10.1 (x) is the uniform PDF, and far (x)) N Σn d( --T./V is the KDE of the normalized arrival h1 times T.VV of Z, obtained by applying time-rescaling and concatenation to X. hi > 0 is parameter called the bandwidth, and (-) denotes the standard normal PDF. We compute the integral using numerical techniques, such as the trapezoidal rule (Stoer et al., 1980).
Par (Xret) = 2 min (p'm ( (Xess); Par (x
(10)
where the left-sided p- value pLu (Xwa) is defined similarly to the above (Xtes). except that the inequality in (q, is reversed. Throughout this paper, p-values such as Pur (Xies) are truncated at whenever they exceed it.
It can be observed that Sar exhibits sensitivity to reductions in event count, effectively overcoming major limitation of KS statistics. For example, if all events of Z in the subin- terval [ V] are removed, then the remaining normalized arrival times T will be confined to [0, 11 resulting in pronounced deviation between the KDE fan and the uni- form PDF. Nevertheless, by definition, 8am is less sensitive to relatively uniform arrival times, limitation also shared by the 3S statistic as well as the Q+ and Q- statistics.
For another score Sint, the two-sided p-value, denoted as Pint Pin (-uws), can be calculated in the same way. To leverage the complementary sensitivities of San and 8im to different abnormal patterns, we combine them for OOD detection using the Bonferroni corrected p-value (Shaffer, 1995):
To address this issue and further utilize the information of the SPP, we introduce another non-conformity score, Sints
Poor (Xuss) = min min(2(1 (2(1 + E Pan (Xws), 2(1 + E (Xrest }.
(11)
4

---

Conformal Anomaly Detection in Event Sequences
where 0 parameter. Following Magesh al. (2023), we introduce the factor (1 + E) in (q.(1 to provide strong false positive guarantees conditioned on the calibration set, as detailed in Theorem 3.4. The value of ε depends on the calibration set size Neaat a smaller & improves detection power but requires a larger faaal to maintain these guarantees. Conversely, when Teaal is small, a larger e is needed to ensure the guarantees, making the detection more conservative.
Algorithm 1 CADES: Conformal Anomaly Detection in Event Sequences
Input: Clean dataset Darai U Deat, test sequence X target significance level a E (0, 1).
1: Train a neural TPP with CIFs (t) M on Dunini
2: Apply time-rescaling and concatenation for Deat and X
3: Calculate the scores 8an (7) and 8im (8) for Deat and Xiest
For given significance level E (0.1), 1), we declare the test sequence Xiest as OOD if Poor Poor(Xien (Xtea) S This ensures a more accurate detection of anomalies by identifying dis- crepancies in the normalized arrival times T // relative to the uniform distribution and in the inter-event times w, rela- tive to the exponential distribution, We summarize our test procedure in Algorithm 1 and provide rigorous theoretical guarantees subsequently.
4: Compute the Bonferroni corrected p-value Peor (1I.
Output: Declare Xust as OOD or abnormal if Poor (-Xest) S a; otherwise, conclude it is ID or normal.
3000
20000
0.1
4020
15003
7
3000
10000
3.3. Theoretical Guarantees
57
3.2
5.4
0.5
0.3
01
02
0.4
05
We first prove the validity of the p-value used in our CADES method, which ensures control of the marginal FPR. Build- ing upon this, we then demonstrate that, under certain con- ditions, our method also controls the dllirrttionccondiiinn FPR with high probability.
(a) E= 0.2
(b) E= 0.5
Figure I. Calibration set size neat that guarantees the calibration-
conditional FPR is bounded by with probability 1 - 5.
where (a, b) denotes the CDF of the Beta(a, b) distribu-
can be verified that the p-value Poor (Xest) in (1 re- mains super-uniform and, therefore, a valid p-value. We state this result formally in the following proposition:
tion. If sare (X) and s (X) are continuously distributed, then for a new sequence Xtest, the probability of incorrectly identifying Xtest as OOD conditioned on Deal while using Algorithm is bounded by a with probability
Proposition 3.3. Suppose the test sequence Xtest and the dataset D are i.i.d., then the p-value Poor Peoo(X (X1est) is valid, for every a E (0, P Ho (Xees.))))) S aa) S a.
8, i.e.
P [P Hr, (deelare OOD| Deaa) I a] 2 1 21-6 5. (14)
The proof can be found in Appendix B. In deed, Peor Penn (Xies) is marginally valid because it depends on the calibration set Deat Since IP Ho (declare OOD)- = Ho (Poor (Xees) S a) S
We present detailed proof in Appendix B.2. This theorem builds upon a result from (Vovk, 2012; Bates et al., 2023), which states that the CDF of the conformal p-value con- ditioned on the calibration set follows Beta distribution. Additionally, due to symmetry. this property holds for both the left-sided and right-sided conformal p-values. Consider- ing the form of the Beta distribution's CDF, is difficult to express the dependence of Neat on a, and in closed form. We show the calibration set size Teaal given by (13) when 0.2 and 0.5 for different values of 5 in Figure 1.
a, we obtain:
(declare OOD)= = E [P H (declare OOD| (Dai)] S a,
(12) where the expectation is over Deat- This indicates that the marginal FPR (i.e. the probability IP H. (declare OOD)) con- trol is averaged over all possible calibration sets. However, it does not guarantee that the target level a is maintained for the particular calibration set used.
Remark. The valuus in this paper, such as (X)ee) in (9), are evaluated on the sequences from the calibration set Deat, rather than the entire dataset D, as used in previous works (Shchur et al., 2021: Zhang et al., 2023). Using the en- tire dataset D may lead invalid p- p-vauues because sur (Xisst is not exchangeable with (Xar(X) X E DJ. The problem is that Sur has already depended fined based on a neural TPP model trained on Dirain). but it has not yet seen the test sample Xist (Tibshirani, 2023). In contrast, using the hold-out calibration set Deal preserves the exchangeability of san (Xusa) and {sar (Xea)) Xeal E D ensuring the validity of p-values (Papadopoulos al., 2002).
Inspired by (Magesh et al., 2023; Bates et al., 2023), we further provide stronger guarantees that Algorithm can control the calibration-comddiioni FPR (i.e. the conditional probability P H. (declare OOD Dea)) with high probability, under certain conditions on the size of the calibration set.
Durain (since it de-
Theorem 3.4. Let å E (0, 1) and 2 0. Let Deal be calibration set of size Neabh a [(Neal + 1) a([45] J. b- =
Neal + +1-a, a, and p a+b. For a given 8% > 0, let neal be such that
5
I( 6) 2 1
(13)
4'
5

---

Conformal Anomaly Detection in Event Sequences
arrival
inter-avent
Chi sallsquerdd
statistic
statistic
statistic
CADES
DecreasingRate
Stepping
RenewalA
Hawes
SelfCorrecting
Uniform
AUROC
08
3.0
0.5
0.5
64
6.3
6.1
00
02
04
8.5
20
10
0.0
0.2
8.4
2.5
3.0
1.0
0.2
0.2
2A Detectability
3.6
0.
1.5
0.0
222
2/4 Detectabiiity
0.6
0.
1.2
0.5
0.2
24 Detectability
0.5
0.5
1.0
0.2
022
2.4
0.6
10
Detectability Dit
Detectability
Detectability
Figure 2. Porformance of GOF test for the standard Poisson process measured by AUROC (higher is better).
4. Experiments
Experimental Setup. For this GOF test, training neu- ral TPP model is unnecessary, since the distribution of the null hypothesis (i.e. the SPP) is known. Hence, we treat the entire dataset D as the calibration set. Moreover, time- rescaling and concatenation operations are also not required. We perform the GOF test following steps 3 and 4 in Algo- rithm 1. Due to the limited sample size in the benchmark dataset, ensuring calibration-condiiioni FPR control would require large in E1), which could make the detection overly conservative. Therefore, in the experiments, we use the marginally valid p-value (Xess) and set E = 0 to improve detection power. The bandwidth values for &air (X) and sinc (X) are determined by a grid search. More details of the implementation can be found in Appendix C.4.
In this section, we conduct extensive experiments to evalu- ate our CADES method, including GOF test for SPP (Sec-
tion 4.1), anomaly detection in synthetic data (Section 4.2) and real-world data (Section 4.3), and FPR control (Sec- tion 4.4). In addition, we perform ablation studies to verify the effectiveness of combining two proposed scores and us- ing two-sided p-values in Section 4.5. Appendix D contains runtime comparisons and additional experimental results.
4.1. GOF Tests for SPP under Nine Alternatives
Existing goodness-of-fit (GOF) statistics for the standard Poisson process (SPP) are either insensitive to the event count or to relatively uniform spacings. As described in Section 3.1, the two proposed non-conformity scores can overcome these limitations. Hence, in this subsection, we empirically evaluate the performance of the GOF test for the SPP using the combination of these two scores.
Evaluation Metric. In line with previous works (Shchur et al., 2021; Zhang et al., 2023), we evaluate the perfor- mance of distinguishing ID and OOD sequences by the area under the receiver operating characteristic curve (AUROC),
Results. The AUROC scores across six scenarios with vary- ing detectability parameters are shown in Figure 2, while the results for the remaining three scenarios are detailed in Appendix D.2 due to space constraints. The results indicate that KS arrival, KS inter-event, and Chi-squared statistics lose their detection capability when the number of events changes substantially, as observed in the DecreasingRate and Stopping scenarios. The 3S statistic and the Q+ and Q- statisties perform poorly in scenarios with relatively uniform spacings, such as the SelfCorrecting and Uniform cases. Specifically, we explain below why the 3S statistic does not perform well under the Uniform scenario. Accord- ing to Proposition 1 in (Shchur et al., 2021), the value of the 3S statistic for an SPP realization (i.e. an ID sequence) is around 2. In the Uniform scenario, the spacings of the OOD sequences are identical. For example, when the detectability parameter 7) = 0.5, the spacings are all 2. In this case, the 3S statistic for an OOD sequence is equal to 2, making it unable distinguish between ID and OOD sequences.
Datasets. ID sequences and OOD sequences are generated from the SPP and the alternative distribution on the interval [0, 100), respectively. We consider nine choices for the alter- native distribution: (1) DecreasingRate, (2) IncreasingRate, (3) InhomoggencouuPoisson (4) Stopping, (5) RenewalA, (6) RenewalB, (7) Hawkes, (8) SelfCorrecting, and (9) Uniform. For each alternative distribution, detectability parameter 7 E [o, 1] is defined. The first eight scenarios follow previ- ous work (Shchur et al., 2021), where a higher value of y) indicates a greater dissimilarity of the alternative distribu- tion from the SPP. In the Uniform scenario, we generate the OOD sequence with equal inter-event times. For all scenar- ios, 'D consists of 1000 ID sequences, while DID and DOOD consist of 1000 ID sequences and 1000 OOD sequences, respectively. Detailed descriptions of these alternative dis- tributions are provided in Appendix C.I.I.
Baselines. We consider six test statisties: (1) KS statistic on arrival times, (2) KS statistic on inter-event times, (3) Chi- squared statistic on arrival times (Cox, 1955), (4) 3S statistic (Shchur et al., 2021), (5) Q+ statistic (Zhang et al., 2023), and (6) Q_ statistic (Zhang et al., 2023). See Appendix C.2 for the definitions of these statistics. For each statistic, the two-sided p-vuuue for the test sequence is calculated with its empirical distribution function (EDF) on the ID dataset D.
In contrast, our CADES method consistently achieves the best or nearly the best performance across all 9 scenarios. Notably, in the Stopping case, CADES significantly outper- forms baselines due to the sensitivity of the proposed score Sur to reductions in event count. Additional experimental analyses are provided in the ablation study in Section 4.5.
6

---

Conformal Anomaly Detection in Event Sequences
KS arrival
inter-event
Chisqquared
Log-lleelincod
35 statisiic
MuDAD--.
MultAD-O Q-
CADES
ServerStop
ServerOverload
Latency
Traie
0.9
AUROC
0.8
0.8
0.7
0.6
0.5
0.5
0.2
0.4
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.8
1.0
0.0
0.4
0.6
0.8
1.0
Detectability
Detactability
Detectability
Detectability
Figure 3. Performance of OOD detection on synthetic datasets measured by AUROC (higher better).
Table AUROC (%) for OOD detection on saal-world& datasets, Best results in bold and second best are underlined.
Dataset
KS arrival
KS inter event
Chi-square
3S statistic
MultiAD-Q+
MultAD---- Q-
CADES (ours)
LOGS
Packet commution (1%)
47.24
71.80
90.92
95.03
96.61
96.48
LOGS
Packet corruption (10%)
64.96
98.72
49.35
98.98
99.30
99.31
99.53
99.48
LOGS
Packet duplication (1%
61.88
79.59
21.26
81.97
91.46
91.24
78.15
92.88
LOGS
Packet delay (frontend)
90.31
95.70
99.55
96.
97.97
95.27
98.
LOGS
Packet delay (all services)
95.13
96.60
94.35
96.30
99.16
99.59
99.31
99.33
STEAD Anchorage, AK
62.31
78.44
70.75
88.16
91.73
84.00
99.16
99.31
STEAD Aleutian Islands, AK
53.37
86.48
64.17
97.08
99.80
99.86
99.84
99,95
STEAD Helmet, CA
61.94
98.83
73.62
96.96
93.82
70.71
99.13
99.30
Average Rank
7.50
5.63
7.00
4,25
3.63
3,50
3,00
1,50
4.2. Detecting Anomalies in Synthetic Data
Experimental Setup. We randomly split D into two disjoint subsets of equal size: a training set Drain and a calibration set Deat. Unlike the baselines that train a neural TPP on the entire dataset DD we train on the subset 'D Drain- We conduct OOD detection as described in Algorithm To ensure a fair comparison,
In this subsection, we evaluate the performance of CADES in detecting anomalous event sequences on synthetic data, which corresponds to performing OOD detection. Different from the GOF test for SPP in the previous subsection, OOD detection requires training a neural TPP model because the distribution under the null hypothesis is unknown.
employ the same neural TPP (LogNor- mMix, Shchur et al. (2020)) as the baselines, utilizing the identical model architecture and training procedure. Model and training details can be found in Appendix C.3.
Datasets. We consider four synthetic datasets introduced in (Shchur et al., 2021). The ServerStop. ServerOverload, and Latency datasets simulate anomaly detection in server logs, while the SpikeTrrains dataset evaluates the detection of anomalies caused by event mark swaps. For each sce- nario, detectability parameter 1) is defined as before. See Appendix C.1.2 for dataset details.
Results. As shown in Figure 3, our method achieves the best or comparable performance across four synthetic datasets. In particular, CADES significantly outperforms baselines on the ServerStop and ServerOverload datasets, even when the detectability 0.05, where OOD sequences are obtained by changing only 5% of the time interval in ID sequences.
Baselines. We compare our CADES method with the fol- lowing baselines:
4.3. Detecting Anomalies in Real-World Data
KS arrival, KS inter-event, Chi-squared, Log-likelihood and 3S statistie: Shchur et al. (2021) train a neural TPP on and compute test statistics on the transformed sequence 7, including KS statistics, the Chi-squared statistic, the log- -ggliikei of the learned model, and their proposed 3S statistic. The two-sided p-values are calculated using the EDF of each statistic on 'D. We refer to each detection method by the corresponding test statistic.
We further evaluate CADES on two benchmark real-world datasets, LOGS and STEAD, both introduced in (Shchur et al., 2021). See Appendix C.1.3 for detailed descriptions of these datasets. The experimental setup remains identical to that in Section 4.2.
Results. Table 1 summarizes the experimental results. It can be observed that CADES consistently achieves the best or near-best AUROC across all scenarios, with an average rank significantly outperforning baselines. Notably, CADES demonstrates strong performance even in challenging sce- narios such as "STEAD Helmet, CA" and "LOGS- Packet duplication (1%)", where competing methods like MultiAD- Q+ and MultiAD-Q exhibit limitations, respectively.
MultiAD-Q+ and MultiAD-Q Zhang et al. (2023) train a neural TPP on D and conduct multiple testing on the rescaled sequence (m) for m E M. Two-sided v-vauues are calculated using the kernel cumulative distribution function of their proposed Q+ or Q- statistic on D.
7

---

Conformal Anomaly Detection in Event Sequences
4.4. FPR Control
CADES-art
CADES-int
CADES
Stopping
RenewalA
We treat OOD sequences as positives and ID sequences as negatives. The false positive rate (FPR) control guarantees crucial for safety-critical applications, such as fraud de-
1.0
1.0
0.9
0.9
ALROC
0.8
0.0
tection and medical diagnosis. For example, in electronic health records (EHRs), maintaining low FPR is vital. High false positives could lead to incorrect diagnoses or unnec- essary treatments, potentially compromising patient safety. Our method ensures that detected anomalies are more likely to be genuine, helping healthcare professionals make more reliable decisions.
0.7
0.7
0.6
0.6
0.5
0.4
0.6
0.8
1.0
1.0
Detectability
Detectability
Figure 5. Performance of CADES-arr and CADES-int for the GOF test in the Stopping and RenewalA scenarios.
We empirically investigate whether CADES and the base- lines can control the FPR. The target significance level a is taken from 0.05 to 0.5, with a step size of 0.05, For each a, we conduct five repeated experiments on the real-world datasets LOGS and STEAD and present the FPR boxplots in
CADES-arrr
CADES-ar
CADES-Int
RenewalA
SelfCorrecting
1.0
0.9
0.8
0.7
0.7
0.6
0.6
Figure 4. As shown, the FPR of CADES is upper bounded by average. However, the baseline methods. the 38 statistic and MultiAD-Q, +. exhibit much higher FPR than the pre-specified level a on the LOGS dataset. Since the baselines also control FPR on the STEAD dataset, we fur- ther provide their true positive rate (TPR), i.e., detection power, at the commonly used level a = 0.05 in Table 2. We can observe that CADES significantly outperforms the baselines in identifying true anomalous sequences. These results highlight the effectiveness of CADES in controlling FPR while achieving superior detection power.
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.1
0.0
0,2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Detectabillty
Detactability
Figure 6. Performance of CADES arr-r for the GOF test in the RenewalA and SelfCorrceting scenarios.
three model variants: (1) single score Sar with the two- sided p-value, (2) single score Sint with the two-sided p- value, and (3) single score 8ar with the right-sided p-value. We denote these three variants as CADES-ar, CADES-int and CADES-ar-r, respectively.
statiatk
MUHAD-O.
CADES
LOGS
STEAD
We evaluate CADES- -arr and CADES-int in the Stopping and RenewalA scenarios. As shown in Figure 5, CADES- arr achieves strong performance in the Stopping scenario, where the number of events in the OOD sequence decreases, while CADES-ing int excels in the RenewalA scenario, where the spacings in the OOD sequence are relatively uniform. These successes can be attributed to the sensitivity of 8ar to reductions in event count and the sensitivity of sint to relatively uniform spacings. By contrast, CADES, which integrates these two scores, demonstrates superior overall performance across both scenarios. We also conduct similar experiments real-world data, with the results provided in Appendix D.3. These findings further validate the advan- tages of using 8an and 8in: simultaneously.
2.2
2.8
57
2.6
0.5
3.5
3.6
0.2
22
0.3
2.0
Target
Target level
Figure 4. Boxplots of FPR for OOD detection on two real-world datascts under different target levels a.
Table 2. TPR (%) for OOD detection on the STEAD dataset under the target level a
0.05.
Dataset
38 statistic
MultAD-Q+
CADES
For CADES-arr-r, we focus on the RenewalA and SelfCor- recting scenarios, where the spacings of the OOD sequence are more uniform than those of the ID sequence generated from the SPP. As shown in Figure 6, CADES- arr-r performs poorly in these two scenarios, with significant classification errors. We explain this as follows. The distributions of sar in these two scenarios are presented in Figure 7. In both cases, smaller values of sur tend to indicate OOD sequences. Con- sequently, the right-sided p-value in CADES-arr- assigns
STEAD - Anchorage, AK
74.30
67.14
95.46
STEAD Alcutian Islands, AK
100
100
100
STEAD- Helmet, CA
69.20
6.50
98.38
4.5. Ablation Study
In this subsection, we conduct ablation studies to validate the effectiveness of combining two non con-corfoym scores and using two-sided p-values. Specifically, we consider
8

---

Conformal Anomaly Detection in Event Sequences
RenewalA
SelfCorrecting
in Neural Information Processing Systems, 36:23047- 23074, 2023.
30
Frequency
Angelopoulos, A. N., Barber, R. F., and Bates, S. Theoreti- cal foundations of conformal prediction. arXiv preprint arXiv:2411.1182, 2024.
20
10
Auer, A., Gauch, M., Klotz, D., and Hochreiter, S. Con- formal prediction for time series with modern hopfield networks. Advances in Neural Information Processing Systems, 36:56027-56074, 2023.
0.0
0.1
0.2
0.3
0.4
0.5
0.00
0.05
0.10
0.15
0.20
0.25
Spr (0)
(X)
Figure 7. Distribution of the score Sar(X) in the RenewalA and SelfCorrectig secnarios at the detectability = 0.5.
Barnard, G. Time intervals between accidents-a note on maguire, pearson and wynn's min's paper. Biometrika, 40(1-2): 212-213, 1953.
larger p- p-values OOD sequences than ID sequences, resulting in incorrect classification. A similar analysis can be conducted in the Uniform scenario. Additionally, from Figure 6, we observe that CADES-arr, which uses the two- sided p-value, significantly outperforms CADES- int in the SelfCorrectng scenario. This also highlights the necessity of using the two-sided p-value.
Bates, S., Candès, E., Lei, L.,, Romano, Y., and Sesia, M. Testing for outliers with conformal p-values. The Annals of Statistics, 51(1):149-17) 2023.
Brown, E. N., Barbieri, R., Ventura, V., Kass, R. E., and Frank, M. The time-rescaling theorem and its applica- tion to neural spike train data analysis. Neural computa- tion, 14(2):325-346, 2002.
5. Conclusion
We introduced CADES, a novel conformal anomaly detec- tion method for contnuuus-timme event sequences. CADES combines two newly designed non-conformity scores with provably valid p-values for hypothesis testing and provides theoretical guarantees on the cnllbration-conntiionaal FPR. Extensive experiments demonstrate that CADES outper- forms state-of-the-art methods in both GOF test for SPP and anomaly detection synthetic and rerl-world datasets, while controlling the FPR at pre-specified level.
Casella, G. and Berger, R. Statistical inference. CRC Press, 2024.
Cox, D. Some statistical methods connected with series of events. Journal of the Royal Statistical Society: Series B, 1955.
Cox, D. and Lewis, The statistical analysis of series of events. Monographs on Applied Probability and Statistics, 1966.
Acknowledgements
Daley, D. J., Vere-Jones, D., al. An introduction the theory of point processes: volume I: elementary theory and methods. Springer, 2003.
The authors would like to thank the anonymous reviewers and area chairs for their valuable comments and suggestions. This work was partially supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB0680101). the National Natural Science Foundation of China (62472416, 62376064, U2336202, 62402491). and the CAS Project for Young Scientists in Basic Research (YSBR-00S).
Dheur, V., Bosser, T., Izbicki, R., and Ben Taieb, S. Distribution conformal joint prediction regions for neural marked temporal point processes. Machine Learn- ing, 1130))7055-7102 2024.
Gerhard, F, and Gerstner, W. Rescaling, thinning or comple- menting? on goodness-of-fit procedures for point process models and generalized linear models. Advances in Neu- ral Information Processing Systems, 23, 2010.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Gerhard, F., Haslinger, R., and Pipa, G. Applying the multi- variate time-rescaling theorem to neural population mod- els. Neural Computation, 1452-14814448 2011.
References
Gibbs, I. and Candes, E. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34: 1660-1672, 2021.
Angelopoulos, A., Candes, E., and Tibshirani, R. J. Con- formal pid control for time series prediction. Advances

---

Conformal Anomaly Detection in Event Sequences
Haroush, M., Frostig, T., Heller, R., and Soudry, D. A statis- tical framework for efficient out of distribution detection in deep neural networks. In International Conference Learning Representations, 2022.
Lin, X., Cao, Y., Sun, N., Zou, L.,, Zhou, C., Zhang, P., Zhang, S., Zhang, G., and Wu, J. Conformal graph-level out-of-distribution detection with adaptive data augmen- tation. In THE WEB CONFERENCE, 2025.
Hawkes, A. G. Spectra of some self-exciting and mutually exciting point processes. Biometrika, 58(1):83-90, 1971.
Liu, S. and Hauskrecht, M. Event outlier detection in con- tinuous time. In International Conference on Machine Learning, 2021.
Isham, V. and Westcott, M. A self-correcting point process. Stochastic processes and their applications, 8(3):335- 347, 1979.
Lüdke, D., Biloš, M.. Shchur, O., Lienen, M. and Günnemann, S. Add and thin: Diffusion for temporal point processes. Advances in Neural Information Pro- cessing Systems, 11756784-588011 2023.
Ishimtsev, V., Bernstein, A., Burnaev, E., and Nazarov, I. Conformal k-nn anomaly detector for univariate data streams. In Conformal and Probabilistic Prediction and Applications, pp. 213-227. PMLR, 2017.
Ma, X., Zou, X., and Liu, W. A provable decision rule for duttof-distribution detection. In International Conference on Machine Learning. 2024.
Kaur, R., Jha, S., Roy, A., Park, S., Dobriban, E., Sokolsky, O., and Lee, idecode: In in-distribution equivariance for conformal nut-of-distribution detection. In Proceedings of the AAAI Conference Artificial Intelligence, 2022.
Magesh, A., Veeravalli, V. V., Roy, A., and Jha, Principled -iistri-iitiiitionnnn detection via multiple testing. Journal of Machine Learning Research, 24(378):1-35, 2023.
Marandon, A., Lei, L., Mary, D., and Roquain, E. Adaptive novelty detection with false discovery rate guarantee. The Annals of Statistics, 52(1):157-183, 2024.
Kingman, J. F. C. Poisson processes, volume 3. Clarendon Press, 1992.
Laxhammar, R. and Falkman, G. Inductive conformal anomaly detection for sequential detection of anomalous sub-trajectories. Annals of Mathematics and Artificial Intelligence, 74:67-94, 2015.
Mei, H. and Eisner, J. M. The neural hawkes process: A neurally self-modulating multivariate point process. Advances in Neural Information Processing Systems, 30, 2017.
Lee, J., Popov, and Ren, Z. Full-conformal novelty detection: A powerful and non-random approach. arXiv preprint arXiv:2501.0270) 2025.
Nath, S., Lui, C.,, and Liu, S. Unsupervised event outlier detection in continuous time. In NeurlPS 2024 Workshop SSL, 2024.
Lewis, P. A. Some results on tests for poisson processes. Biometrika, 52(1-2):67-77 1965.
Naumzik, C. and Feuerriegel, S. Detecting false rumors from retweet dynamics on social media. In Proceedings of the ACM Web Conference 2022, pp. 2798-2809, 2022.
Li, S., Xiao, S., Zhu, S., Du, N., Xie, Y., and Song, L. Learn- ing temporal point processes via reinforcement learning. Advances in Neural Information Processing Systems, 31, 2018.
Ogata, Y. Statistical models for earthquake occurrences and residual analysis for point processes. Journal of the American Statisticul association, 83(401):9-27, 1988.
Li, Z., Xu, Xu, Z., Mei, Y., Zhao, T., and Zha, H. Beyond point prediction: Score matching-basee based lseudolikelihood estimation of neural marked spatio-temporal point pro- cess. In International Conference on Machine Learning, 2024.
Papadopoulos, H., Proedrou, K., Vovk, V., and Gammerman, Inductive confidence machines for regression. In Machine learning: European Conference on Machine Learning, pp. 345-356. Springer, 2002.
Pillow, J. Time-rescaling methods for the estimation and assessment of non-poisson neural encoding models. Ad- vances in Neural Information Processing Systems, 22, 2009.
Liang, Z., Sesia, M., and Sun, W. Integrative conformal p-values for -uittf-disttibuutin testing with labelled out- liers. Journal of the Royal Statistical Sociery Series B: Statistical Methodology, 86(3):671-693 2024.
Scott, D. W. Multivariate density estimation: theory, prac- tice. and visualization. John Wiley & Sons, 2015.
Lin, X., Cao, J., Zhang, P., Zhou, C., Li, Z., Wu, J., and Wang, B. Disentangled deep multivariate hawkes process for learning event sequences. In International Conference on Data Mining, pp. 369. IEEE, 2021.
Shaffer, P. Multiple hypothesis testing. Annual Review of 1995.
Psychology,
10

---

Conformal Anomaly Detection in Event Sequences
Shchur, O., Bilos, M., and Günnemann, S. Intensity-free learning of temporal point processes. International Con- ference Learning Representations, 2020.
Xu, C. and Xie, Y. Conformal prediction for time series. IEEE Transactions on Pattern Analysis and Machine In- telligence, 45(10):11575-111577 2023a.
Shchur, O., Turkmen, A. C.,, Januschowski, T., Gasthaus, J.. and Günnemann. S. Detecting anomalous event se- quences with temporal point processes. Advances in Neu- ral Information Processing Systems, 34:13419-13431, 2021.
Xu, and Xie, Sequential predictive conformal in- ference for time series. In International Conference on Machine Learning, pp- 38707-38727. PMLR, 2023b.
Yang, Y., Yang. C., Li, B., Fu, Y., and Li, S. Neuro-symbolic temporal point processes. International Conference on Machine Learning, 2024.
Shi, X., Xue, S., Wang, K., Zhou, F., Zhang, J., Zhou, J., Tan, C., and Mei, H. Language models can improve event prediction by few-shot abductive reasoning. Advances in Neural Information Processing Systems, 36, 2023.
Zaffran, M., Féron, O., Goude, Y., Josse, J., and Diculeveut, A. Adaptive conformal predictions for time series. In In- ternational Conference on Muchine Learning, pp. 25834- 25866. PMLR, 2022.
Smith, J., Nouretdinov, I., Craddock, R., Offer, C.,, and Gam- merman, A. Conformal anomaly detection of trajectories with a multi-class hierarchy. Statistical Learning and Data Sciences: Third International Symposium, SLDS 2015, Egham, UK, April 20-23, 2015, Proceedings pp. 281-290. Springer, 2015.
Zeng. M., Coates, M., et al. Interacting diffusion processes for event sequence forecasting. In International Confer- ence on Machine Learning. 2024.
Zhang, S., Zhou, C., Zhang, P., Liu, Y., Li, and Chen, H. Multiple hypothesis testing for anomaly detection in multi-type event sequences. In International Conference on Data Mining, pp. 808-817. IEEE, 2023.
Stankeviciute, K., M Alaa, A., and van der Schaar, M. Con- formal time-series forecasting. Advances in Neural Infor- mation Processing Systems, 34:6216-6222 2021.
Zhang, S., Zhou, C., Liu, Y. A., Zhang, P., Lin, X., and Ma, Z.-M. Neural jump-diffusion temporal point processes. In International Conference on Machine Learning, 2024a.
Stoer, J.. Bulirsch, R., Bartels, R., Gautschi, W., and Witz- gall, C. Introduction to numerical analysis, volume 1993. Springer, 1980.
Zhang, W., Panum, T., Jha, S., Chalasani, P., and Page, D. Cause: Learning granger causality from event sequences using attribution methods. In International Conference on Machine Learning, pp. 11235-1 1245. PMLR, 2020.
Tao, L.. Weber, K. E., Arai, K., and Eden, U. T. A common goodness-off of fit framework for neural population models using marked point process time-rescaling. Journal of Computational Neuroscience, 45:147-162, 2018.
Zhang, Y., Fang, G., and Yu, W. On robust clustering of
temporal point process. arXiv preprint arXiv:2405.17822 2024b.
Tibshirani, R. Conformal prediction. UC Berkeley, 2023.
Trivedi, R., Farajtabar, M., Biswal, P., and Zha, H. Dyrep: Learning representations over dynamic graphs. In Inter- national Conference on Learning Representations, 2019.
Zhu, S., Yuchi, H. S., and Xie, Adversarial anomaly detection for marked spatio-temporal streaming data. In International Conference Acoustics, Speech and Sig- IEEF, 2020.
nal Processing, pp.
Vovk, V. Conditional validity of inductive conformal pre- dictors. In Asian Conference on Machine Learning, pp. PMLR, 2012.
Zuo, S., Jiang. H., Li, Z., Zhao, T., and Zha, H. Transformer hawkes process. In International Conference on Machine Learning, pp. 11692-11700 PMLR, 2020.
Vovk, V., Gammerman, A., and Shafer, G. Algorithmic learning in random world, volume 29. Springer, 2005.
Xiao, S., Yan, J., Yang, X., Zha, H., and Chu, S. Modeling the intensity function of point process via recurrent neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Xu, C. and Xie, Y. Conformal prediction interval for dy- namic temesseris. In International Conference on Ma- chine Learning, pp. 11559-1 11569. PMLR, 2021.

---

Conformal Anomaly Detection in Event Sequences
A. Related Work
Conformal Anomaly Detection. Conformal anomaly detection (Smith et al., 2015; Laxhammar & Falkman, 2015;
Ishimtsev et al., 2017), also referred to as conformal -utto--disttibuttinn (OOD) detection (Kaur al., 2022: Lin et al., 2025), is an application of conformal inference/prediccion (Vovk et al., 2005) in the context of anomaly detection or OOD detection. It employs non-conformity score to quantify how different input is from the training (in-distribution) data, and then computes the conformal p-value to assess the abnormality. Recent advances in this field have gained considerable attention (Kaur et al., 2022; Bates et al., 2023; Ma et al., 2024; Marandon et al., 2024; Liang et al., 2024; Lee et al., 2025). However, these works mainly focus on image data, and none have yet considered continuous-time event sequences. To fill this gap, we design powerful non- non-conformity scores tailored to event sequences and develop a valid test procedure based on the inductive conformal inference. Moreover, we provide calitration-condition
FPR guarantees (Magesh et al., 2023), which are stronger than marginal FPR guarantees (Kaur et al., 2022; Haroush et al., 2022; Lin et al., 2025).
Anomaly Detection in Event Sequences. Temporal point processes (TPPs) are probabilistic models for continuous-timg event sequences (Zuo et al., 2020; Lin et al., 2021; Lüdke et al., 2023). With the development of neural networks, neural TPPs have become promising method for modeling complex event dynamics (Li et al., 2024; Zhang et al., 2024a; Zeng et al., 2024; Yang et al., 2024). Naumzik & Feuerriegel (2022) developed a mixture marked Hawkes model to detect false rumors on social media. Liu & Hauskrecht (2021); Nath et al. (2024) focused on detecting specific anomalous events in a sequence using neural TPPs, whereas our work addresses the detection of entire anomalous sequences. The most relevant studies ours are (Shchur et al., 2021: Zhang et al., 2023), which perform hypothesis testing to identify anomalous event sequences. Compared these works, we address the non-identifiability issues of their test statistics and provide theoretical guarantees for both marginal FPR and calibration-comditionaal FPR.
Goodness-of-Fit Test for TPPs. Many popular goodness-of-fit (GOF) tests for TPPs rely on the time-rescaling theorem (Ogata, 1988; Brown et al., 2002; Gerhard et al., 201 Tao et al., 2018), reducing the problem of GOF testing for an arbitrary TPP to determining whether the rescaled sequence follows the standard Poisson process (SPP). In practice, GOF tests for the SPP typically use the Kolmogorov-Smie -miinoo (KS) statistic to check if the arrival times are distributed uniformly (Barnard, 1953) or if the inter-event times follow exponential distribution (Cox & Lewis, 1966). However, the KS statistic performs poorly in scenarios involving changes in the event count (Shchur et al., 2021; Zhang et al., 2024b). More failure modes of (3S) statistic (Shchur et al., 2021) and
KS statistics can be found in (Pillow, 2009). While the wwwooo-uuuudd-ppaetaaeeeeeeeeeee the Q+ and Q- statistics (Zhang et al., 2023) address some shortcomings of KS statistics, they are insensitive to relatively uniform spacings, which significantly reduces their statistical power. In this paper, we show that our two non-conformity scores capture complementary sensitivities to the event count and relatively uniform spacings. To make full use of their respective strengths, we apply the Bonferroni correction to combine these two scores for hypothesis testing.
Conformal Prediction for Time Series. Conformal prediction (CP) has gained traction in the field of time series forecasting (Stankeviciute et al., 2021: Angelopoulos et al., 2023; Auer et al., 2023), providing powerful tool for uncertainty quantification. 'rraditionally, CP methods require exchangeability of data, condition often violated by time series data due to inherent temporal dependencies. To address this challenge, various approaches have been developed to adapt CP for time series (Xu & Xie, 2021; Gibbs& & Candes, 2021; Zaffran et al., 2022; Xu & Xie, 2023a;b). While both time series and event sequences are types of sequential data, they differ significantly (Xiao et al., 2017: Zhang et al., 2024a), with event sequences commonly modeled using TPPs. Recently, Dheur et al. (2024) employed CP and neural TPPs to predict event times and marks, offering valid coverage guarantees. In contrast, our work focuses on the application of CP for anomaly detection in event sequences.
B. Proofs
B. Proof of Proposition 3.3
Proof. We aim to show that the p-value Peer (Xies) in Eq.(11) is stochastically larger than or equal to the uniform distribution under the null hypothesis Ho, i.e., Xies ~ Px. Since the conformal p-values, such as pha (Xies) and (X Xesx), are valid under Ho (Papadopoulos al., 2002; Vovk et al., 2005), it follows that for every a E (0, 1).
PHo (Xuss) S <a)=
S a and PHo
S a) S a.
(15)
These inequalities follow from the exchangeability of the non-conformity scores Sun (Niea) and (San (Neu) Xaa E Dea)
12

---

Conformal Anomaly Detection in Event Sequences
Using the Union Bound, we obtain the following for every a (0,
P Ho (Par xea) S =a) Hoo ( (min {2p'mn (Xwa)) 2 (Xeu)) } S sa)
(16)
no ((2parr (Xies) sa} U 22 am (Xtes) S sa])
(17)
Hoo (2pam(Xusa) aa) + PHo (2p am (Xua) S a)
(18)
0 +
(19)
2 2
a.
(20)
Similarly, we also have PHo (Pest) aa) S a. Then, for every a (0, 1) and 2 O. we obtain:
PHo (Dear (reea) 5 x)) Ho ( min {2(1 + Pa (Xtes), 2(1 + E)Piin (Xiest) S 5a)
(21)
S Hoo Purr (Xies) S
+ Ho Pim (Xieu) S 2(1
a 2(1 +
(22)
+ 2(1 +
(23)
2(1 + E)
S a.
(24)
This completes the proof.
B.2. Proof of Theorem 3.4
Proof. According to Algorithm 1, the probability of declaring that Xiest is OOD, conditioned on the calibration set Deat, is given by:
Pho (declare OOD (Det) PHo (Pece (Xies) S a Demn).
(25)
From the Union Bound, we derive
PHo (Deor (Xret) S a Dai) = IH ( (min {2(1 + P)r (Xrea); 2(1 + 8 E) (Xesa) S a Deal)
(26)
-Ho (U i=1
(Upea(Kess) (ie(t))))= _ 2(1 +
a
} (eal)
(27)
PHo (pi(X
S 2(1 +
Deall
(28)
i=1
2
2
$
P Pio (p((Xiea) S 4(1 +1
a (Deal
(29)
i=1
j=1
2
(30)
Here, we use the notation i E {1, 21 to represent "arr" and "int", and j E {1, to represent "I" and "y" for convenience. We denote rf = Pro (p/ (es) S पा+ Dama) for i,j E 21.
When the non-conformity score follows a continuous distribution, the cumulative distribution function (CDF) of the conformal p-value conditioned on the calibration set follows a Beta distribution (Vovk, 2012: Bates al., 2023). Due to symmetry, this property holds for both the left-sided and right-sided conformal p-values. Specifically, for each r/. we have r? Beta(a, b), where a (Neat + 1) 4[++)] and b = Neal + - a. The mean of this distribution is μ = a+b
Let E represent the event
P s 7
E
(31)
Then,
2 2
2 >Σ
2 P
1 1-P(E) =1
4) = =ΣΣ I (4(a,b)
(32)
1=
13

---

and Conformal Anomaly Detection in Event Sequences
where (a, b) denotes the CDF of the Beta(a, b) distribution. Since Real satisfies the condition in (13) and μ is upper
bounded by 4(1+5) obtain
8 4
- Ig (a, 1 - ((++)) (a,
(33)
Thus, it follows that
1 1-P(E) S
4 6.
(34)
i=j=1
Then, under event E, i... with probability at least - 6, we have
2
PHo (declare OOD D <
a
(35)
1=1
1=1
This completes the proof.
C. Experimental Details
C.1. Dataset Descriptions
C. 1.1. ALTERNATIVE DISTRIBUTIONS
In Section 4.1, we consider the following nine alternative distributions, where 7) E [0, 1] is the detectability parameter.
RecreasingRate: Homogeneous Poisson process with intensity = 0.5%..
IncreasingRate: Homogeneous Poisson process with intensity À = 1 + 0.5mj.
InhomogennouusPoison:: Inhomogeneous Poisson process with intensity X(t) = + sinn(w)), where w
2m 50
8 = 2n.
Stopping: Events occurring in [fstup: T] are removed from the SPP sequence, where fstop T(1 0.37) and T 100.
RenewalA: renewal process, where inter-event times т; are sampled i.i.d. from a Gamma distribution with shape A: - and scale
1z. In this case, [ ko 1 and Var[7]] k02 = 1/. Thus, the expected inter-event time remains constant 1, but the variance of inter-event times decreases for higher 1).
RenewalB: A renewal process, where inter-event times т; sampled i.i.d. from Gamma distribution with shape I. -- The expected inter-event time remains constant, but the variance increases for higher
k: == 1 and scale 0
Hawkes: Hawkes process with intensity X (1) = μ + a Er,er exp(-(t l:)), where μ and = 17.
SelfCorrecting: Self-correctngg process with intensity X1 (t) exp(ut - Et, <t a), where 10-5 and a
Uniform: The inter-event times т; in a sequence are all 1 + 27, i.e., Ti = 1 21).
C.1.2. SYNTHETIC DATASETS
We provide an overview of four synthetic datasets and two real-world datasets below. More detailed descriptions can be found in the original paper (Shchur et al., 2021). Table summarizes statistics of these datasets.
ServerStop and ServerOverload: ID sequences for both scenarios are generated by multivariate Hawkes process with M = 3 marks on the interval [0, 100], modeling network traffic among hosts. In OOD sequences, the influence matrix is changed to simulate two scenarios: (1) a host goes offline (ServerStop). and (2) a host goes down and the traffic is routed to another host (ServerOverload), higher detectability 17 indicates that the change in the influence matrix occurs earlier.
14

---

Conformal Anomaly Detection in Event Sequences
Table Statistics of synthetic and real-world datasets.
Dataset
# Marks
[D]
test
ServerStop
1000
1000
1000
ServerOverload
1000
Latency
1000
1000
1000
SpikeTrrains
50
500
96
96
LOGS
1668
502
110
STEAD
4000
1000
3000
Latency: Each sequence contains two marks on the interval [0, 100). In ID sequences, the first mark, the "trigger", is generated by a homogeneous Poisson process with intensity À = 3. The second mark, the "response", is obtained by shifting the arrival times of the first mark by offsets, which are independently sampled from Normal(j 1, = 0.1). In OOD sequences, the offsets are instead sampled from Normal(j 1 + 0.5m, a 0.1), introducing an increased latency between the "trigger" and "response" events.
SpikeTraiin: ID sequences consist of the firing times of 50 neurons observed over 20-second period, with each neuron represented by distinct mark. OOD sequences are obtained by switching event marks (e.g., switching marks I and 2). A higher detectability 7) corresponds to greater number of switches.
C.1.3.
DATASETS
Event sequences represent the timestamps of 8 types of log entries. Each type of log entry is treated as a mark, and each sequence is observed in 30 seconds. ID sequences correspond to normal operations, while OOD sequences are generated by injecting failures using a chaos testing tool Pumba. OOD sequences contain types of injected anomalies (e.g., packet corruption, increased latency), See Table 1 for the list of anomalies. D, Dis and Des consist of 1668 ID sequences, 502 ID sequences, and 22 OOD sequences per each failure injection scenario, respectively.
LOGS:
STEAD (Stanford Earthquake Dataset): Event sequences contain the occurrence times of earthquakes within a 350km radius of 4 geographical locations: San Mateo, CA; Anchorage, AK; Aleutian Islands, AK; and Helmet, CA. Each sequence is unmarked and observed over 72 hours. The sequences corresponding to San Mateo, CA, are used as ID data, and the remaining 3 locations as OOD data. D, Dis and DOOD consist of 4000 sequences, 1000 ID sequences, and 1000 OOD sequences per each remaining location, respectively.
To ensure a fair evaluation of the anomaly detection performance, we excluded three normal (ID) sequences with the shortest lengths (0, 120, and 196) from the test set of the LOGS dataset. These sequences are significantly shorter than the average length of approximately 5000 for normal sequences and are therefore not representative of typical normal data patterns.
C.2. Test Statistics
Let Z = (Ti,..., TN) be an unmarked event sequence on the interval [0, VJ. with To = 0 and TN+1 = V. Let wy = Ti-1 for = 1,..., N + 1. The test statistics, used as baseline methods in this paper, are provided below.
KS arrival:
7.) = VN N sup TE(O,V]
(T)(T) (T))
N 1 1m)) (T) and = =
where (T)(T)
Σ
1=1
KS inter-event:
Kimt (2) V
[in(ww) - Finn (w))
sup wE(0,00)
N+1
where (w) N41
Σ
1 (c) (w) and Fin(w) = -xp(-ww)
15

---

Conformal Anomaly Detection in Event Sequences
Chi-squared: Following Shchur al. (2021), the interval [0, V is partitioned into B 10 disjoint buckets of equal length, and then the observed event count No in each bucket is compared with the expected amount I. V/B Thus, the Chi-squared statistic is defined as
(No - 2)
x² =
L
b=1
Sum-o--quareddspacingg (3S) statistic:
N+1
1
(Z) V
wf.
i=1
Q+ statistic:
N+1
Q+ V
1(w5+1) )-
Q- statistic:
Q- N+1 w,Ws+1)
N
W
Different from the above statistics, the log-likelihood statistic (used in Sections 4.2 and 4.3) is computed directly on the marked event sequence using the condition intensity function of the learned neural TPP model. Specifically, the log- log-likelihood function of marked event sequence X {(t,, mz) on the interval [0, 7] defined follows:
Log-likelihood:
M5
( L (o)
((X) = Σ log Xo, (14) - Σ
172
C.3. Training Details
In the case of the GOF test for SPP (Section 4.1), there is no need to train a neural TPP model, as the distribution under the null hypothesis is known. In contrast, for OOD detection (Sections 4.2 and 4.3), where the distribution under the null hypothesis is unknown, we train neural TPP (LogNormMix, Shchur et al. (2020)) using the same model architecture as the baselines (Shchur et al., 2021; Zhang et al., 2023). For completeness, we reiterate the details here. The inter-event time distribution is parameterized with a mixture of 8 Weibull distributions, the mark embedding size is set to 32, and the RNN hidden size is set 64 for all experiments.
We randomly split the dataset D into two disjoint subsets of equal size: training set Dirain and calibration set D Unlike the baselines that optimize the model parameters by maximizing the log-likelihood of the sequences in the entire dataset D, we train the model using the sequences in Drrin The following training procedure is the same for both our method and the baselines. The batch size is set to 64, the optimizer is Adam with a learning rate of 10-3, and the L2 norm of the gradient is clipped to 5. We set the maximum number epochs to 500 and perform early stopping if the training loss does not improve for 5 epochs.
C.4. Implementation Details
All experiments in this paper are conducted on an NVIDIA RTX 3090 Ti GPU using PoTorch. For kernel density estimation (KDE) in our test procedure, we use "scipy.stats. gaussian kde" with the parameter "bw method" set to hy for sur(X) and for for Sint (X). The values of h1 and f22 are selected from the grid (0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1.0]. Following the baselines, the results reported in Section 4.1 and Section 4.2 are averaged over 10 random seeds. In Section 4.3, train the neural TPP model with 5 different random initializations compute the average results.
D. Extra Experimental Results
D.I. Runtime Comparison
We compare the inference runtime of CADES with the baselines on two real-world datasets. The experimental results, averaged over five runs, are presented in Table 4. CADES requires slightly longer inference times compared to the baselines.
16

---

Conformal Anomaly Detection in Event Sequences
The additional time taken by CADES be attributed its use of two non-conformity scores simultaneously and the need for numerical integration. Despite this slight increase in runtime, the results indicate that the computational overhead introduced by CADES is acceptable and does not diverge significantly from the baselines.
Table Inference runtime averaged over runs (in seconds).
Dataset
3S statistic
MultiAD-Q.
CADES
LOGS
24.49
30.36
38.32
STEAD
19.14
22.27
25.98
D.2. Additional GOF Test for SPP
The AUROC scores for the IncreasingRate, Inhomogeneouusoiissn,, and RenewalB scenarios shown in Figure 8. These results are consistent with the conclusions presented in Section 4.1. In these cases, CADES achieves superior comparable performance, validating its effectiveness across patterns distinct from the SPP.
Chi-square
statistic
Rnee
InhomogeneousPoiissn
RenewalB
1.0
1.0
1.0
0.9
0.9
0.9
AUROC
0.8
0.7
0.7
0.6
0.6
0.6
0.5
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
1.0 0.4
0.6
0.8
Detectability
Detectability
Detectability
Figure 8. Performance of GOF test for the SPP under other three scenarios.
D.3. Ablation Study on Real- World Data
The results of CADES-an arr and CADES-int on two real-world datasets are presented Table 5. CADES-ar and CADES-in -nnt refer the variants that use single non-conformity score (X) and smm (X), respectively. We can observe that CADES achieves superior overall performance. Notably, in certain scenarios, both CADES-art and CADES-int perform worse than CADES, one of them performs poorly such as CADES-arT in the "LOGS Packet duplication (1%)" scenario. These observations highlight the benefits of using Sam (X) and Sim (X) simultaneously.
Table 5, AUROC (%) results of the CADES variants for OOD detection on real-world datasets.
Dataset
CADES-ar arr
CADES CADES-int
LOGS Packet corruption (1%)
94. 14
90.59
96.48
1.OGS Packet corruption (10%)
99.60
97.84
99.48
LOGS Packet duplication (1%)
58.43
92.98
92.88
LOGS Packet delay (frontend)
98.24
96.99
98.
LOGS Packet delay (all services)
98.22
99.24
99.33
STEAD Anchorage, AK
84.60
98.92
99.31
STEAD Aleutian Islands,
99.85
99.82
99.95
STEAD Helmet, CA
82.67
99.67
99.30
Average Rank
2.25
2.25
1.50
17


Reviews:
[{"review_id": "Cer3DvliIU", "reviewer": "Reviewer_T5Ha", "summary": "This paper proposes a novel test procedure based on conformal inference for detecting anomalous event sequences, with rigorous control over the false positive rate (FPR), a crucial factor for deploying anomaly detection methods in safety-critical applications. Specifically, it designs two new non-conformity scores tailored to event sequences, which capture complementary sensitivities to different abnormal patterns. By combining these scores with Bonferroni correction, the proposed method CADES overcomes the non-identifiability limitations of existing methods. Theoretically, this paper proves the validity of CADES and provides guarantees on calibration-conditional FPR. Experimental results validate the effectiveness of CADES across multiple benchmark datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and questions. We provide answers to your concerns as follows:\n\n**Q1**: It is suggested to explain the reason behind the poor performance of the 3S statistic under the Uniform scenario in Section 4.1.\n\n**R1**: This is because the 3S statistic is not sensitive to relatively uniform spacings (i.e. inter-event times). Specifically, according to Proposition 1 in [1], the value of the 3S statistic for a standard Poisson process realization (i.e. an ID sequence) is around 2. In the Uniform scenario, the spacings of the OOD sequences are identical. For example, when the detectability parameter $\\eta = 0.5$, the spacings are all 2. In this case, the 3S statistic for an OOD sequence is equal to 2, making it unable to distinguish between ID and OOD sequences. We will add this explanation in Section 4.1.\n\n**Q2**: In Figure 7, why are the distributions of ID scores different in the RenewalA and SelfCorrecting scenarios?\n\n**R2**: Since the bandwidth of the non-conformity score $s_{\\text{arr}}(X)$ is selected differently in the RenewalA and SelfCorrecting scenarios, the value of $s_{\\text{arr}}(X)$ is different, leading to differences in the distribution of ID scores.\n\n**Q3**: Could you provide the computational time of the CADES method?\n\n**R3**: During the training phase, for fairness, CADES employs the same neural TPP model as the baseline methods. Thus, its training time is similar to that of the baselines. We compare the inference runtimes of CADES and the baseline methods on real-world datasets in Appendix D.1, showing that CADES achieves comparable runtimes. For convenience, we also present the experimental results below:\n\n- Inference runtimes (in seconds)\n\n| Dataset | 3S statistic | MultiAD-$Q_+$ | CADES |\n| --- | --- | --- | --- |\n| LOGS | 24.49 | 30.36 | 38.32 |\n| STEAD | 19.14 | 22.27 | 25.98 |\n\n**Q4**: How would CADES perform in scenarios where event sequences have sparse data?\n\n**R4**: Our method is flexible, allowing any neural TPP model to be plugged into CADES. Therefore, we can use more advanced neural TPP models or models specifically designed for sparse event data to capture the distribution of the normal training data, and then apply the proposed test procedure for inference on the test data.\n\n\nReference:\n\n[1] Shchur O, Turkmen A C, Januschowski T, et al. Detecting anomalous event sequences with temporal point processes. NeurIPS, 2021.", "claims_and_evidence": "This paper provides clear and convincing evidence supporting its claims. The theoretical analysis of the proposed method, including guarantees on the marginal FPR and calibration-conditional FPR, is solid. The experiments demonstrate that CADES outperforms baseline methods, overcomes the non-identifiability limitations, and effectively controls FPR.", "methods_and_evaluation_criteria": "The proposed method and evaluation criteria are reasonable and effective for detecting anomalous event sequences.", "theoretical_claims": "The authors provide rigorous theoretical analysis, proving the validity of the proposed test procedure and ensuring calibration-conditional FPR control.", "experimental_designs_or_analyses": "The experimental design is thorough, using multiple benchmark datasets with various anomaly types. The results highlight the superior performance of CADES compared to existing methods, both in terms of AUROC score and FPR control.", "relation_to_broader_scientific_literature": "The authors successfully fill a gap in the literature by applying conformal inference to anomaly detection in event sequences. The two proposed non-conformity scores overcome the non-identifiability limitations of previous test statistics, thereby ensuring more accurate and robust detection results. The paper has the potential to inspire other researchers to further advance the field of anomaly detection.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "Strengths:\n\n1. Applying conformal inference and neural temporal point processes to tackle the problem of anomaly detection in event sequences is both innovative and promising. This approach is backed by rigorous statistical theories, enhancing its reliability in real-world deployment.\n\n2. The proposed two non-conformity scores, which overcome the non-identifiability limitations of existing test statistics, are well motivated and reasonable.\n\n3. Extensive experiments across multiple benchmark datasets validate the effectiveness and reliability of the proposed method.\n\nWeaknesses:\n\nSome experimental results require further clarification, such as:\n\n1. The reason behind the poor performance of the 3S statistic under the Uniform scenario in Section 4.1.\n\n2. In Figure 7, why are the distributions of ID scores different in the RenewalA and SelfCorrecting scenarios?", "other_comments_or_suggestions": "1. Could you provide the computational time of the CADES method?\n\n2. How would CADES perform in scenarios where event sequences have sparse data?", "questions_for_authors": "Please see above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JBMweByUCV", "reviewer": "Reviewer_A6ji", "summary": "The paper is the first to extend conformal inference to anomaly detection in event sequences and proposes a novel method called CADES, which provides statistical guarantees of validity. Notably, CADES addresses the severe non-identifiability issues found in previous methods by developing two new powerful non-conformity scores. The authors provides both rigorous theoretical analysis of FPR and thorough evaluations, demonstrating CADES' strong performance in event sequence anomaly detection.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your recognition and feedback. We provide answers to your concerns as follows:\n\n**Q1**: It would be more comprehensive if the literatures on applying conformal inference to time series were discussed.\n\n**R1**: Thank you for your valuable suggestion and the references you provided. We will incorporate a discussion of them in the paper. In this work, we specifically apply conformal inference to event sequences. Although both time series and event sequences are types of sequential data, they differ significantly [1]. Specifically, in time series, time serves only as the index to order the sequence of values for the target variable. In event sequences, time is treated as a random variable representing the timestamps of asynchronous events, commonly modeled using temporal point processes. In addition, the references you mentioned primarily focus on prediction tasks, while our work addresses the anomaly detection task.\n\n**Q2**: What would the experimental performance be like if the KL divergence in Eq.(7) and Eq.(8) is replaced by the $L_2$ norm? \n\n**R2**: We conducted experiments by replacing the KL divergence with the $L_2$ norm on real-world datasets, referring to this method as CADES-$L_2$. The results demonstrate that CADES with KL divergence outperforms CADES-$L_2$ in terms of AUROC.\n\n- AUROC (\\%) results\n\n| Dataset | CADES-$L_2$ | CADES |\n| --- | --- | --- |\n| LOGS - Packet corruption (1\\%) | 90.75 | 96.48 |\n| LOGS - Packet corruption (10\\%) | 94.72 | 99.48 |\n| LOGS - Packet duplication (1\\%) | 92.03 | 92.88 |\n| LOGS - Packet delay (frontend) | 93.66 | 98.15 |\n| LOGS - Packet delay (all services) | 93.49 | 99.33 |\n| STEAD - Anchorage, AK | 97.95 | 99.31 |\n| STEAD - Aleutian Islands, AK | 99.82 | 99.95 |\n| STEAD - Helmet, CA | 97.46 | 99.30 |\n\n\nReference:\n\n[1] Xiao S, Yan J, Yang X, et al. Modeling the intensity function of point process via recurrent neural networks. AAAI, 2017.", "claims_and_evidence": "The claims are supported by rigorous theoretical analysis and superior empirical performance on both synthetic and real-world datasets.", "methods_and_evaluation_criteria": "The proposed method is built on a statistically sound framework (conformal inference), and the evaluation criteria used (AUROC, TPR and FPR) are appropriate for the anomaly detection task.", "theoretical_claims": "I have checked the correctness of all proofs supporting the theoretical claims, which are both clear and solid.", "experimental_designs_or_analyses": "The experiments are well-designed and the results show that CADES outperforms existing methods on both synthetic and real-world datasets. The ablation study further supports the importance of using the two proposed non-conformity scores and two-sided p-values.", "relation_to_broader_scientific_literature": "The paper clearly explains how it is related to previous work. The paper draws a new connection between conformal inference and event sequence anomaly detection, and then proposes an effective detection method with statistical guarantees.", "essential_references_not_discussed": "While the paper cites relevant works well, it would be helpful to also discuss related work on applying conformal inference to time series (see Weaknesses for detailed references).", "other_strengths_and_weaknesses": "Strengths:\n\n- The idea of detecting anomalies in event sequences with conformal inference is novel and interesting. The paper is well-written and the presentation is coherent.\n\n- The motivation is clear. This paper provides theoretical guarantees and empirical validation for FPR control, and proposes new non-conformity scores to address previous non-identifiability issues.\n\n- The effectiveness of CADES is thoroughly evaluated on experiments, showcasing superior performance compared to existing approaches.\n\nWeaknesses:\n\nI don't find any obvious weaknesses. It would be more comprehensive if the literatures on applying conformal inference to time series were discussed.\n\n[1] Xu C, Xie Y. Conformal prediction interval for dynamic time-series. ICML, 2021.\n\n[2] Zaffran M, Féron O, Goude Y, et al. Adaptive conformal predictions for time series. ICML, 2022.", "other_comments_or_suggestions": "No further comments or suggestions.", "questions_for_authors": "What would the experimental performance be like if the KL divergence in Eq.(7) and Eq.(8) is replaced by the $L_2$ norm?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2nUOco9Xee", "reviewer": "Reviewer_aGKQ", "summary": "The paper introduces CADES, a novel anomaly detection method for continuous-time event sequences under the conformal inference framework. The authors propose two new non-conformity scores tailored to event sequences based on the time-rescaling theorem, which address the non-identifiability issues of existing test statistics. CADES combines these scores with Bonferroni correction to conduct statistical hypothesis testing and provides theoretical guarantees on the false positive rate (FPR). Extensive experiments on synthetic and real-world datasets demonstrate that CADES outperforms state-of-the-art methods while maintaining FPR control.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation of our work and suggestions. We provide answers to your concerns as follows:\n\n**Q1**: While the authors provide theoretical guarantees on FPR control, the practical implication and significance of these guarantees in real-world scenarios could be explained in more detail.\n\n**R1**: FPR control guarantees are crucial for safety-critical applications, such as cybersecurity, finance, and healthcare. For example, in electronic health records (EHRs), maintaining low false positive rates is vital. High false positives could lead to incorrect diagnoses or unnecessary treatments, potentially compromising patient safety. Our method ensures that detected anomalies are more likely to be genuine, helping healthcare professionals make more reliable decisions. We will add this content in our paper. \n\n**Q2**: The paper could explore the potential for extending the proposed method to other types of sequential data, such as time series or spatial-temporal data, to broaden its applicability.\n\n**R2**: In this work, we primarily focus on anomaly detection in event sequences. Unlike other types of sequential data, event sequence data is asynchronous and of variable length, commonly modeled using temporal point processes. As you suggested, we plan to extend our method to other types of sequential data as future work.\n\n**Q3**: It would be clearer to denote the new observation $X$ in the problem statement of Section 2 as $X_{\\text{test}}$.\n\n**R3**: Thank you for pointing this out. We have already made the replacement in our manuscript.", "claims_and_evidence": "The claims are well-supported by both theoretical guarantees and empirical results.\n\n- The paper provides rigorous proofs for the validity of the p-values used in CADES and offers theoretical guarantees on calibration-conditional FPR control.\n\n- The experimental results show that CADES achieves superior detection performance compared to state-of-the-art methods while controlling the FPR at a pre-specified level.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense for the problem of anomaly detection in event sequences. \n\n- The application of conformal inference and the design of two non-conformity scores effectively address the limitations of existing methods, particularly their inability to control the FPR and the non-identifiability issues. \n\n- The evaluation criteria on synthetic and real-world datasets, including AUROC, FPR control, and runtime comparisons, are appropriate for assessing the performance of anomaly detection methods.", "theoretical_claims": "The theoretical claims about the validity of the p-values used in CADES and the guarantees on calibration-conditional FPR are backed by rigorous proofs. The authors also provide additional theoretical insights into the conditions under which these guarantees hold, especially with regard to the size of the calibration set.", "experimental_designs_or_analyses": "The experimental designs and analyses are sound and well-executed. It is worth noting that the summary and visualization of the experimental results are neat and clear.\n\n- The paper conducts experiments on synthetic datasets to evaluate CADES' performance in GOF testing for the SPP and addressing the non-identifiability issues, as well as on real-world datasets to demonstrate its practical applicability in detecting various types of anomalies and controlling the FPR.\n\n- The ablation studies verify the effectiveness of combining two non-conformity scores and using two-sided p-values.", "relation_to_broader_scientific_literature": "The key contributions of this paper lie in its exploration of conformal inference and anomaly detection in event sequences. The paper effectively bridges the gap between these two domains, making a significant advancement, which could also inspire future work on anomaly detection in other types of sequential data.", "essential_references_not_discussed": "No. The paper covers the essential references related to conformal inference, event sequence anomaly detection, and temporal point processes.", "other_strengths_and_weaknesses": "Strengths:\n\n- **Originality**: The paper establishes a novel connection between conformal inference and anomaly detection in event sequences. It combines two newly designed non-conformity scores with Bonferroni correction to conduct statistical hypothesis testing and addresses the non-identifiability issues of existing approaches.\n\n- **Theoretical Analysis**: The paper offers theoretical support for FPR control of anomaly detection in event sequences. Furthermore, the paper provides guarantees on calibration-conditional FPR control, which are stronger than the marginal FPR guarantees.\n\n- **Evaluation**: The experimental evaluations are thorough and well-designed. The results demonstrate that CADES outperforms state-of-the-art methods on both synthetic and real-world datasets while effectively controlling the FPR at a pre-specified level.\n\n- **Presentation**: The paper is well-organized. It presents a clear explanation of the motivation, methods, and experiments. This makes the paper easy to follow.\n\nWeaknesses:\n\n- While the authors provide theoretical guarantees on FPR control, the practical implication and significance of these guarantees in real-world scenarios could be explained in more detail.", "other_comments_or_suggestions": "Suggestions:\n\n- The paper could explore the potential for extending the proposed method to other types of sequential data, such as time series or spatial-temporal data, to broaden its applicability.\n\n- It would be clearer to denote the new observation $X$ in the problem statement of Section 2 as $X_{\\text{test}}$.", "questions_for_authors": "Please refer to weaknesses and suggestions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents CADES, a method for anomaly detection in event sequences using conformal inference. The approach introduces non-conformity scores tailored to event data and combines them using Bonferroni correction to address challenges related to non-identifiability and FPR control. Reviewers agree that the paper is well-motivated, with theoretical analysis and comprehensive empirical validation on both synthetic and real-world datasets. The theoretical guarantees on calibration-conditional FPR and the ablation studies were noted as particular strengths. Some reviewers requested clarification on specific experimental results and suggested a broader discussion of related work on conformal inference in time series. The authors addressed these points in the rebuttal, providing additional explanations and experimental comparisons. While some aspects, such as the practical implications of FPR control, could be discussed in more detail, the overall assessment is that the paper makes a solid contribution to anomaly detection in sequential data.
Decision: Accept (poster)