Title: Causal Discovery from Event Sequences by Local Cause-Effect Attribution
Venue: NeurIPS 2024
Accepted: yes
Keywords: causality

causal discovery

event sequences
Abstract: Sequences of events, such as crashes in the stock market or outages in a network, contain strong temporal dependencies, whose understanding is crucial to react to and influence future events. In this paper, we study the problem of discovering the underlying causal structure from event sequences. To this end, we introduce a new causal model, where individual events of the cause trigger events of the effect with dynamic delays. We show that in contrast to existing methods based on Granger causality, our model is identifiable for both instant and delayed effects.

We base our approach on the Algorithmic Markov Condition, by which we identify the true causal network as the one that minimizes the Kolmogorov complexity. As the Kolmogorov complexity is not computable, we instantiate our model using Minimum Description Length and show that the resulting score identifies the causal direction. To discover causal graphs, we introduce the Cascade algorithm, which adds edges in topological order. Extensive evaluation shows that Cascade outperforms existing methods in settings with instantaneous effects, noise, and multiple colliders, and discovers insightful causal graphs on real-world data.

Causal Discovery from Event Sequences by Local Cause-Effect Attribution

Abstract

Sequences of events, such as crashes in the stock market or outages in a network, contain strong temporal dependencies, whose understanding is crucial to react to and influence future events. In this paper, we study the problem of discovering the underlying causal structure from event sequences. To this end, we introduce a new causal model, where individual events of the cause trigger events of the effect with dynamic delays. We show that in contrast to existing methods based on Granger causality, our model is identifiable for both instant and delayed effects.

We base our approach on the Algorithmic Markov Condition, by which we identify the true causal network as the one that minimizes the Kolmogorov complexity. As the Kolmogorov complexity is not computable, we instantiate our model using Minimum Description Length and show that the resulting score identifies the causal direction. To discover causal graphs, we introduce the CASCADE algorithm, which adds edges in topological order. Extensive evaluation shows that CASCADE outperforms existing methods in settings with instantaneous effects, noise, and multiple colliders, and discovers insightful causal graphs on real-world data.

1 Introduction

Suppose we are considering a multivariate event sequence. What caused a specific event to happen? Which variables are causes of each other? Data-driven methods can infer causal relationships from observed data. Existing methods for discovering causal networks from event sequence data [1-3] are based on Granger causality [4]. This purely predictive notion defines a variable X to be a cause of another variable Y if the past of X helps to predict the future Y . It is a relatively weak notion of causality that excludes instantaneous effects and is often unable to discover true causal dependencies; in Granger causality, baking a cake is causal to a birthday.

In this paper, we instead build upon Pearl's model of causality, which assumes the existence of an an underlying causal structure in the form of a directed acyclic graph (DAG) [5]. In our context, such a graph describes the causal relationships between types of events, such as alarms in a network. We propose a new causal model for event sequences based on a one-to-one matching of individual events, where we model the process of one individual event of a certain type possibly causing an

Sascha Xu ◦

Jilles Vreeken

individual event of another type. In our model, we take into account the uncertainty of whether an event is actually caused or independently generated, the uncertainty of an event actually causing an effect or failing to do so, and the uncertainty of the delay between cause and effect. As we will show, our model has several advantages, such as a clear notion of what event caused another and the identifiability for both instant and non-instant effects.

We base our theory on the Algorithmic Markov Condition (AMC) [6], which postulates that the true causal model achieves the lowest Kolmogorov complexity. As Kolmogorov complexity is not computable, we instantiate it via the Minimum Description Length (MDL) principle [7]. We show that our score is consistent, identifies the true causal direction for both instantaneous and delayed effects, and formally connect it to Hawkes processes. To discover causal networks in practice, we introduce the CASCADE algorithm, which adds edges in topological order. Through extensive empirical evaluation, we show that CASCADE performs well in practice and outperforms the state of the art by a wide margin. On synthetic data, CASCADE recovers the ground truth without reporting spurious edges, and on real-world data, it returns graphs that correspond to existing knowledge.

2 Preliminaries

We write i → j when S i is a cause of S j and pa ( j ) for the set of parents of node j . We assume faithfulness, sufficiency, and the causal Markov condition [8].

Information-Theoretic Causal Discovery The Algorithmic Markov Condition (AMC) postulates that the factorization of the joint distribution according to the true causal network achieves the lowest Kolmogorov complexity [6]. The Kolmogorov complexity K ( x ) of a binary string x is the length of the shortest program p for a universal Turing machine U that computes x and halts [9]. For a distribution P , it is the length of the shortest program that uniformly approximates P arbitrarily well,

The AMC states that the Kolmogorov complexity of the joint distribution P ( X ) is the sum of the complexities of the conditional distributions P ( X i | pa ( i )) of the true DAG G ∗ , i.e.

up to a constant independent of the input. Due to, among others, the halting problem, Kolmogorov complexity is not computable, but we can approximate it from above. A statistically well-founded way to do so is by Minimum Description Length (MDL) [7, 10]. For a fixed class of models H , MDL identifies a description length L of encoding data X together with its optimal model,

S

i

:

S

j

:

0.2

0.3

N

j

∞

∞

Next, we introduce the assumed data generating process, its corresponding model class H and encoding length function L , and show under which conditions it can be identified.

3 Theory

To be able to infer causal relationships from observational data, we need to make assumptions about the underlying data-generating process [5]. The key assumption we make here is that an individual event of type i at time t with probability α i,j causes an individual event of type j at time t ′ ≥ t . To illustrate, we give a toy example in Fig. 1 in which event sequence S i causes event sequence S j . The individual events in S i occur uniformly at random. The first and third events in S i cause events in S j , resp. with a delay of 0.2 and 0.3. The other two events in S i do not cause events in S j , denoted by a delay of ∞ . The final event in S j is due to noise, marked by N j .

Next, we formally describe the causal mechanism. We differentiate between source and effect nodes.

Source nodes are nodes i in G ∗ with an empty parent set pa ( i ) . For source nodes i we assume that the events in S i occur uniformly at random with a rate of λ i events per time unit. This mechanism, commonly known as a homogeneous Poisson process , is used, for example, as a model for accident rates requiring hospital admission [11]. In this work, we focus on the delay times between individual events, denoted as d k for the delay t k -t k -1 and as ∆ i → i = { d k } n i k =1 for the sequence. For a Poisson process, the delay times are independently and exponentially distributed. Thus, we model a source event sequence S i as

Effect nodes are nodes j in G ∗ with at least one parent pa ( j ) . For each effect node j , the individual events in S j are either caused by an individual event in an S i with i ∈ pa ( j ) or due to noise. That is, reasoning from the causing node i , every event t k ∈ S i may trigger an event of the effect S j with a probability of α i,j . If triggered, an individual event in S j will occur after a random delay d k , drawn from a cause-effect specific delay distribution Φ i,j parameterized by θ i,j , e.g. the rate λ of an exponential distribution, and α i,j . If no event is triggered, then we model the delay as infinite, i.e. d k = ∞ . The sequence of delays ∆ i → j from S i → S j is modeled as

where ϕ i,j ( d ) denotes the density of the delay distribution. Thus, given the event sequence S i of the cause and delays ∆ i → j , the individual events in S j caused by S i are obtained by adding the delays d k to the time stamps t k of the individual cause events, with the reconstruction function f as

̸

In addition, individual events in S j can also be due to noise. Like for source nodes, we assume these a Poisson process as per Eq. (1) with rate λ j , i.e. N j ∼ Poisson ( λ j ) . Putting this together, given causal structure G ∗ , an effect event sequence S j is generated by taking the union of the individual delays from the causal parents pa ( j ) and the time stamps due to noise N j ,

Next, we instantiate an MDL score for this causal model and consider its identifiability.

3.1 Minimum Description Length Instantiation

We now develop a score for our causal model using MDL [7]. It consists of the cost of the data given the model, L ( S | Θ) , i.e. the negative log-likelihood of the data, and the cost of the model, i.e. that of the parameters, L (Θ) , and that of the graph, L ( G ) , all measured in bits.

Data Cost The cost of data in bits directly corresponds to its negative log-likelihood, i.e. the likelihood of each delay as per Eq. (2) over all the event sequences corresponding to the parents of node j and that of the noise events. Formally, we have

The first term encodes those events that were caused by the parent S i through the delays ∆ i → j . Here, we use a Shannon-optimal coding that requires -log( ϕ i,j ( d k )) bits per sample [7]. In the second term, we encode all remaining events as noise using the delay distribution of a Poisson process. For source events, i.e. variables without any parents, only the noise term is present.

The cost of all sequences is then simply L ( S | G, Θ) = ∑ j ∈ [ p ] L ( S j | S pa ( j ) , Θ) .

Parameter Cost Next, we define the costs of the DAG, L ( G ) , and that of the parameters, L (Θ) . We encode the DAG in topological order. Per node we encode its number of parents | pa ( i ) | and identify which those are, i.e. L ( G ) = ∑ d -1 k =0 ( log( k ) + log ( k | pa ( i ) | ) ) . Depending on their type, we encode the parameters θ ∈ Θ . For parameters θ ∈ N we use L N , the MDL-optimal encoding for integers [12]. It is defined as L N ( z ) = log ∗ z +log c 0 where log ∗ z is the expansion, log z +loglog z + · · · in which we only include positive terms. To ensure this is a valid encoding, i.e. one that satisfies the Kraft inequality, we set c 0 = 2 . 865064 [12]. For parameters θ ∈ R we use L R ( θ ) = L N ( d )+ L N ( ⌈ θ · 10 d ⌉ ) + 1 as the number of bits needed to encode a real number up to a user-specified precision [7]. For an edge i → j , the parameters are the trigger probability α i,j and those of the delay distribution ϕ . For the cost of an edge we hence have L ( i → j ) = L R ( α i,j ) + ∑ θ ∈ ϕ i,j L ( θ ) . For Θ as a whole, we have L (Θ) = ∑ i → j ∈ G L ( i → j ) .

The overall MDL score is then

3.2 Identifiability

We now study the identifiability guarantees of our model and score, i.e. under what conditions we can identify from a given pair which is the cause and which the effect. Consider a pair of event sequences S i and S j , where S i → S j and the cause S i is a source event while S j is an effect event.

Instant Effects We begin with the case of instant effects only. Instant effects are observed when the sampling frequency of the data, e.g. a daily time scale, is insufficient to pick up a difference in time, such as a financial crash that can spread across the globe within hours. It is well-known that Granger causality cannot identify the causal direction for instant effects [13]. In Pearl's causal framework, on the other hand, the causal direction between two binary variables is identifiable [14-16]. We can build upon these results and show that our causal model and MDL-based score can identify the causal direction for non-deterministic instant effects.

Theorem 1. Let S i be an event sequence generated by a Poisson process as per Eq. (1) and S j be an effect of S i as per Eq. (3) , with, low noise λ j < (1 -α i,j ) λ i , and a trigger probability α i,j < 1 .

In the case of exclusively instant effects, i.e. ϕ i,j ( d ) = δ ( d ) , where δ ( d ) is the Dirac delta function, the MDL score in the true causal direction is lower than in the anti-causal direction, i.e.

We provide the full proof in the Appx. A.1, the general idea is under a non-deterministic trigger mechanism, i.e. α i,j < 1 . Then, in the causal direction, we can fully explain S j with S i , but not vice-versa, as the cause is generated by a Poisson process. If α i,j = 1 , i.e. the process is deterministic, we always observe cause and effect together, making them indistinguishable.

Delayed Effects Next, we consider the case of exclusively delayed effects. Here, there is an inherent asymmetry in the benefit of knowing the cause versus the effect. As shown by Didelez [17] for marked point processes, and later used by Xu et al. [1], Eichler et al. [18] for Granger causality in Hawkes processes, the intensity of observing the cause after an event of the effect is unchanged. That is, the future of the cause is independent of the past of the effect, while if a cause triggers an effect, the intensity of the effect is increased by the cause. We have the following identifiability guarantee.

Theorem 2. Let S i be an event sequence generated by a Poisson process as per Eq. (1) and S j be an effect of S i as per Eq. (3) , such that H ( ϕ j,j ) > H ( p (; θ i,j )) + α -1 i,j H ( B ( α i,j )) + α -1 j,j H ( B ( α j,j )) , where H denotes the entropy and B the Bernoulli distribution.

Then the matching in the anti-causal direction ∆ j → i of the effect S j to the cause S i has a worse MDL score than the true matching ∆ i → j , i.e.

We provide the full proof in the Appx. A.2. In the anti-causal direction S j → S i , the delay times follow the same exponential distribution of Exp ( λ i ) , leading to no gain in score compared to the

self-delay encoding. On the other hand, in the true causal direction, knowing the times of the cause leads to a better knowledge of the delay and hence a lower cost, so long as the delay distribution ϕ i,j provides a better description than treating it as noise. This requirement is closely related to the algorithmic Markov condition, which postulates that the shortest description of a variable is given through its parents.

3.3 Connection to Hawkes Processes

Hawkes processes [19] are analytically convenient and well-suited for modeling real-world processes where events trigger further events, e.g. earthquakes triggering aftershocks. Consequently, the majority of methods focusing on Granger causality are based on Hawkes processes [1-3]. The Hawkes process extends the Poisson process by incorporating the influence of past events on the intensity, i.e. the rate of occurrence of future events. This is done by means of excitation functions v i,j ( t -t k ) , which increase/inhibit the intensity of future events based on past events. The intensity function of a Hawkes process under a DAG structure is given by

Each event t k ∈ S i increases the intensity of seeing an effect by v i,j ( t -t k ) . The main difference between our model and a Hawkes process is our direct trigger model from cause to effect. In a Hawkes process, an event of type i increases the intensity and, therewith, the probability of effect events occurring. That is, contrary to our framework, in a Hawkes process there is no explicit one-to-one relationship between causing and effect events, i.e. no one event can be attributed solely to causing another. Nonetheless, in Appendix A.5 we show how to identify S i as a parent of S j by constructing a sequence of delays ∆ i → j with the most-influential past event and therewith ϕ i,j . If ϕ i,j fulfills Theorem 2, we can identify S i as a parent of S j . Hence, should the data be generated by a Hawkes process, our method can still pick up the causal relationship between the two event classes, so long as there are sufficiently many events where S i is the primary cause.

4 Algorithm

With our model in place, we now turn to the problem of discovering the underlying causal structure from an observed sequence of events. In recent years, several methods that find and proceed on a topological ordering of the true graph have been introduced [20-22], which outperform other score-based frameworks such as GES [23] in terms of accuracy. We here propose the CASCADE algorithm that instantiates this idea for information-theoretic scores. We prove that in the limit, it recovers not only the correct topological ordering but also the correct parent set of each node.

CASCADE derives its guarantees from the gain in bits of adding an edge i → j to the model, i.e.

The edge cost L ( i → j ) is constant and independent of the number of samples n i . In the limit n i → ∞ , the gain inherits the identifiability guarantees from Sec. 3.2, such that g ( i → j | Θ) > g ( j → i | Θ) if S i is a true ancestor of S j . In other words, the gain of an edge is greater in the causal than in the anti-causal direction.

4.1 High Level Overview

CASCADE initializes the model Θ with an empty graph G and without any causal edges. During the search, we maintain a set of nodes C = [ p ] , from which we remove nodes in a topological order of G ∗ . We iterate over the following four steps until C is empty.

Source Node Selection : Select that node i ∈ C with minimal gain for any edge j → i , j ∈ C , i.e.

Edge Adding : Add all outgoing edges from i → j , j ∈ C , to G that improve our score.

Edge Pruning : Remove all incoming edges j → i from G that harm our score.

4. Node Set Update Remove i from C .

Each iteration, CASCADE selects that node i , which has the minimal achievable gain when adding any edge j → i to the current graph G , expressed in Eq. (4); below, we will show that under our causal model this node is guaranteed to be a true source of the graph G ∗ . We then add all edges from i to nodes j ∈ C that improve our score; provided that all true causal edges i → j were added, there is now at least one node j ∈ C whose parents are all accounted for, that in the next iteration can be identified as a source. We remove edges j → i from G to remove shortcuts. By repeating this process, CASCADE proceeds in a topological order of the true graph G ∗ . In total, CASCADE requires p iterations, leading to an overall cubic complexity O ( p 3 ) .

Source Node Selection. To identify a source node in the graph, we can use the identifiability guarantees from Sec. 3.2. They show that the gain g ( i → j | Θ) correctly orients the edge i → j in the unconfounded bi-variate case. We additionally require that the edge gain is pathwise oracle , i.e. it can identify the direction of the path from i to k .

Theorem 3. Given an event sequence S generated by a causal structure G ∗ , let S i be a source node of G ∗ and S v be a descendant of S i , where there exists a path i → j →··· → v in G ∗ .

Then, the gain in the causal direction of the path g ( i → v | Θ) -g ( v → i | Θ) is greater.

We provide the proof in Appx. A.3. We can now show that the criterion in Eq. (4) selects nodes in a topological ordering of G ∗ . Initially, CASCADE has to identify a true source of G ∗ , i.e. a node i without parents. For that node i , all other nodes j are either ancestors or independent of i . If i is an ancestor of j , then g ( j → i | Θ) -g ( i → j | Θ) < 0 , i.e. the gain in the anti-causal direction is lower. If i is independent of j , then g ( j → i | Θ) = 0 and g ( i → j | Θ) = 0 . Hence, the maximum achievable gain for a node without parents is zero.

Now consider a node v which does have a parent. For this node, there exists an ancestor u which is a true source. Hence, for that pair g ( u → v | Θ) -g ( v → u | Θ) > 0 . Consequently, the maximum achievable gain is positive, whilst for a source node, we can maximally achieve zero, allowing us to identify true sources with Eq. (4).

In the next step, we add all outgoing edges from the source i to G that improve the score. As G ∗ is a DAG, we are now guaranteed to have another node j , whose incoming edges are all accounted for in G . Then, as per the causal model from Eq. (3), the only events that remain are those of the noise N j . Hence, j is now a source node for which the guarantees from above apply. By repeating this process, CASCADE thus follows a topological order of G ∗ .

Edge Addition Given a source i , CASCADE adds all outgoing edges i → j that improve the score. We restrict the set to nodes j ∈ C from the candidate set only, i.e. to nodes further down the topological order. By the Algorithm Markov Condition, the description length of the true set of parents of a node j is smaller than the description length of any other set of parents, and hence the gain of the true edge is positive in the limit of n i →∞ .

When adding an edge i → j , where there is already an edge v → j , we use an Expectation Maximization approach to attribute all events to their respective cause. That is, we first find the bi-variate alignment ∆ i → j using all events in S j . Now, it is very likely that there are conflicts between ∆ i → j and ∆ v → j , as the same event can be attributed to both i and v . In those cases, we choose that event where the density ϕ i,j / ϕ v,j is higher and set the delay to infinity in the other matching. After re-assigning all events, we refit the delay distribution function ϕ i,j using the new matching.

Edge Pruning Lastly, we deal with removing any shortcuts that have been added in the previous iteration. With the previous two steps, we are guaranteed to have a superset of all true causal edges incoming to i . Fortunately, we can prune such edges directly with MDL by removing any incoming edge i → j that does not improve the MDL score. In the chain graph i → j → v , we would remove i → v as the edge j → v is sufficient to explain the data. In practice, given the current set of parents of i in G , we search for the true set

i

j

v

of parents by starting with the empty set and greedily adding only those edges that improve the score. As we show in Appx. A.3, a shortcut always has a lower gain than the true edge and hence will not be

re-added. In this manner, we are asymptotically left with only the true causal parents. We can now finally show the consistency of CASCADE.

Theorem 4. Given an event sequence S , where each individual subsequence S i was generated as per Eq. (3) by an underlying causal graph G ∗ . Assuming all ∆ i → j are the true causal matchings. Under the Algorithmic Markov Condition, CASCADE recovers the true graph G ∗ for n →∞ .

We postpone the proof to Appx. A.7. In the experiment section, we show that CASCADE recovers the true DAG even in challenging settings and works well on real-world data.

5 Related Work

Causal discovery on observational data is an active research topic. Two main research directions exist: constraint-based [5] and score-based [23, 24] methods. Our approach belongs to the latter and is based on the Algorithmic Markov Condition [6]. While Kolmogorov complexity is uncomputable, Marx and Vreeken [10] formally showed that if we instantiate the AMC with two-part MDL [7], we, on expectation, achieve the same results. MDL has been successfully used for bivariate causal inference [25, 15, 26], causal discovery [27], identifying hidden confounding [28], identifying mechanisms shifts [29], and identifying selection bias [30].

In this paper, we consider point processes. Particularly close to our method are Hawkes processes [31] as a way to model the influence of past events onto future events. As such, our work is also related to the concept of transfer entropy [32], which measures the influence in terms of Shannon entropy. Budhathoki and Vreeken [33] proposed an MDL-based method for bivariate causal inference on event sequences, which is unsuitable for learning a global causal structure.

Existing methods for discovering causal graphs from event sequence data focus on different instantiations of Granger causality and can mostly be categorized by different intensity functions. Most common are parametric approaches with different regularizing [34, 2, 1]. ADM4 [34] uses the nuclear matrix norm in combination with lasso, THP [2] uses BIC for regularization. The method MDLH by Jalaldoust et al. [3] is most closely related, as they also use MDL for regularization. NPHC [35] takes a non-parametric approach by using a moment matching method to fit second and third-order integrated cumulants. A recent development is neural point processes. Mei and Eisner [36] propose a deep neural network that learns the dependencies [36], which Xiao et al. [37] extended to include attention mechanisms. Zhang et al. [38] first learn a neural point process and then use a feature importance attribution method to obtain a weight matrix of pairwise variable influence.

6 Experiments

We evaluate CASCADE on both synthetic and real-world data. CASCADE is implemented in Python. We provide the source code, along with the synthetic data generator and the used real-world datasets online. 3 We compare our method to four of state of the art methods: THP [2] as representative for the regularized parametric approaches, CAUSE [38] as representative for the neural point processes and NPHC [35] as a representative non-parametric approach, and MDLH [39] who also rely on MDL, as our most closely related competitor. CAUSE and NPHC do not return a graph but rather a weight matrix where the weight indicates the strength of the causal relation. On synthetic data, we can obtain a graph by thresholding such that we optimize the F 1 score.

6.1 Evaluation

We evaluate the estimated graphs in terms of structural similarity by the Structural Hamming Distance (SHD) [40], in terms of causal similarity by the Structural Intervention Distance (SID) [41], and predictive performance by F1 score. To compare graphs of different sizes, we report the scores normalized by the maximally achievable SHD/SID and show the unnormalized scores in Appendix B.

NPHC, CAUSE, and MDLH can and often do return cyclic graphs. As SID is strictly only defined for acyclic graphs, we omit these methods from the SID evaluation.

5

10

15

20

30

40

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

SHD

5

10

15

20

30

40

0

0

.

2

0

.

4

0

.

6

0

.

8

1

SID

5

10

15

20

30

40

0

0

.

2

0

.

4

0

.

6

0

.

8

1

F1

(a)

0.9

0.8

0.7

0.6

0.5

0

0.1

0.2

0.3

SHD

0.9

0.8

0.7

0.6

0.5

0

0.2

0.4

0.6

SID

0.9

0.8

0.7

0.6

0.5

0

0

.

2

0

.

4

0

.

6

0

.

8

1

Noise level

a

F1

(b)

50

60

70

80

90 100 150 200

0

0.001

0.002

0.003

SHD

MDLH

THP

CAUSE

NPHC

CASCADE

50

60

70

80

90 100 150 200

0

.0001

.0002

.0003

.0004

SID

50

60

70

80

90 100 150 200

0

0.2

0.4

0.6

0.8

1

F1

(c)

6.2 Synthetic Data

We begin by comparing all methods on data with known ground truth. To this end, we generate synthetic data. We generate both data within and outside our causal model and vary aspects such as noise intensity, number of event types and the number of parents of a variable. We describe the full data-generating process in Appendix B.

Sanity Check We start with a sanity check on data without any structure over 20 variables, CASCADE correctly does not report any causal edge. THP reports in 45% of the cases at least one spurious edge. We omit the results of CAUSE and NPHC as it is unclear how to choose a meaningful, non-trivial threshold, in this setting. MDLH did not terminate within 96 hours.

Scalability We evaluate how well each method scales under an increasing number of variables. We vary the number of nodes, which correspond to the number of unique event types, from 5 to 50 and report the results in Fig. 3a. As MDLH did not terminate within 96 hours for 15 variables, we omit it from here on out. For a lower number of nodes, both CASCADE and THP obtain far better results than NPHC and CAUSE. With increasing event types, all methods SID and F1 scores decrease. Amongst all methods, CASCADE scales best with an increasing number of nodes, whereas Granger causality based methods such as THP and NPHC find many spurious edges of connected but not causal variables. On the other hand, CASCADE is the most accurate method for a higher number of nodes, showing the efficacy of its causal model and MDL-based approach.

Noise Next, we assess the impact of noise, which are events that is not caused by any parent. To this end, we vary the 'cause' probability and the fraction of events due to additive noise. We do so by varying a noise parameter a , adding an additional n i · a events to S i (additive noise), and by setting the 'cause' probability α = 1 -a , i.e. we decrease additive noise and increase trigger probability. We show the results in Fig. 3b. We observe that CASCADE does quite well for high noise and that for noise levels of a = 0 . 7 and lower, it (mostly) recovers the true DAG. All other methods perform considerably worse.

Colliders Matching an effect event to the correct parent, resp. modeling the correct excitation, becomes increasingly challenging for a larger number of parents. We test this through a setting where half ( ⌈ p -1 2 ⌉ ) the variables converge into a collider, and the other half ( ⌊ p -1 2 ⌋ ) are independent. We vary the total number of variables, p and we show the results in Fig. 3c. We observe that CASCADE achieves almost perfect results. THP is robust, but with an increasing number of nodes, it starts to miss edges. Beyond 100 variables, it does not terminate within 24 hours. To validate that our method can recover structures with multiple colliders, we repeat the same experiment where 10% of nodes are colliders. That is, for 50 event types, 5 are colliders and 23 direct causes of all 5 colliders. The remaining 22 are independent. Resulting in an F 1 score of 0.97 for 50 event types, slightly decreasing to 0.82 for 200 event types; as such CASCADE can deal well with multiple colliders.

Instantaneous Effects Next, we evaluate performance under instantaneous effects. First, we consider data with exclusively instant effects. CASCADE achieves an average unnormalized SHD of 32.8. The second best-performing method, NPHC, achieves 46.85. Next, we generate a setting where 90% of the effects are instantaneous and the others occur with a small delay. CASCADE improves to an SHD of 19.45, while NPHC achieves the second lowest average with 47.5. We provide all results in the Appendix B.

Hawkes Processes Finally, we evaluate how effectively CASCADE recovers the true DAG on data generated by a Hawkes process. We vary the intensity of the excitation function, i.e., the expected number of events generated per cause. We show the results in Figure 4. We observe that CASCADE performs best when our assumptions hold, when there is one effect per cause or fewer, but still demonstrates strong performance across all settings.

6.3 Real-World Data

0.4 0.7 1.0 1.3 1.6 1.9 2.2 2.5 2.8 3.1

0

0

.

5

1

Expected Events per Cause

F1

We evaluate CASCADE on three distinct datasets of real-world event sequences. We begin by evaluating CASCADE on a dataset of network alarms, where the causal structure is known.

Network Alarms This data was provided by Huawei for the NeurIPS 2023 CSL-competition 4 and consists of data from a simulated network of devices in which alarms can cause other alarms. We run all methods and get an (unnormalized) SHD score of 42 for CASCADE, 127 for THP, 214 for NPHC, and 1564 for CAUSE. As the network connectivity structure is known, we can take it into account during the search. THP supports this natively, CASCADE can be trivially constrained to only consider the given edges. CASCADE correctly identifies 142 out of 147 causal edges, THP 20. Neither method reports spurious edges. We show the full recovered graph in Appendix B.5.

Global Banks Second, we run CASCADE on a daily return volatility dataset [42], we follow the preprocessing of Jalaldoust et al. [39], specifically we turn the time series into an event sequence by rolling a one year window over the data and register an event if the last value is among the top 10%. The dataset includes the 96 world's largest publicly traded banks. We show the largest discovered subgraph in Fig. 5. In addition, three unconnected sub-graphs are discovered, one covering two banks in Australia and two others connecting banks in Japan, which we provide in Appendix B.5.

Daily Activities We run CASCADE on a dataset of recorded daily activities [43]. Our method reports plausible causal connections such as Sleeping End → Showering Start → Showering End → Breakfast Start → Breakfast End , etc. We show the complete graph in the Appendix B.5. This result reinforces the suitability of our causal model and CASCADE for real-world data, and illustrates the potential of our method to discover causal structures in a wide range of applications.

7 Conclusion

Westudied the problem of causal discovery from event sequences, we propose a cause-effect matching approach to learn a fully directed acyclic graph (DAG). To this end, we introduced a new causal model

HSBC

BNP

Paribas

Barclays

RBS

Santander

Goldman

Sachs

Intesa

Sanpaolo

BBVA

Pohjola

Bank

JPMorgan

Chase & Co

Deutsche

Bank

Credit

Agricole

SocGen

ING

Lloyds

Credit

Suisse

Nordea

Commerzbank

Danske

Svenska

Handelsbanken

SEB

KBC

Fifth Third

Erste

Group Bank

UBS

Bank of

America

Citigroup

Wells

Fargo

Morgan

Stanely

TD Bank

Standard

Chartered

CIBC

BNY

Mellon

U.S.

Bancorp

PNC

Capital

One

State Street Corporation

BB&T Corp

National

Bank of

Canada

SunTrust

American

Express

DNB ASA

BMO

Regions

Financial

ANZ

RBC

Bank of

Nova Scotia

Malayan

Banking

Berhad

Asia

Europe

America

Australia

Top 10 Banks

and an MDL based score. We proposed the CASCADE algorithm to discover causal graphs through a topological search from observational data. Finally, we evaluated CASCADE on synthetic and realistic data. On synthetic data, we find that CASCADE is either the best or close to the best-performing method across all settings, both within and outside our causal model. In particular, whenever conditions get challenging, e.g. due to noise or with multiple colliders, CASCADE outperforms all other methods by a significant margin. We examined how CASCADE performs on real world event sequences, where the true data-generating process may lie outside our causal model. We found that CASCADE recovers meaningful graphs that match with a common understanding of the world.

Limitations As is necessary, we have to make causal assumptions. The most prominent in our work is the direct matching between a cause event and an effect event - which precludes modeling of a single event causing multiple other events, as well as multiple events jointly causing a single effect event and that we only consider excitatory effects - which precludes modeling the absence of events due to a cause. Our proof of identifiability for instantaneous effects depends on the strengths of the trigger resp. noise probabilities. The identifiability of the model seems provable via the independence of these, but how to operationalize this into an effective score and search algorithm are open questions.

Future Work Currently, our structural equations are 'or' relations over the parent's variables. An interesting future direction would be to explore 'and' relations, e.g., A and B together cause C. This raises several questions, like how close to each other A and B have to occur or if the order matters. Another interesting future direction is to allow matching of multiple causing events to one event, where each parent could have caused the event. This would allow us to answer counterfactual questions, such as if a causing event had not occurred, would we nevertheless observe its effect? This strongly relates to the firing squad example by Pearl [5], where multiple guards shoot a prisoner at the same time; if one guard did not shoot, the prisoner would still have died.

References

Hongteng Xu, Mehrdad Farajtabar, and Hongyuan Zha. Learning granger causality for hawkes processes. In International conference on machine learning , pages 1717-1726. PMLR, 2016.

Ruichu Cai, Siyu Wu, Jie Qiao, Zhifeng Hao, Keli Zhang, and Xi Zhang. Thps: Topological hawkes processes for learning causal structure on event sequences. IEEE Transactions on Neural Networks and Learning Systems , 2022.

Amirkasra Jalaldoust, Kateˇ rina Hlaváˇ cková-Schindler, and Claudia Plant. Causal discovery in hawkes processes by minimum description length. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6978-6987, 2022.

Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society , pages 424-438, 1969.

Judea Pearl. Causality: Models, Reasoning and Inference . Cambridge University Press, 2nd edition, 2009.

Dominik Janzing and Bernhard Schölkopf. Causal inference using the algorithmic markov condition. IEEE Transactions on Information Theory , 56(10):5168-5194, 2010.

Peter Grünwald. The Minimum Description Length Principle . MIT Press, 2007.

Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques . MIT press, 2009.

Ming Li, Paul Vitányi, et al. An introduction to Kolmogorov complexity and its applications , volume 3. Springer, 2008.

Alexander Marx and Jilles Vreeken. Formally justifying mdl-based inference of cause and effect. In AAAI Workshop on Information-Theoretic Causal Inference and Discovery (ITCI'22) , 2022.

Donald C Weber. Accident rate potential: An application of multiple regression analysis of a poisson process. Journal of the American Statistical Association , 66(334):285-288, 1971.

Jorma Rissanen. A universal prior for integers and estimation by minimum description length. 11(2):416-431, 1983.

Jonas Peters, Dominik Janzing, and Bernhard Schlkopf. Elements of Causal Inference: Foundations and Learning Algorithms . The MIT Press, 2017. ISBN 0262037319.

Jonas Peters, Dominik Janzing, and Bernhard Scholkopf. Causal inference on discrete data using additive noise models. IEEE Transactions on Pattern Analysis and Machine Intelligence , 33(12):2436-2450, 2011.

Kailash Budhathoki and Jilles Vreeken. Accurate causal inference on discrete data. In Proceedings of the IEEE International Conference on Data Mining (ICDM'18) . IEEE, 2018.

Murat Kocaoglu, Alexandros Dimakis, Sriram Vishwanath, and Babak Hassibi. Entropic causal inference. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 31, 2017.

Vanessa Didelez. Graphical models for marked point processes based on local independence. Journal of the Royal Statistical Society Series B: Statistical Methodology , 70(1):245-264, 2008.

Michael Eichler, Rainer Dahlhaus, and Johannes Dueck. Graphical modeling for multivariate hawkes processes with nonparametric link functions. Journal of Time Series Analysis , 38(2): 225-242, 2017.

Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika , 58(1):83-90, 1971.

Peter Bühlmann, Jonas Peters, and Jan Ernest. Cam: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics , 42(6):2526-2556, 2014.

Paul Rolland, Volkan Cevher, Matthäus Kleindessner, Chris Russell, Dominik Janzing, Bernhard Schölkopf, and Francesco Locatello. Score matching enables causal discovery of nonlinear additive noise models. In International Conference on Machine Learning , pages 18741-18753. PMLR, 2022.

Spencer Compton, Kristjan Greenewald, Dmitriy A Katz, and Murat Kocaoglu. Entropic causal inference: Graph identifiability. In International Conference on Machine Learning , pages 4311-4343. PMLR, 2022.

David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research , 3(Nov):507-554, 2002.

Joseph Ramsey, Madelyn Glymour, Ruben Sanchez-Romero, and Clark Glymour. A million variables and more: the fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. International journal of data science and analytics , 3:121-129, 2017.

Alexander Marx and Jilles Vreeken. Telling cause from effect using mdl-based local and global regression. In 2017 IEEE international conference on data mining (ICDM) , pages 307-316. IEEE, 2017.

Sascha Xu, Osman Mian, Alexander Marx, and Jilles Vreeken. Inferring cause and effect in the presence of heteroscedastic noise. In Proceedings of the International Conference on Machine Learning (ICML) . PMLR, 2022.

Osman A Mian, Alexander Marx, and Jilles Vreeken. Discovering fully oriented causal networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 8975-8982, 2021.

David Kaltenpoth and Jilles Vreeken. Nonlinear causal discovery with latent confounders. In Proceedings of the International Conference on Machine Learning (ICML) . PMLR, 2023.

Sarah Mameche, David Kaltenpoth, and Jilles Vreeken. Learning causal models under independent changes. In Proceedings of Neural Information Processing Systems (NeurIPS) . PMLR, 2023.

David Kaltenpoth and Jilles Vreeken. Identifying selection bias from observational data. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) . AAAI, 2023.

Alan G Hawkes. Spectra of some self-exciting and mutually exciting point processes. Biometrika , 58(1):83-90, 1971.

Thomas Schreiber. Measuring information transfer. Physical review letters , 85(2):461, 2000.

Kailash Budhathoki and Jilles Vreeken. Causal inference on event sequences. In Proceedings of the 2018 SIAM International Conference on Data Mining , pages 55-63. SIAM, 2018.

Ke Zhou, Hongyuan Zha, and Le Song. Learning social infectivity in sparse low-rank networks using multi-dimensional hawkes processes. In Artificial Intelligence and Statistics , pages 641-649. PMLR, 2013.

Massil Achab, Emmanuel Bacry, Stéphane Gaïffas, Iacopo Mastromatteo, and Jean-François Muzy. Uncovering causality from multivariate hawkes integrated cumulants. Journal of Machine Learning Research , 18(192):1-28, 2018.

Hongyuan Mei and Jason M Eisner. The neural hawkes process: A neurally self-modulating multivariate point process. Advances in neural information processing systems , 30, 2017.

Shuai Xiao, Junchi Yan, Mehrdad Farajtabar, Le Song, Xiaokang Yang, and Hongyuan Zha. Learning time series associated event sequences with recurrent point process networks. IEEE transactions on neural networks and learning systems , 30(10):3124-3136, 2019.

Wei Zhang, Thomas Panum, Somesh Jha, Prasad Chalasani, and David Page. Cause: Learning granger causality from event sequences using attribution methods. In International Conference on Machine Learning , pages 11235-11245. PMLR, 2020.

Amirkasra Jalaldoust, Kateˇ rina Hlaváˇ cková-Schindler, and Claudia Plant. Causal discovery in hawkes processes by minimum description length. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6978-6987, 2022.

Markus Kalisch and Peter Bühlman. Estimating high-dimensional directed acyclic graphs with the pc-algorithm. Journal of Machine Learning Research , 8(3), 2007.

Jonas Peters and Peter Bühlmann. Structural intervention distance for evaluating causal graphs. Neural computation , 27(3):771-799, 2015.

Mert Demirer, Francis X Diebold, Laura Liu, and Kamil Yilmaz. Estimating global bank network connectedness. Journal of Applied Econometrics , 33(1):1-15, 2018.

Fco Javier Ordónez, Paula De Toledo, and Araceli Sanchis. Activity recognition using hybrid generative/discriminative models on home environments using binary sensors. Sensors , 13(5): 5460-5477, 2013.

Dominique MA Haughton. On the choice of a model to fit data from an exponential family. The annals of statistics , pages 342-355, 1988.

A Theory

A.1 Proof - Identifiability on Instant Effects

Theorem 1. Let S i be an event sequence generated by a Poisson process as per Eq. (1) and S j be an effect of S i as per Eq. (3) , with, low noise λ j < (1 -α i,j ) λ i , and a trigger probability α i,j < 1 .

In the case of exclusively instant effects, i.e. ϕ i,j ( d ) = δ ( d ) , where δ ( d ) is the Dirac delta function, the MDL score in the true causal direction is lower than in the anti-causal direction, i.e.

Proof. Let n i be the number of events in S i and n j the number of events in S j . As n i →∞

where B is the Bernoulli distribution. In the reverse direction, we have

To show

α i,j = α i,i since every event that does not cause an i, can not be explain by j in the reverse direction, hence

For H ( ϕ i,i ) < H ( ϕ j,j ) to hold n i > n j .

It directly follows that H ( ϕ i,i ) < H ( ϕ j,j ) , and hence for n i →∞

A.2 Proof - Identifiability on Delayed Effects

Theorem 2. Let S i be an event sequence generated by a Poisson process as per Eq. (1) and S j be an effect of S i as per Eq. (3) , such that H ( ϕ j,j ) > H ( p (; θ i,j )) + α -1 i,j H ( B ( α i,j )) + α -1 j,j H ( B ( α j,j )) , where H denotes the entropy and B the Bernoulli distribution.

Then the matching in the anti-causal direction ∆ j → i of the effect S j to the cause S i has a worse MDL score than the true matching ∆ i → j , i.e.

We will show that the delays between S i itself, i.e. ∆ i → i , and the delays between S j to S i , i.e. ∆ i → j , are equivalent.

Proof. We consider a source event S i with exponentially distributed delays, i.e. ∆ i → i ∼ exp( λ i ) . Consider any event t k ∈ S j , then the intensity of observing an event in S i at time t > t k is given by λ i . The distribution of the delay to the next event in S i is exponential with λ = λ i . Thus, the difference between

It remains to show that the likelihood in the causal direction is better when conditioning effect on the cause, i.e. L ( S j | S i ) < L ( S j ) .

Gain by i → j For i to cause j it has to provide information about j , that is the cost of selecting which i events cause j , and with what dealys. Additional it has to ofset the cost which j events do not have to be encoded as a self delay. Formally this is,

where n i,j are the number of events in j caused by i , and H ( p (; θ i,j )) is the entropy of distribution described by the pdf p .

As n →∞ ,

To show,

we can substitute n i H ( ϕ i,j ) = n i,j H ( p (; θ i,j )) + n i H ( B ( α i,j ))

Now, note that the number of caused items α i,j n i = n i,j and α j,j n j = n i,j , then it follows

Hence, we show that i → j is identifiable.

A.3 Proof - Path Identifiability

Theorem 3. Given an event sequence S generated by a causal structure G ∗ , let S i be a source node of G ∗ and S v be a descendant of S i , where there exists a path i → j →··· → v in G ∗ .

Then, the gain in the causal direction of the path g ( i → v | Θ) -g ( v → i | Θ) is greater.

Proof. We begin by proving that the path identifiability holds for a triplet of nodes i → j → v , by constructing a new alignment from i → v .

From ∆ i → j and ∆ j → v we can construct ∆ i → v . For each d k ∈ ∆ i → j , there is a corresponding d l ∈ ∆ j → v , i.e. the trigger time of the triggered event. To construct ∆ i → v , we consider the following cases:

If d k = ∞ for d k ∈ ∆ i → j , then d k ∈ ∆ i → v , is set to d k = ∞ .

̸

Let d l ∈ ∆ j → v be the delay of event a of type j where a has been caused by delay d k . If d k = ∞ for d k ∈ ∆ i → j and d l = ∞ then then d k ∈ ∆ i → v , is set to d k = ∞ .

̸

Let d l ∈ ∆ j → v be the delay of event a of type j where a has been caused by delay d k . If d k = ∞ for d k ∈ ∆ i → j and d l = ∞ then then d k ∈ ∆ i → v , is set to d k = d k + d l .

̸

As ∆ i → v is another valid alignment, and i remains a source node, the guarantees of Theorem 1 and 2 hold, i.e. the path is identifiable. This extends to a path of arbitrary length. Consider an additional edge v → w , then we construct the alignment ∆ i → w by considering the delays of ∆ i → v and ∆ v → w .

Furthermore, we can show that the true path j → v has a better gain than the shortcut i → v , so that we can remove the shortcut in the pruning stage. (1) Since i → j and j → v are independent processes, it follows either Var( ϕ i,v ) > Var( ϕ j,v ) , or α i,v > α j,v and by that a more costly description of v .

If j → v / ∈ G , we can construct a new function f i,v = f j,v ( f i,j ( S i , ∆ i → j ) ∪ N j , ∆ j → v ) by Theorem 2 it follows that edge i → v improves our score.

(3) Assume j → v ∈ G then each individual v event that is matched to by i → v is already matched to by j → v , and from (1) we know it does so cheaper, hence we get no gain by adding the shortcut i → v .

A.4 Consistency

Proof. Here we will show that L ( S n , Θ) asymptotically behaves like BIC . L ( S, Θ) directly corresponds to the log likelihood, which we rewrite as log p ( S n | Θ , G ) Our approach can be instantiated with arbitrary delay distribution, to show consistency we have to upper bound the number of parameters by O (log n ) , this trivially holds for the parametric setting we focus on in this paper, because | pa ( i ) | ∈ O (log n ) [27]. The encoding of the graph G is independent of n , i.e. fixed for a given network, hence in O (1) . Finally this results at,

we set c = d 2 where d is the number of free parameters, arriving at the BIC score.

From Haughton [44] and Chickering [23] we know that BIC identifies a Markov equivalence class of the true DAG. For the identifiability of undirected edges we refer to Theorem 1 and 2.

A.5 Connection to Hawkes Processes

The key difference between a linear Hawkes process and our model is the assumption of direct triggers, that is the mechanism of one event and one event only causing another. In a linear Hawkes process 'cause events' increase the intensity and therewith the probability of events occurring. However, one can generally not label for a specific event another as the 'cause', as each event is the result of a multitude of causes.

In this section, we are going to explore under which conditions we can identify a Hawkes process under our causal model.

Given an event sequence S j = { t k } n j t k =0 generated by a linear Hawkes process,

We construct a set of primary causes for each event t k ∈ S j as

where we consider as primary cause of an event that past event with the highest influence at time point t from a causal parent i ∈ pa ( j ) . Using these delays, we construct an alignment (mapping of delays) as

,

where t = arg max t ∈ S j ∧∃ ( t,i,t k ) ∈ C j v i,j ( t -t k ) if ∄ ( t, i, t k ) ∈ C j then β ( t k ) = ∞ . We consider t k ∈ S i a cause of an event t ∈ S j if it is the primary cause of t and t k has no stronger influence on any other event of S j . Finally, the influence has to be stronger than that of the base intensity of S j .

To be able to identify a causal edge between S i and S j the improvement gained by this alignment must outweigh the edge cost L ( i → j ) . For this, there are two conditions: firstly, the number of primary cause events from i to j , i.e. those instances where an event from S i has the maximum influence on an event from S j , must be large enough. This is the case as long as | ∆ i → j | increases with n j , i.e. the total number of events of S j . Then, in the limit n j → ∞ the number of primary cause events is large enough to offset the constant edge cost.

The achievable score gain is obtained by constructing a delay distribution from | ∆ i → j | as

If this density ϕ i,j ( d ) fulfills the conditions of Theorem 2, we can identify S i as a parent of S j in the limit of n j →∞ .

In conclusion, CASCADE can identify a causal pair generated under a Hawkes process, if there exist sufficiently many events from S i → S j , where v i,j ( t ) has the strongest influence on λ j ( t ) for some of the t . By aligning the delays of these events, we can identify the causal edge and recover the underlying causal structure.

A.6 Empirical Evaluation

[From main paper] To empirically evaluate how effectively CASCADE recovers the true DAG on data generated by a Hawkes process, we generate synthetic data using the tick library 5 . We vary the intensity of the excitation function, i.e., the expected number of events generated per cause. We show the results in Figure 6. We observe that CASCADE performs best when our assumptions hold, when there is one effect per cause or fewer, but still demonstrates strong performance across all settings.

A.7 Consistency of Algorithm

Theorem 4. Given an event sequence S , where each individual subsequence S i was generated as per Eq. (3) by an underlying causal graph G ∗ . Assuming all ∆ i → j are the true causal matchings. Under the Algorithmic Markov Condition, CASCADE recovers the true graph G ∗ for n →∞ .

Proof. We begin by proving that in the first step, CASCADE identifies a true source node of G ∗ . We denote the parents of i in the true causal graph G ∗ as pa ( i ) , whilst we write for the parents in the graph maintained by CASCADE as pa ′ ( i, G ) .

Let i ∈ [ p ] be a node of G ∗ without parents, i.e. a source. By Theorem 3, for a path i →··· → v ∈ G ∗ the gain in the causal direction g ( i → v | Θ) is greater than the gain in the reverse direction g ( v → i | Θ) .

̸

For all nodes v ∈ [ p ] , v = i , v is either an descendant of i or unrelated.

If v is a descendant of i , then the gain of g ( i → v | Θ) is greater than the gain of g ( v → i | Θ) .

If v is unrelated to i , then the gain in both sides is 0 .

0.4 0.7 1.0 1.3 1.6 1.9 2.2 2.5 2.8 3.1

0

0

.

5

1

Expected Events per Cause

F1

Hence it follows, that for a node i where pa ( i ) = ∅ ,

̸

On the other hand, consider a node i with parents pa ( i ) = ∅ . Then, as G ∗ is a DAG, there exists an ancestor v of i , where v is a source node, i.e. pa ( v ) = ∅ . For that v , it holds that the gain from v to i is greater than the gain from i to v . Hence, for a node i with parents, it holds that

Therefore, by taking the argmin over all nodes, CASCADE identifies the true source node of G ∗ , i.e.

Edge Addition Now, we show that CASCADE always identifies a true causal edge for n i →∞ . First, note that given a source node i , there do not exist any incoming causal edges in the graph G ∗ ,

Hence, by fitting outgoing edges only, we test all possible edges for i and never add a false oriented edge,

Finally, we recall that the true causal graph G ∗ is the graph that minimizes the description length of the data as per the Algorithm Markov Condition. Hence, adding a true causal edge to the graph will result in a lower description length, i.e.

Edge Removal Consider the node i , where pa ( i ) ∩ C = ∅ , and given a graph G where all true causal edges have been added, i.e.

It follows, that pa ′ ( i, G ) ⊇ pa ( i ) . By the Algorithm Markov Condition, the shortest description length of the data is achieved by the true causal graph G ∗ . Hence, a superset of the true parents of i will result in a higher description length, and it holds that

Therefore, by testing that subset of the parents of i results in a lower description length, CASCADE identifies the true parents of i .

Overall Consistency For n i →∞ , we note that in each step for the node i it holds that

i has no parents in the candidate set pa ( i ) ∩ C = ∅ .

We add no false oriented edges to the graph, as pa ( i ) ∩ C = ∅ = ⇒ ∄ j ∈ C : j → i ∈ G ∗ .

We add all true edges i → j to the graph G , i.e. ∀ j ∈ C, i → j ∈ G ∗ : L ( S j | S pa ′ ( j,G ) , Θ ′ ) > L ( S j | S pa ′ ( j,G ) ∪ i , Θ ′ ∪ θ i,j ) .

For i , the current graph G contains a superset of all true parents, i.e. pa ′ ( i, G ) ⊇ pa ( i ) , while the description length of the data is minimized by the true graph L ( S i | S pa ′ ( i ) , Θ ′ ) > L ( S i | S pa ( i ) , Θ) .

Hence, by repeating the edge addition and pruning in a topological order, in the limit of n i →∞ under our causal model and by the Algorithm Markov Condition, CASCADE identifies the true causal graph G ∗ .

Then, for i it holds that

B Experiments

In this section we provide additional detail on the synthetic data generation and the experiment setup. Additional we provide further metrics on the synthetic experiments. For the real-world data we provide additional results.

B.1 Synthetic Experiments

We generate synthetic data according to our causal model. We discretize the timestamps to 1 million unique timestamps. Throughout the experiments we vary the following parameters:

Variables: Number of unique events types p .

Edges: The total number edges in the generating causal graph G ∗ .

Delay Distribution: For all synthetic experiments we generate delays according to geometric distribution (as a discretize instantiation of the exponential).

Delay Distribution Parameter: For each causal edge we sample the rate λ uniformly from a specified range.

Cause probability: For each causal edge we sample α uniformly from a specified range.

# Source Events: Number of events sampled for source nodes (variables without any parents in the DAG).

Additive noise parameter: percentage of additional added events to the caused events, also applies to source nodes, where # Source Events are considered as 'caused'.

Instant effect: Except for the 'Instant Effect' experiments no instant effects are created.

For all experiments, unless otherwise stated a random DAG is generated. And for each parameterization 20 independent samples are generated.

Sanity Check We set the number of types to 20 and generate 100 root events per source node (in this every node is a source node).

Increase of Event Types In this experiment we increase the number of event types p from 5 to 40, We set the number of edges to ( d 2 -d ) / (2 ∗ 5) , that is 20% of all possible edges. To avoid overly many events in the colliders we set the number of root events to 20, for 40 variables this results in up to ≈ 30 . 000 events. We do not include any additive noise and set α = 1 . For the delay distribution, we sample λ from a range between of [0 . 3 , 1] .

Decrease of Noise In this experiment we increase the probability of α , and decrease the fraction of additive noise. We set the number of variables to 20 and set the number of root events to 100. We sample λ from a range of [0 . 1 , 0 . 4] .

Distribution Misspecification To further evaluate robustness of CASCADE we test recovery on generated data where the actual distribution does not match the assumed distribution. To this end, we change the assumed distribution of CASCADE and use the same setup as the previous experiment (Increase of Event Types) with 20 unique events. For the, true, exponential we observe an average F1 score of 0 . 82 , for the Poisson 0 . 81 , with a Normal distribution 0 . 76 , and uniform 0 . 75 . While recovery is best when assumed and generating distribution match CASCADE still performs well under misspecification.

Multiple Parents For this experiment we specify a DAG, where ⌈ n -1 2 ⌉ are direct parents and ⌊ n -1 2 ⌋ are independent, the n th node is the collider. We plant 30 events per root cause and increase the number of variables from 50 to 200. We add 30 % of additive noise and set the cause probability randomly between 0.9 and 0.6. We repeat the same experiment where 10% of nodes are colliders. That is, for 50 event types, 5 are colliders and ⌈ p -5 2 ⌉ direct causes of all 5 colliders. The remaining ⌊ p -5 2 ⌋ are independent. We show the results in Table 1.

Instant Effects For the instant effects experiments we again use 20 variables with 100 root events, we shift the geometric delay distribution and set λ = 0 . 9 , such that 90% of the events are generated at the same timestamp. We randomly sample the trigger probability between 0 . 7 and 0 . 5 . For the exclusively instant effects we set λ = 1 . We show the full results in Table 2 and

B.2 Method Parameterization

CASCADE We set the precision parameter for all experiments to 2. For all synthetic experiments we consider events as potential causes of at most 100 timestamps. For all experiments we consider a geometric distribution, which we shift back to cover instant effects.

MDLH For the results of the Increase event types experiment we use the sparse version, where we set the maximum degree to the true maximal degree and set T =1000. In an effort to reduce runtime with higher number of types (i.e. nodes), we tested it with T=100, where it also did not terminate within 96 hours.

Other For all other competing methods we used the default parameters.

B.3 Compute Recourses

All experiments where executed on a internal cluster on compute nodes equipped with a AMD EPYC 7773X 64-Core Processor (2.2 GHz; Turboboost: 3.5 GHz), with 2 TB of RAM, while in practice a fraction of that was necessary. We provide the average runtimes below.

B.4 Network Alarms

In the provided dataset each event happens on a specific device. In addition to the event sequences a topology T over the devices is provided. An event can cause an event on each neighboring device, in addition to the device where the event occurred. To support this we can include a matching for connected devices. That is if { a, b } ∈ T we include ∆ ( a,b ) i → j and ∆ ( b,a ) i → j , we include both directions since events on a can cause events on b and events on b can cause events on a . For all devices we include the self loop ∆ ( a,a ) i → j .

B.5 Real World Experiment

In this section we provide Causal Graphs reported by CASCADE and for the Global Banks dataset additionally the result of THP.

5

10

15

20

30

40

0

100

200

300

SHD

5

10

15

20

30

40

0

200

400

600

800

1

,

000

SID

5

10

15

20

30

40

0

0

.

2

0

.

4

0

.

6

0

.

8

1

F1

0.9

0.8

0.7

0.6

0.5

0

50

100

150

SHD

0.9

0.8

0.7

0.6

0.5

0

50

100

150

200

250

SID

0.9

0.8

0.7

0.6

0.5

0

0

.

2

0

.

4

0

.

6

0

.

8

1

Noise

F1

50

60

70

80

90 100 150 200

0

50

100

150

200

250

SHD

MDLH

THP

CAUSE

NPHC

CASCADE

50

60

70

80

90 100 150 200

0

25

50

SID

50

60

70

80

90 100 150 200

0

0

.

2

0

.

4

0

.

6

0

.

8

1

F1

Grooming

end

Grooming

start

Leaving

end

Spare

Time/TV

start

Leaving

start

Lunch

end

Lunch

start

Showering

end

Showering

start

Sleeping

end

Toileting

start

Toileting

end

Sleeping

start

Snack

end

Snack

start

Spare

Time/TV

end

(a)

Breakfast

end

Breakfast

start

Dinner

end

Dinner

start

Grooming

end

Grooming

start

Leaving

end

Leaving

start

Lunch

end

Lunch

start

Showering

end

Showering

start

Sleeping

end

Toileting

start

Spare

Time/TV

start

Toileting

end

Sleeping

start

Snack

end

Snack

start

Spare

Time/TV

end

(b)

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

Correctly Recovered

Not Recovered

Westpac

Banking

Commonwealth Bank of

Australia

Japan

HSBC

BNP

Paribas

Barclays

RBS

Santander

Goldman

Sachs

Intesa

Sanpaolo

BBVA

Pohjola

Bank

JPMorgan

Chase & Co

Deutsche

Bank

Credit

Agricole

SocGen

ING

Lloyds

Credit

Suisse

Nordea

Commerzbank

Danske

Svenska

Handelsbanken

SEB

KBC

Fifth Third

Erste

Group Bank

UBS

Bank of

America

Citigroup

Wells

Fargo

Morgan

Stanely

TD Bank

Standard

Chartered

CIBC

BNY

Mellon

U.S.

Bancorp

PNC

Capital

One

State Street Corporation

BB&T Corp

National

Bank of

Canada

SunTrust

American

Express

DNB ASA

BMO

Regions

Financial

ANZ

RBC

Bank of

Nova Scotia

Malayan

Banking

Berhad

Asia

Europe

America

Australia

Top 10 Banks

Mitsubishi

Fukuoka

Mizuho

Resona

Sumitomo

Trust

Sumitomo

Group

Bank Of

Yokohama

Shizuoka

Bank

Chiba

Bank

HSBC

Lloyds

Commerzbank

ING

SocGen

BNP

Paribas

Credit

Suisse

National

Australia

Bank

JPMorgan

Morgan

Stanley

ANZ

Deutsche

Bank

National

Bank of

Canada

SunTrust

Bank of

America

Citigroup

U.S.

Bancorp

TD Bank

RBC

BMO

Fukuoka

Resona

Sumitomo

Trust

Capital

One

PNC

American

Express

Fifth

Third

BB&T Corp

Bank Of

Yokohama

Chiba Bank

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: All claims made in the abstract and introduction are supported by the findings and the main body of the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss limitations and assumptions in the conclusion.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We clearly state all theorems and proof them in the appendix.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Our results can be fully reproduced, we provide all the data and code online.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide the full code along with the synthetic data generator. All used dataset are publicly available.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide an experiment setup description in the appendix. We did not do any hyperparameter tuning. Our method does not require data splitting.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We show Box-Plots for all (synthetic) experiments.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes, we provide a detailed description of the compute recourse in the appendix.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research complies with NeurIPS Code of Ethics, we do not expect any harmful consequences resulting from our research.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification:

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We provided citations sources for all datasets.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We use only already publicly available datasets, and do not introduce any new.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No Human Subjects where involved.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Peer reviews:
comments: SUMMARY:
This paper introduces a new causal model in which individual events of the cause variable trigger events of the effect variable with dynamic delays. The authors propose a cause-effect matching approach to learn a fully directed acyclic graph, named the CASCADE algorithm. The algorithm performs a topological search on observational data.

STRENGTHS:
This paper presents a comprehensive theory and algorithm, and conducts extensive experiments, particularly with real data, to validate the effectiveness of the proposed method. The analysis and the algorithm are presented in a logical way.

WEAKNESSES:
The proposed method is the direct matching between a cause event and an effect event, which precludes modeling a single event causing multiple other events, as well as multiple events jointly causing a single effect event.  This limits the applicability of the algorithm.

QUESTIONS:
**1.** 
Assuming ``an individual event ... causes an individual event'' in Line 64, and in Line 88: Equation 2.
Do these indicate that an effect event can only be caused by a single cause event? If so, the use of $pa(*)$ later in the manuscript is confusing. Can an effect event be caused by more than one event?

In the results of real experiments, there is an event that is caused by more than one event, which contradicts the assumption. How are the results obtained?


**2.**
Are timestamps in events erased during actual use?
As far as I can see, the algorithm proposed in the manuscript does not use timestamps.

LIMITATIONS:
The authors discuss in detail the limitations and applicability of their algorithm.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article employs the Algorithmic Markov Condition alongside Kolmogorov\ncomplexity for causal discovery from event sequences. It focuses on a specific scenario in\nwhich the sequence of events is divided into source and effect variables. The principal\ncontribution of this study is its innovative application of Pearl's causality model with\ncombination of AMC method, in contrast to the traditional Granger causality approach,\nenabling the identification of both instantaneous and delayed effects.\n\nSTRENGTHS:\n1. Originality: The author employs Pearl's model of causality, diverging from\ntraditional Granger causality, to innovatively incorporate instantaneous effects\ninto the analysis of sequential events for causal relationship discovery.\n2. Quality: The article is with good quality and honest about its strength and\nlimitation on their work.\n3. Clarity: The article presents its algorithm with well-defined logic and\nsubstantiated proofs.\n4. Significance: The article offers an innovative approach to integrating\ninstantaneous effects into the causal discovery of sequential events, proposing a\npotential method to enhance causal discovery techniques under such conditions.\nHowever, it imposes strict limitations on the scenarios involving event sequences.\n\nWEAKNESSES:\n1. Significance: As mentioned in the limitation section by the author, strict\nassumptions like direct matching between a cause event and an effect event leads\nto challenges and possible violations in practical application, and it lacks\nflexibility.\n2. Section 3.3, which discusses the connection to Hawkes Processes, might be better\nplaced in an appendix or in a section dedicated to comparing different\nmethodologies. Its current placement in the theoretical part of the paper is\nsomewhat abrupt, especially since there is no direct focus on these processes in\nyour model.\n3. The experimentation section lacks depth. It would be beneficial to evaluate and\nreport on the robustness of your model when its assumptions are challenged\nduring real-world applications.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn their work, the authors are concerned with recovering causal relations, where cause and corresponding effects occur in varying temporal distances. The authors leverage information theoretic formulations and properties of the algorithmic Markov condition to recover the causal graph via minimum description length principled. To this end, the authors present the 'CASCADE' algorithm, which recovers the topological ordering of the causal structure and proof identifiability results. The algorithm is evaluated on multiple synthetic data setups to examine the algorithm's performance under different varying noise, event type, and collider settings. Lastly, the algorithm is tested on a banking and daily activity data set to demonstrate robust performance on real-world data.\n\nSTRENGTHS:\nThe paper is well-written and introduces the problem setup and formalisms intuitively. The authors consider the challenging problem of modeling causal event sequences. The information-theoretic treatise and causal modeling of the event-generating process via minimum description length encodings are well described and follow common notation from related work. While I am not an expert on the topic of time series event causality, relevant related work seems to be sufficiently discussed and compared to.\n\nThe overall intuition on all proofs is well described. To the best of my knowledge, proofs of theorems 1, 3 and 4 seem to be correct. (Please see minor comments on Thm. 2 below). The presented CASCADE algorithm seems to be sound and its robustness is evaluated via multiple real-world and synthetic experiments, varying the noise and number of event types.\n\nWEAKNESSES:\nWhile the authors present strong theoretical identifiability results, these guarantees are tied to a restrictive set of assumptions (faithfulness, sufficiency, low noise) and hold only for a specific type of event process (single excitation, no suppressing effects). While the authors state all assumptions explicitly, the paper could be improved by discussing the possible implications and reasonability of real-world applications.\n\n\nProof of Theorem 2 (Sec. A.2; second line of l. 496): As all other terms seem to be taken over from the line above, it is unclear to me where the canceled term on the left side of the inequality is coming from. (Since all terms are positive, I believe the transformation to be still correct.) Furthermore, it is not obvious to me how the equation following l.497 and the noise ratio of $n_{i,j}/n_j$ leads to the desired result. The paper could be improved by providing a more detailed explanation of this step.\n\n\nThe experiments seem to demonstrate consistently better results compared to related algorithms. However, from the experimental description in B.1, it seems that the experiment on the especially challenging identification of colliders --due to unclear parent assignment-- only considers a setting with a single collider. The authors might want to demonstrate algorithm performance for settings where multiple colliers exist, to better examine the algorithm's robustness regarding unclear EM assignments.\n\n\nMinor:\n* It would be helpful to mention the definition of H() in Sec. A.1 as the entropy, which is only mentioned afterward in A.2.\n* Typos in the Proof of Thm. 2 (sec. A.2 l.490): \"dealys\", \"ofset\"; and the Conclusion (l.340) \"discovers\" -> \"discover\".\n* In Sec. 4.1 l.201 text and formula disagree on the complexity: \"[...] leading to an overall quadratic complexity $O(p^3)$\".\n\nQUESTIONS:\nMy questions mainly concern the weaknesses mentioned above. I would kindly like to ask the authors to comment on the following:\n\n1) How realistic are the assumptions made in the paper (e.g., low noise in real-world settings)? How would one test for them to hold true? How robust would the algorithm be in the presence of other event types, such as suppressing events or multi-effect events?\n\n2) Proof Thm. 2: Could the authors provide further details regarding the proof of theorem 2 - in detail, the derivation of the final step?\n\n3) Regarding my comments above, could the authors give further insights on the algorithm's performance with an increased number of colliders?\n\n4) Figures. 4, 6 and 8 seem to feature few colliders. This seems unreasonable to me, especially for the global banking data set, which I assume to be highly interconnected (possibly violating the DAG assumptions). Could the authors comment on this possible bias? Is it a result of the assumptions made, and how could it be reduced?\n\nLIMITATIONS:\nLimitations with regard to the applicability of the algorithm are discussed. Assumptions required for identifiability of the considered causal models are stated explicitly but might be hard to check in real-world settings. The work might be improved by discussing societal impacts from applying the algorithm under possible assumption violations in real-world settings.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method for identifying causal relationships in event sequences. The authors presents a causal model that handles both instantaneous and delayed effects, contrasting it with existing methods like Granger causality. This algorithm is evaluated on both synthetic and real-world datasets.\n\nSTRENGTHS:\n1. The theoretical foundation based on the AMC and MDL principle is provided.\n\n2. The proposed CASCADE algorithm is evaluated through extensive experiments.\n\n3. The paper is well-organized, with clear explanations of the proposed model, theoretical underpinnings, and algorithmic steps. The use of illustrative examples and detailed proofs enhances understanding.\n\nWEAKNESSES:\n1. The paper acknowledges assumptions such as the direct matching between cause and effect events and the focus on excitatory effects. However, it could provide more discussion on the impact of these assumptions and potential ways to address them.\n\n2. Scalability and computational complexity: The paper demonstrates the algorithm's performance on datasets with a moderate number of variables and events. An evaluation of its scalability to very large datasets, which are common in real-world applications, is less emphasized. The computational complexity of the algorithm, particularly for large datasets with many event types, is a concern. The quadratic complexity in the number of event types may limit its applicability to very large-scale problems.\n\n3. Parameter sensitivity is not provided: How sensitive is the CASCADE algorithm to the choice of parameters for the delay distribution and cause probability?\n\nQUESTIONS:\n1. How sensitive is the CASCADE algorithm to the choice of parameters for the delay distribution and cause probability?\n\n2. What are the practical limits of the CASCADE algorithm in terms of the number of event types and the size of the datasets?\n\n3. How does the algorithm handle high levels of noise in the data, and are there specific noise thresholds beyond which performance degrades significantly?\n\nLIMITATIONS:\nThe authors discussed the limitation in the Conclusion section. The identifiability of instantaneous effects relies on the strengths of the trigger and noise probabilities, which may be challenging to estimate accurately in practice.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}