Title: Causal Discovery in Semi-Stationary Time Series
Venue: NeurIPS 2023
Accepted: yes
Keywords: time-series causal discovery

constraint-based causal discovery
Abstract: Discovering causal relations from observational time series without making the stationary assumption is a significant challenge. In practice, this challenge is common in many areas, such as retail sales, transportation systems, and medical science. Here, we consider this problem for a class of non-stationary time series. The structural causal model (SCM) of this type of time series, called the semi-stationary time series, exhibits that a finite number of different causal mechanisms occur sequentially and periodically across time. This model holds considerable practical utility because it can represent periodicity, including common occurrences such as seasonality and diurnal variation. We propose a constraint-based, non-parametric algorithm for discovering causal relations in this setting. The resulting algorithm, PCMCI$_{\Omega}$, can capture the alternating and recurring changes in the causal mechanisms and then identify the underlying causal graph with conditional independence (CI) tests. We show that this algorithm is sound in identifying causal relations on discrete time series. We validate the algorithm with extensive experiments on continuous and discrete simulated data. We also apply our algorithm to a real-world climate dataset.

Causal Discovery in Semi-Stationary Time Series

Abstract

Discovering causal relations from observational time series without making the stationary assumption is a significant challenge. In practice, this challenge is common in many areas, such as retail sales, transportation systems, and medical science. Here, we consider this problem for a class of non-stationary time series. The structural causal model (SCM) of this type of time series, called the semistationary time series, exhibits that a finite number of different causal mechanisms occur sequentially and periodically across time. This model holds considerable practical utility because it can represent periodicity, including common occurrences such as seasonality and diurnal variation. We propose a constraint-based, nonparametric algorithm for discovering causal relations in this setting. The resulting algorithm, PCMCI Ω , can capture the alternating and recurring changes in the causal mechanisms and then identify the underlying causal graph with conditional independence (CI) tests. We show that this algorithm is sound in identifying causal relations on discrete-valued time series. We validate the algorithm with extensive experiments on continuous and discrete simulated data. We also apply our algorithm to a real-world climate dataset.

1 Introduction

In modern sciences, causal discovery aims to identify the collection of causal relations from observational data, as in Pearl [1980], Peters et al. [2017] and Spirtes et al. [2000]. One of the most popular causal discovery approaches is the so-called constraint-based method. Constraint-based approaches assume that the probability distribution of variables is causal Markov and faithful to a directed acyclic graph called the causal graph. Given large enough data, they can then recover the corresponding Markov equivalence class by exploiting conditional independence relationships of the variables. See Peters et al. [2017]. There are many constraint-based algorithms such as PC and FCI algorithms Spirtes et al. [2000]. The standard assumption of these approaches is that data samples are independent and identically distributed, which makes it possible to perform CI tests. See Bergsma [2004], Zhang et al. [2012] and Shah and Peters [2020].

Recently, there have been numerous efforts to extend such constraint-based algorithms to accommodate time series data. For instance, PCMCI in Runge et al. [2019] and LPCMCI in Gerhardus and Runge [2020] are the PC-based algorithms for time series. Inspired by FCI algorithms, approaches designed for time series include ANLSTM in Chu and Glymour [2008], tsFCI in Entner and Hoyer [2010] and SVAR-FCI in Malinsky and Spirtes [2018]. This setup is relevant in several industrial applications since many data points have an associated time-point, such as root-cause analysis in Ikram et al. [2022]. Most of the existing causal discovery algorithms make the stationary assumption.

See Chu and Glymour [2008], Hyvärinen et al. [2010], Entner and Hoyer [2010], Peters et al. [2013], Malinsky and Spirtes [2018], Runge et al. [2019], Pamfil et al. [2020]and Assaad et al. [2022].

Non-stationary temporal data makes causal discovery more challenging since the statistics are time-variant, and it is unreasonable to expect that the underlying causal structure is time-invariant. Identifying causal relations from non-stationary time series without imposing any restriction on the data is difficult. Here, we focus on a specific class of non-stationary time series, called the semistationary time series, whose structural causal model (SCM) exhibits that a finite number of different causal mechanisms occur sequentially and periodically across time. One example is illustrated in Fig.1, where the time series X 1 has three different causal mechanisms across time, shown as red edges, green edges, and blue edges, respectively. Similarly, time series X 2 has two alternative causal mechanisms. This setting holds considerable practical utility. Periodic nature is commonly observed in many real-world time series data. See Han et al. [2002], Nakamura et al. [2003], Carskadon et al. [2005] and Komarzynski et al. [2018]. Here are a few additional intuitive examples: poor traffic conditions often coincide with commute time and weekends; household electricity consumption typically follows a pattern of being higher at night and lower during the daytime. Consequently, it is reasonable to expect periodic changes in the causal relations underlying this type of time series without assuming stationarity. Here, the constraint-based methods in Chu and Glymour [2008], Entner and Hoyer [2010], Malinsky and Spirtes [2018] and Runge et al. [2019], designed for stationary time series, may fail. Given observational data with periodically changing causal structures, it is hard to apply CI tests directly. Most of the other algorithms designed for non-stationary time series rely heavily on model assumptions, as in Gong et al. [2015], Pamfil et al. [2020], and Huang et al. [2019]. These algorithms are discussed further in the related work.

In this paper, we propose an algorithm to address this problem, namely non-parametric causal discovery in time series data with semi-stationary SCMs. The key contributions of our work are:

We develop an algorithm to discover the causal structure from semi-stationary time series data where the underlying causal structures change periodically. Our algorithm systematically uses the PCMCI algorithm proposed for the stationary setting in Runge et al. [2019]. The resulting algorithm is hence named PCMCI Ω where Ω denotes periodicity.

We validate our method with synthetic simulations on both continuous-valued and discretevalued time series, showing that our method can correctly learn the periodicity and causal mechanism of the synthetic time series.

We utilize our method in a real-world climate application. The result reveals the potential existence of periodicity in those time series, and the stationary assumption made by previous works could be relaxed in some practical situations.

1.1 Related Work

PCMCI has been applied in diverse domains to investigate atmospheric interactions in the biosphere, as demonstrated in Krich et al. [2020], global wildfires as explored in Qu et al. [2021], water usage as studied in Zou et al. [2022], ultra-processed food manufacturing as examined in Menegozzo et al. [2020], and causal feature selection as discussed in Peterson [2022], among other applications. See Arvind et al. [2021], Gerhardus and Runge [2020], Castri et al. [2023a,b]. While PCMCI has achieved considerable success, it is not without its limitations. One notable assumption that can be challenged is the concept of causal stationary , that is, causal relations are time-invariant. PCMCI exhibits robustness when applied to linear models with an added non-stationary trend. See also Runge et al. [2019]. However, there is an ongoing exploration to enhance its performance in a wider range of non-stationary settings.

Although not as extensively as the stationary case, causal discovery in non-stationary time series has been studied by some authors. However, many of those algorithms rely on parametric assumptions such as the vector autoregressive model in Gong et al. [2015] and Malinsky and Spirtes [2019]; linear and nonlinear state-space model in Huang et al. [2019]. One non-parametric algorithm in the literature is CD-NOD proposed by Huang et al. [2020], which has been extended to recover time-varying instantaneous and lagged causal relationships. In very recent work, Fujiwara et al. [2023] proposed an algorithm JIT-LiNGAM to obtain a local approximated linear causal model combining algorithm LiNGAM and JIT framework for non-linear and non-stationary data. To the best of our knowledge, no other non-parametric approaches can discover causal relations underlying time series without

Z1

Z3

Z4

x

I

一

X2

W

t:

4

5

7

8

9

10

11

12

13

14

15

assuming stationarity and can also allow for sudden changes in causal mechanisms. Our proposed approach does not directly enforce the stationary assumption on the time series. The SCM also integrates a finite set of causal mechanisms that exhibit periodic variations over time.

2 PCMCI Ω : Capturing Periodicity of the Causal Structure

In this section, we formulate the problem of learning the causal graph on multi-variate time series data when the SCM exhibits periodicity in causal mechanisms. In section 2.1, we present the necessary definitions and provide an overview of the problem setting. In section 2.2, we introduce the required assumption.

2.1 Preliminaries

Let G ( V, E ) denote the underlying causal graph, and for each variable X ∈ V , we denote the set of all incoming neighbors as parents, denoted by Pa( X ) .

For any two variables X,Y ∈ V and S ⊂ V , we denote the CI relation X is independent of Y conditioned on S , by X ⊥ ⊥ Y | S .

For simplicity's sake, define sets: [ b ] := { 1 , 2 , ..., b } and [ a, b ] := { a, a +1 , ..., b } , where a, b ∈ N .

̸

In the time series setting, let X j t ∈ R 1 denote the variable of j th time series at time t , X j = { X j t } t ∈ [ T ] ∈ R T denote a univariate time series and X t = { X j t } j ∈ [ n ] ∈ R n denote a slice of all variables at time point t . V = { X j } j ∈ [ n ] = { X t } t ∈ [ T ] ∈ R n × T denotes a n -variate time series. By default, we assume n > 1 and hence X j ⊊ V , and p ( V ) = 0 , where p ( . ) denotes the probability or probability density.

Definition 2.1 ( Non-Stationary SCM) . A Non-Stationary Structural Causal Model (SCM) is a tuple M = ⟨ V, F , E , P ⟩ where there exists a τ max ∈ N + , defined as:

such that with this τ max , each variable X j t>τ max ∈ V is a deterministic function of its parent set Pa( X j t>τ max ) ∈ V and an unobserved (exogenous) variable ϵ j t>τ max ∈ E :

and there exist at least two different time points t 0 , t 1 ∈ [ τ max +1 , T ] satisfying

̸

where f j,t , f j,t 0 , f j,t 1 ∈ F and { ϵ j t } t ∈ [ T ] are jointly independent with probability measure P . τ max is the finite maximal lag in terms of the causal graph G .

Definition 2.2 ( Semi-Stationary SCM) . A Semi-Stationary SCM is a Non-Stationary SCM that additionally satisfies the following conditions. For each j ∈ [ n ] , there exists an ω ∈ N + such that:

are satisfied for all t ∈ [ τ max +1 , T ] , N ∈ { N : N ∈ N , t + Nω ≤ T } . This means that a finite number of causal mechanisms are repeated periodically for every univariate time series X j in V . One example of this model is illustrated in Fig.1. For X 1 in Fig.1, three causal mechanisms are reiterated periodically with ω 1 = 3 , represented by red, green, and blue edges, respectively.

The minimum value that satisfies the above conditions for X j is defined as the periodicity of X j , denoted by ω j . Furthermore, for an n -variate time series V , Ω denotes the minimum periodicity across all time series X j , j ∈ [ n ] . The number of causal mechanisms occurring sequentially and periodically of univariate time series X j is ω j , and that number of causal mechanisms of multivariate time series V is Ω . For X j , the causal mechanisms are associated with each variable X j t . However, for V , the causal mechanisms are related to each time slice vector X t as a whole. The relationship between Ω and ω j can be captured by:

where LCM ( . ) is an operation to find the least common multiple between any two or more numbers. Here, Ω is the smallest common multiple among { ω 1 , ...ω n } . In Fig.1, Ω = LCM (3 , 2 , 1) = 6 .

Definition 2.3 ( Time Partition ) . A time partition Π j ( T ) of a univariate time series X j in a SemiStationary SCM with periodicity ω j is a way of dividing all time points t ∈ [ T ] into a collection of non-overlapping non-empty subsets { Π j k ( T ) } k ∈ [ ω j ] such that:

where mod denotes the modulo operation. For instance, 5 mod 3 = 2 .

We can observe that the variables in { X j t } t ∈ Π j k ( T ) share the same causal mechanism. Since the number of potentially different causal mechanisms of variables in X j is ω j , the number of such time partition subsets is ω j . For simplicity, notations Π j and Π j k are used instead of Π j ( T ) and Π j k ( T ) . In Fig.1, Π 1 1 = { 4 , 7 , 10 , 13 , .., 4 + 3 N,... } , Π 1 2 = { 5 , 8 , 11 , 14 , .., 5 + 3 N,... } and Π 1 3 = { 6 , 9 , 12 , 15 , .., 6 + 3 N,... } where N ∈ N + . The nodes X 1 t are classified into their associated time partition subsets by the matching colors.

Definition 2.4 ( Illusory Parent Sets ) . For a univariate time series X j ∈ V with Semi-Stationary SCM having periodicity ω j > 1 , parent set index pInd j k ∈ [ ω j ] is defined as:

where n ′ = | Pa( X j t )) | , τ i is the time lag and y i is the variable index. Given pInd j k , Illusory Parent Sets are defined as:

Put simply, the illusory parent sets of X j t are the time-shift version of the parent set of other variables in X j that have a different causal mechanism from X j t . Note that the illusory parent sets are constructed specifically in the Semi-Stationary SCM. For stationary SCM, there is no illusory parent set needed. To maintain consistency in notation, for time points t ∈ Π j k , the notation Pa k ( X j t ) can also be extended to encompass the true parent set of X j t :

By doing so, Pa( X j t ) ⊂ ∪ k ∈ [ ω j ] Pa k ( X j t ) . In Fig.1, by observing Pa( X 1 7 ) , we have pInd 1 1 = { (1 , 1) , (2 , 2) } ; by observing Pa( X 1 8 ) , we have pInd 1 2 = { (1 , 1) , (3 , 1) } and finally by observing Pa( X 1 9 ) , pInd 1 3 = { (1 , 1) , (1 , 2) } . Based on those indexes, Pa 1 ( X 1 7 ) = { X 1 7 -1 , X 2 7 -2 } = { X 1 6 , X 2 5 } , Pa 2 ( X 1 7 ) = { X 1 7 -1 , X 3 7 -1 } = { X 1 6 , X 3 6 } and Pa 3 ( X 1 7 ) = { X 1 7 -1 , X 1 7 -2 } = { X 1 6 , X 1 5 } . The first one is the true parent set of X 1 7 and the latter two are the illusory parent sets. The order of those parent sets is not important.

At last, we need to further define a series of Markov chains that are associated tightly with SemiStationary SCM. The presence and characteristics of these Markov chains are thoroughly examined in the supplementary materials. The motivation behind creating such Markov chains is to introduce assumptions on them rather than directly on the original data V .

Definition 2.5 ( Time Series as a Markov Chain ) . For time series V with Semi-Stationary SCM, there are (potentially) δ different Markov chains { Z q n } n ∈N , q ∈ [ δ ] :

where N := { n ∈ N + : τ max + q -1 + nδ ≤ T } , δ = ⌈ τ max +1 Ω ⌉ Ω . Note that in { Z q n } , q is used to indicate a specific Markov chain, while n serves as the running index for that particular Markov chain. Such a Markov chain { Z q n } , q ∈ [ δ ] exists as long as Pa( Z q n ) ⊂ Z q n ∪ Z q n -1 for all n . This is a finite state Markov Chain if all time series in V are discrete-valued time series. The state space of { Z q n } is the set containing all possible realization of { X τ max + q +( i -1)+( n -1) δ } i ∈ [ δ ] ,n ∈ N . The transition probabilities between the states are the product of associated causal mechanisms based on Assumption A2 , which is elaborated by an example in section C.1 (Eq.(10)-(11)) of the supplementary material.

2.2 Assumptions for PCMCI Ω

A1. Sufficiency : There are no unobserved confounders.

A2. Causal Markov Condition : Each variable X is independent of all its non-descendants, given its parents Pa( X ) in G .

A3. Faithfulness Condition (Pearl [1980]) : Let P be a probability distribution generated by G . ⟨G , P ⟩ satisfies the Faithfulness Condition if and only if every conditional independence relation true in P is entailed by the Causal Markov Condition applied to G .

A4. No Contemporaneous Causal Effects : Edges between variables at the same time are not allowed.

A5. Temporal Priority : Causal relations that point from the future to the past are not permitted.

A6. Hard Mechanism Change : If at time points t 1 and t 2 , the causal mechanisms of X j t 1 and X j t 2 are different, then their corresponding parent sets can not be transformed to each other by time shifts:

̸

̸

A7. Irreducible and Aperiodic Markov Chain : The Markov chains { Z n } of V are assumed to be irreducible (Serfozo [2009]): for all states i and j of { Z n } , ∃ n so that

and aperiodic(Karlin [2014]): for every state i of { Z n } , d ( i ) = 1 , where the period d ( i ) of the state i is the greatest common divisor of all integers n for which p ( n ) ii > 0 .

Assumptions A1-A5 are conventional and commonly employed in causal discovery methods for time series data. On the other hand, our approach requires additional Assumptions A6-A7 to be in place. To clarify, A6 is essential because our method may encounter challenges in distinguishing distinct causal mechanisms for variables in { X j n } n ∈ [ T ] if they share identical parent sets after time shifts. As for A7 , it serves a crucial role in establishing the soundness of our algorithm.

3 PCMCI Ω Algorithm

In this section, we propose an algorithm called PCMCI Ω , and in section 3.1, we present a theorem demonstrating the soundness of PCMCI Ω and its ability to recover the causal graph. Our algorithm

Algorithm 1 PCMCI Ω

PCMCI Ω builds on the Algorithm PCMCI in Runge et al. [2019]. Additional details about PCMCI are provided in the supplementary material.

Overview of Algorithm 1 PCMCI Ω . We assume that the periodicity and time lag are upper bounded by ω ub and τ ub respectively. Using PCMCI Runge et al. [2019], we obtain a superset of parents for every variable X j t denoted by ̂ S Pa( X j t ) (line 2). Our goal is to identify the correct set of parents along with its periodicity for every variable in V . For a variable X j t , we guess its periodicity ω by iterating over all possible values in [ ω ub ] . Next, we construct time partition subsets ̂ Π j k , k ∈ [ ω ] based on the guess of periodicity ω . In each time partition subset, we maintain a parent set, denoted by ̂ Pa ω ( X j t ) , initializing it with the superset ̂ S Pa( X j t ) . Then we test the causal relations between X i t -τ ∈ ̂ Pa ω ( X j t ) and X j t using a CI test on the sample t ∈ ̂ Π j k (lines 6-10).

For each guess ω , every variable in X j should have its estimated parent set (line 9), and there are total ω potentially different parent set index pInd j k ∈ [ ω ] in X j . We return an estimate ˆ ω j that maximizes the sparsity of the causal graph (Lemma 3.4). Therefore, we select the value of ω ∈ [ ω ub ] that minimizes the maximum value of | ̂ Pa ω ( X j t ) | , t ∈ [ T ] as the estimator of ω j (line 12).

3.1 Theoretical Guarantees

Our main theorem shows that PCMCI Ω recovers the true causal graph on discrete data. There are three important lemmas. We provide all the detailed proof in the supplementary material.

Theorem 3.1. Let ˆ G be the estimated graph using the Algorithm PCMCI Ω . Under assumptions A1-A7 and with an oracle (infinite sample size limit), we have that:

almost surely.

Lemma 3.2 and Lemma 3.3 jointly state that if CI tests are conducted on samples generated by different causal mechanisms, the obtained parent sets ̂ S Pa( X j t ) should be the superset of the union of the true and illusory parent sets ∪ k Pa k ( X j t ) . That is, the estimated graph should be denser than the correct graph. The true parent set can then be obtained by directly testing the independent relations between the target variable X j t and the variables in ̂ S Pa( X j t ) , assuming a consistent CI test. Note that the CI tests in our algorithm are assumed to be consistent given i.i.d. samples. We do not assume

the consistency of CI tests with respect to semi-stationary data. Therefore, any CI tests that maintain consistency with i.i.d. samples can be seamlessly integrated into our algorithm.

Lemma 3.2. Denote that { Pa k ( X j t ) } k ∈ [ ω j ] contain the true and illusory parent sets, where ω j is the true periodicity of X j . For any random variable X j t with large enough t , under assumptions A1-A7 and with an oracle (infinite sample size limit), we have:

̸

Here, p ( X j t | ∪ ω j k =1 Pa k ( X j t )) = lim T →∞ ˆ p ( X j t | ∪ ω j k =1 Pa k ( X j t )) where ˆ p ( X j t | ∪ ω j k =1 Pa k ( X j t )) is an estimated conditional distribution using all samples t ∈ [ τ max +1 , T ] :

✶ Proof sketch. We argue that the estimated conditional distribution in Eq.(15) can be written as a linear combination of ˆ p ( X j t | Pa( X j t )) where t ∈ Π j k , k ∈ [ ω j ] , i.e., as a mixture of conditional distributions. The coefficients in the linear function, say α k , k ∈ [ ω j ] , can be further decomposed based on a finer time partition called the homogenous time partition , which consists of subsets constructed according to the Markov chains { Z q n } q ∈ [ δ ] corresponding to the time series. Based on Assumption A7 , the Markov chains are stationary and ergodic. Therefore, after sufficiently large time steps, the distribution of { Z q n } q ∈ [ δ ] will be invariant across n as it achieves unique equilibrium. With this type of stationary sample, we can express α k by joint distributions instead of the indicators. Then, we can complete the proof of our inequality claim in Eq.(14) using Assumption A2 and Bayes theorem.

Lemma 3.3. Let ̂ S Pa( X j t ) denote the estimated superset of parent set for X j ∈ V obtained from the Algorithm 1 (line 2). { Pa k ( X j t ) } k ∈ [ ω j ] contain the true and illusory parent sets, where ω j is the true periodicity of X j . Under assumptions A1-A7 and with an oracle (infinite sample size limit), we have that:

almost surely.

Proof sketch. Assume the contrary, i.e., there exists s ∈ ∪ k Pa k ( X j t ) \ ̂ S Pa( X j t ) . From Lemma 3.2, we have X j t ✚ ✚ ⊥ ⊥ s ∣ ∣ ∣ ∪ ω j k =1 Pa k ( X j t ) \ s . By the Definition 2.4, we have Pa( X j t ) ⊂ ∪ ω j k =1 Pa k ( X j t ) . If s ̸∈ Pa( X j t ) , by the causal Markov property (Assumption A2 ), the dependence relation can not be true because s is a non-descendant of X j t . If s ∈ Pa( X j t ) , our Algorithm would have concluded that X j t ✚ ✚ ⊥ ⊥ s ∣ ∣ ∣̂ S Pa( X j t ) (line 2), evident from the causal Markov property, contradicting our assumption. Hence, the lemma.

Based on Lemma 3.2 and Lemma 3.3, we can identify the true ω j for X j through Lemma 3.4.

Lemma 3.4. Let ω j denote the true periodicity for X j ∈ V and ̂ Pa ω ( X j t ∈ Π j k ) denote the estimated parent set for X j t obtained from Algorithm 1 line 9, where t ∈ Π j k . Define:

Under assumptions A1-A7 and with an oracle (infinite sample limit), we have that ˆ ω j = ω j , ∀ j ∈ [ n ] almost surely.

̸

Proof sketch. Assume the contrary that ˆ ω j = ω j , then in the Algorithm 1, we have an incorrect time partition ̂ Π j . Hence, CI tests that are performed use samples with different causal mechanisms.

̂

1

2

3

4

5

6

7

8

9

10

ωmax

0.0

0.2

0.4

0.6

0.8

1.0

Accuracy of

ω

n = 5, τmax = 5

T=500

T=2000

T=4000

(a) Accuracy of

ˆ

ω

1

2

3

4

5

6

7

8

9

10

ωmax

0

50

100

150

Time (seconds)

n = 5, τmax = 5

T=500

T=2000

T=4000

(b) Runtime (in sec.) when

ω

varies

ˆ p ( X j t | ∪ ω j k =1 Pa k ( X j t )) in Eq.(15) is estimated from a mixture of two or more time partition subsets, say Π j 1 and Π j 2 . We can apply Lemma 3.2 with ∪ 2 k =1 Pa k ( X j t ) . With Lemma 3.3, ∪ 2 k =1 Pa k ( X j t ) ⊆ ̂ Pa ˆ ω j ( X j t ) . Hence, for ˆ ω j , | ̂ Pa ˆ ω j ( X j t ) | ≥ | ∪ 2 k =1 Pa k ( X j t ) | using mixture samples t ∈ ∪ 2 k =1 Π j k . For true ω j , we have | ̂ Pa ω j ( X j t ) | = | Pa( X j t ) | . With Assumption A6 the Hard Mechanism Change, | ∪ 2 k =1 Pa k ( X j t ) | > | Pa( X j t ) | so that ω j always leads to a smaller size of estimated parent sets than ˆ ω j , contrary to the definition of ˆ ω j . Hence, ˆ ω j = ω j .

4 Experiments

4.1 Experiments on Continuous-valued Time Series

To validate the correctness and effectiveness of our algorithm, we perform a series of experiments. The Python code is provided at https://github.com/CausalML-Lab/PCMCI-Omega. In this section, we test four algorithms 1 , PCMCI Ω , PCMCI Runge et al. [2019], VARLiNGAM Hyvärinen et al. [2010] and DYNOTEARS Pamfil et al. [2020], on continuous-valued time series with Gaussian noise. The experiments for continuous-valued time series with exponential noise and binary-valued time series are in the supplementary material.

Following Runge et al. [2019], we generate the continuous-valued time series in three steps:

̸

Construct an n -variate time series V with length T using independent and identical (Standard Gaussian or Exponential) noise temporarily. Determine τ max and ω max where ω max = max { ω j } j ∈{ n } . After making sure that one univariate time series, say X j , has periodicity ω max , the periodicity of the remaining time series X i , i = j is randomly selected from { 1 , · · · , ω max } respectively.

Randomly generate ω j binary edge matrices with shape ( n, τ max ) for each time series X j , j ∈ [ n ] . 1 denotes an edge and 0 denotes no edge. Each binary matrix represent one parent set index pInd j k , k ∈ [ ω j ] . Randomly generate ω j coefficient matrices with shape ( n, τ max ) for each time series X j , j ∈ [ n ] . One binary edge matrix and one coefficient matrix jointly determine one causal mechanism. Hence, total ω j causal mechanisms are constructed. Here, make sure that V satisfies Assumption A6 .

Starting from time point t > τ max , generate vector X t over time according to all the causal mechanisms of V , until t achieves T .

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 8000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

PCMCI

VARLiNGAM

DYNOTEARS

(a) Gaussian noise

12

5

10

15

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

1)

12

5

10

15

Precision

T = 2000,

max = 5, n = 5

12

5

10

15

Recall

2 5

10

15

20

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

2)

2 5

10

15

20

n

Precision

T = 2000,

2 5

10

15

20

Recall

PCMCI

PCMCI

VARLiNGAM

DYNOTEARS

(b) Performance when

τ

or

n

varies

Following the previous work in Huang et al. [2020], F 1 score, Adjacency Precision, and Adjacency Recall are used to measure the performance of the algorithms. The details of calculating these metrics are described in the Appendix. All the performance statistics are averaged over 100 trials. The standard error of the averaged statistics is displayed either by color filling or by error bars.

A correct estimator ˆ ω is the prerequisite for obtaining the correct causal graph. Fig.2(a) shows the accuracy rate of ˆ ω for different time lengths T . Here, elements in { Nω j } where N ∈ [ ω ub ω j ] are all treated as correct estimations. By Definition 2.2 and 2.3, the multiple of ω j is still associated with a correct causal graph. However, it leads to a finer time point partition Π j , decreasing the sample size used in each CI test from approximately T/ω j to approximately T/ ( Nω j ) . The accuracy rate is sensitive to ω max for small T . This result verifies that algorithm PCMCI Ω has the capacity to detect the true periodicity of each X j ∈ V with a large enough time length.

We evaluate the performances of PCMCI Ω on continuous-valued time series with Gaussian noise shown as Fig.3(a). As T increases, it is natural to see a continuous improvement in performance. The sub-figures show that all three evaluation metrics decrease when ω max gets larger. The precision of PCMCI Ω is always far better than other algorithms when ω max is not equal to 1. Given the fact that the parent sets ̂ Pa( X j t ) ∀ j, t obtained from PCMCI Ω are subsets of the parent set ̂ S Pa( X j t ) ∀ j, t estimated from PCMCI, the recall rate of PCMCI should be the upper bound of the recall rate of PCMCI Ω . This assertion has been verified as the red recall line of PCMCI Ω is always below the blue recall line of PCMCI as T increases.

In Fig.3(a), the recall of PCMCI Ω is worse than PCMCI for T = 500 . In this regime, the accuracy rate of ˆ ω is low, shown as the dark blue line in Fig.2(a). Small sample sizes in CI tests may result in a sparser causal graph. Hence the number of true positive edges may decrease. This is a common problem for many constraint-based algorithms, but it hurts PCMCI Ω the most because in PCMCI Ω , the sample sizes in each CI test are approximate T/ ˆ ω instead of T . As T increases, the red recall line of PCMCI Ω push forward to the blue recall line of PCMCI. The high value of both adjacent precision and recall rate with large T verify that PCMCI Ω can identify the correct causal graph.

We also observe the performance of our algorithm as τ max and N varies in Fig.3(b). As the performance of PCMCI Ω is consistent over n -variate time series with different n , large τ max may lead to a smaller precision and recall rate.

4.2 Case Study

Here, we construct an experiment with a real-world climate time series dataset. In Runge et al. [2019], the authors tested dependencies among monthly surface pressure anomalies in the West Pacific and surface air temperature anomalies in the Central Pacific, East Pacific, and tropical Atlantic from 1948 to 2012. Our application explores the causal relations among the monthly mean of the same set of variables from 1948-2022 with 900 months. Let X wp t denote the monthly mean of surface pressure in the West Pacific, X cp t , X ep t and X ta t denote the monthly mean of air temperature in the Central Pacific, East Pacific, and tropical Atlantic, respectively.

The parent sets for each variable obtained from PCMCI Ω algorithms are shown in Table 1. Sets of true and illusory parents of a variable at time t are separated by curly braces. For instance, variable X wp t with ˆ ω wp = 1 means that the causal mechanism of the surface pressure in the West Pacific remains invariant over time with the estimated parent set { X wp t -1 , X wp t -2 , X ep t -1 , X ta t -1 } . Only time series X cp has three different parent sets, including one true parent set and two illusory parent sets, which appear periodically over time. The three parent sets of X cp t imply that the causal effect from the tropical Atlantic air temperature X ta t -1 to the Central Pacific air temperature X cp t would disappear every quarter of a year. Note that we do not have a ground truth in this case, and we do not possess the necessary knowledge in this area, so the significance of these results is under-explored. More discussion about this application can be found in the supplementary materials.

5 Conclusions

In this paper, we propose a non-parametric, constraint-based causal discovery algorithm PCMCI Ω designed for semi-stationary time-series data, in which a finite number of causal mechanisms are repeated periodically. We establish the soundness of our algorithm and assess its effectiveness on continuous-valued and discrete-valued time series data. The algorithm PCMCI Ω has the capacity to reveal the existence of periodicity of causal mechanisms in real-world datasets.

References

Judea Pearl. Causality: models, reasoning, and inference, 1980.

Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms . The MIT Press, 2017.

Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search . MIT press, 2000.

Wicher Pieter Bergsma. Testing conditional independence for continuous random variables . Citeseer, 2004.

Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv:1202.3775 , 2012.

Rajen D Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure. 2020.

Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances , 5 (11):eaau4996, 2019.

Andreas Gerhardus and Jakob Runge. High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems , 33:12615-12625, 2020.

Tianjiao Chu and Clark Glymour. Search for additive nonlinear time series causal models. Journal of Machine Learning Research , 9(5), 2008.

Doris Entner and Patrik O Hoyer. On causal discovery from time series data using fci. Probabilistic graphical models , pages 121-128, 2010.

Daniel Malinsky and Peter Spirtes. Causal structure learning from multivariate time series in settings with unmeasured confounding. In Proceedings of 2018 ACM SIGKDD workshop on causal discovery , pages 23-47. PMLR, 2018.

Azam Ikram, Sarthak Chakraborty, Subrata Mitra, Shiv Saini, Saurabh Bagchi, and Murat Kocaoglu. Root cause analysis of failures in microservices through causal discovery. Advances in Neural Information Processing Systems , 35:31158-31170, 2022.

Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, and Patrik O Hoyer. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research , 11(5), 2010.

Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Causal inference on time series using restricted structural equation models. Advances in neural information processing systems , 26, 2013.

Roxana Pamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Georgatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics , pages 1595-1605. PMLR, 2020.

Charles K Assaad, Emilie Devijver, and Eric Gaussier. Survey and evaluation of causal discovery methods for time series. Journal of Artificial Intelligence Research , 73:767-819, 2022.

Fang Han, Shyam Subramanian, Edwin R Price, Joseph Nadeau, and Kingman P Strohl. Periodic breathing in the mouse. Journal of Applied Physiology , 92(3):1133-1140, 2002.

Akira Nakamura, Yasuichiro Fukuda, and Tomoyuki Kuwaki. Sleep apnea and effect of chemostimulation on breathing instability in mice. Journal of Applied Physiology , 94(2):525-532, 2003.

Mary A Carskadon, William C Dement, et al. Normal human sleep: an overview. Principles and practice of sleep medicine , 4(1):13-23, 2005.

Sandra Komarzynski, Qi Huang, Pasquale F Innominato, Monique Maurice, Alexandre Arbaud, Jacques Beau, Mohamed Bouchahda, Ayhan Ulusakarya, Nicolas Beaumatin, Gabrièle Breda, et al. Relevance of a mobile internet platform for capturing inter-and intrasubject variabilities in circadian coordination during daily routine: pilot study. Journal of Medical Internet Research , 20 (6):e204, 2018.

Mingming Gong, Kun Zhang, Bernhard Schoelkopf, Dacheng Tao, and Philipp Geiger. Discovering temporal causal relations from subsampled data. In International Conference on Machine Learning , pages 1898-1906. PMLR, 2015.

Biwei Huang, Kun Zhang, Mingming Gong, and Clark Glymour. Causal discovery and forecasting in nonstationary environments with state-space models. In International conference on machine learning , pages 2901-2910. PMLR, 2019.

Christopher Krich, Jakob Runge, Diego G Miralles, Mirco Migliavacca, Oscar Perez-Priego, Tarek El-Madany, Arnaud Carrara, and Miguel D Mahecha. Estimating causal networks in biosphereatmosphere interaction with the pcmci approach. Biogeosciences , 17(4):1033-1061, 2020.

Yuquan Qu, Carsten Montzka, and Harry Vereecken. Causation discovery of weather and vegetation condition on global wildfire using the pcmci approach. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS , pages 8644-8647. IEEE, 2021.

Liangfeng Zou, Yuanyuan Zha, Yuqing Diao, Chi Tang, Wenquan Gu, and Dongguo Shao. Coupling the causal inference and informer networks for short-term forecasting in irrigation water usage. Water Resources Management , pages 1-23, 2022.

Giovanni Menegozzo, Diego Dall'Alba, and Paolo Fiorini. Causal interaction modeling on ultraprocessed food manufacturing. In 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE) , pages 200-205. IEEE, 2020.

Sayeh Peterson. Comparison of Lasso Granger and PCMCI for Causal Feature Selection in Multivariate Time Series . PhD thesis, The University of Arizona, 2022.

DK Arvind, Sharan Maiya, and P Andreu Sedeño. Identifying causal relationships in time-series data from a pair of wearable sensors. In 2021 IEEE 17th International Conference on Wearable and Implantable Body Sensor Networks (BSN) , pages 1-4. IEEE, 2021.

Luca Castri, Sariah Mghames, Marc Hanheide, and Nicola Bellotto. Causal discovery of dynamic models for predicting human spatial interactions. In Social Robotics: 14th International Conference, ICSR 2022, Florence, Italy, December 13-16, 2022, Proceedings, Part I , pages 154-164. Springer, 2023a.

Luca Castri, Sariah Mghames, Marc Hanheide, Nicola Bellotto, et al. Enhancing causal discovery from robot sensor data in dynamic scenarios. 2023b.

Daniel Malinsky and Peter Spirtes. Learning the structure of a nonstationary vector autoregression. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 2986-2994. PMLR, 2019.

Biwei Huang, Kun Zhang, Jiji Zhang, Joseph D Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Schölkopf. Causal discovery from heterogeneous/nonstationary data. J. Mach. Learn. Res. , 21(89):1-53, 2020.

Daigo Fujiwara, Kazuki Koyama, Keisuke Kiritoshi, Tomomi Okawachi, Tomonori Izumitani, and Shohei Shimizu. Causal discovery for non-stationary non-linear time series data using just-in-time modeling. In 2nd Conference on Causal Learning and Reasoning , 2023.

Richard Serfozo. Basics of applied stochastic processes . Springer Science & Business Media, 2009.

Samuel Karlin. A first course in stochastic processes . Academic press, 2014.

Appendix

A PCMCI Algorithm

The PCMCI algorithm is proposed by Runge et al. [2019], aiming to detect time-lagged causal relations in a window causal graph. There are two stages of PCMCI: the condition-selection stage and the causal discovery stage. In the first stage, unnecessary edges are removed based on the conditional independencies from an initialized partially connected graph where Assumption A4-A5 should be satisfied. In the second stage, Momentary Conditional Independence tests (MCI) are used to further remove the false positive edges caused by autocorrelations in time series data. More specifically, these two steps can be briefly formalized as follows:

PC 1 in Algorithm A1: Condition selection stage. PC 1 is a variant of the skeleton-discovery part of the PC algorithm in a more robust version named stable-PC Le et al. [2016]. The goal in this stage is to obtain a superset of the parents ̂ Pa( X j t ) for all variables X j ∈ [ n ] t ∈ [ τ max +1 ,T ] ∈ V . Initialize ̂ Pa( X j t ) = { X i t -τ } i ∈ [ n ] ,τ ∈ [ τ max ] . ̂ Pa( X j t ) will remove X i t -τ if

MCI in Algorithm A2: Causal discovery stage. In this stage, do MCI tests for all variable pairs ( X i t -τ , X j t ) with i, j ∈ [ n ] and time delays τ ∈ [ τ max ] :

where ̂ Pa( X j t ) and ̂ Pa( X i t -τ ) are estimated from the PC 1 stage.

Note that τ max in this section is the same as τ ub in the main paper, serving as the upper bound for the time lag that exhibits causal effects. On the other hand, τ max in the main paper denotes the maximum time lag observed within the multivariate time series. Essentially, in the main paper, τ ub is a parameter that must be fed into the algorithm, and τ max is observed from the true causal graph. As a default, we assume τ ub is configured with a value greater than τ max , ensuring that the algorithm uncovers the correct causal relations. See Fig.1 for more detail.

X2

t:4

5

6

7

8

9

10

11

12

13

14

15

Tmax = max{t1 =2,T2 =3,T3=1} = 3

Tub=5

Algorithm A1 PC q max

Algorithm A2 MCI

1: Input: A n -variate time series V = ( X 1 , X 2 , X 3 , ..., X n ) , sorted parents ̂ Pa ( X j t ) for all variables X j estimated with Algorithm A1, maximum time lag τ max , maximum number p X of parents of variable X i , and conditional independence test function CI

2: for all ( X i t -τ , X j t ) with i, j ∈ { 1 , ..., n } , τ ∈ { 0 , ..., τ max } , excluding ( X j t , X j t ) do

3: Remove X i t -τ from ̂ Pa ( X j t ) if necessary

4: Define ̂ Pa p X ( X i t -τ ) as the first p X parents from ̂ Pa ( X i t ) , shifted by τ

5: Run MCI test to obtain ( p -value , I ) ← CI ( X i t -τ , X j t , Z = { ̂ Pa ( X j t ) , ̂ Pa p X ( X i t -τ ) } )

6: end for

7: Optionally adjust p -value of all links by False Discovery Rate-approach (FDR)

8: return p -value and MCI test statistic values

B PCMCI Ω

For simplicity's sake, define sets: [ b ] := { 1 , 2 , ..., b } and [ a, b ] := { a, a +1 , ..., b } , where a, b ∈ N .

Algorithm B1 PCMCI Ω

C Soundness of PCMCI Ω

C.1 Stationary Markov Chain

Claim : Any discrete-valued time series V with Semi-Stationary Structural Causal Model (SCM) satisfying assumption A1, A2, A4, A5 can be written as a Markov chain { Z n } as long as this Markov chain satisfies Pa( Z n ) ⊂ Z n ∪ Z n -1 for all n , where Z n is a set of variables in V . This Markov chain has a finite number of states if all time series in V are discrete-valued time series.

Note that when the notation n is related to a Markov chain Z n , it means the running index. In the context of X j ∈ [ n ] t , n represents the index of component time series within the n -variate time series.

To simplify, assume that one associated Markov chain of V = { X , Y } has Z n = { X t , Y t , X t -1 , Y t -1 } with t ∈ { t ∈ N + : , t ≤ T } satisfying Pa( Z n ) ⊂ Z n ∪ Z n -1 . Here, the notation for the time points of variables is simplified as t and t -1 , even though it should be a function of n , the running index of the Markov chain. Note that Z n -1 = { X t -2 , Y t -2 , X t -3 , Y t -3 } rather than { X t -1 , Y t -1 , X t -2 , Y t -2 } , as the simplified notation could erroneously suggest the latter sequence. A simple proof is shown below through Markov assumption ( A2 ).

Z1

Z3

Z4

x

I

一

X2

W

t:

4

5

7

8

9

10

11

12

13

14

15

Proof.

Assume that the space of both X t and Y t with t < T are { 1 , 2 } . There are total 2 4 = 16 states of Markov Chain { Z n } = {{ X t , Y t , X t -1 , Y t -1 }} . The transition probability P for this Markov Chain is illustrated as a 16 × 16 matrix:

where (1 , 1 , 1 , 1) means X t = 1 , Y t = 1 , X t -1 = 1 , Y t -1 = 1 . Each row in this transition probability matrix is a conditional distribution of Z n given one realization of Z n -1 . Each entry is a probability of having one specific realization of Z n given one realization of Z n -1 . This probability can be decomposed by conditional distributions based on Markov assumption ( A2 ). Take p 1 , 1 as an example:

where Pa( . ) here are realizations, not random variables.

For time series V with Semi-Stationary SCM, there are (potentially) δ different Markov chains { Z q n } , q ∈ [ δ ] :

where n ∈ { n : n ∈ N + , τ max + q -1 + nδ ≤ T } , δ = ⌈ τ max +1 Ω ⌉ Ω . As proved in the claim, such a Markov chain exists as long as Pa( Z q n ) ⊂ Z q n ∪ Z q n -1 for all n . The value of δ can guarantee the existence of such Markov chain because δ is larger than τ max + 1 and is a multiple of Ω , that is, a multiple of all { ω j } j ∈ [ n ] . By doing so, Pa( Z q n ) ⊂ Z q n ∪ Z q n -1 is satisfied; for any variable X j t , there exists q ∈ [ δ ] and n ∈ N + such that variable X j t and its parent set Pa( X j t ) can be included in Z q n ; and the causal mechanism generating Z q n is invariant for different n . The state space of { Z q n } is the set containing all possible realizations of { X τ max + q +( i -1)+( n -1) δ } i ∈ [ δ ] ,n ∈ N . The transition probabilities between the states are the product of associated causal mechanisms based on Markov assumption ( A2 ).

Determined by the starting slice X t where τ max < t ≤ τ max + δ , there should be δ potentially different Markov chains { Z q n } where 1 ≤ q ≤ δ . To be more specific, those Markov chains are:

.

.

.

Given Irreducible and Aperiodic Markov Chain assumption ( A7 ), discrete-time Markov chain { Z q n } 0 <n , q ∈ [ δ ] with finite states should be a stationary and ergodic Markov chain, and there is a unique stationary distribution π q (Bertsekas and Tsitsiklis [2008], Karlin [2014]). Additionally, the large power of the associate transition matrix P q will eventually converge to a matrix in which each row is the stationary distribution π q . Equivalently,

where S is the state space of Z q n .

In other words, after a sufficiently long time, equivalently, n is large enough, the distribution of { Z q n } does not change with increasing n . That is, for large enough n :

Returning from the stationary and ergodic Markov chains { Z q n } , q ∈ [ δ ] back to the original data V through Eq.(12) to Eq.(14), the distribution of the original data V must adhere to the following condition:

p ( X τ max + q + n 1 δ , X τ max + q +1+ n 1 δ , ..., X τ max + q + δ -1+ n 1 δ ) = p ( X τ max + q + n 2 δ , X τ max + q +1+ n 2 δ , ..., X τ max + q + δ -1+ n 2 δ ) (17) for any q ∈ [ δ ] and n 1 , n 2 > n .

Given these clarifications, we can naturally introduce a more refined time partition that is based on, yet finer than, the time partition defined in Definition 2.3 in the main paper.

Definition C.1 ( Homogenous Time Partition ) . For a univariate time series X j in a Semi-Stationary SCM with periodicity ω j , the time partition Π j k of X j can be further divided into a series of nonoverlapping and non-empty subsets { π j ( k,s ) } 1 ≤ s ≤ δ ω j . For each t ∈ [ τ max + 1 , T ] , there exists k ∈ [ ω j ] so that t ∈ Π j k and further there exists s ∈ [ δ ω j ] so that t ∈ π j ( k,s ) . π j ( k,s ) can be written as:

With this definition, we have ∪ δ ω j s =1 π j ( k,s ) = Π j k . While time partition Π j k guarantees that all variables in { X j t } t ∈ Π j k share the same causal mechanism, homogenous time partition π j ( k,s ) guarantees that all variables in { X j t } t>t ′ ,t ∈ π j ( k,s ) share the same distribution where t ′ represent the steps needed by the associated Markov chain to achieve equilibrium.

Fig.2 shows a partial causal graph for a 3-variate time series with Semi-Stationary SCM. τ max = 3 means that the causal mechanisms start from t = 4 , and the random variables with t ∈ { 1 , 2 , 3 } are random noises. For the first time series X 1 , the periodicity ω 1 is 3. And the periodicity of the time series X 2 and X 3 is 2 and 1, respectively. The periodicity of the whole time series V is obtained by LCM (3 , 2 , 1) = 6 . δ = ⌈ τ max +1 Ω ⌉ Ω = ⌈ 3+1 6 ⌉ × 6 = 1 × 6 = 6 .

In Fig.2, periodicity ω 1 = 3 means that the causal mechanisms repeat every three time points and hence there are three time partition subsets Π 1 k , k ∈ [3] . More specifically, Π 1 1 = { 4 , 7 , 10 , 13 , ..., 4 + 3 N,... } , Π 1 2 = { 5 , 8 , 11 , 14 , ..., 5 + 3 N,... } , Π 1 3 = { 6 , 9 , 12 , 15 , ..., 6 + 3 N,... } where N ∈ N + . Random variables { X 1 t } with t in the same time partition subset share the same causal mechanism. However, they may not share the same marginal distribution.

Still in Fig.2, based on the definition of homogenous time partition, time partition subset Π 1 1 for X 1 can be further decomposed as π 1 ( k =1 ,s =1) = { 4 , 10 , ..., 4 + δN,... } , π 1 ( k =1 ,s =2) = { 7 , 13 , ..., 7 + δN,... } . where s ∈ [ δ ω 1 ] . After a long run n , Z 1 n and Z 1 n +1 will eventually share the same distribution, that is, all the variables inside Z q n will share the same joint or marginal distribution as the corresponding variables inside Z q n +1 . To illustrate this, we assume that this Markov chain has already achieved its equilibrium at time point t = 4 . Based on Eq.(12) and Eq.(17), we have:

From the identical joint distribution, we can further have:

as X 1 4 ∈ X 4 , X 1 10 ∈ X 10 and X 1 16 ∈ X 16 .

Therefore, for sufficiently large values of t ensuring that Z 1 n has reached its stationary distribution, all variables within { X j t } t ∈ π j ( k,s ) will share the same distribution.

In Fig.2, there are 6(= δ ) potentially different Markov chains { Z q n } , q ∈ [ δ ] in V . For any time window with length δ , { X t , ..., X t + δ -1 } , there exists q ∈ [ δ ] , n ∈ N + , so that this time window can be completely included in Z q n . For instance, set { X 5 , X 6 , ..., X 10 } is in Z 2 1 , which is the first element of Markov chain { Z 2 n } .

Constructing Markov chains and applying the Irreducible and Aperiodic Markov Chain assumption ( A7 ) enable us to obtain a consistent estimator for the conditional and joint distributions of interest.

C.2 Consistent Estimator

The conditional distributions for variables in { X j t } t ∈ Π j k are the same, that is, p ( x j t 1 | Pa( x j t 1 )) = p ( x j t 2 | Pa( x j t 2 )) , ∀ t 1 , t 2 ∈ Π j k . For simplicity, denote

Consider an indicator function such that ✶ ( x j t , Pa( x j t )) = 1 if configuration ( x j t , Pa( x j t )) has realized, otherwise ( x j t , Pa( x j t )) = 0 .

✶ Since every t ∈ π j ( k,s ) is apart from each other with Nδ steps where N ∈ N + , and there must exist q ∈ [ ω j ] and n 1 ∈ N + so that { x j t , Pa( x j t ) } ∈ Z q n 1 , then for the same q , there must exist another n 2 so that { x j t + Nδ , Pa( x j t + Nδ ) } ∈ Z q n 2 . Hence, we have { ✶ ( x j t , Pa( x j t )) } t ∈ π j ( k,s ) = { f ( Z q ( t ) n 1 ( t ) ) } t ∈ π j ( k,s ) with some function f : R n × δ → R 1 satisfying E | f ( Z q ( t ) n 1 ( t ) ) | < ∞ . Since the value of t determines q and n 1 , we use q ( t ) and n 1 ( t ) to emphasize their relations. For large enough t > t ′ ,

{ ✶ ( x j t , Pa( x j t )) } t>t ′ ,t ∈ π j ( k,s ) are identical samples where t ′ is the time point needed by the associate Markov chain to achieve its equilibrium after n 1 ( t ′ ) steps.

Without loss of generality, we assume T is a multiple of δ all the time.

We can construct an estimator of p ( x j t , Pa( x j t )) with large enough t as:

where k, s is determined by t and there must exist one and only one k, s satisfying t ∈ π j ( k,s ) . Now, we are going to show this estimator is consistent.

We first decompose the estimator into two parts: time point t ≤ t ′ and t > t ′ , where t ′ represents the time point when the equilibrium of the associated Markov chain is achieved.

Take a limit of Eq.(23), we have:

Denote

(29)

Based on the definition of homogenous time partition and time partition, p t ∈ π j ( k,s ) ( x j t | Pa( x j t )) = p t ∈ Π j k ( x j t | Pa( x j t )) , ∀ s ∈ [ δ ω j ] .

Similar to Eq.(22), one estimator of p t ∈ Π j k ( x j t | Pa( x j t )) , ∀ k = [ ω j ] is

Take a limit of Eq.(37), we have:

Hence, ˆ p t ∈ Π j k ( x j t | Pa( x j t )) is a consistent estimator of p t ∈ Π j k ( x j t | Pa( x j t )) .

Similarly, we construct an estimator of p ( x j t | ∪ h Pa h ( x j t )) where t ∈ [ T ] :

✶ We will prove that this estimator is converged as T goes to infinity in Lemma D.2. Hence, it is a consistent estimator.

In this section, we have proved that ˆ p ( x j t , Pa( x j t )) in Eq.(22) is a consistent estimator of p ( x j t , Pa( x j t )) using samples with t in the same homogenous time partition subset and ˆ p ( x j t | Pa( x j t )) in Eq.(37) is a consistent estimator of p ( x j t | Pa( x j t )) using samples with t in the same time partition subset.

D Theorem

Theorem D.1. Let ̂ G be the estimated graph using the Algorithm PCMCI Ω . Under assumptions A1-A7 and with an oracle (infinite sample size limit), we have that:

almost surely.

Lemma D.2. Denote that { Pa k ( X j t ) } k ∈ [ ω j ] contain the true and illusory parent sets, where ω j is the true periodicity of X j . For any random variable X j t with large enough t , under assumptions A1-A7 and with an oracle (infinite sample size limit), we have:

̸

Here,

p

(

X

j

t

| ∪

ω

k

j

=1

Pa

k

(

X

j

t

)) = lim

T

→∞

ˆ

p

(

X

j

t

| ∪

ω

k

j

=1

Pa

k

(

X

j

t

))

.

Proof. We first prove that there exist a sequence of coefficients { α k } k ∈ [ ω j ] satisfying ∑ ω j k =1 α k = 1 so that: ∀ configuration ∪ h Pa h ( x j t ) ,

If this is correct, then ˆ p ( x j t | ∪ h Pa h ( x j t )) would be a consistent estimator of p ( x j t | ∪ h Pa h ( x j t )) . Based on Eq.(46), we have:

Using the same logic in Eq.(30)-(35), we can decompose the numerator and denominator of α k with homogenous time partition until each component converges to a stationary distribution.

Without loss of generality, assume y ∈ Pa( x j t ) , where t ∈ Π 1 j and y / ∈ Pa( x j t ) , where t / ∈ Π 1 j . Then we have

Similarly, we have:

̸

Proving p ( x j t | ∪ h Pa h ( x j t )) = p ( x j t | ∪ h Pa h ( x j t ) \ y ) is equal to proving:

̸

Substitutes Eq.(60) and Eq.(67) in Eq.(72), we have the following derivation:

After equating the denominators, the numerator is:

For the sake of simplicity, denote

After substituting the simple notations in Eq.(76):

̸

̸

̸

That is, V t contains all the nodes before time point t .

Denote { b t i } i ∈ [ n ] = ∪ h Pa h ( x j t ) and assume { b t i } 1 ≤ i ≤ n 1 <n = Pa 1 ( x j t ) , where t ∈ Π j ( k,s )

We express p t ∈ Π j ( k,s ) ( ∪ h Pa h ( x j t )) by marginalizing all other random variables occurring before the latest variables in ∪ h Pa h ( x j t ) and utilizing the Causal Markov assumption ( A2 ):

Note that x j t ′ ∈ V h \ { b t i } i ∈ [ n ] .

This joint distribution is now represented by conditional distributions of one related variable given its parents.

Similarly, assume y = b t 1 , we have

̸

̸

̸

̸

Note that x j t ′ ∈ V t n \ { b t i } i =1 .

Define

p t ∈ Π j 1 ( x j t | Pa( x j t ) \ y ) can also be represented by those conditional distributions based on Bayes rule.

where

Note that t ∈ Π j 1 for distributions in above section from Eq.(94) to Eq.(104) and that x j t ′ ∈ V h \ { b t i } i ∈ [ n 1 ] .

Hence, every term in Eq.(83) can be expressed as a function of those conditional distributions. Substituting Eq.(88), Eq.(93) and Eq.(100) in Eq.(83), we have a polynomial equation only composed of conditional distributions { p ( x j t ′ | Pa( x j t ′ ) ) } j ∈ [ n ] ,t ′ ≤ t except the joint distribution of the starting points p ( V τ max ) . Note that the conditional distributions of variables in { X j t } t ∈ Π j k , j ∈ [ n ] , k ∈ [ ω j ] are the same. Since sets do not allow duplicate values, set { p ( x j t ′ | Pa( x j t ′ ) ) } j ∈ [ n ] ,t ′ ≤ t contains only different conditional distributions. There should be potentially total ∑ n j =1 ω j different causal mechanisms. The total number of conditional probabilities should be jointly determined by the number of causal mechanisms and also the number of realizations that variables can take. After adjusting those conditional distributions by the linear restriction ∑ y p ( x | y ) = 1 , all components in the set { p ( x j t ′ | Pa( x j t ′ ) ) } j ∈ [ n ] ,t ′ ≤ t are mutually independent, and p ( V τ max ) is also independent of all the causal mechanisms because the first starting points are random noises. That is, upon adjustments, all the terms in Eq.(83) should be rendered independent of each other, without any imposed constraints across them.

After expanding all the summations in Eq.(83), the coefficients of this polynomial equation are either 1 or -1 . Each coefficient is accompanied by one unique monomial as index ( k, s ) in the joint distribution p t ∈ π j k,s determined a unique product of conditional distributions, i.e., with a different pair of ( k, s ) , the product should be different. Considering all random and independent conditional

distributions in { p ( x j t ′ | Pa( x j t ′ ) ) } j ∈ [ n ] ,t ′ ≤ t , the polynomial is not identically zero, and the probability of choosing a root of this polynomial is zero.

Denote the polynomial equation in Eq.(83) as A, we have:

̸

Back to the original Eq.(73), we finally have p ( p ( x j t | ∪ h Pa h ( x j t )) = p ( x j t | ∪ h Pa h ( x j t ) \ y ) ) = 1 , ∀ y ∈ ∪ h Pa h ( x j t ) .

Lemma D.3. Let ̂ S Pa( X j t ) denote the estimated superset of parent set for X j ∈ V obtained from the Algorithm B1 (line 2). { Pa k ( X j t ) } k ∈ [ ω j ] contain the true and illusory parent sets, where ω j is the true periodicity of X j . Under assumptions A1-A7 and with an oracle (infinite sample size limit), we have:

almost surely.

Proof. Assume the contrary, i.e., there exists s ∈ ∪ k Pa k ( X j t ) \ ̂ S Pa( X j t ) . From Lemma D.2, we have X j t ✚ ✚ ⊥ ⊥ s ∣ ∣ ∣ ∪ ω j k =1 Pa k ( X j t ) \ s . By the Definition 2.4, we have Pa( X j t ) ⊂ ∪ ω j k =1 Pa k ( X j t ) . If s ̸∈ Pa( X j t ) , by the causal Markov property ( A2 ), the dependence relation can not be true, because s is a non-descendant of X j t . If s ∈ Pa( X j t ) , our Algorithm would have concluded that X j t ✚ ✚ ⊥ ⊥ s ∣ ∣ ∣̂ S Pa( X j t ) (line 2) with a consistent CI test, evident from the causal Markov property, contradicting our assumption. Hence, the lemma.

LemmaD.4. Let ̂ Pa ( X j t ) denote the estimated parent set for X j ∈ V obtained from the Algorithm B1 (line 19) assuming that true ω j has obtained (line 17). { Pa k ( X j t ) } k ∈ [ ω j ] contain the true and illusory parent sets. Under assumptions A1-A7 and with an oracle (infinite sample size limit), we have:

almost surely.

In Runge et al. [2019], the author proved ̂ Pa ( X j t ) = Pa ( X j t ) if we run PCMCI on stationary time series. Using the same logic, we have the following proof.

Suppose X i t -τ / ∈ ̂ Pa ( X j t ) but X i t -τ ∈ Pa ( X j t ) . With a consistent conditional independence test and correct time partition, the MCI test (line 8 in Algorithm B1) will remove X i t -τ from ̂ Pa ω j ( X j t ) if and only if:

Based on Eq.(107), the rule is equivalent to removing X i t -τ from ̂ Pa ω j ( X j t ) if and only if:

Based on Causal Markov Condition assumption ( A2 ) and Faithfulness Condition ( A3 ), from Eq.(110) we have X i t -τ / ∈ Pa ( X j t ) . In other words, if X i t -τ / ∈ ̂ Pa ( X j t ) then X i t -τ / ∈ Pa ( X j t ) . That is, Pa ( X j t ) ⊆ ̂ Pa ( X j t )

Suppose X i t -τ ∈ ̂ Pa ( X j t ) but X i t -τ / ∈ Pa ( X j t ) . By the contraposition of Faithfulness ( A1 ), we know that X i t -τ ✚ ✚ ⊥ ⊥ X j t ∣ ∣ ∣ ̂ Pa ( X j t ) \ { X i t -τ } , ̂ Pa ( X i t -τ ) . Denote W = { ̂ Pa ( X j t ) \ { Pa ( X j t ) , X i t -τ } } ∪ { ̂ Pa ( X i t -τ ) \ Pa ( X i t -τ ) } . Since X i t -τ / ∈ Pa ( X j t ) , based on Causal Markov Condition assumption ( A2 ),

This is contrary to the assumption so that there is no such X i t -τ satisfying X i t -τ ∈ ̂ Pa ( X j t ) but X i t -τ / ∈ Pa ( X j t ) . In other words, if X i t -τ ∈ ̂ Pa ( X j t ) , then X i t -τ ∈ Pa ( X j t ) . That is, ̂ Pa ( X j t ) ⊆ Pa ( X j t ) . Combined with the previous conclusion that Pa ( X j t ) ⊆ ̂ Pa ( X j t ) , we have ̂ Pa ( X j t ) = Pa ( X j t ) .

Based on Lemma D.2, Lemma D.3 and Lemma D.4, we can identify the true ω j for X j through Lemma D.5.

Lemma D.5. Let ω j denote the true periodicity for X j ∈ V and ̂ Pa( X j t ∈ Π j k ) denote the estimated parent set for X j t obtained from Algorithm B1 where t ∈ Π j k . Define:

Under assumptions A1-A7 and with an oracle (infinite sample limit), we have that ˆ ω j = ω j , ∀ j ∈ [ n ] almost surely.

̸

Proof. Assume the contrary that ˆ ω j = ω j , then in the Algorithm B1, we have an incorrect time partition ̂ Π j . Hence, CI tests that are performed use samples with different causal mechanisms. ˆ p ( X j t | ∪ ω j k =1 Pa k ( X j t )) in Eq.(50) is estimated from a mixture of two or more time partition subsets, say Π j 1 and Π j 2 . We can apply Lemma D.2 where ∪ ω j k =1 Pa k ( X j t ) is replaced by ∪ 2 k =1 Pa k ( X j t ) and then in Lemma D.3, ̂ S Pa( X j t ) is replaced by ̂ Pa ˆ ω j ( X j t ) and hence ∪ 2 k =1 Pa k ( X j t ) ⊆ ̂ Pa ˆ ω j ( X j t ) where ̂ Pa ˆ ω j ( X j t ) is obtained from samples with t from the mixture of two different partition subsets (line 8). Hence, with ˆ ω j , | ̂ Pa ˆ ω j ( X j t ) | ≥ | ∪ 2 k =1 Pa k ( X j t ) | using mixture samples t ∈ ∪ 2 k =1 Π j k . However, with true ω j , we have | ̂ Pa ω j ( X j t ) | = | Pa( X j t ) | based on Lemma D.4. With Assumption A6 the Hard Mechanism Change, | ∪ 2 k =1 Pa k ( X j t ) | > | Pa( X j t ) | so that ω j always leads to a smaller size of estimated parent sets than ˆ ω j , contrary to the definition of ˆ ω j . Hence, ˆ ω j = ω j .

With those lemmas, we can prove Theorem 1.

Proof. Assuming that a correct ω j has already been obtained, from Lemma D.4 we have

From Lemma D.5, we know that a correct ω j must be obtained with consistent CI tests, that is, ˆ ω j = ω j , ∀ j ∈ [ n ] . Therefore from Algorithm B1, we have

Therefore ̂ G = G .

If ∃ ω j > 1 , we can simply separate the whole graph G into sub graphs {G ω j k } k ∈ [ ω j ] consisting of only target variable X j t with corresponding t ∈ { Π j k } k ∈ [ ω j ] and parent variables X i t ′ ∈ Pa( X j t ) . Focusing only on one time partition subset Π j , k ∈ [ ω ] , we have

If the causal mechanism is fixed across time, i.e., ω j = 1 , j ∈ [ n ] , the proof of PCMCI Runge et al. [2019] showed that for all X j ∈ V ,

k j

for any k ∈ [ ω j ] and j ∈ [ n ] based on the proof of Proposition 1 in the supplementary materials of Runge et al. [2019].

Each sub-graph G ω j k includes only variable X j t , the edges entering X j t for time points t ∈ Π j k and the corresponding parent variables X i t ′ ∈ Pa ( X j t ) . Given Π j = ∪ k ∈ [ ω j ] Π j k and V = ∪ j ∈ [ n ] X j , we have:

On the basis of Eq.(112), we finally have:

E Turning Points

Given infinite samples, our estimate ˆ ω j (line 17 in Algorithm B1) is the exact value ω j (see Lemma D.5). However, for finite samples, estimating ω j by the equation in line 17 in Algorithm B1 does

not yield good performance when T is small. While searching, larger guesses ω lead to finer time partitions in Π j , resulting in smaller sizes for Π k j (see Line 5 in Algorithm B1). Due to the power limit of CI tests on a smaller sample given by Π k j , the number of false negative edges increases. In order to solve this issue, we introduce turning points . A turning point is a guess ˆ ω satisfying:

where | ̂ Pa ˆ ω ( X j t ) | is the estimated parent set for X j t with periodicity guess ˆ ω . See line 19 in Algorithm B1.

We illustrate it with a special example in Fig.3. If there are several turning points, then ˆ ω j is the first turning point. If there is no turning point, then we obtain ˆ ω j using Line 17 of Algorithm B1.

The concept of the turning point is not based on any formal theorem but rather on experimental observations. In experiments, the turning point often corresponds to a multiple of the true periodicity when T is not large. This occurs due to the limitations of CI tests on finite samples. In such cases, the causal graph can still be correct because the estimated time partition remains accurate. In these experiments, the accuracy rate is calculated by considering { Nω j } N ∈⌊ ω ub ω j ⌋ as correct estimations.

F Computational Complexity

Executing the PCMCI algorithm on the entire time series constitutes the initial phase of the proposed approach (Algorithm B1 line 2). The algorithm's worst-case overall computational complexity is O ( n 3 τ 2 ub ) + O ( n 2 τ ub ) , discussed in Runge et al. [2019]. Here, the symbol n denotes n -variate time series and τ ub represents the upper boundary for time lags.

The subsequent computational load stemming from the remaining components of our algorithm follows a complexity of O ( ω 2 ub n 2 τ ub ) . This encompasses the O ( n 2 τ ub ) complexity associated with conducting Momentary Conditional Independence (MCI) tests on all n univariate time series. The parameter ω 2 ub here arises due to the search procedure involving ω , iterating through values from 1 to ω ub for all n univariate time series.

The runtime of the computation is further influenced by the scaling behavior of the CI test concerning the dimensionality of the conditioning set and the temporal series length T . For further details, see section 5.1 in Runge et al. [2019].

G Experiments

All experiments, including those detailed in the main paper, are conducted on a single node with one core, utilizing 512 GB of memory in the Gilbreth cluster at Purdue University.

Here, we describe how to calculate the metrics ( F 1 score, Adjacency Precision, and Adjacency Recall) in our setting. In stationary time series, the output of the causal discovery algorithm is typically an adjacency matrix with dimensions [ n, n, τ max +1] . Within the three-dimensional binary array, the value 1 signifies an edge pointing from one variable to another with a specific time lag, while 0 indicates the absence of an edge. For instance, if element [ i, j, k ] in the matrix is 1, then there is an edge pointing from X i t -k to X j t . In semi-stationary time series, due to the presence of multiple causal mechanisms, the binary edge matrix is a four-dimensional array with dimensions [ n, Ω , n, τ max +1] , where Ω is defined as Eq.(7) in the main paper. This expanded binary matrix is constructed based on the edge matrix of each variable X j t , j ∈ [ n ] , through repetition. For instance, if Ω = 2 ωj , setting the third dimension of the large binary matrix to j should yield ω j potentially different parent sets (including illusory and true parent sets), each appearing twice.

We should have two such binary arrays, one representing the ground truth with dimensions [ n, Ω , n, τ max + 1] and one obtained from the algorithm with dimensions [ n, ̂ Ω , n, τ max + 1] . If the estimator ̂ Ω is incorrect, those two binary arrays will have different sizes, so we can not directly compare them. To solve this problem, we do the same operation and calculate the least common multiple of Ω and ̂ Ω . Denoting this least common multiple as LCM( Ω , ̂ Ω ), we create two four-dimensional binary arrays with dimensions of [ n, LCM (Ω , ̂ Ω) , n, τ max +1] based on the true edge array and the

estimated edge array, respectively, through repetition. The metrics are then computed by comparing the values in these two arrays.

G.1 More Discussion regarding the Case Study

As stated in the main paper, we express our inability to comment on the significance of the case study results. We open a door for the related experts; if assumptions A1-A7 are satisfied, the stationary assumption may not hold in this real-world dataset, and such periodicity exists. However, if the finding is not correct from an expert's viewpoint, the following assumptions may be violated:

Assumption A4 No Contemporaneous Causal Effects: There is a possibility of potential causal effects from X ta t to X cp t that the algorithm is unable to capture.

Assumption A6 Hard Mechanism Change combined with limited power of CI tests: If there is a soft mechanism change in the variables, the reliability of the CI test of two variables given their parents will be influenced by the skewed distribution of the parent variables. This effect will be exacerbated by the fact that the sample size will be shrunk by ˆ ω .

We provide a sound and robust algorithm for experts in various fields who are interested in validating the presence of periodicity within the causal mechanisms specific to their domain.

G.2 Experiments on Continuous-valued Time Series with Exponential Noise

Considering that VARLiNGAM is a temporal extension of LiNGAM and LiNGAM is an algorithm designed for non-Gaussian data, following the work in Pamfil et al. [2020], we also construct experiments on continuous-valued time series data with Exponential noise. Shown as Fig.4(a), the performance of PCMCI Ω , PCMCI and VARLiNGAM, are quite similar with their performance on Gaussian noise. The recall rate of DYNOTEARS, however, gets worse with Exponential noise.

G.3 Experiments on Binary Time Series

Similar to the process of generating continuous-valued time series, the generation of binary time series also involves three steps. However, the main difference lies in the last two steps. In the third step, we simulate the conditional distributions of each child variable based on all possible combinations of parent variable values. Subsequently, we randomly generate the value of the child variable by considering the corresponding conditional distribution given its parent sets.

For discrete-valued time series, a longer time length is required. To evaluate performance, we conduct a series of experiments following the same methodology as described in section 4.1. Fig.4(b) illustrates the variation in comprehensive performance with respect to ω max . PCMCI Ω demonstrates a similar performance to PCMCI in terms of the F 1 score, indicating a well-balanced trade-off between precision and recall. This outcome is expected since discrete-valued time series demand larger sample sizes, and the increases in ω max negatively impact the power of MCI tests. This observation is further supported by Fig.5(a), where an increase in time length T from 4000 to 12000 does not lead to a significant improvement in the accuracy rate of ˆ ω , while the accuracy decreases rapidly with higher values of ω max .

Comparing these results to the experiments conducted on continuous-valued time series, it becomes evident that the demand for efficient samples is even more substantial for binary time series, and the influence of increasing ω max on performance becomes more pronounced.

Fig.5(b) shows how the performance of the algorithm varies across τ max and the same trade-off between recall and precision has been shown.

G.4 More experiments on Continuous-valued time series

In this section, we conduct more experiments with continuous-valued time series with Gaussian noises.

In Fig.6(a), we test our algorithm with and without utilizing the turning point rule. See lines 13-14 in Algorithm B1 and section E for more information about the turning point rule. Let PCMCI Ω TP

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 8000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

PCMCI

VARLiNGAM

DYNOTEARS

(a) Exponential noise

1

2

3

4

5

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 4000

1

2

3

4

5

T = 8000

1

2

3

4

5

T = 12000

1

2

3

4

5

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

2

3

4

5

1

2

3

4

5

n = 3,

PCMCI

PCMCI

(b) Binary time series

1

2

3

4

5

ωmax

0.0

0.2

0.4

0.6

0.8

1.0

n = 3, τ

T=4000

T=8000

T=12000

(a) Accuracy of

ˆ

ω

1

2

3

4

5

0.0

0.5

1.0

F

1

Score

T = 4000,

= 3, n = 3

1

2

3

4

5

0.0

0.5

1.0

Precision

1

2

3

4

5

0.0

0.5

1.0

Recall

PCMCI

PCMCI

(b) Performance when

τ

varies

denote the version of PCMCI Ω that the turning point rule is utilized in choosing ω . PCMCI Ω non-TP means that the turning point rule is not applied and ω is chosen directly according to Lemma D.5.

Fig.6(a) shows that the algorithm PCMCI Ω non-TP and PCMCI Ω TP have similar performance with various T and ω max . With T = 500 , PCMCI Ω non-TP yields slightly larger standard errors for those metrics, compared to PCMCI Ω TP. As time length T increases, the performance of the algorithm PCMCI Ω non-TP has consistently increased and is even slightly better than PCMCI Ω TP.

The consistent performance of PCMCI Ω under different chosen rules of ω supports our theoretical result; that is, the correct periodicity leads to the most sparse causal graph.

In Fig.6(b), non-stationary time series are produced instead of semi-stationary ones. Consequently, the causal mechanisms for each univariate time series no longer appear sequentially and periodically. The proposed method performs slightly better in terms of F1 score and precision. However, the recall rate is the worst compared to other baselines.

In Fig.6(c), we conduct experiments in the nonlinear setting. The proposed algorithms PCMCI Ω TP and PCMCI Ω non-TP perform the best.

In Fig.6(d), with ω ub < ω max , the performance of the proposed algorithm is significantly worse compared to the scenario where ω ub > ω max . However, with ω ub < ω max , the proposed algorithm can still detect a less dense graph in comparison to other baselines. Based on these outcomes, it is essential to maintain a slightly higher ω ub without significantly impacting the number of efficient samples utilized in each CI test.

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 8000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

TP

PCMCI

non-TP

PCMCI

VARLiNGAM

DYNOTEARS

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 8000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

TP

PCMCI

non-TP

PCMCI

VARLiNGAM

DYNOTEARS

<

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 4000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

TP

PCMCI

non-TP

PCMCI

tsFCI

(c) Performance in non-linear SCM

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

F

1 Score

T = 500

1

3

5

7

9

T = 2000

1

3

5

7

9

T = 8000

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Precision

1

3

5

7

9

1

3

5

7

9

1

3

5

7

9

0.0

0.2

0.4

0.6

0.8

1.0

Recall

1

3

5

7

9

1

3

5

7

9

n = 5,

PCMCI

ub

>

PCMCI

ub

<

PCMCI

VARLiNGAM

DYNOTEARS

(d) Performance when

ω

ub

> ω

and

ω

ub

ω

References

Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances , 5 (11):eaau4996, 2019.

Thuc Duy Le, Tao Hoang, Jiuyong Li, Lin Liu, Huawen Liu, and Shu Hu. A fast pc algorithm for high dimensional causal discovery with multi-core pcs. IEEE/ACM transactions on computational biology and bioinformatics , 16(5):1483-1495, 2016.

Dimitri Bertsekas and John N Tsitsiklis. Introduction to probability , volume 1. Athena Scientific, 2008.

Samuel Karlin. A first course in stochastic processes . Academic press, 2014.

Roxana Pamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Georgatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics , pages 1595-1605. PMLR, 2020.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe problem of causal discovery is tackled for time series data in those cases where stationarity of multivariate time series data can not be assumed. A problem formulation based on structural causal model is given by considering the case of semi-stationary time series. Under this assumptin, the paper considers the case where a finite number of different causal mechanisms happen in a sequential manner and periodically across time. A constraint based, and non parametric, structural learning algorithm is designed and developed by leveraging on the PCMCI, described in the specialized literature. Such an algorithm is extended and adapted to cope with the considered semi-stationary setting. The causal discovery problem is tackled by standard conditional independence tests. The paper proves that the proposd algorithm is sound when asked to discover and identify causal relationships between variables of discrete time series. Numerical experiments, both from synthetic and real world data, are reported and described to comment on the performance of the proposed approach. The main contributions of the paper are in my humble opinion the following:\n- designed and developing a new causal discovery algorithm for semi-stationary time series where the causal graph changes periodically.\n- validation of the proposed approach through syntethic data\n\nSTRENGTHS:\n- the tackled problem which is of both theoretical and practical interest\n- the theoretical framework of the considered problem\n\nWEAKNESSES:\n- the proposed algorithm wisely exploits the PCMCI algorithm while not being original, it looks a little as an incremental result\n- numerical experiments seem to compare the proposed algorithm to algortihms which are not conceived to tackle the considered problem\n- numerical experiments on real world data are disappointing. Indeed, the paper itself recognizes that not having knowledge about the considered domain the results obtained by the proposed algorithm can not be validated nor commented on.  Therefore, I ask myself which is the value of presenting such experiments?\n- computational complexity is not mentioned at all\n- the paper structure and organization makes it not trivial to follow it\n- I found many typos\n\nQUESTIONS:\n- which is the coputational complexity of the algorithm you proposed?\n- how does the proposed algrithm scales with respect to the number of variables of the multivariate time series?\n- could you please explain why the following works had not been taken into account? maybe they are slighlty out of the scope but still linked to the problem you tackled (Learning Continuous Time Bayesian Networks in Non-stationary Domains), (On-homogeneous  dynamic bayesian networks  with  bayesian  regularization  for  inferring  gene  regulatory  networks  with gradually time-varying structure) (Learning non-stationary dynamic bayesian networks)\n- line 104: could you please explain the meaning of setting P(V) different from 0?\n- line 105: the notation used for the pair of variables is not properly coherent, indeed Xj is a univariate timeseries contained in V, which has not been defined. Furthermore, what do you mean by variables here? two time serires? \n- line 115: what do you mean by finite maximum lag?\n- would you be that kind to comment explicitly and not only formally on the assumptions made about the time when nonstationarity occurs, i.e., the casual graph changes its structure? synchronous? asynchronous for each time series? could you discusse more on this aspect? what about parameters non stationarity? I guess you should consider this is excluded to happen, or not?\n- line 156: the definition of n is a little confusing. Indeed, n belongs to a set that is defined as a function of n again, would you be that kind to help me understand?\n- formula (14), the expernal p is the same probability ad the p inside brackets?\n- line 243, I read \"continuous time series in four steps\", but I found only three of them\n- would you also consider the structrual hamming distance to measure the performance of the proposed algorithm\n\nLIMITATIONS:\n- computational complexity of the proposed is not mentioned at all\n- comparison has not been performed with algorithm natively designed for semi-stationary time series data. Maybe no such algorithms exist?\n- results on real world data are of no help because no ground thruth is available and no domain expert comments are given to corroborate it", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper describes a constraint-based causal discovery method for semi-stationary SCMs, where  the causal structure is periodically changing over time.\nThis type of structure is relevant in real-life scenarios where seasonal or diurnal variation is present for example.\nThe problem is rigorously formalized and and algorithm is give to reconstruct the SCM. As a first step the algorithm identifies a superset of parents using PCMCI naively, \nthen searches for the correct periodicity for all variables by minimizing parent set size. The author provides theoretical guarantees that the reconstructed causal graph \nis indeed the generating graph in the infinite sample size limit, if all assumptions hold.\n\nSTRENGTHS:\nThe paper describes a novel extension of PCMCI for semi-stationary SCMs, and gives theoretical guarantees. The probrem formulation is sufficiently rigorous, and the algorithm is described in sufficient details. This makes the works reproducible.\n\nThe method relaxes a quite frequent assumption of causal discovery methods in the temporal domain: stationarity. Even if this new relaxed assumption also likely violated by many real life dataset, it is clearly an improvement  in cases when stationarity is clearly a wrong assumption.\n\nWEAKNESSES:\nThe main limitation I see is the following: For example, if the causal effect changes by the day of the week but the samples are collected in every hour, this periodicity is hard to represent in the presented framework. Given the process inherent timescale is hourly, we cannot just subsample the data as this would induce transitive dependencies. However in the current form the method will try to fit a new causal mechanism in every hour (until the period is reached of course).\nIt would be good to know what happens if this type of data was fed to the algorithm, as this is hard to avoid. To avoid the identification of the transitive dependencies as genuine we are motivated to work as fine timescale as available.\n\nSimilar question applies for the \"sudden\" mechanism change. In real life most likely the sudden change will transform to a smooth change as the sampling frequency increases (for example seasonal effect in a daily scale). Most likely the effect of the Atlantic air temperature on the Central Pacific air temperature does not disappears suddenly.\n\nQUESTIONS:\nl257: **Following the previous work in Huang et al. [2020], F1 score, Adjacency Precision, and Adjacency Recall are used to measure the performance**\nPlease specify in details how you compute these metrics. Classical methods reconstruct a single causal mechanism per variable, while the ground truth has multiple.\n\nl259: **The standard error of the averaged statistics, displayed either by color filling or by error bars is usually too small to be observed.**\nYou mean here the standard error of the mean estimate? You can use the sample variance instead.\n\nFig 2a, In horizontal axis $\\omega_{max}$ is shown. Has the code been executed with the same $\\omega_{ub}$ for all cases, or the $\\omega_{ub}$ was smaller for smaller $\\omega_{max}$ trials?\nIn general what hyper parameters (upper bounds, significance thresholds) were selected? What CI test was used?\n\nIn the Case Study, if you have monthly data, and Central Pacific has 3 parent sets, it seems to me the causal effect is different in every month say the time partitions are {January,April, July, .. } {February,May, August, ..} {March, June, September, ...}, how you come to the conclusion that: **\"the causal effect from the tropical Atlantic air temperature to the Central Pacific air temperature would disappear every quarter of a year\"*** ? I am not sure I understand this point.\n\nMinor comment:\nI would suggest to use \"discrete valued\" and \"continuous valued\" terminology instead of \"discrete time series\" / \"continuous time series\", as it can be mistaken with discrete and continuous time.\n\nHow the  sampling frequency influences the reconstruction of the SCM? (See weaknesses part.)\n\n**Rebuttal**: I have read the author's rebuttal and we exchanged further comments. I still see the paper having potential interest for the audience of this conference. I had one main open question, the result of the case study.  Its result is unfortunately quite implausible, therefore I cannot further increase my score.\n\nLIMITATIONS:\nThe author points out some limitation of the method. For example the higher sample complexity of the method relative to vanilla PCMCI.\nIt is not clear how realistic the present semi-stationarity assumption.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n## Summary\nThe paper addresses the problem of discovering causal relations in semi-stationary time series data, where a finite number of causal mechanisms occur sequentially and periodically across time. The authors propose a constraint-based, non-parametric algorithm called $PCMCI_\\Omega$ to handle this type of time series data. \n\nThe main contributions are \n1. The proposed algorithm, PCMCIΩ, is an extension of the PCMCI algorithm designed for stationary time series data. It leverages the PCMCI algorithm to systematically discover the superset \nand use CI test to find out the correct periodicity and remove unnecessary parents.\n\n2. The soundness of the PCMCIΩ algorithm is demonstrated through a theoretical analysis, showing that it can recover the true causal graph under specific assumptions.\n\n3. The authors validate the effectiveness of the PCMCIΩ algorithm through experiments on both continuous and discrete synthetic time series data. Although I think the baselines can be stronger.\n\nSTRENGTHS:\nStrengths of the paper:\n1. Originality and significance: The paper proposes a novel algorithm, PCMCIΩ, to address the challenging problem of causal discovery in semi-stationary time series data. Fewer work have considered such settings and most of the previous method assume stationarity. This is a step towards generalising causal discovery to non-stationary time series. \n\n2. Soundness: This work provides a theoretical guarantees on the soundness of the proposed method, and validate it with experiments.\n\nWEAKNESSES:\nThe main weakness of this paper are\n 1. Limited scope of experiments: Although the authors have performed experiments on synthetic continuous and discrete time series data, as well as a real-world climate dataset. However, the baseline selection is too simple. For example, VARLinGaM and DYNOTEARS are both linear models with stationarity assumptions. It is expected to outperform those two methods. I suggests including more stronger baselines, such as Granger causality baselines [1], and SCM-based method [2]. \n\n2. The paper is hard to understand when I first read it. There are many definitions defined in section 2, without knowing why they are important. The graphical illustration helps a little bit, but still confuses me on why we have to define them. For example, I am not sure I understand why Illusory Parent sets requires a subscript. Like why we need $Pa_2(X_7^1)$ and $Pa_3(X_7^1)$. Isn't it the same as $Pa_1(X_8^1)$? Since the following theoretical analysis extensively uses these definitions, I recommend the author to be very clear on these definitions and should make them easier to understand for the readers. \n\n3. How does your method performs when there are mismatches in the assumptions? For example, the ground-truth is stationary/non-stationary. When the lag upper bound and period upper bound are shorter than the truth. etc. More ablation study would be helpful.\n\n4. What are the potential limitations of the proposed method? Some discussion should be added. For example, instantaneous effect can be important when aggregation happens. Computational complexity with dimensionality, etc. \n\n[1] Khanna, S., & Tan, V. Y. (2019). Economy statistical recurrent units for inferring nonlinear granger causality. arXiv preprint arXiv:1911.09879.\n\n[2] Gong, W., Jennings, J., Zhang, C., & Pawlowski, N. (2022). Rhino: Deep causal temporal relationship learning with history-dependent noise. arXiv preprint arXiv:2210.14706.\n\nQUESTIONS:\n1. I am a bit confused on line 123, why minimum periodicity has the same symbol as the number of causal mechanisms in $V$?\n\n2. In definition 2.3, eq.8, the subscript $k$ is not showing in the latter equations. What does this k means?\n\n3. Can you make the definition of Illusory parent sets easier to understand and explain the intuition behind it?\n\n4. For figure 1, why $Z_2^1$ is identical to $Z_1^7$? I thought $Z_2$ defines the Markov chain for second time series? Also, you use superscript in $X$ to indicate time series number, I think $Z$ should follow the same pattern to avoid confusion.\n\nLIMITATIONS:\nThe author does not explicitly provide a discussion on the limitations. Hence, I suggest the author to include a limitation section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The authors propose a causal discovery algorithm for a class of periodic random processes called semi-stationary process. The algorithm is built upon an existing algorithm assuming stationarity. Consistency results of the algorithm is provided.

STRENGTHS:
1. Consistency results of the algorithm was shown even though the algorithm is relatively complicated.

2. The algorithm shows superior performance against the baselines.

WEAKNESSES:
1. The motivation for generalizing PCMCI to handle semi-stationary processes is not clear, especially from a theoretical perspective. The description of PCMCI is completely moved to the supplementary. I think including a brief description and the key procedures (e.g., how the superset is estimated) is helpful for understanding why PCMCI is good fit for the considered setting. 

2. As mentioned in Section F. from Appendix, the proposed algorithm does not yield good performance in finite samples until the heuristic method $turning\text{ } points$ is introduced. To support the theoretical results, it should be shown that the algorithm starts to work when the sample size is sufficiently large. 

typo: 
Definition 2.3 is not correct: $\Pi_{k}^{j}(T)$ depends on $k$, but its definition in (8) does not depend on $k$.

QUESTIONS:
Since PCMCI assumes stationarity, in Algorithm 1, why PCMCI is used to estimate the superset $\widehat{SPa}(X_{t}^{j})$ for semi-stationary processes?

LIMITATIONS:
No potential negative societal impact.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07