DyCAST: Learning Dynamic Causal Structure from Time Series
Authors: Yue Cheng
Bochen Lyu
Weiwei Xing
Zhanxing Zhu
Venue: ICLR 2025 Poster (2025)
Keywords: dynamic causal discovery; time series
Primary area: causal reasoning

Abstract: Understanding the dynamics of causal structures is crucial for uncovering the underlying processes in time series data. Previous approaches rely on static assumptions, where contemporaneous and time-lagged dependencies are assumed to have invariant topological structures. However, these models fail to capture the evolving causal relationship between variables when the underlying process exhibits such dynamics. To address this limitation, we propose DyCAST, a novel framework designed to learn dynamic causal structures in time series using Neural Ordinary Differential Equations (Neural ODEs). The key innovation lies in modeling the temporal dynamics of the contemporaneous structure, drawing inspiration from recent advances in Neural ODEs on constrained manifolds. We reformulate the task of learning causal structures at each time step as solving the solution trajectory of a Neural ODE on the directed acyclic graph (DAG) manifold. To accommodate high-dimensional causal structures, we extend DyCAST by learning the temporal dynamics of the hidden state for contemporaneous causal structure. Experiments on both synthetic and real-world datasets demonstrate that DyCAST achieves superior or comparable performance compared to existing causal discovery models.

Published as a conference paper at ICLR 2025
DYCAST: LEARNING DYNAMIC CAUSAL STRUCTURE FROM TIME SERIES
Yue Cheng Bochen Ly
Weiwei Xing
Zhanxing 3hu
Beijing Jiaotong University DDtaCanvvas University of Southampton yuecheng(bjt jtu. edu. en bochen 1v@gmail....... com wwxingeb wtw.:itjutuuuu...omo.oo cn
2. z.zh@soton.ac.. ac..oo.nnccu....
ABSTRACT
Understanding the dynamics of causal structures is crucial for uncovering the derlying processes in time series data. Previous approaches rely on static assump- tions, where contemporaneous and time-lagged dependencies are assumed to have invariant topological structures. However, these models fail to capture the evoly- ing causal relationship between variables when the underlying process exhibits such dynamics. To address this limitation, we propose DyCAST, novel frame- work designed to learn dynamic causal structures in time series using Neural Or- dinary Differential Equations (Neural ODEs). The key innovation lies in model- ing the temporal dynamics of the contemporaneous structure, drawing inspiration from recent advances in Neural ODEs constrained manifolds. We reformu- late the task of learning causal structures at each time step as solving the solution trajectory of Neural ODE the directed acyclic graph (DAG) manifold. To ac- commodate high-dimensional causal structures, we extend DyCAST by learning the temporal dynamics of the hidden state for contemporaneous causal structure. Experiments on both synthetic and real-world datasets demonstrate that DyCAST achieves superior or comparable performance compared to existing causal discov- ery models.
INTRODUCTION
Learning causal structures from time series data has been recognized as fundamental and chal- lenging problem due to its widespread use in various domains, such as traffic (Cheng et al., 2024b), biology (Sachs et al., 2005; Yu al., 2023), healthcare (Lucas et al., 2004) etc. Recently, Pamfil et al. (2020); Sun et al. (2023); Gao et al. (2022) have made significant efforts in this direction of causal discovery through the directed acyelic graphs (DAGs), which provide important clues about the relationship between variables particularly in multivariate dynamical system.
Existing methods typically adhere DYNOTEARS (Pamfil et al., 2020) paradigm that classifies causal structures in time series into contemporaneous (intra-slice) and time-lagged (inter-slice) de- pendencies. As shown in Figure. (b), the weight of each edge between nodes (drawn as solid lines) at the same time / represents the intra-slice dependencies, while the weight of edges between nodes (drawn as dashed lines) at different times captures inter-slice. However, most such results (Pamfil et al., 2020; Sun et al., 2023; Li et al., 2024) mainly focused on static DAGs, i.c. intra- and inter-slice structures with invariant topology over a given set of variables. A major challenge in dynamic data in real-world is its inherently non-stationary properties, in which DAG topological structures (i.e. variables' causality) can evolve over time. Examples include traffic networks, where roads may have periodically variable lanes, and the network can undergo significant changes due to sudden accidents or temporary traffic controls (For more details, please refer to the visualization results in Section 4.3). In these scenarios, modeling evolutionary intra-slice causal patterns is crucial for accurately capturing the underlying dynamic properties of the system.
significant challenge in learning dynamic causal structures is that the edges in DAGs can emerge or disappear over time while still maintaining the directed acyclic property, see Figure. 1 (c) for the
"Contributed during an internship DataCanvas.

---

Published as a conference paper at ICLR 2025
ba
X1
43
X3
(4) Original multivariate time-serios
(b) Static intra- and inter-slice structure
(4) Dynamic intra-alice and static inter structure
Illustration of the intra-slice (solid lines) and inter-slice (dashed lines) dependencies in a SEM with d = 3 variables and autoregression order p 1 under different dynamical settings. (a) The original multivariate time series. (b) The causal discovery framework with static setting.
Figure 1:
(c) Our causal discovery framework with dynamic intra-slice dependencies and static inter-slice dependencies settings.
dynamic of the DAGs across time. However, current methodologies are still insufficient for model- ing such a dynamic causal structure. More recent efforts have concentrated on recovering dynamic inter-slice structures, as their continuous evolution avoids the hard constraints maintaining DAG structure (Song et al., 2009). For instance, Gao & Yang (2022) proposed time-varying Granger causal network, where the inter-slice structures are learned via kernel-reweighhed group lasso; such approach requires prior knowledge for appropriately selecting the kemeel and does not model intra- slice dependencies. Nevertheless, we still expect to recover dynamic intra-slice structures from multivariate time series. This is the main goal of our work.
To this end, we propose DyCAST, a dynamic causal structure learning approach for time series based Neural Ordinary Differential Equations (Neural ODEs). Targeting the dynamic intra-slice structures, we devise a constrained Neural ODE to model the temporal latent states dynamics of DAGs. See Figure. for the difference between previous work and our proposed DyCAST. Com- pared with DYNOTEARS, we enforce DAG manifold constraint for the dynamics function to model how the change of intra-slice structure over time. Further, the dynamic function is transformed into latent space to model the complex structure's 's change for nigh-dimension time series. Empirical results across synthetic dynamic and real world datasets substantiate the effectiveness of DyCAST.
RELATED WORK
Static causal structures learning. Inspired by the success of NOTEARS-baseee methods of causal discovery (Zheng et al., 2018), a series of NOTEARS variants were proposed to recover the causal structures in time series. Specifically, DYNOTEARS (Pamfil et al., 2020) pioneered extended instan- taneous linear NOTEARS exploiting autoregression. NTS-NOTEARS (Sun et al., 2023) proposed capture nonlinear relations with 1-D CNN. IDYNO (Gao et al., 2022) and TECDI (Li al., 2023) incorporated intervention data into the DYNOTEARS framework to enhance identifiability. How- ever, the above existing methods for learning temporal causal structure typically assume that the intra-slice and inter-slice topologies remain static.
Dynamic causal structures learning. With the assumption that the causal structures are varing across time, works used various methods for recovering dynamic structures. TVDBN (Song et al., 2009) and KWgL (Gao & Yang. 2022) proposed adopt reweghere fr-regularized auto- regressive procedure for modeling the structurally varying. TVGL. (Hallac et al., 2017) derived scalable passage-passing algorithm infer time-varying structures. However, these works only model the dynamics of inter-slice, and none of them address the dynamics of intra-slice.
Neural ODEs. Neural ODEs were originally introduced to model time series with continuous latent dynamics, offering natural way to capture the dynamical behavior of systems (Chen et al., 2018). However, they have not been extensively explored in the context of dynamic causal structures, as most prior work has applied them to model node interactions in time series forecasting tasks. For instance, HOPE (Luo et al., 2023) incorporated second-order graph ODEs to model higher-order temporal dependencies, while GDERec implicitly captures the temporal evolution of user-item in-
2

---

Published as a conference paper at ICLR 2025
teraction graphs using GNN-based ODEs. SGODE (Chen et al., 2024) extended signed graph Neural ODEs to leverage both positive and negative node information during continuous dynamics. LCCM (Brouwer et al., 2021) leveraged latent dynamics for causal discovery but is limited to static or quasi-static relationships, overlooking dynamic intra-slice structures. Despite these advancements, most existing models focus primarily time series forecasting, rather than addressing changes in intra-slice causal structures.
2 PRELIMINARIES
Notations. We let denote the space of DAGs on d variables. Given N independent realizations of multivariate time series represented X' = {X,,0, X.,,,".. X..," )., where X
. I:N E R is the recording of d variables at time step L, where d represents the number of variables in the dataset. With a slight abuse of notation, we will use X, interchangeably.
:.i.N..:f..... "i,,,,''.... .ii...:
Causal structure model for time series. Let Y = (X,-1l... --.1--..(-pppll be the N x pd matrix of p-order time-lagged version of Xt, W be the d x d matrix of intra-slice and A = [A]] ... | be the pd x d matrix of inter-slice. A causal structure model for time series can be equivalently represented by a structural equation model (SEM) (Pamfil et al., 2020):
-,W + Y + Ze,
(1)
where t {1,2, ,)} with horizon T. Z. N(0,0²1) is noise vector with independent elements across time. The intra-slice structure is defined by the nonzero elements in W, i.c. X is dependent on X if and only if the coefficient W is nonzero. The inter-slice structure is defined by the nonzero elements in Ag, i.e. X; is dependent on X² if and only if the A is nonzero.
Neural ODEs. Neural ODEs (Chen et al., 2018) are a class of deep learning models that effectively learn temporal dynamics for time series modeling. They use neural network to map Zy into hidden space 20, where a continuous model fo governs the dynamics, described with following:
20 J. fo(zs) ds where ((ro), and = E(=c):
where & and Ç are parametrized neural networks that model the relationship + 20 and 2, r respectively. The dynamics the hidden state, fe, are specified by another learnable neural network.
LEARNING DYNAMIC CAUSAL STRUCTURE
3.1 PROBLEM DEFINITION
We will target recovering the dynamic causal structures. That is, the intra-slice dependencies be- tween variables in time and t-1 is not fixed, and W changes over time. Following the practice in causal structure discovery (Pamfil et al., 2020), we formulate the problem finding a series snapshots of DAG W (Wo.... WT} capture the time-dependent causal structure, and a series of graph A = [AI...., A .Ap) for time-lagged. Each graph W, = (), &,, W,) or A = (V,E, A&) is a weighted graph with shared variable set V, a causal link set E, and weighted adjacency matrix W, E ddddd A₁ E gdxdd Thus, a linear form of the dynamic causal discovery model SEM is:
X, X.W, + YA Z.-
(3)
In order to accurately manipulate the time- denendependent causal structure of time series, we use a smooth dynamics assumption to characterize the changes in intra-slice structure. Then, we further find function Fg to model the evolution dynamic causal structure,
W, (t, Wo) s.t. W, E D.
(4)
Thus, minimizing the least-squares loss under the DAG constraint leads to the following optimiza- tion problem:
T
arg min L(W (O), A) A.A
Σ
X.
X.W.(0) - YA S.I. W. DD
(5)
2NT
1=0
3

---

Published as a conference paper at ICLR 2025
3.2 CONSTRAINED NEURAL ODE FOR DYNAMIC DAGS
The change of intra-slice can reflect the underlying dynamics of time series data. To fully capture the dynamic causal structure, inspired by the idea of Neural ODEs, we propose adopt a continuous dynamical system Se for parameterizing the function Fg. i.e.
W = Wo + L fo( s) ds, s SE (0 s.t. W. E D.
(6)
where Wo is also a learnable matrix that denotes the intra-slice matrix when time t = 0; fo(:) specifies the dynamics of time-dependent intra-slice structure and is neural network be learned.
Since our setting introduces additional complexity compared to DYNOTEARS, we found that the difficulty in solving the dynamical system in Eq. (6) is the directed acyclicity constraint on W.. We are confronted with a primary question: how to guarantee the solution trajectory for the Neural ODE governed by fe consistently satisfies the constraint?
Dynamic acyclicity constraint. We adopt an equivalent formulation via the trace exponential func- tion studied (Zheng et al., 2018) to characterize the DAG constraint,
h(W)= = tr(e )
where "o" denotes the Hadamard product of two matrices. They show that the function h satisfies h(W) = 0 if and only W is acyclic. Replacing the acyclicity constraint with the equality con- straint h(W)= 0, we regard the Neural ODE Eq. (6) as underlying ODE (Rheinboldt, 1984) to address the main concern. We further reformulate the vector field so with an invariant manifold M (W, E Rxdd ((W) 0) constrainted Neural ODE (i.e. underlying Neural ODE),
W, = Wo + ( so - W.W.W.hWWWW.... ds, 8 [0,
where 7 2 0 is a scalar parameter, 5(-) Rddd + Ra in our paper is a stabilization matrix to guarantee the manifold M constraint gradually and consistently satisfied. Note that the underlying Neural ODE (8) is equivalent to the neural ODE (6) on the manifold M in the sense that they have the same (analytical) solution set for W, on M. By controlling the trajectory of the dynamic causal structure to asymptotically satisfy DAG constraints, hard enforcement techniques, such as the augmented Lagrangian, are unnecessary.
Inspired by White et al. (2023), we choose the Moore-Penrose pseudoinverse of the Jacobian matrix G(W) of function h as the stabilization matrix, where G(W) Vh(W) = (w.' 2W. This matrix aims to guarantee that S(W)G(W) is symmetric positive definite with the smallest eigenvalue bounded away from zero near M and compatible with gradient-based optimization of 0 as part of neural underlying ODE. Then, for each time f, a G++ (W.) matrix is employed to obtain the trajectories of dynamic causal structures belonging M,
W, = Wo + [( fo(W., s, - - (W.))WWW ds, s E [0 where G+ (W.) = GT (W.) (W.))
3.3 CONSTRAINED NEURAL LATENT ODE FOR DYNAMIC DAGS
When inferring dynamic causal structure, fo W., s) can be expensive solve directly using stan- dard Dormand- -Pinne methods (Hartman, 2002), especially in real-world applications. This is because W. can be very high-dimensional. Fortunately, variety of works provided both theo- retical (Holmes, 2012) and empirical evidence (Noack et al., 2011; Sholokhov et al., 2023) that many dynamical systems evolve on a latent space with lower dimensions. Therefore, we follow the Preeee-Proooss--eeeeeeee
(Battaglia al., 2018) fashion to design the neural latent ODE for the intra-slice structure, as sketched in Figure. 2. We now present the details of each component.
Encoder.
The goal of the encoder is to efficiently compress the intra-slice structure into compact, diver-dimensional representation, while preserving essential dynamic characteristics, leveraging the inherent sparsity of the DAG structure. For the initial time step lg. the intra-slice structure Wo
4

---

Published as a conference paper at ICLR 2025
Rdxd derived from the observation Xg. represents a high-dimensional state, especially when d is sufficiently large. To represent the causal properties of each variable more explicitly, we transform the initial intra-slice structure Wo into S Rdx2d
So = CO, W
(10)
Note that the i-th row of Wo indicates which variables depend on 2; at the initial time 0, while the i-
W
W.
Static inter-slice A Cousal Structure
Decoder Vy
Latent ODE
(2,.)) s.t. h(W
Encoder 4y
Reconstructed
Observed
Initial intra-slice Wn
1r, X,W
Time Regularized Squares
Figure 2: Constrained Neural Latent ODE for constructing causal structures from hidden states.
th column shows which variables æ; depends on. Therefore, the i-th row So reflects the variables that x; is linked to in the DAG within the entire slice. Then, consider using encoder d to distill S. into key features within a r-dimensional (r < d) latent space. Given the nigh-dimensional intra-slice structure Wo. vectorized latent representation zg Rrd is obtained as:
= FLATTEN (REUUS - (o
(11)
where dg Rd + Rdr is mapping parameterized by neural network, which involves linear transformation P E 2dxx followed by a ReLU activation function.
Processor: latent ODE. In the Processor, the latent state z, is determined by the mapping é from the high-dimensional casual structure matrix W. Therefore, we can apply the chain rule to derive the dynamics of 2,. dz, dz ds, dW, dz, dsc ds, dW, dt ds, dW. (o. (z),
(12)
Additionally, since 2, corresponds to W., and the trajectory of W, must lie within the DAG mani- fold, the dynamics of must satisfy some kind of constraint to ensure that the corresponding We remains a DAG,
h(vo f))
(13)
where the is the decoder neural network to map the compressed latent state 2, at a given time back the corresponding intra-slice structure W. and its details will be described later. Thus, we can integrate such constraint into the latent neural ODE for as follows:
2, = Zo + S 8) - (vho(z, +))k((((2, 8)) ds, E [0,t]
(14)
This Neural ODE aims to learn the dynamic causal structure of time series by exploiting latent states, which can be estimated using a numerical ODE solver,
ODESolver(fe. (t.o............................ (to....,tt)
(15)
Decoder. In some real-world scenarios, e.g. traffic, the dynamics of the W, do not only depend on 2., but also explicitly on the time due to the periodicity of dependency relationship between variables. To deal with this case, concatenate the latent state 2, with the time variable to form a temporal-conditions latent representation & E R ddd++1
CONCAT(ZE,
(16)
5

---

Published as a conference paper at ICLR 2025
This approach enables the decoder to effectively incorporate time-dependent dynamics in its recon- struction of the intra-slice structure W. Utilizing the temporal-condtioned latent representation ž,, the intra-slice structure W, can be reconstructed as follows:
(17) where tyg IRd++1 + Rdd is a mapping parameterized by L-layers neural network with non- linear activation function o. We note that if W. exhibits periodic dynamic behavior, the activation function in the should effectively capture this periodicity. In such cases, recommend utilizing the SIREN activation function (Sitzmann et al., 2020); otherwise, the SiLU activation function can be employed.
W. o(Pr ......00000 o (PP),)) tho (2,,
Optimization problem. To further enhance the model, we introduce & penalty terms to enforce sparsity in both W; and A, leading to the following regularized optimization problem:
T
arg min L(A, Wo, W;) 0.W.
2NT Σ (. YA + A Σ W1(0))+++
A2
(18) where 8 represents all the trainable parameters involving in W., including the encoder Φ9. decoder the and vector field function Eg. We use the Adam algorithm (Kingma & Ba, 2015) to solve this objective function.
Extension of DyCAST. Our DyCAST modeling framework is flexible, which can be easily incor- porated into other casual discovery approaches, particularly those for modeling complex inter-slices temporal causal discovery. When p 1, the summary graph coincides with the inter-slice graph, allowing the use of Granger-causal methods for learning inter-slice structures. For instance, Dy- CAST can be introduced to CUTS+ (Cheng et al., 2024a), a nonlinear model for inferring inter-slice relationships with summary graphs. Following Section 3.3 as well as the work of NTS- -TS-NOSEARS (Sun al., 2023), the foundational model (3).(3) DyCAST can be extend as:
X X.W, +p: (X X 2-P-
X (pppe- + Z;
(19)
where the i-th neural network sP predicts the expectation E. ; X,W, of the target variable X at each time step i, conditioned preceding variables. Here, 9i can be implemented as d separate MLPs or LSTMs to ensure disentanglement. Instead modeling the full-time DAG, we utilize CUTS+ as sp extract summary graph that captures the causal effects of the lagged version on x
X; = X,W, + ((X M) M)+ + Z
(20)
= X,W, + [..., Linear, (MLP (X (x--p--,, hb; D)) Z
where hig is the initial value of GRU hidden states and irrelevant to X1-pt-1, M is a binary causal matrix, where m., = 1 denotes i-th hidden states Granger cause the prediction. Notably, when the preceding variables span only a single time step, the summary graph is equivalent to the full-time DAG.
EXPERIMENTAL RESULTS
We evaluate the efficacy of DyCAST through extensive experiments on both synthetic and real-world datasets. For synthetic data, we perform series of simulation experiments with known ground truth. To demonstrate the broad applicability of our method, we apply two real-world datasets: NetSim (Smith et al., 201 I) and CausalTime (Cheng et al., 2024b). Additional experiments on variable counts, sequence lengths, and noise robustness, along with ablation studies, are provided in Appendix C. We also demonstrate DyCAST's ability to detect dynamic nonlinear interactions using the Human3.66 6M dataset, detailed in Appendix E.
Evaluation Metrics. We assess the performance of our proposed method for learning dynamic causal structures using three key metrics: 1) Fl score, representing the harmonic mean preci- sion and recall. 2) Structural Hamming Distance (SHD), which counts discrepancies (e.g., reversed, missing, or redundant edges) between two DAGs; Since the number of potential non non-causal rela- tionships vastly outnumbers true causal relationships in real datasets, also utilize Area Under

---

Published as a conference paper at ICLR 2025
the Precision-Bei Recall Curve (AUPRC) and Area Under the ROC Curve (AUROC) to evaluate the effectiveness of DyCAST in identifying genuine causal relationships.
A (Intar-alice)
W; (Intra-slice)
W (a(Intra-slice)
W; (i(lnmm-sliee)
W-((nira-sllie)
Ground Truth
11
Ground Leamed
-05
-2
Figure 3: Example results using DyCAST on data with Gaussian noise, consisting of N 500 samples, T time steps, d = 5 variables, and p 1 autoregressive term. The First Row: The ground truth I-order dynamic causal structure. The Second Row: The difference between ground truth and learned dynamic structures over time.
4.1 SYNTHETIC DATASETS
We first follow the procedure outlined in Appendix A.1 generate data with Gaussian noise, con- sisting of N 500 samples, 7 = 8 time steps, d = 5 variables, and p autoregressive term. The evolution function F' is represented by a linear transformation matrix, with both intra- and inter-slice DAGs simulated using an Erdös-Rény: Rényi (ER) model (Erdös & Rényi. 1960), where the mean degree set to We implement DyCAST this dataset using regularization parameters λ = X = 0.05 and scalar parameter
1. In Figure. 3, demonstrate the performance of DyCAST synthetic dataset. We can clearly see that the estimated weights closely match the ground truth for both W. and A. Furthermore, the temporal dynamics of W, are effectively captured.
this
205
d-10
d=15
3.0
0.8
0.8
0.0
0.6
0.4
0.4
0.2
0.2
0.0
R5 95 W9, Wy W. No Wr. We
Ri Wy R6, We R5, Mi.
Wr
We
Wr
R7,
We
RS.
WL
We
89
R2
W9
W5
H6
R.
W.
Figure 4: Fl scores for different temporal causal discovery algorithms and different numbers of variables E (5, 10, 13, 20) on synthetic dataset with dynamic causal structure. Each panel contains results for both intra- and inter-slice structures. Every marker corresponds the mean performance across 4 algorithm runs, each on a different simulated dataset.
Performance on time series with dynamic causal structure. We compare DyCAST against sev- eral causal discovery baselines, including DYNOTEARS (Pamfil al., 2020), NTS-NOTEARS (Sun et al., 2023) and TECDI (Li et al., 2023), on synthetic dataset with variables dimension time steps, and = autoregressive term. The results presented in the Figure 4. The vertical axis indicates the performance of each algorithm measured by the Fl score, computed separately for intra- and inter-slice structures. is clear that DyCAST outperforms the others, consistently attaining F1 scores near over time. The selection of hyperparameter values for the four algorithms discussed in Appendix B.
(5, 10, 15, 20). N 500 samples,
Performance time series with static causal structure. Here show that DyCAST can also naturally adapt to time series with static causal structure. We apply it to a synthetic dataset similar to that in Pamfil al. (2020), where only W, remains fixed over time. As anticipated, DyCAST

---

Published as a conference paper at ICLR 2025
Intra-slice
200
Intra slice
4520
Inter-slice
0,80
30
500
30
20
Number variables (d)
Number variables (d)
variables (d)
Number variables
NTS-NGTEARS
DyCAST
Intra-slice
Intenslice
Figure 5: Left two panels: the intra-slice Fl scores and SHD metries for various temporal causal discovery algorithms across different variable counts d E {5, 10, 20, 50) synthetic dataset with static causal structure. Right two panels: the inter-slice Fl scores and SHD metrics for various temporal causal discovery algorithms.
Table 1: Ablation study synthetic datasets with d d=3 5.
DYNOTEARS
w/o So
w/o Latent state
DyCAST
Structure
TPR
SHD
TPR
SHD
TPR
SHD
TPR
SHD
W;
0.40
0.67
0.86
0.75
1.00
0.67
1.00
1.00
DYNAMIC
W4
0.40
3
0.69
0.86
0.75
1.00
2
0.78
1.00
1.00
We
0.80
0.68
0.86
0.75
1.00
0.82
1.00
1.00
DATASET
W
0.40
0.68
1,00
0.59
1,00
0.93
1.00
1.00
A
0.20
3
0.00
1.00
12
0.37
1.00
0.86
1.00
0.97
0.80
0.89
1.00
7
0.53
1.00
0.89
1.00
1.00
DATASET
1.00
The best bold.
emerges as the sest-perforing algorithm in Figure 5, demonstrating high F1 scores and low SHD values. The second-best algorithm is NTS-NOTEARS. However, its performance tends to degrade as we add the number of variables. Moreover, DyCAST reduces the running time by more than 20% across various variable dimensions. See Appendix C.6 for more details.
Ablation study. We conduct an ablation study by evaluating DyCAST without latent states, as shown in Table 1. Specifically, we run DYNOTEARS, DyCAST, DyCAST w/o So (which means Zo = (RULLEN
and DyCAST w/o Latent states (which means 2o := FLATTEN Wo)) on two datasets: 5-variable synthetic dataset with dynamic causal structure, and 5-variable synthetic dataset with static causal structure. The results from these experiments provide insights into the contribution of latent states and So to the overall performance of DyCAST. The gray shadows in tables that are highlighted in all tables indicate our choice for our method. We find that So plays important role in improving the performance, while latent states further enhance the effect. See Appendix D for real world datasets ablation study.
4.2 NETSIM DATASETS
Baseline methods. For NetSim datasets, we compare DyCAST with GC (Granger, 1969), DYNOTEARS (Pamfil al., 2020), NTS-NOTEARS (Sun al., 2023), PCMCI (Peters al., 2013), NGC (Tank et al., 2022), CUTS+ (Cheng et al., 2024a), LCCM (Brouwer et al., 2021), eSRU (Khanna & Tan, 2020), TECDI (Li et al., 2023).
NetSim is an fMRI dataset aimed at learning gradual changes in brain networks to enhance the understanding of brain functions and intelligence (Smith et al., 2011). contains 28 simulation datasets, from which we select 17 simulations with the same sequence length, each featuring 50 independent time series recordings for d = (5. 10, 15} nodes over 200 time steps. A detailed comparison of AUPRC is shown in Table 2, see Appendix C0 for AUROC results and Appendix D for ablation study. Given that the NetSim dataset explores brain network causal relationships, whose structure does not fully adhere the DAG constraint. We also evaluated variant of DyCAST without the DAG manifold constraint, referred to as DyCAST (No DAG). While TECDI performs better than other variants of NOTEARS (DYNOTEARS, NTS-NOTEARS), DyCAST outperforms
8

---

Published as a conference paper at ICLR 2025
Table 2: Performance on NetSim Dataset Under AUPRC.
Mosic
DATASET
mosormaatss
FCMEI
NGC
cuts+
LCCM
RycAST
DyCAST (OO)
DycASPCUTTS
Sint
0 Mane
083son
0 0.laas
0eluse
(9
Sid
Bifxco
336
D2
0.7
Si6
4 Muais
0.9g
522
0
S1el1
0.5
0.7
0 Monn
0.P
0
035
0.77
033
50a
D42grm
0 21
0.8
05
0337
140
0.0
$55.5
042
04laes:
040mm
023.mm
138
072
S1a21
DA3
038
D4
0.5500
037.x
852
38025
2.2-4x
$1624
0.3000
B4T
033
D24
050-x
0.7kox
035
0.7oo0t
Aveage
TECDI on 82.4% of datasets. Overall, DyCAST achieves an average AUPRC increase of 16.43%, demonstrating adaptability to low-resolution datasets with significant time intervals.
Additionally, we found that DyCAST (Not DAG) can obtain an AUROC value close to 1.0 on some sub-datasets of the NetSim dataset. This because after eliminating the DAG constraint, DyCAST can identify the influence relationship between variables, but it is not accurate in identifying the direction of the influence. Therefore, the AUPRC of DyCAST (Not DAG) is relatively low. More- over, due to the highly nonlinear relationships between variables in the NetSim dataset, nonlinear causal modeling methods like CUTS+ and LCCM demonstrate higher accuracy compared to linear methods. Similarly, the DyCAST variant integrated with CUTS+ achieves average 2.4% higher accuracy than the vanilla DyCAST.
4.3 CAUSALTIME DATASETS
Baseline methods. For CausalTime datasets, we compare DyCAST with GC (Granger, 1969), SVAR, NTS-NOTEARS (Sun et al., 2023), PCMCI (Peters et al., 2013), Rhino (Gong et al., 2023), CUTS (Cheng et al., 2023), CUTS+ (Cheng et al., 2024a), NGC (Tank et al., 2022), NGM (Bellot et al., 2022), LCCM (Brouwer et al., 2021), eSRU (Khanna & Tan, 2020), SGCL (Xu et al., 2019), TCDF (Nauta et al., 2019), TECDI (Li et al., 2023).
0.1
0.90
X1
Xs
20
40
20
40
20
40
Traffic Flow
0.88
X5
0.1
0.0
Xs 3g
0.06
20
20
20
0.84
0.05
0.05
0.00
xs - X/5
0.82
0
20
40
20
40
20
40
0.00
0.80
0.05
As Xy8
20
20
40
20
40
20
40
Time
Time
Figure 6: Visualization of the dynamic causal structure on Traffic. Left: the original traffic time series. Right: the periodic changes of some part edge links in intra-slice structures discovered by DyCAST.
We evaluate DyCAST on the CausalTime dataset, a real-world temporal causal discovery benchmark (Cheng et al., 2024b). CausalTime includes three distinct scenarios: weather (AQI subset), traffic (Traffic subset), and healthcare (Medical subset). The Traffic and Medical subsets each consist of = 20 variables, while the AQI subset contains d = 36 variables. Table 3 shows the performance Appendix D for ablation study. We can observe that:
comparison on temporal causal discovery. (1) Among all the compared methods, DyCAST performs best in the Traffic subset and second- best in Medical. One possible reason is that DyCAST is more effective datasets with rapidly

---

Published as a conference paper at ICLR 2025
changing causal structures. See Figure 6 for more details. (2) Our DyCAST achieves the best aver- age performance, likely due to the explicit incorporation of both instantancous and lagged causality information, which enhances its ability capture complex causal dynamics.
Table 3:
Performance CausalTime Dataset Under AUROC and AUPRC. DyCAST-CUTS com- bines DyCAST for dynamic intra-slice discovery and CUTS+ for nonlinear inter-slice discovery, achieving the best performance.
AUROC
AUPRC
Methods
AQI
Traffic
Medical
AQI
Traffic
Medical
GC
0,45
0.42
0.57
0.63=0002
0.28=0000
0.42
SVAR
0.62:1001
0,6310000
0.71
0.79
0.58
0.6820.04
NIS-NOTEARS
0.57=0022
0.63++003
0.71
0.712
0.58=0005
0.46:20.2
PCMCI
0.53+007
0.54+
0.70+oo
0.67=20.04
0.35=a.06
0.51
Rhino
0.67
0.63
0.65
0.76
0.38
0.49
CUTS
0.602000
0.62.t0.22
0.37
0,51
0.15=0002
0. 0.1520000
CUTS+
0.89
0.62.10.07
0.82+1002
0.80=0008
0.64=20.12
0.55
NGC
0.72
0.60
0.57
0.72=0001
0.36=:0.5
0.46=0001
NGM
0.67 0.602
0.47
0,56100z
0.48
0.28
0.47
LCCM
0.86+0007
0.55
0.80+zoz
0.93=0002
00.5
0.76=2002
eSRU
0.83
0.76:004
0.72=000
0.49=0003
0.74
SCGL
0.49
0.59
0.50
0.36
0.45
0.48
TCDF
0.41
0.50+000
0.63
0.65=0.0
0.36=am0
0.55
TECDI
0.56:002
0.60+0000
0.63
0.65=0000
0.63=a0.1
0.49
DyCAST (Not DAG)
0.85
0.68
0.74
0.70=00.03
0.60=a002
0.69
DyCAST
0.85
0.63
0.81 0.00
0.82.20.00
0.70=
0.74
DyCAST-CUTS+
0.91
0.651
1.01
0.9000.00
0.73
0.77
The hest results hold the second best underlined.
Since our method focuses primarily on the dynamic intra-slice structure, we do not heavily address the inter-slice structure. To comprehensively account for both aspects, we combine DyCAST with other methods that emphasize inter-slice structure. As expected, DyCAST-CUTS+ achieves the best performance across all three datasets.
Visualization of the dynamic causal structure. Figure 6 provides visualizations of the dynamic causal structure learned by DyCAST, using the Traffic subset as an illustrative example. The left panel shows the changes in the original traffic time series within day. The right panel shows the periodic changes in the intra-slice edge links between 2G and other variables found by DyCAST. We blue to represent Fı, orange represent 33, green to represent Jrs, and red to represent 218- We can observe that within a period of day, the causal relationship between intersection 22 and other intersections 11, 33, 3z, Ti8 shows obvious dynamics and periodicity.
CONCLUSION
In this work, we introduce DyCAST, novel constrained latent Neural ODE-based temporal causal discovery framework, specifically designed to learn dynamic causal structures from time series data. We conduct extensive experiments on dynamic synthetic, static synthetic, and real-world time series datasets, demonstrating DyCAST's superior performance in recovering causal structures across all scenarios. In the future, plan explore new architectural designs for neural networks in the latent dynamics model constrained by the DAG manifold. Another direction involves integrating both intra-slice and inter-slice latent dynamics within unified neural network framework.
ACKNOWLEDGEMENT
This work supported by the Beijing Natural Science Foundation under Grant (No.0231005), and by the National Key Research and Development Program of China under Grant (No. No.20224YF33112200)
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Gonzaez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprini 2018.
Alexis Bellot, Kim Branson, and Mihaela van der Schaar. Neural graphical modelling in continuous- time: consistency guarantees and algorithms. In International Conference on Learning Represen- tations, 2022. URL https:://///tcc. http:///ppppeeee.eeeeee/eeee./ep net/ http://wwwnne/euumteeeeeeea forummid-SslBkeeeeee
Edward De Brouwer, Adam Arany, Jaak Simm, and Yves Moreau. Latent convergent cross
mapping. In International Conference Learning Representations, 2021. URL https:
// //openreview... wwwpnneeiee.nee/fooooooooe d
Lanlan Chen, Kai Wu, Jian Lou, and Jing Liu. Signed graph neural ordinary differential equa- tion for modeling continuous-time dynamics. Proceedings of the AAAI Conference on Artificial Intelligence, 38(8):8222-8301. 8301, 2024.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Yuxiao Cheng, Runzhao Yang. Tingxiong Xiao, Zongren Li, Jinli Suo, Kunlun He, and Qionghai Dai. CUTS: Neural causal discovery from irregular time-series data. In The Eleventh International Conference on Learning Representations, 2023.
Yuxiao Cheng, Lianglong Li, Tingxiong Xiao, Zongren Li, Jinli Suo, Kunlun He, and Qionghai Dai. Cuts+: High-dimensional causal discovery from irregular time-series. Proceedings of the AAAI Conference Artificial Intelligence, 38(10): (5000):111211111 11533, Mar. 2024a.
Yuxiao Cheng, Ziqian Wang, Tingxiong Xiao, Qin Zhong, Jinli Suo, and Kunlun He. Causaltime:
Realistically generated time-series for benchmarking of causal discovery. In The Twelfth Interna- tional Conference on Learning Representations, 2024b.
Paul Erdös and Alfréd Rényi. On the evolution of random graphs. Publ. Math. Inst. Hungar. Acad. Sci, 5:17-61, 1960.
Zhuangyan Fang, Shengyu Zhu, Jiji Zhang, Yue Liu, Zhitang Chen, and Yangbo He. On low-rank directed acyclic graphs and causal structure learning. IEEE Transactions on Neural Networks and Learning Systems, 454444492444937 2024. doi: 10.1 1109/NNNLS.200233277353
Tian Gao, Debarun Bhattachariya, Elliot Nelson, Miao Liu, and Yue Yu. IDYNO: Learning non- parametric DAGs from interventional dynamic data. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 6988-7001. 17-23 Jul 2022.
Wei Gao and Haizhong Yang. Time-varying group lasso granger causality graph for high dimen- sional dynamic system. Pattern Recognition, 130:108789, 2022.
Wenbo Gong, Joel Jennings, Cheng Zhang, and Nick Pawlowski. Rhino: Deep causal temporal relationship learning with history-dependent noise. In The Eleventh International Conference on Learning Representations, 2023.
Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pp. 424 438, 1969.
David Hallac, Youngsuk Park. Stephen Boyd, and Jure Leskovec. Network inference via the time- varying graphical lasso. In Proceedings of the 23rd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 205-213, 2017.
Philip Hartman. Ordinary differential equations. SIAM, 2002.
11

---

Published as a conference paper at ICLR 2025
Philip Holmes.
Turbulence, coherent structures, dynamical systems and symmetry. Cambridge university press, 2012.
Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 3.6: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence.
1eo0:1332313333 2014.
Saurabh Khanna and Vincent Y. F. Tan. Economy statistical recurrent units for inferring nonlinear granger causality. In International Conference on Learning Representations, 2020. URL https: ///ppnneeiiww.c.cmmmm/e.e net wow.mmmd-yyxx//a/FF
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Peiwen Li, Yuan Meng, Xin Wang, Fang Shen, Yue Li, Jialong Wang, and Wenwu Zhu. Causal discovery in temporal domain from interventional data. In Proceedings of the 32nd ACM Inter- national Conference on Information and Knowledge Management, CIKM '23, pp. 4074-4078, 2023.
Peiwen Li, Xin Wang, Zeyang Zhang, Yuan Meng, Fang Shen, Yue Li, Jialong Wang, Yang Li, and Wenweu Zhu. Realted: Temporal causal discovery from interventional data with large language model, 2024.
Peter J.F. Lucas, Linda C. van der Gaag, and Ameen Abu-Hanna. Bayesian networks biomedicine and health-care. Artificial Intelligence in Medicine, 30(3):201-214, 2004.
Xiao Luo, Jingyang Yuan, Zijie Huang, Huiyu Jiang, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. HOPE: High-order graph ODE for modeling interacting dynamics. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 23124-2313332 23 29 Jul 2023.
Meike Nauta, Doina Bucur, and Christin Seifert. Causal discovery with attention-based convolu- tional neural networks. Machine Learning and Knowledge Extraction, 1(1):19, 2019.
Bernd R Noack, Marek Morzynski, and Gilead Tadmor. Reduced-order modelling for flow control, volume 528. Springer Science & Business Media, 2011.
Roxana Pamfil, Nisara Sriwattanaworachai Shaan Desai, Philip Pilgerstorfer, Konstantinos Geor- gatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure learning from time-series data. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statis- tics, volume 108, pp. 1595-1605, 2020.
J. Peters and P. Bühlmann. Identifiability of Gaussian structural equation models with equal error variances. Biometrika, 101(1):219 2014.
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Causal inference on time series using restricted structural equation models. In Advances in Neural Information Processing Systems, volume 26, 2013.
Robert J. Prill, Daniel Marbach, Julio Saez-Rodriguez, Peter K. Sorger, Leonidas G. Alexopoulos, Xiaowei Xue, Neil D. Clarke, Gregoire Altan-Bonnet, and Gustavo Stolovitzky. Towards a rig- orous assessment of systems biology models: The dream3 challenges. PLOS ONE, 5(2):1-18, 2010.
WC Rheinboldt. Differential-algebraic systems as differential equations on manifolds. Mathematics of computation, 43(118):473-482, 1984.
Karen Sachs, Omar Perez, Dana Pe'er, Douglas A. Lauffenburger, and Garry P. Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721): 523-529, 2005.
12

---

Published as a conference paper at ICLR 2025
Aleksei Sholokhov, Yuying Liu, Hassan Mansour, and Saleh Nabi. Physics-informen neural ode (pinode): embedding physics into models using collocation points. Scientific Reports, 13(1): 10166, 2023.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. In Advances in Neural Information Processing Systems, volume 33, pp. 7462-7473, 2020.
Stephen M Smith, Karla L. Miller, Gholamreza Salimi-Khorshidd Khorshidi, Matthew Webster, Christian Beckmann, Thomas E Nichols, Joseph D Ramsey, and Mark W Woolrich. Network modelling methods for fmri. Neuroimage, 54(2))875-811, 2011.
Le Song, Mladen Kolar, and Eric Xing. Time-varying dynamic bayesian networks. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 22, 2009.
Xiangyu Sun, Oliver Schulte, Guiliang Liu, and Pascal Poupart. Nts-notears: Learning nonparamet- ric dbns with prior knowledge. Proceedings The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 1942-
1964, 25-27 Apr 2023.
Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaie, and Emily B. Fox. Neural granger causality. IEEE Transactions Pattern Analysis and Machine Intelligence, 44(8):4267-422 2022.
Alistair White, Niki Kilbertus, Maximilian Gelbrecht, and Niklas Boers. Stabilized neural differen- tial equations for learning dynamics with explicit constraints. In Advances in Neural Information Processing Systems, volume 36. pp. 12929-12950, 2023.
Chenxiao Xu, Hao Huang, and Shinjae Yoo. Scalable causal graph learning through a deep neural network. In Proceedings of the 28th ACM International Conference on Information and Knowl- edge Management, pp. 1853-1862, 2019.
Yue Yu, Xuan Kan, Hejie Cui, Ran Xu, Yujia Zheng. Xiangchen Song, Yanqiao Zhu, Kun Zhang, Razieh Nabi, Ying Guo, Chao Zhang, and Carl Yang. Deep dag learning of effective brain nectivity for fmri analysis. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pp. 1-5, 2023.
Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. Dags with no tears: Contin- uous optimization for structure learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianche and R. Garnett (eds.), Advances Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
13

---

Published as a conference paper at ICLR 2025
APPENDIX
A IMPLEMENTATION DETAILS
In this section, we first provide the specific steps and details of synthetic data generation. Secondly, we introduce the structure of each component of DyCAST in its specific implementation.
A.1
DATA GENERATION PROCESS
We provide more details about the data generation process that we use in our numerical experiments form Section 1.
Dynamic intra-slice model. As in Zheng et al. (2018), we use the Rrdós-Rényí (ER) model to generate the initial intra-slice DAG Wo- Similarly, the intra-slice DAG Wr at the final time step generated using the same approach sample uniformly from the intervals [-2.0, -0.5) U [0.5, 2.0]. Given the assumption of smooth causal transitions, generate the intra-slice DAGs for the remaining t- transition times using linear interpolation. At the i-th time step, the interpolated matrix W; is defined as follows:
the initial structure. To assign weights the DAG,
W = (1 - f.)) Wo + t, W.
(21)
where t; = - with representing the interpolation step. As i increases, the interpolation coeffi- cient t linearly progresses from to 1, smoothly transitioning from Wo to Wr.
Static inter-slice model. We employ a directed Erdös-Rényi (ER) model to generate inter-slice DAGs. Specifically, we sample entries of the binary adjacency matrix A using independent and identically distributed (i.id.) Bernoulli trials with probabilities of k/d. Given the binary inter-slice adjacency matrix Abin, we sample edge weights uniformly from a specified interval that depends on the parameter p. More precisely, edge weights from slice L - p to slice are sampled from the intervals 1-0.5o.- -0.3a] U [0.3xx, 0.5a]. where a 1/1p-1 and parameter 7 effectively reduces the influence of variables that are farther back in time relative to the current time slice.
1.5. This weight decay
Once we have W, and A, we utilize the SEM from Eq. (1) to generate a data matrix X of size T x d. The noise term Zi in Eq. (1) consists of independent and identically distributed (i..d.) random variables.
A.2 DYCAST IMPLEMENTATION DETAILS
The encoder part: Similar to Neural LAD, we solve the constrained Neural latent ODE with defined Φe, which is parameterized
single layer MLP with ReLU.
The vector field part: The vector field of DyCAST is constructed by stacking multiple linear layers, each followed by tanh activation function. This structure enables the model to capture complex dynamics of hidden states.
The decoder part: stack multiple linear layers with non-linear activation function to param- eterize the neural network th.. Specifically, we configure the first layer as a multi-layer perceptron (MLP) with an input dimension of d and output dimension corresponding the hidden layer. The final layer has an input dimension equal to the hidden layer and an output dimension of d x to generate the intra-slice structure. Typically, the decoder employs the SiLU activation function. However, if the intra-slice structure is known to exhibit periodicity, we select the SIREN activation function instead.
B EXPERIMENTS DETAILS
We show the detailed settings of papammtrsssserss including learning rate, hidden dimensions, and stable matrix scale factor. We run all experiments an NVIDIA GeForce RTX 4090 GPU. It is worth that the DyCAST converges faster than the other variants NOTEARS, so it achieves better performance earlier than baselines.
14

---

Published as a conference paper at ICLR 2025
Table 4: Detailed hyper-parameter settings of all networks all datasets.
Type
Dataset
Lags
Ir
?
Activation
d=5
1.00E-03
1
SiLU
d=10
1.00E-03
SiLU
DYNAMIC
d=15
I 1.00E-03
12
1
SiLU
d=20
I
1.00E-03
16
1
SiLU
d=50
I
1.00E-03
SiLU
d=100
I
1.0E-03
80
0.1
SiLU
d=200
I
1.00-033
180
0.01
SiLU
d=300
1.00E-03 I
240
0.01
SILU
d=5
1
1,00E-03
SiLU
d=10
1.00--03 I
SILU
STATIC
d=20
1.00E-03 I
16
SiLU
d=50
1.00E-03 1
40
SiLU
d=5
1
1.00E-03
1
SiLU
NETSIM
d=10
1,00-033
SiLU
d=15
1.00F-03
SiLU
d=20 (Traffic)
1.00E-03
Siren
CAUSALTIME
d=36 (AQI)
I
1.00E-03
30
1
SiLU
d=20 (Medical)
I
1.00E-03
16
1
SiLU
HUMAN3.6M
d=16
1.00E-03
16
1
SiLU
C ADDITIONAL QUANTITATIVE RESULTS
In this section, we add several quantitative experiments, including analyses on variable counts, sam- ple sizes, sequence lengths, the rank of causal structures, unknown autoregressive order, noise ro- bustness, and running times.
C.I LARGE-SCALE VARIABLE OF DATASETS.
This section provides additional results of DyCAST on the large-scale variable datasets for Section 4.1, as shown in Table 5. We run DyCAST and other baselines on the synthetic datasets with variables dimension E (50, 100, 200, 300), N 500 samples, autoregressive term.
time steps, and
Table 5: Performance on large-scale variable datasets.
50
100
200
200
Marbods
NTS NOTEARS
1.40.000
2500.00
0,00sm00
1,0020
2500grm
6.00pmm
We can see that DYNOTEARS, NTS-NOTEARS, and TECDI fail to converge when the number of variables reaches 50, incorrectly inferring causal relationships for all edges. In contrast, DyCAST achieves average Fl score of 0.73 under the same conditions. Scaling up to 300 variables, these baselines remain non-convergent, while DyCAST maintains an average score of 0.56 and a low SHD 49.1 14.144 ), demonstrating robustness in handling large-scale causal structures.
C.2 SMALL SAMPLE SIZES DATASETS.
This section provides additional results of DyCAST on small sample sizes for Section 4.1, as shown in Figure 7. We DyCAST and other baselines the synthetic datasets with sample size n E (20, 50, 100, 200, 300, 400), d 20 variables dimension, time steps, and p = autoregressive term.
15

---

Published as a conference paper at ICLR 2025
= 20
1.0
400
0.8
Average F1 Score
300
0.6
Average SHD
250
200
150
100
400
300
200
100
50
400
300
200
100
20
(Sample size)
(Sample size)
TECDI
NTS NOTERSS
DYNOTEARS
DyCAST
Figure 7: Performance small sample sizes datasets. Left panel: Average Fl scores for different temporal causal discovery algorithms and different sample sizes E {400, 300, 200, 100, 50, 20} on synthetic dataset with dynamic causal structure. Right panel: Average SHD for different temporal causal discovery algorithms and different sample sizes on the synthetic dataset with dynamic causal structure.
As sample size decreases, accuracy drops across all methods, but DYNOTEARS, NTS-NOTEARS, and TECDI fail converge with fewer than 100 samples. In contrast, DyCAST achieves erage Fl score of 0,41 with just 20 samples, maintaining a low average SHD (20) per time step, highlighting its data efficiency compared to other methods.
C.3 DIFFERENT TIME STEPS DATASETS.
This section provides additional results of DyCAST on different time steps of datasets for Section 4.1, as shown in Figure We run DyCAST and other baselines on the synthetic datasets with time steps T {8, 16, 32, 64, 128], d 10 variables dimension, N 500 samples, and p autoregressive term.
As time steps increase, the complexity of intra-slice dynamics grows, leading to gradual decline in DyCAST's accuracy. Nonetheless, even with ultra-long time steps (T = 128), DyCAST achieves average Fl score of 0.5, while avoiding the SHD surge observed in baseline algorithms.
C.4 DIFFERENT NOISE STRENGTH AND TYPES OF DATASETS.
This section provides additional results of DyCAST on noise strength and types of datasets for Section 4.1, shown in Figure 9.
As noise increases, the performance of causal graphs estimated by DyCAST declines. Notably, the intra-slice matrix of the first time step the most sensitive to noise. While the accuracy of intra-slice matrices for subsequent time steps also decreases, the overall performance remains high. This is because the first time step relies solely on initial conditions, lacking prior variables to aid in estimating its dynamics. Additionally, we also run DyCAST on datasets with various types of noise, and the results demonstrated that DyCAST performs robustly across different noise conditions, as shown in the right two panels of Figure 9.
C.5 AUTOREGRESSIVE ORDER.
This section provides additional results of DyCAST on the correct value the autoregressive order p is unknown, shown in Figure 10 and 12. We first follow the procedure outlined in Appendix A. to generate data with Gaussian noise, consisting of 500 samples, T = 8 time steps, d = 10 variables, and Ptrac autoregressive term, as shown in the first of Figure 10. We then
16

---

Published as a conference paper at ICLR 2025
10
d 10
1.0
100
0.8
80
Average F1 Score
Average SHD
0.6
60
0.4
0.2
20
0.0
32
128
(Time step)
(Time step)
TECDI
DyCAST
Figure 8:
Performance on small sample sizes datasets. Left panel: Average Fl scores for different temporal causal discovery algorithms and time steps T (8, 16, 32, 64, 128) synthetic dataset with dynamic causal structure. Right panel: Average SHD for different temporal causal discovery algorithms and different time steps the synthetic dataset with dynamic causal structure.
1.0
3.0
1.0
115c0
Gunte
Doutton
Escorance
W; W1
Figure 9: Left two panels: Fl scores and SHD for different temporal causal discovery algorithms and different noise strengths 10.1, 0.5, 1. 1.5, 2, 5, 10) of the synthetic dataset with dynamic causal structure. Right two panels: F1 scores and SHD for different temporal causal discovery algorithms and different noise types (Gaussian, Exponential, Gumbel, Uniform} ) of the synthetic dataset with dynamic causal structure. Each panel contains results for both intra- and inter-slice structures. Every marker corresponds to the mean performance across 4 algorithm runs, each different simulated dataset.
17

---

Published as a conference paper at ICLR 2025
4s (frtendion)
(lntersliee)
(limadding)
(latradiio))
Groud Truh
Lanal
Crunt Tran Lamaal
Figure 10: Example results using DyCAST on data with Gaussian noise, consisting of N 500 samples, T = time steps, d 10 variables, and = 1 autoregressive term, while the p=3. The The ground truth 1-order dynamic causal structure. The Second Row: The learned 3-order dynamic causal structure. The third Row: The difference between ground truth and learned dynamic structures over time.
First Row:
DyCAST to this dataset using an estimate autoregressive term p 3, regularization parameters d = A2 = 0.05 and scalar parameter y 1. We can clearly see that the estimated weights closely match the ground truth for both W, and A, the estimated inter-slice matrices reveal that only the 2nd-order matrix A₂ contains entries with very small values, while the 3rd-order matrix A3 aligns closely with the ground truth. Thus, we suggest that when p unknown, opting for a slightly larger value is preferable.
0.4
L(Wo. W,, A)
max(A_3
> Ptrue
p > Perue
0.2
0.1
9 10
0.0
1
2
5
6
10
Figure 11: Left panel: Objective value function of p. Right panel: Largest absolute value in Ap as function of
Following DYNOTEARS, we also illustrate two diagnostic methods for selecting p in a simulated dataset with Pure 5 as shown in Figure 12. In the left panel, the objective function decreases as p increases, plateauing when > Punes indicating no improvement in model with additional complexity. For real-world data, where Pine is unknown, plateaus in the BIC score can guide the selection of p. p can also estimated by analyzing the weights of the inter-slice matrix, as shown in the right panel. When is unknown, increment until the entries of A, become negligible.
C.6 PARAMETER SELECTION.
The hyperparameters in DyCAST include the & sparsity terms, X and A2. the scale coefficient of the y stable matrix, and the embedding dimension of the hidden state. For the sparsity terms X
18

---

Published as a conference paper at ICLR 2025
and X2 adopt the values reported in the DYNOTEARS (Pamfil et al., 2020) to ensure fair comparison. For the remaining parameters, r and y, which are specific to DyCAST, we conducted experiments the simulated dataset, using the Fl score the evaluation metric.
Rank 30%
Rank 60%
1.0
1.0
0.8
Average F1 Score
Average Fl Score
0.6
0.4
(Embedding
(Embedding dim)
Figure 12: Left panel: Fl score values under different embedding dimensions under dynamic synthetic dataset. Right panel: Fl score values under different stable matrix scale coefficient under dynamic synthetic dataset.
Fang et al. (2024) observed that causal DAGs often exhibit a central structure, resulting in low rank. To evaluate this, we conducted experiments on low-rank and high-rank simulated datasets with d = 10, where the rank of the W, matrix corresponds to 30% and 60% of the number of variables, respectively. The results are presented Figure 12.
Embedding-tee -telaiable Dimension Ratio (r)
Score under different value of y
E1 score Score
0.8
score Score
0.8
0.6
0.6
0.4
0.4
0.0
0.0
10%
40%
60%
80%
100%
0.01
2
Ratio
Value
Figure 13: Left panel: Fl score values under different embedding dimensions under dynamic syn- thetic dataset. Right panel: Fl score values under different stable matrix scale coefficient under dynamic synthetic dataset.
In both low-rank and high-rank scenarios, increasing the embedding dimension r improves DAG accuracy. However, in the low-rank case, even when r is only 20% of the number of variables, the average Fl score still reaches around 0.8. Notably, when the rank is only 10% of the number of variables, as shown the left panel Figure 13, DyCAST relatively insensitive to both hy- perparameters. To ensure that the hidden states remain in a low-dimensional space, we select as 80% of the number of variables. For the seale coefficient y, as shown in the left panel of Figure simply choose value of I, it has minimal impact on the final recovery performance of the intra-slice structure.
C.7 RUNNING TIMES
In this section, we provide some illustrative running times for different numbers of variables d in Figure 14.
19

---

Published as a conference paper at ICLR 2025
Running Time
Time (log(minutes))
10
20
50
100
Number of variables (d)
MTS NOTEARS
DYNOTEARS
DyCAST
Figure 14: Running times for the simulations from Figure 4 (N 500 samples, T = 8 time steps,
and p autoregressive term).
We ensure consistent density and edge weights for both intra-slice and inter-slice connections across graphs with varying numbers of variables. The runtime of DyCAST scales gradually with the num- ber of nodes, maintaining lead of approximately 20%. Notably, TECDI's runtime exceeds 12 hours when the number of variables reaches 100. While DYNOTEARS achieves faster runtimes with fewer variables, it fails converge when the variable count exceeds 50, leading shorter but incomplete runs. Additionally, we test the runtime of the DyCAST extension against DyCAST, Dy- CAST (Not DAG), and DyCAST-CUTS+ on synthetic dataset. Each model runs for 1000 epochs, and the total runtime is presented in Table 6.
Table 6: Runtime comparison of DyCAST variants synthetic data with d 20, T 8 time steps.
Methods
DyCAST
DyCAST (Not DAG) DYCAST-CUTS+
Running times
16.67 min
16.53min
17.34min
The integration with CUTS+ introduces minimal overhead, ensuring DYCAST-CUTS+ remains computationally efficient despite the added complexity of modeling non-linear inter-slice relation- ships.
C.8 VISUALIZATION OF THE HIDDEN STATES
We visualize the hidden states of intra-slice structures under predicted dynamic synthetic dataset and static synthetic dataset expansion in Figure 15. The left panel shows the hidden states weights of periodic dynamic intra-slice structure, and the right is the hidden states weights of static intra- slice structure. We can see that the hidden states under a periodic dynamic intra-slice structure can capture the obvious mutation point, which is the midpoint of the periodicity. Correspondingly. under the static ground truth, the hidden states show the same values.
C.9 CHALLENGES OF EVALUATING SEPARATE LEARNING ON DYNAMIC SYNTHETIC DATA.
We observe from the experimental results in Figure that NOTEARS and its variants perform poorly datasets with dynamic modes. To address this, applied multi-stage trick to these methods
20

---

Published as a conference paper at ICLR 2025
Figure 15: Visualization of the hidden states on synthetic datasets. Left: the hidden states weights of a periodic dynamic intra-slice structure with d 10, r 9, T 24. Right:the hidden states weights of a static intra-slice structure. The vertical axis represents the time step, and the x-axis represents the dimension of the hidden states.
d-20
1.0
FI Score
Score
Score
W: W. W. Mi N, W, W. Ms
W. W, N, N: W. W. W. H, A
Ni M; W, W. Ms M, W; W,
W. 9, W, 8. W, W. Ri N5
Causal structure over time
Causal structure time
Causal structure over time
Causal structure time
TECH
NTS NOT-NOSEARS
DYNOOTEARS
DyCAST
Figure 16: Fl scores for different temporal causal discovery algorithms and different numbers of variables {5, 10, 15, 20} on synthetic dataset with dynamic causal structure. Each panel contains results for both intra- and inter-slice structures. Every marker corresponds to the mean performance across 4 algorithm runs, each on different simulated dataset.
and then compared them with DyCAST. Specifically, apply the baseline method T times on dataset of length T to obtain T intra-slice DAGs. Then we compare it with DyCAST obtained from single rum, and the results are shown Figure 16. We show that the separate learning trick improves the causal discovery performance of DYNOTEARS, NTS-NOTEARS, and TECDI on dynamic datasets but requires running their respective algorithms times per dataset. Despite this, DYNOTEARS and TECDI with the trick still fall short of or only close to DyCAST. Notably, while NTS-NOTEARS slightly outperforms DyCAST on intra-slice relationships when the variable count 20, its inter-slice performance remains significantly inferior. In conclusion, DyCAST's ability to jointly learn from all time steps while capturing the dynamic evolution of intra-slice matrices over time provides significantly better results in terms of both accuracy and generalization.
C.10 NETSIM EVALUATION UNDER AUROC
Due to space constraints in the main text, we report the AUROC results for each method on the NetSim dataset in Table 7. It can be seen that under the AUROC evaluation criteria, DyCAST still achieved the best results.
We also provide the results of running cLSTM, LINGAM, and CUTS on the NetSim dataset, as shown in Table 8. In each column of results, the first column is the AUROC value, followed by the AUPRC value.
D ADDITIONAL ABLATION STUDY
This section provides additional ablation study results of DyCAST the NetSim and CausalTime dataset, as shown in Table and Table 10.
We conduct the ablation study by evaluating DyCAST without latent states and Sp. Moreover, the causal graph of NetSim and CausalTime is not necessarily DAG, so we added the ablation of DAG constraint. In addition, since DyCAST can be combined with other Granger causality-
21

---

Published as a conference paper at ICLR 2025
Table 7: Performance on NetSim Dataset Under AUROC.
INCAST
Smal
007
0.55ax
0.20ooc
Sine7
0.75
0
0. 19
07
07fars
0.72
4.6
0,82
082aam
0.5
DAlae
0
0.5
0.
07
0.7
Stm.2
0.35xx
00646.
Sie13
0.2
0 0.60
04
0.Raad
0d
Noo
0.5
0.0
Seell
085am0
Stm.3
0.5
085ax
Sta:6
000
04500
0.3san
0.75a
077
0.5
8.7
Siel7
05
077am
077
0344xm
045emn
Sia21
0.4
044630
064kan
B77
035ia
037
0.0.111
Sie30
066
0M
8Max
d8taxo
032axx
S1824
0,5mc
033ra
04hma
033aam
0.7xxx
0.hxx
0.7ton
046ext
065
62ixen
0N:
0.5
03
017
Talwanoths hedd the wood ralctadd
Table 8: Performance of other algorithms on the NetSim dataset.
Metric
DATASET
cLSTTM
LiNGAM
SRU
CUTS
Siml
0.64aa02
0.41
0.66aaas / 0,43
0.62#a13 / 0,39
0.74
0.77
Sim2
0.67aam1
/ 0.28
0.69 / 0.30aam1
0.66san / 0,26
0.91=nas
0.84
Sim3
0,70ga12
0.24
0,73 / 0.27
0.69 / 0.22
0.962nm
0.86
Sim8
0,61 0.39
0.61aam1 / 0,36
0.61 / 0.38
0.65=nJo
0.70
Sim10
0.64g016
0.42.a0.5
0.662a15
0.42a0.0
0.63am1 / 0.42a015
0.74gum
0.77
AUPRC & AUROC
Sim11
0.66
0.24
0.68
0.25
0.65 / 0.23
0.77=am 0.63
Sim12
0.66
0.27=o0.1
0.69 / 0.30
0.65 / 0,26
0.78:na:
0.68
Sim13
0.59
0.47=2.13
0,59 / 0.42:0033
0.58 / 0.43
0.74 0,80=001
Sim14
0.
0.11
0.6612.1 / 0.42:10.13
0.62gau5 / 0.39
0.66:nas / 0.7 1:000
Sim15
0,67
0.45
0,70 / 0.48
0.64:am6 / 0.43
0.60=mm 0.72
Sim16
0.59
1.45
0.601a011
0.46.1..0
0.58g917 / 0.45
0.63ammn 0.76
Sim17
0.75
0.37-
0.78
0.42+0.1
0.71 / 0.34
0.89=am:
0.80=ao0
Sim18
0.64=a0.6
0.41
0.67#aus / 0.43
0.62#a.6 / 0.39
0.63
0.69:000
Sim21
0.62
0.40
0.65gaus / 0.42.a015
0.61am13 / 0.38
0.75
0.73
Sim22
0,37
/ 0.34
0.60 / 0.37
0.56gau3 / 0,34
0.77=n1o / 0.73
Sim23
0,67
0.43
0,68aa15 / 0.47
0.61 / 0.41
0.66=mm
0.7 sa0n
Sim24
0.55
0.34
0.57 g.35
0.54ga13 / 0.33
0.63gnm
0.73ga0
Average
0.64
0.37.103
0.662813 0.39
0.62:a14
0.36
0.77=am 0.74
Table 9: Ablation study on NetSim datasets.
Sim1
Sim2
Sim3
Methods
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
DYNOTEARS
0.64=20.12
0.41
0.81
0.33
0.85
0.3
w/o So
0.65
0.68
0.63
0.61
0.71 0.003
0..10
w/o Latent State
0.73=0.02
0.79
0.74-t0.2
0.82=2005
0.69
0.75
DyCAST (No DAG)
0.77=0.02
0.79
0.95
0.85
0.98
0.87
DyCAST
0.79=0.01
0.
0.
0.91
0.75
00
DyCAST-CUTS+
0.81=
0.92
0.80
0.89
0.80++.01
0.85
22

---

Published as a conference paper at ICLR 2025
based methods, we also added the results of DYCAST-CUTS+ for comparison. Specifically, DYNOTEARS, DyCAST w/o Latent states, DyCAST w/o S,, DyCAST without DAG constraint, and DyCAST SimNet and CausalTime.
Table 10: Ablation study on CausalTime datasets.
AQI
Traffic
Medical
Methods
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
DYNOTEARS
0.58
0.65=
0.60=a0.7
0.55
0.65
0.40
w/o So
0.73
0.61
0.65=0001
0.57
0.74:10.01
0.63
w/o Latent State
0.80=0.07
0.65=
0.67=0.02
0.60
0.77-10.06
0.70=o.02
DyCAST (No DAG)
0.85=0003
0.70=
0.68
0.60=
0.74
0.69
DyCAST
0.85
0.82
0.63=0001
0.70
0.81
0.74+008
DyCAST-CUTS+
0.91
0.93
0.65=a001
0.73
0.84
0.77=00s
The results from these experiments provide insights into the contribution of latent states and So to the overall performance of DyCAST. We observe that latent states are crucial in DyCAST, with So being particularly significant. This is because each row of the matrix derived from So fully preserves the influence and dependency information of the variables. Embedding S, enhances the latent states' ability to capture the intra-slice matrix dynamics. Furthermore, on the NetSim and CausalTime datasets, DyCAST achieves comparable results even without enforcing DAG constraints. However, the absence of DAG constraints makes it harder to identify directed causal relationships, leading to higher AUROC but relatively lower AUPRC.
E
CASE STUDY: HUMAN3.6M
In this section, We apply DyCAST to human motion dataset of Human3.6M 3.6 (Ionescu al., 2014). The dataset contains activities by 11 professional actors in 17 scenarios, as well as provides accu- rate 3D joint positions. Although the Human3.6M dataset lacks ground truth for the causal matrix between human joints, human kinematics offers a theoretical foundation for qualitative analysis. Combined with our intuitive understanding of human activities, we can effectively evaluate the plau- sibility of the dynamic causal DAGs inferred by the algorithm.
We select the Smoking scenario which the amplitude changes before and after the characters complete the more obvious actions and run DyCAST for analysis, shown in the right panel of Figure 17. It can be easy to see that, during the smoking action, the key joint system comprising the elbow (No. joint), shoulder (No.11 joint), and head (No. 10 joint) exhibit dynamic behavior distinct from other joints. These dynamic joints are highlighted with larger red dots in the original trajectory shown in the first row of Figure 17. Our experiment focuses on analyzing the causal relationships within this dynamic system.
We apply DyCAST with p- T 50, and embedding dim r=12, and also run DYNOTEARS and CUTS+ for more intuitive comparison. The static method CUTS+ captures constant, inappropriate causal links between the hands and head, even after the smoking action ends, as shown in Figure 17 second row. Similarly, DYNOTEARS, despite identifying the lagged influence of the left elbow and its link to the head, misses the dynamic relationship with the shoulder, as shown Figure 17 third row. In contrast, DyCAST captures the intra-slice relationships among the elbow, shoulder, and head, as well as the inter-slice dynamics of the elbow during the smoking action. When the smoking action ends at the 40th second, the intra-slice relationships disappear, leaving only the elbow's 's inter-slice dynamics. This provides a realistic and accurate dynamic representation of the smoking action at the joint level. Thus, in this system with evident complex dynamics, DyCAST, specifically designed for discovering dynamic intra-slice relationships, proves particularly suitable and performs exceptionally well.
23

---

Published as a conference paper at ICLR 2025
t=1
t=10
t=30
[=40
t=50
Human motion trajectory
Static causal with CUTS+
Static causall with DYNOTEARS
Smoking
Dynamic causal with DyCAST
Figure 17: Visualization of the dynamic intra-slice relationships of the Smoking scenario in Hu- man3.6M. 6. Left three rows: Original human motion trajectories across time, alongside static causal relationships of the joint inferred by CUTS+ and DYNOTEARS, and dynamic causal relationships by DyCAST. Right rows: A complete motion trajectory in the Smoking scene across the 50s.
F DREAM CHALLENGE
Baseline methods. For DREAM-3 datasets (Prill et al., 2010), we compare DyCAST with GC (Granger, 1969), DYNOTEARS (Pamfil et al., 2020), NTS-NOTEAR: (Sun al., 2023), PCMCI (Peters et al., 2013), NGC (Tank et al., 2022), CUTS+ (Cheng et al., 2024a), LCCM (Brouwer al., 2021), eSRU (Khanna & Tan, 2020), TECDI (Li et al., 2023).
In this section, we apply DyCAST to the DREAM-3 network inference challenge, which aims to infer gene regulatory networks from gene expression data. The DREAM-3 dataset consists of independent datasets, each containing 10 time series recordings from 100 genes across 21 time steps. These datasets widely used for evaluating causal discovery methods. As shown in Table 11, DYCAST-CUTS+ outperforms other methods, demonstrating its effectiveness.
Table II: Performance on the DREAM-3 dataset.
Methods
DREAM-3 (d=100)
GC
0.50=
DYNOTEARS NTS-NOTEARS
0.50
0.52
PCMCI
0.55=0.03
NGC
0.56=0.03
CUTS+
0.64
LCCM
0.00
NGM
0.55
eSRU
0.56=0.03
CUTS
0.59=0m0
TEDCI
0.50=0.08
DyCAST
0.55=0.05
DyCAST (Not DAG)
0.57
DyCAST-CUTS+
0.65 0.05
24

---

Published as a conference paper at ICLR 2025
From Table 11, observe that the real-world dataset with d = 100. DyCAST maintains highly competitive performance, significantly outperforming similar methods like DYNOTEARS and NTS- NOTEARS. Given the complexity and nonlinearity of the DREAM-3 dataset, integrating it with the nonlinear CUTS+ further enhances performance, achieving the best results.
G IDENTIFIABILITY OF DYCAST
According to the identifiability result [Peters Bühlmann (2014), Theorem 1]. it shows that when the sample size goes to infinity, there is other DAG structure that generate the same distribu- tion. When observational data is generated through the structural equation model defined in (I. the graph structure W; is identifiable up to Markov equivalence class and the coefficients W can be reconstructed for all i and j.
25


Reviews:
[{"review_id": "1FSXKFsGvL", "reviewer": "Reviewer_4Eyh", "summary": "The authors introduced the DyCAST algorithm to infer time-variant DAGs from nodal time series, considering both intra-slice and inter-slice dependencies among variables. They demonstrated its performance using both synthetic and real-world datasets and compared it with other causal discovery methods.", "questions": "1. The authors used traffic as a motivation and test example; however, traffic networks are not DAGs. Similarly, brain networks also contain many bidirectional edges and feedback cycles. How would the algorithm address situations where the underlying network is a directed graph rather than a DAG?  \n2. How should the value of p (i.e., the order of time lag) be set in the algorithm? Is the ground truth for p assumed to be known for the inference process?  \n3. How much additional data is needed for inference when considering both intra- and inter-slice dependencies? If unlimited data were available, many other factors could be incorporated into causal discovery. Thus, the amount of required data is critical for addressing general inverse problems. The authors should specify how many realizations are necessary to achieve a given performance for a specific problem defined by the number of variables and time steps. Additionally, if p is unknown during the inference process, the algorithm may require even more data (i.e., independent realizations).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors addressed limitations of previous methods by considering both intra-slice and inter-slice dependencies among variables. \n2. They offered clear and plausible theoretical motivation for their DyCAST algorithm. \n3. The authors tested DyCAST's performance and compared it with previous methods.", "weaknesses": "1. The authors tested their DyCAST algorithm on datasets with a limited number of variables, which restricts its applicability in real-world scenarios.\n2. While incorporating inter-slice dependencies is beneficial, the inference requires more data to be effective.\n3. The algorithm relies on multiple independent realizations (runs) of system dynamics (i.e., nodal time series). For instance, in the first synthetic dataset, there are only d=5 variables and T=7 time steps (i.e. 7 snapshots of DAG), yet it requires N=500 independent realizations for inference. This is unrealistic, as many real-world situations may only provide one trial or a single trajectory of each variable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ex3NH1E7Qs", "reviewer": "Reviewer_AYt4", "summary": "This paper introduces DyCAST, a method for determining time-varying causal graphs using constrained neural ODEs. In particular, DyCAST determines the causal graph for a set of variables $\\mathbf{X}_t$ as a linear combination of dynamic \"intra-slice\" causes, $\\mathbf{X}_t \\mathbf{W}_t$, and static \"inter-slice\" causes, $\\sum\\_{k=1}^P \\mathbf{X}\\_{t-k} \\mathbf{A}\\_k$, plus additive noise. The dynamics of $W_t$ are modeled as a constrained neural ODE, which is extended to a latent ODE using the encoder-process-decoder framework. The latent state models an autonomous system, but may be concatenated with the time $t$ in decoding for systems with periodic components. Finally, a sparsity penalty is added, and DyCAST is combined with CUTS(+), a Granger-causal method for time-varying inter-slice graphs. Extensive experiments are performed on some limited synthetic data and several real or simulated data sets, where DyCAST with CUTS(+) is typically the best-performing method in terms of AUROC and AUPRC.", "questions": "- My main question, outlined in weaknesses, is **to what extent can the performance of DyCAST be attributed to the latent state**? For example, in Table 1, the latent state is shown to significantly improve performance; could similar ablation studies be performed in the other experiments?\n- There is a brief mention of running time on page 7 (\"moreover, DyCAST reduces the running time by more than 20% across various variable dimensions\"). This claim is not particularly clear to me: **with respect to what is the running time reduced? At what variable dimension is this noticeable?** Similar claims are made in Appendix A.2.1, but without any specific figures. In general, I think it would improve the manuscript to report the run time of DyCAST with respect to the baselines.\n- **What are implementation details for DyCAST-CUTS?** And how does this affect runtime? How are dynamic inter-slice graphs handled outside of the $p=1$ case?\n- It seems like the most straightforward approach in the current framework for inter-slice dynamics would be another (independent?) neural ODE on $\\mathbf{A}$. Is there a reason why this doesn't work?\n- It seems surprising to me that the $F_1$ score is so insensitive to $r$ -- **how would the authors interpret this? Would they expect very small $r$ to be more detrimental in problems with faster dynamics?**\n- The methods for comparison are different in NetSim and CausalTime datasets; for example, TECDI is included in NetSim but not CausalTime, and vice versa for LCCM. **Is there a reason for the differing benchmark methods in these sections?** Since both of these methods appear to be the closest competitors to DyCAST, it would be great to standardize the benchmarks, and include both TECDI and LCCM in all comparisons.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The issue of time-varying causal structure is under-addressed and practical problem. Using constrained neural ODEs is an elegant solution, and the results are impressive.\n- The paper is well-written and introduces concepts from neural ODEs very well. The included illustrations are helpful, and the figures are instructive.\n- Incorporating time-varying structures can be extremely interpretable in some scenarios. For example, I found Figure 6 very compelling, with causal influences smoothly following the time of day.", "weaknesses": "**On the Combination With Other Methods**\n\nThe combination of DyCAST with CUTS+ is one of the more important contributions in the experiments section of the paper, but is not discussed much. This is especially difficult to me because CUTS+ is a Granger-causal method, and to the best of my knowledge does not actually learn what are referred to as \"inter-slice\" graphs, except in the trivial $p=1$ case.\n\nIt would improve the presentation of the paper to include more details of how DyCAST can be combined with other methods like CUTS+, and also how this changes the computational requirements and potential interpretations. Moreover, as an aside, the paper inconsistently refers to DyCAST-CUTS as using CUTS or CUTS+.\n\n**A Lack of Ablation on Most Datasets**\n\nAs mentioned in the strengths, the performance of DyCAST is impressive, and it is apparent that it is beneficial in dynamic problems. That said, I find the ablation study in Section 4.1 very interesting, and would like to see a similar experiment in the NetSim and CausalTime experiments. In particular, from Table 1, it is clear that the latent state provides at least some benefit, even in static problems, but this isn't explored outside of the simplistic synthetic case. I view this as critical to understanding *why* DyCAST is successful.\n\n**Minor Typos** \n\nThere are several typos throughout the manuscript; ones I noticed include:\n- (Page 2, Line 085) NOTEATS should be NOTEARS.\n- (Page 2, Line 087) DYNOTEATS should be DYNOTEARS.\n- (Page 3, Line 124) I think $X_{t-k}^j$ should be bolded.\n- (Equation 18) In the first summand, there is an $\\rvert$ instead of $\\rVert$.\n- (Page 8, Line 398) NTS-NOTEATS should be NTS-NOTEARS.\n- (Page 8, Line 398) TEDCI should be TECDI.\n- (Page 10, Line 404) \"In future\" should be \"In the future\".\n- (Page 13, Line 657) Erdős–R'enyi should be Erdős–Rényi.\n\n**A Few Other Minor Points**\n\n- (Related Work) The approach of CCM is of course much different than the proposed approach -- nevertheless, latent CCM (Brouwer et al., 2021) seems like a relevant application of neural differential equations to causal learning to be mentioned in related work. \n- (Implementation and Code) While I think the details in Appendix A would be sufficient to reproduce most of the paper's experiments, I hope that the authors consider releasing their code with a finalized version of the paper, as I think it strengthens the impact and reproducibility of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jIjZTQNels", "reviewer": "Reviewer_gcbX", "summary": "This work presents DyCAST, a novel framework leveraging Neural Ordinary Differential Equations (Neural ODEs) to learn dynamic causal structures, showing superior performance compared to existing models across various datasets.", "questions": "1. As someone who is not an expert in Neural ODEs, I find the model presented in Equation (4) somewhat unconvincing. Why not learn $W_t$ based on the data at each time step? Are there theoretical justifications or ablation studies to support this modeling choice? If this matter is cleared up, I will be happy to raise my score.\n\n2. Are there additional examples that demonstrate the necessity of dynamic intra-slice graphs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The algorithm effectively supports dynamic contemporaneous structures.\n\n2. The experimental results are compelling, though there are inconsistencies in the choice of baseline methods across different sections (see weaknesses).", "weaknesses": "1. While DyCAST claims to support dynamic intra-slice graphs, the ground truth causal graphs in the Netsim and CausalTime datasets do not contain such dynamics. Therefore, although I appreciate the results in Figure 6, I am concerned that the claim is not fully validated in the quantitative experiments.\n\n2. Several baseline methods are included in the CausalTime experiments but not in the Netsim experiments. Additionally, the baseline choices differ between the Netsim and Synthetic experiments. Is there a specific reason for this? If not, more baseline comparisons should be included. If time is limited during the rebuttal period, incorporating the most recent works would be acceptable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EU3vkV9how", "reviewer": "Reviewer_e4ie", "summary": "This manuscript investigates dynamic causal structure in time series using Neural ODEs, a topic of broad interest. The authors reformulate the problem of learning causal structures at each time step as finding the trajectory of a Neural ODE on a directed acyclic graph. Finally, simulations were performed on both synthetic and dynamical models to validate the proposed approach.", "questions": "1)\tThe rationale of Eq.10. What would happen if the concatenation was removed? An ablation study addressing this would be helpful.\n2)\tIs this approach applicable to large-scale systems? The results presented in the manuscript only cover 5/50/100 nodes. \n3)\tLine 182, is the correct phrasing \"neural underlying ODE\" or \"underlying Neural ODE\"?\n4)\tNo information about the $\\lambda$ parameter in Eq. 18. \n5)\tWhy do negative weights appear in Figure 3? This seems unreasonable and counterintuitive. Using a BA synthetic network might be more appropriate.\n6)\tHow does the noise strength affect the performance of the proposed method?\n7)\tIt is unclear why time steps are discussed, but time points are not mentioned. The causal structure detection largely depends on factors like the length of the time series considered, particularly the information at each time point. However, these key details are missing from the manuscript, which raises concerns about the accuracy and applicability of the proposed method.\n8)\tMore rigorous numerical experiments are needed to substantiate the claims made. The current experiments are insufficient to demonstrate the robustness of the proposed method.\n9)\tI am confused why the authors mentioned time steps, but not mentioned time points at all. Causal structure detection is related to how much information, e.g., the length of time sequences you considered. All these important information is lacking. \n10)\tI am surprised that the NetSim dataset includes 100 nodes. Can you provide more details about this dataset?\n11)\tFigure 5 requires sublabels, and in part (b), there seems to be an error (0, 30, 80, 00). Please review and correct it.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This paper proposes a method for identifying dynamic causal structure by learning their trajectory using L1 normalization. Both intra-slice and inter-slice structures are learned simultaneously. The method demonstrates higher performance compared to other approaches it is evaluated against.", "weaknesses": "The concept of using activity trajectories to identify causal structure seems intriguing. However, due to a lack of clarity, the novelty of the proposed method remains uncertain. The method is not well explained. There are insufficient numerical experiments to support the proposed approach. Several key details are missing. The writing and certain claims need refinement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)