Generating Hypotheses of Dynamic Causal Graphs in Neuroscience: Leveraging Generative Factor Models of Observed Time Series
Authors: Zachary C. Brown
David Carlson
Venue: ICML 2025 poster (2025)
Keywords: neuroscience
hypothesis generation
generative models
causal discovery
Primary area: applications->neuroscience_cognitive_science
TL;DR: We present a novel algorithm for generating hypotheses of causal relationships in systems featuring dynamic causal interactions (esp. the brain), along with state-of-the-art results over multiple datasets and case studies on real brain recordings.

Abstract: The field of hypothesis generation promises to reduce costs in neuroscience by narrowing the range of interventional studies needed to study various phenomena. Existing machine learning methods can generate scientific hypotheses from complex datasets, but many approaches assume causal relationships are static over time, limiting their applicability to systems with dynamic, state-dependent behavior, such as the brain. While some techniques attempt dynamic causal discovery through factor models, they often restrict relationships to linear patterns or impose other simplifying assumptions. We propose a novel method that models dynamic graphs as a conditionally weighted superposition of static graphs, where each static graph can capture nonlinear relationships. This approach enables the detection of complex, time-varying interactions between variables beyond linear limitations. Our method improves f1-scores of predicted dynamic causal patterns by roughly 22-28% on average over baselines in some of our experiments, with some improvements reaching well over 60%. A case study on real brain data demonstrates our method's ability to uncover relationships linked to specific behavioral states, offering valuable insights into neural dynamics.

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience: Leveraging Generative Factor Models of Observed Time Series
Zachary C. Brown David Carlson
Abstract
regions by using Granger causality on time series recorded from each region (Seth, 2007), which can be extended to many brain regions to capture more complete network rela- tionships. Granger causality describes whether the history of one variable can help predict future information about another variable (Löwe et al., 2022; Assaad al., 2022). Granger causal models are more statistical in nature than causal (Pearl, 2009); however, we seek to generate hypothe- ses about causal relationships to then inform causal infer- ence / assays, hence the use of Granger causal models.
The field of hypothesis generation promises to re- duce costs in neuroscience by narrowing the range of interventional studies needed to study various phenomena. Existing machine learning methods can generate scientific hypotheses from complex datasets, but many approaches assume causal rela- tionships are static over time, limiting their appli- cability to systems with dynamic, state-dependent behavior, such as the brain. While some tech- niques attempt dynamic causal discovery through factor models, they often restrict relationships to linear patterns or impose other simplifying as- sumptions. We propose novel method that mod- cls dynamic graphs conditionally weighted superposition of static graphs, where each static graph can capture nonlinear relationships. This approach enables the detection of complex, time- varying interactions between variables beyond lin- ear limitations. Our method improves fl fl-scores of predicted dynamic causal patterns by roughly 22-28% average over baselines in some of our experiments, with some improvements reaching well over 60%. A case study on real brain data demonstrates our method's is ability to uncover re- lationships linked to specific behavioral states, offering valuable insights into neural dynamics.
Current machine learning (ML) methods do not capture the brain's 's full complexity, which has well-documented nonlin- ear interactions and dynamic causal graphs that vary by state and task. Existing discovery methods tend to assume that either a) (potentially dynamic) linear models are sufficient for modeling causal behavior (Huang et al., 2019; Gallagher et al., 2021) or that b) causal graphs between variables are static and not state-dependent (Zhang et al., 2017b; Sadeghi et al., 2024). These limitations reflect gap existing ML methodology, with implications outside neuroscience for any field seeking to automate hypothesis generation for potentially nonlinear, state-dependent dynamical systems.
Thus, we focus on generating hypotheses about mechanisms governing nonlinear and state-dependent (or "dynamic") behavior in time series systems (hereafter referred to as "temporal systems" or simply "systems" to conserve space). While understandable concerns may be raised about the detectability of such mechanisms in even simple systems (sec our discussion on identifiability in Appendix A.1), prior literature has demonstrated that using automated hypothe- sis generation for these mechanisms can still yield useful insights which aid scientific discovery (Mague et al., 2022). Indeed, we accurately estimate causal graphs in observed systems without making restrictive assumptions as to the state-dependence lineariyynoolinearrty of causal relation- ships. We do this by combining key concepts from both factor-based and nonlinear models of causal time series.
1. Introduction
Causal discovery methods applied to brain dynamics hold great promise for designing targeted interventions in var- ious neurological diseases. A typical approach is to gen- erate hypotheses about causal relationships between brain
Department Electrical and Computer Engineering, Duke University, Durham, NC 15213, USA 'Departments of Civil and Environmental Engineering; Biostatistics and Bioinformatics: and Computer Science, Duke University, Durham, NC 15213, USA. Correspondence to: Zachary Brown braw. duke. David Carlson warvii..arlson&& duke.edu>>.
In summary, we seek to reduce the space of hypotheses that must be tested before a causal relationship successfully identified. As such, we present 1) a novel approach to hy- pothesis generation for dynamic causal graphs using a deep generative factor model, 2) methods for including auxiliary
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
variables such as behavior- as supervised labels to improve hypotheses and utility, 3) neuroscience-inspen inspiredd relational discovery challenges and datasets for improving ML meth- ods, and 4) empirical results comparing leading methods on under-explored hypothesis generation paradigms. We only assume our data consists of regularly sampled time series of scalar-valued nodes (see Section 3. 3. These time series may be noisy, and we make no assumptions regarding the underlying generative processes. In Section 3.5, we assume the availability of "global state" labels for each time step that relate to the dominant generative process.
use them as baselines in this paper. Our contributions are orthogonal to this line of work, as we foeus capturing dynamics of graphs over different points in time rather than estimating static graph. In fact, our factor-based based approach can use these structures as individual graphs whose strength modulates over time to make a composite graph.
There is also prior work on identifying dynamic graphs over time. For example, Fujiwara al. (2023) developed an approach to estimate how the strength of causal relation- ships change over time, but did not capture dynamic graph structures. Other approaches based on factor models or matrix factorizations - capture dynamic graph structures, but are restricted to discovering linear relationships between variables (Fox et al., 2011: 1; Mague et al., 2022: Gallagher et al., 2021; Calhoun et al., 2014). Similarly, the very recent DyCAST method is tested solely on synthetic data with lin- ear relationships (Cheng et al., 2025). Building on this work, we use factor model approaches on top of ANN base units to capture time-evolving graphs with nonlinear relationships. This allows us to more realistically capture key properties of many real-world temporal systems, including brain models. Additionally, our framework allows for the inclusion of aux- iliary labels, similar what has been done previously with systems limited to linear relationships (Talbot et al., 2023).
2. Related Work
Causal discovery may be considered to be a sub-field of hypothesis generation, and is the sub-field most closely related to our aim of forming hypotheses about causal re- lationships. Specifically, our proposed method should be viewed as preparatory analysis technique preceding more complex methods such as Dynamic Causal Models (DCMs) (Friston et al., 2003). This is because we assume we have no access to any interventional inputs required to construct a DCM model (Stephan et al., 2010). In such a setting, it is recommended that Granger causal models (such ours) be used to inform DCM models (Friston et al., 2013).
There is also work using dynamic nonlinear models through time, such as State-Dependent Causal Inference (SDCI) (Balsells- Rooses et al., 2022), Switching Neural Network Trackers (SNNTs) (Ghimire, 2018), and Constraint-basee causal Discovery from Nonstationarr/llettrgggoooo Data (CD-NOD) (Zhang al., 2017a). While SDCI-related con- cepts inform our work, SDCI itself incompatible with our use case because it models interactions between embedded latent variables (which are difficult to tie back to physical en- tities), each with their own state labels (a granularity which behavioral assays in neuroscience often lack). Meanwhile, SNNTs are probabilistic nature instead of being designed for causal discovery and use discrete states instead of su- perposition of factors used in our Section 4.3 experiments and the design of our proposed method. In contrast, the development of CD-NOD was largely motivated by the task of performing causal discovery on neurological (esp. fMRI) data, and thus shares similarities with our work in much of the problem setup and certain assumptions. Still, there are enough differences between our approach and CD-NOD such as focus autoregressive systems and uncon- strained directionality of underlying causal relationships that our direct comparisons with Regime-PCMCI (Saggioro et al., 2020) can be considered sufficient for our purposes.
There are numerous causal discovery methods which avoid Granger causality (Runge, 2020; Gerhardus & Runge, 2020; Zou & Feng, 2009; Yu et al., 2019). These methods do not assume causality from temporal correlation, and thus are better positioned to handle causal relations such as con- temporaneous effects (Runge, 2020) and latent confounders (Gerhardus & Runge, 2020). However, these methods tend to rely on assumptions that rule out hypotheses we would like to explore in neuroscience; for example, many methods assume causal graphs are acyclical (Runge, 2020; Gerhardus & Runge, 2020; Zou& & Feng. 2009; Yu et al., 2019), which recurrent, multi-time scale systems like the brain may fun- damentally violate- e.g., through cross-frequency coupling (Dzirasa et al., 2010; Allen et al., 201 1). Thus, our approach differs from these methods and we instead report empiri- cal comparisons between our method and Regime-PCMM PCMMI (Saggioro et al., 2020) in Section 4.2.
Vector Autoregressive Models (VARs) are often used to estimate Granger causality, but are limited to linear relation- ships (Shojaie & Fox, 2022). Recent research has used arti- ficial neural networks (ANNs) to perform nonlinear causal discovery in temporal systems (Tank et al., 2021), including tasks from finding predictive graphical models (Löwe et al., 2022) to estimating a subject's 's emotional state via recorded electroencephalogram (EEG) signals (Song et al., 2018). These frameworks tend focus on estimating static causal graphs (Tank et al., 2021; Song al., 2018; Bussmann et al., 2021; Pamfil et al., 2020; D' A'Auntto et al., 2022) and we
Finally, prior work explored using low-dimensional latent spaces to model dynamics in neural activity (Linderman et al., 2017; Keshtkaran et al., 2022). In contrast, we largely ignore our models' latent features beyond applying con-
2

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
straints because our focus is on identifying candidates for causal relationships in recorded data.
Table Common symbols in this paper.
SYMBOL
DESCRIPTION
3. Methods
PREDICTION, FORECAST, AND/OR ESTIMATE
3.1. Problem Statement
"
'BROADCAST MULTIPLICATION (SEC. 3.2) (EMPH.) SCALAR-VECTOR MULTIPLYING
Suppose we must hypothesize a causal model for a dataset D N E N multi-dimensional (i.e.. 'multi-channel") time W -- denote
((..))
AVERAGE VALUE OF (..)
'MSE' "COSSIM'
THE MEAN SQUARED ERROR PENALTY THE COSINE SIMILARITY MEASURE
series recordings X,, Let D (X) the complete dataset of recordings. Each recording Xn E X contains the observed history of n. system variables at regular intervals with T E N temporal measurements. Let Tin E Ngr be the maximal number of time steps over which we estimate causal relationships. For simplicity, assume observations are of scalar features such that Xn,tit+7.. E Rm A 1 S T and n S.N. N.
formally represented during evaluation. We turn to graph theory to express each factor's estimate of the strength of Granger causal relationships ces mapping past variable (i.e., 'node") states/values to the present. The flexibility of graphs lets us also consider an abstract adjacency matrix summarizing the effect of (t() on variables in the true system. Formally, we summarize our models' estimated Granger causal relationships in a series of adjacency matrices A E I X where each Os.j. E Á is the estimated weight of the Granger causal relationship
series of adjacency matri-
We assume our data is noisy realization from time- dependent system (()). We must lear model that a) reasonably approximates & at any point in time and b) esti- mates causal influences between system variables in Φ. We approximate & by learning a factor model f& over the set of observed variables, where the expression of each factor is dynamic over time. Indeed, Sections 4.1, 4.2, and 4.3 we implement as a factor model itself (see Section 3.2 for formalization). The functions comprising each factor can take on many forms including hypothesis generation func- tions. Additionally, the linear combinations in factor models enable us to track statistics such as presence of a factor over time or in relationship to auxiliary variables (c.g., behavior), making these models useful for scientific research.
where the state of node V/j E V from time steps in the past 'causally' effects current node E V. To make quan- titative comparisons, we standardized the representation(s) of haselines' and our models' estimated causal graph(s) by summing over time-lagged features (see Appendix B.2); hence, we define the lag-summed view of Á as
Á =
Σ
A
(1)
Importantly, À and À are summary statistics computed separately from the forecasting operation, and do not imply any linearization assumption in the forecasts themselves.
3.2. General Notation
Table 1 presents important symbols in this work. In particu- lar, we use @f to express 'broadcast multiplication" between vectors and 3-axis tensors, in which we take the Hadamard product (Kolda & Bader, 2009) between vector elements and corresponding
3.3. Base (Single-Factor) Model
Consider the case where our factor model is a single factor function Saz We can use any algorithm for Sa, so long as it learns relevant Granger causal information by forecasting time series. In this paper we implement fè, as cMLP (Tank et al., 2021) due to its versatile design. Formally, the forward computation is given as
tensor slices before sum- ming over all product results; see Appendix A.3 for details. Using this notation, our experiments treat & as
K"
(tt) ((t) F()())))))))))) = Σ ((t))Fa(t)
fØr (Xm,itt+m)) X xn,++(7++1--
(2)
where K* is the number of true factors F& E Fctiing on all system variables V at each time step 1 -i.ee, Fs: : RR RV according to the system state g(t)= IR. K"
Regarding the loss function of this single-factor model, we assume no direct knowledge of the target system's is under- lying causal graph, since we are performing hypothesis generation. Instead, we estimate causal connections in an unsupervised fashion. Hence, our objective function is es- sentially a sum over regularization terms guiding the model towards solutions with desirable qualities, like sparsity.
Our model consists of functions for for each factor, each mapping a record of past variable states to present vari- able states. By predicting present variable states from time- lagged states of (other) variables, each fo, learns Granger causal relationships between variables (Seth, 2007).
Formally, suppose we are given a recorded signal x, e X, the model's corresponding forecast X, E X, and an
We now discuss how estimates of these relationships can be
3

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
estimated adjacency matrix Á & formed via a subset of the model's parameters Φ. Our implementation of So learns A in the first layer of weights, which together take on the shape (ne ne X Tia) and can be interpreted as series of time-lagged adjacency matrices (Tank et al., 2021); details in Appendix B.I. Hence, we include a lag-dependent L1-norm sparsity penalty on the time-lagged slices of A (favoring the discovery of temporally- -ooccl interactions in cases of cyclical or chained causal relationships in the factor) along with a MSE loss over forecasts in our base objective
where we have the ny factor scores (1++))+ E E A being mroadcast-multiplied against the ni factor outputs for (Xn,ti++T.)) E R We refer to the resulting model as RElational Discovery via ConditionaLly Interacting Forecasting Factors (REDCLIFF) model (see Figure 1).
The objective function for training REDCLIFF models has two main components. Modifying Equation 3. the first component C pertains to the generative factors fe, and retains the MSE forecasting penalty, but now includes all 2 factors in the L1-norm penalty. We also include a cosine similarity penalty to induce our prior that each factor should serve different purposes by encouraging factors to dissociate (details in Appendices B.3 and C.6). Specifically,
Thn
L) = Σ
log(t= + 1)IIA...el) MSF( X)
(3)
where 17, E R are chosen hyperparameters. Again, the L1 regularization term here serves two purposes for the factor's causal model: I) to encourage sparsity/parsimony, and 2) to emphasize reliance on more recent time lags via the log term. We use the Adam optimization algorithm to train our model to minimize the above loss function. We also include weight decay (i.e., L2-norm regularization) on all parameters as done in prior work (Tank et al., 2021).
Tin
Ly(0, +sMSE X)
log(t 1)]1* A.
(5)
m1g
Σ (++1)
CosSim(PA - A
+p
where I R_.xX is the identity matrix and P Á and Å are the lag-summed views of the adjacency matrices derived from factors p and q (Eq. I). We set 1), w, and P as real- valued hyperparameters. The second loss component Lg enforces sparsity (offset by 1) in the factor scores of go.
3.4. RElational Discovery via ConditionaLly Interacting Forecasting Factors (REDCLIFF)
Now consider general cases where our model has nk 2 factors. We seek to learn a Granger causal model fo satisfy- ing (x( = Σk=1 O (n,,,,+(mm++)).6 (xm, where mk N is the total number of model factors, for is the k--th factor generator, and (n...)+ R is the coefficient for for at time t + (Tmn +).
N
C,(,, = 1 -1 Σ
[lam:ll). n=1
(6)
where 7 E R is another constant hyperparameter. Putting it together gives the full objective function:
To learn each factor coefficient, we introduce a parameter- ized function go (the 'state model') which generates factor weights (or 'scores') conditioned on the system's 's history (and nothing else; see Balsells-Rodas et al. (2022)'s "state- determined" setting). Note go can be seen as classifying each factor's importance in forecasts. Thus, go is funda- mentally different from the generative factors of fo (which perform regression on the time series itself); indeed, we found that go typically required more historical information than the generative factors to effectively learn their tasks (see Section 3.6 and Appendix A.2). Thus, we provide addi- tional historical information of length Tal E NşT to ge so go sees a combined context window of Tin Tal time steps.
(() = L (0, X,X) C.
In review, the state model go is how a REDCLIFF model adjusts weights of relationships between variables. By em- phasizing factors with different relationships, Ge can model changes in the causal direction of relationships, making it the main function for estimating dynamic causal behavior.
3.5. REDCLIFF-Superris (REDCLIFF-S)
Now suppose DD includes a set Y of "global state" (i.e.. "behavioral state" in our motivating neurobehavioral cases) labels for the span of each recording in X; that is, D [, Y). We assume the set Y takes the form Y = (yn e R AxTYN ) > where B E N the number of global state variables and both N and T are before.
We define the forward computation of go according to (x = (Ta.+) where (nuttt(natt +1) E R are conditional factor scores. Thus, our model forecasts 4 evolution as:
&n =
oF fo (o( (4)
SSbtracting the identity induces an implicit assumption that each variable is involved in each factor (via self-connection): this is reasonable for recurrent systems like the brain and in our Synthetic Systems experiments. Future work should explore alternatives.
Σ Cn (Xn
4

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
We put this all together by re-defining de such that
A
9a(g) By (9 (x).n)
9a (aa)) By (a
90(x)
Task 1: Forecast Factor Effect
Task 2: Estimate
Factor Relevance
(8)
L
(1+1)
where 9a R + R and R + R are invertible and now g/. represents the original function go prior to our inclusion of go and 9y. The predicted behavioral label tb indicates the presence of behavior yn,b b if In,b cb where C₁ is some threshold. We make change to the REDCLIFF forward computation (Equation 4) other than to note the output of go now includes y, which must be ignored in the broadcast multiplication step so just a is used.
V2
www
V3
B
Task
In
Task 2
fØ
To complete the definition of REDCLIFF-S model, we add supervised component L in Eq. 7 to obtain
A
90
((0,0,7) Ly(d, D) + (0 D) AMSE(Y, Y) (9)
where À is real-valued hyperparameter, We hypothesize this new MSE term constrains the Granger causal graph(s) of the relevant factor(s) to correspond to Y (as would any other supervised loss term).
O
supervised y
factor 1 of nk
3.6. Training Strategies and Details
unsupervised
We train REDCLIFF(-S) using the Adam optimizer with weight decay. To effectively learn each model, we first pre-train the state model go using the terms Lg and AMSE(Y, Y) from Equation 9 (setting À = in the unsu- pervised case). Then, we freeze the state model and train the generative factors in fg using Equation 5 (we call this "acelimation"). Following these steps, we employ the full training objective (Equation 9) to jointly update all parameters.
net
net
net
...
nc
Out Twoe
One challenge for hypothesis generation is that you can- not perform model selection explicitly using holdout graph discovery performance on real data as the true graph is un- known. Instead, we stop training once the number of maxi- mal iterations reached when the following criterion minimized on the validation set D. (X,, Y.):
Figure 1. Illustration of the REDCLIFF(-S) algorithm. A) The two primary subtasks performed by a REDCLIFF(SS) model. B) An illustration of ni factor REDCLIFF-S model's forward pass.
We seek to estimate causal patterns linked specific behav- iors in signals. Thus, follow prior literature and assign behaviors to the first B elements of a (Mague et al., 2022). To enforce better separation between the factors for and state model ga and because our behavioral labels may be noisy, we define invertible functions 9c. and such that = yy (g (o..)) Invertibility necessary for fully pre- serving information between these operations, allowing us deduce the relationship between label predictions and factor weights; discussion in Appendix A.2 for further details. When the behavioral label is known quantita- tive variable, we can generate Vn via supervised regression. Alternatively, if y, categorical, we can predict a behavior as 'present' in the signal when the value of one of these behavioral elements crosses a threshold c E TR.
Co D.) wMSE(X,, X.) AMSE Y., Y.) (10)
P A
+p
CosSim
max(P A)
max(1A
0=1==(++1)
For more details on Cs see Appendix B.3.
In terms of computational complexity, the REDCLIFF(SS) implementations we tested in this work scale (in training and testing) as the product of the number of factors ny and the number of forward passes nsim made by each factor
We experimented with having 7ai E N>1 forward passes before updating REDCLIFF-S parameters in some grid searches. In practice, nsim performed best, but include the general form our complexity analyses for rigor.
5

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
plus the computational complexity of the state model. More formally, let Co. represent the maximal (temporal spatial, depending context) complexity of any factor in & and Co be the complexity of the state model; then the temporal complexity of our implementation is ( Visim Co. + Co)
A
Synthetic System Name (n-ne--nd)
6-2-2
and the spatial complexity is the same. Learning can also be parallelized over factors for scalable implementations.
6-4-2
Additional theoretical analyses in Appendix A.2 support our findings that go required more context than each fex.
12-11:5 12-11-2
REDCLIFF-S (cMLP)
4. Experiments Simulated Datasets
cMLP
cLSTM
We now report empirical comparisons of REDCLIFF-S against other methods with additional results given Ap- pendix All algorithms were trained using a local cluster that featured various GPU devices, including A6000 GPUs.
DGCNN
OCSFA-NMF
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Avg. Optimal F1-Score + Std. Err. the Mean
We focus much of our analysis fl-score performance, partly because our neuroscientific application implicitly as- sumes all parts of the brain are causally related to some
B
Synthetic System Name (Ne-ee fie)
12-11-5 12-11-2 6-4-2 6-2-2
degree and the difficulty determining which connections are the most relevant key moments; the f1-score's em- phasis the presence of positive-label relationships is thus well suited to setting. As is common practice, report 'optimal' fl-scores in which we compute each evaluated al- gorithm's 's -c-re using classification threshold (i.e., the
REDCLIFF-S (cMLP)
value at which logits are labeled negative or positive) which
cMLP
CLSTM
yields the highest possible fl-score for that algorithm on the
DGCNN
task. do use the "Adjustment Identification Distance" (or
dCSFA-NMF
AID) family of causal distance metrics (Henckel et al.) in comparisons against several supervised baselines (Table
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35 0.40
Avg. Difference In Optimal F1-Score + Std. Err. of the Mean
2 and Appendix C.3), since all algorithms in this compari- son had access to "global state' information related to causal edge relevancy, possibly making it harder for the fl-score to differentiate algorithms. We also report some ROC-AUC scores, as Table and the appendices.
Synthetic Systems results from select systems. A) Av- erage optimal fl-score and standard error of the mean (SEM) between true estimated vaterlvariable causal relationships. B) Average pairwise improvement and SEM between optimal fl- scores obtained by REDCLIFF-S baselines.
Figure 2.
4.1. Synthetic Data Generation and Analyses
We rely on synthetic data to test REDCLIFF-S' ability to es- timate known causal relationships. To create these datasets, we defined each artificial temporal system as set of Vector Regresegeee (VAR) models (Zivot & Wang, 2006), for each factor. Our implementation allows for edge activations between time-lags to be either linear or nonlinear; in this work, we included nonlinear activations between different time-lags. Specifically, we used the ReLU (that ReLU(z) = x1 [max(x,0) and negative ReLU (that is (ReeU(x)) x1 [min(2,0) (r)) nonlinear activations. For each experiment, the number of VAR models used, how many time series were present, and how many lags were in each VAR (see
Appendix D). Here we set the number of lags for each VAR to time steps for visualization purposes, but any in- teger greater than O would suffice. Defining these systems also meant selecting square adjacency matrices mapping values from past to present system states for each VAR model in the system. We included self-connections each adjacency matrix to better visualize their generated time se- ries. The number inter-variable connections between time series was varied across systems, with connections chosen randomly. Our Synthetic Systems were designed approxi- mate the conditions of the TST dataset (Carlson et al., 2023) discussed in Section 5, with increasing fidelity / complexity (the largest systems had 12 nodes, which corresponded to 12 channels in our preprocessed TST dataset).
selected hyperparameters such as
*Code has been made available ctttub..om//arlson- shnrcaalii-s--yypohhhsiizig
-raaphs
Once we defined system, we sampled recordings and their
6

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
labels for the dataset. First, a random state vector was drawn from uniform distribution for each VAR model in the sys- tem. This vector was recurrently fed through its correspond- ing VAR model for number of 'burn-in' steps, after which we recorded the resulting multivariate time series for T time steps; innovations were simulated throughout by multiply- ing draws from uniform and Gaussian distributions. Record- ings obtained from each VAR model were then weighted over time with linearly interpolated weights (randomly se- lected between 0 and 1). These weighted recordings were added together along with a level of Gaussian noise. We augmented labels by marking with a one-hot vector which factor weight was largest at each time step, meaning all of our Synthelic Systems experiments contain label noise.
A
B
C
D
E
F
G
H
K
L
Correct
Missed
Incorrect
We stratified results by marking systems as low (L), moder- ate (M), or high (H) complexity e, according
Figure 3. Visualizing true and REDCLIFF-S Top-10 estimated inter-variable causal relationships of a factor from Synthetic Sys- tem "12-11-5", REDCLIPF-S captures 7 of 11 true relationships, while incorrect/missed pathways share functional similarities; the fact that REDCLIFF-S extracted these relationships with four other system states/factors adding noise in the dataset marks a sea change in modeling capability over baselines. While we could have high- lighted examples with higher optimal fl-scores, we highlight this one due to its comparable complexity to our TST case study.
C(De: De) = --
Ile
(11)
graph
(n.²
nI.)
where Tgraph is the ratio of true inter-variable interactions in each system state the number of possible vnter-variable interactions. In essence, C is inversely related to the sparsity of relationships in the graph. We provide additional details regarding this complexity score in Appendices C and D.
4.2. Hypothesis Generation for Synthetic Systems
As another test, we re-trained REDCLIFF-S models along with Regime-PCMCI (Saggiono et al., 2020) and supervised versions of 'SLARAC', 'QRBS', and 'LASAR' from the tidybench repository (Weichwald al., 2020) 4 "on our '12- 11-2' Synthetic Systems (Fig. 2A) using 10 different ran- dom seeds and compared their ability to detect unweighted causal edges. In Table 2, we report the average median per- formance of each method on several metrics, with additional details and results given in Appendix C.3.
Figure 2 gives results from high-complexity system (6- 2-2) and three moderate system configurations for which we generated five random repeats (i.e., unique systems) to train each algorithm (more results in Appendix C). Each repeat was designed to have 1040 training samples and 240 validation samples for each factor/system state in the system, with each sample recording 100 sequential time steps. We used the cMLP and cLSTM (Tank et al., 2021); DGCNN (Song et al., 2018); and dCSFA (Talbot et al., 2023; Mague et al., 2022) methods as baselines. Hyperparameters for each algorithm were chosen via grid search on a single repeat in our (low-complexity) n. = 12-node, n. = 33-edge Liter-variable, n = 3-factor Synthetic System dataset. Our grid searches selected parameters which minimized stopping criteria (for DYNOTEARS and REDCLIFF-S) or objective functions (for all other baselines), with ties won by the most 'parsimonious' parameters (see Appendices D and E). The same hyperparameters were used across all the experiments reported in this section, with limited adjustments e.g., to the number of REDCLIFF-S factors or the value of loss coefficients- made account for certain characteristics in the target system (see Appendix E). Each algorithm was trained on given system repeat dataset, after which we computed the average optimal fl fl-score and standard error of the mean (SEM) of each algorithm's 's inter- variable causal predictions against the true causal graphs across factors and repeats for each synthetic system.
4.3. D4IC Multi-State Extension of DREAM4
We also validated REDCLIFF-S on an adaptation of the pub- lic DREAM4 dataset (Marbach et al., 2009). We chose to adapt the DREAM4 dataset due to 1) it's's use as a benchmark dataset in prior hypothesis generation and causal discovery research (Pamfil al., 2020) and 2) its relatively small size since some baseline implementations required that data not be batched- - namely, the DYNOTEARS algorithm (Pamfil et al., 2020) and the NAVAR algorithm (Bussmann et al., 2021) with recursive cl CL.ST STM--based (NAVAR-R) and cMLP- based (NAVAR-P) implementations. We measured the aver- age optimal fl-score of each algorithm's causal estimates between variables, with causal relationships defined by fac- tors taken from different folds of the DREAM4 dataset and presented to the algorithms in a single training session as
*We also tried training 'SELVAR' from tidybench, but could not compile it locally.
7

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table Comparing unweighted inter-variable edge detection hy supervised discovery methods the 12-11-2 Synthetic Systems (Fig. note REDCLIFF-S' fl-scores differ from Fig. 2 which included edge weighting. Edge predictions were determined by optimal fl-score thresholds. Values report mean (over repeats)
median (over random seeds and system factors) statistic SEM. 'Upper' and 'Lower' refer to upper- and lower-triangular portions of adjacency matrices, which split to avoid cycles (see Sec. 4.1 & Appendix B.2). We also report the mean comparative placement (from to 5) each method's mean performance (details Appendix C.3).
(FI THRESH.) ROC-AUC
PARENT
ERROR LOWER
STRUCTURAL HAMMING DIST.
MEAN PLACE,
AL.GORITHM
OPTIMAL. FI
UpPer
UPPER
LOWER
LASAR (SUP.)
0.34410.034
0.579=0.018
7.0=1.995
4.310.460
21.2
20.915.905
3.5
QRBS (SUP.)
0.203=0.016
0.536:10.08
4.5=1.631
3.010.949
35.915.049
38.814.258
3.5
SLARAC (SUP.)
0.169=0.016
0.512=0.010
2.7=1.973
0.7
54.5+5.194
56.3+4.447
3.7
BEGIME-PCMCC
0.407=0.027
0.673=0.026
6.1=1.212
4.820.782
9.1-0.219
9.4=1.033
2.3
REDCLIFF-S
0.383=0.038
0.705=0.022
3.9=0.607
3.120.727
10.9#1.513
11.8+1.753
2.0
4.4. Discussion
REDCLIFF-S
The main Strength of REDCLIFF-S is that its hypotheses be easily tested. Equations and 8 are designed for this in several ways. As discussed, factor scores On, can be given explicit behavioral designations. Factor scores can also be constrained (via Sigmoid or similar activation functions) to be nonnegative, and thereby be viewed as how (physically) present corresponding factors are in predicted signals. Thirdly, trained factors foi rely on fixed graphs explicitly showing relationships between variables. Finally, the sum over factors allows to inspect individual contributions of each factor. Overall, REDCLIFF-S shows performance improvements in many scenarios.
(cMLP)
cMLP-v1
CMLP-V2
cLSTM
DGCNN
OCSFA-NMF
DYNOTEARS
NAVAR-R
NAVAR-P
First, Figure and Table 2 together give the core results on our Synthetic Systems Dataset experiments. Figure 2A presents average optimal fl-scores (across all samples and repeats) of REDCLIFF-S and applicable baselines. Surpris- ingly, the single-factor baselines attain fairly competitive fl f1-scores in some cases, seemingly learning an 'average factor graph' that may not be particularly accurate for any single factor but which performs well 'on average'. Even so, Figure 2B we see the average pairwise improvement of REDCLIFT-S' predictions is at least standard error of the above for all baselines all four systems, suggesting REDCLIFF-S yields improvements with statisti- cal significance. These findings are further supported by the results shown in Table 2. Despite the fact that REDCLIFF-S was the only algorithm trained to predict global state infor- mation (all other baselines in Table 2 had direct access to the state label), REDCLIFF-S is uniquely able achieve competitive performance across all metrics used. This be seen in that when we rank the algorithms based on their mean performance for each metric, REDCLIFF-S ranking (or 'ompaaattive placement' to avoid confusion with 'ma- trix rank') ) tends to be lower than the other methods'. Taken together, our findings Section 4.2 suggest REDCLIFF-S
0.28
0.38 Avg. Optimal F1-Score Std. Err. of the Mean
0.32
0.34
0.36
Figure 4. Average optimal fl fl-score between true and estimated inter-variable causal relationships across the D4IC HSNR dataset.
a single dynamic-causal system. Non-stochastic noise was simulated by adding down-weighted recordings from all but one DREAM4 folds a recording from the dominant fold at three different signal-to-noise ratios: low (LSNR)), moderate (MSNR), and high (HSNR). We refer to the resulting dataset as the "DREAM4 In Silico-Combined" (D4IC) dataset.
Hyperparameters for each algorithm were tuned to a single repeat of the D4IC MSNR dataset prior to training on all D4IC noise levels and repeats. Model selection proceeded as in Section 4.2. although we increased the weight on the cosine similarity penalty in selecting our final REDCLIFF-S model which balanced the scale of variation of the cosine similarity penalty with that of the other penalties (details in Appendix B.3). We include two cMLP baseline models (v1 and v2) with slightly different hyperparameters. Results are visualized in Figure 4. in which we see REDCLIFF-S improves over alternative approaches on this dataset.
"Our early experiments included these constraints, but we omit them here due to different models being selected for by our criteria.
8

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table Summarizing effects of REDCLIFF-S ablations mean
1.00
optimal fl-score across various datasets (Appendix C.6).
acb_Core
0.75
ABLATION 0
Acb_Sh
DATASET
=
a = A = O
IL_x
0.50
SYN.SYS. 6-4-2
SYN.SYS, 6-2-2
f
Receiving Channels
L LVT
0.25
SYN.SYS. 12-11-2
4
Md_ Thal
SYN.SYs. 12-11-5
Pri_Co Cx
D4IC HSNR
L
4
R R_VTA
0.00
alLH Hab
-0.25
IDHlp
generates hypotheses about the presence of dynamic causal relationships that prove accurate at state-of-ther -frth----tee-ttttt rates, and this pattern holds true for systems with differing numbers of variables, inter-variable relationships, and factors.
ISNC
0.50
mDHip
mSNC
0075
Acb Core
Acb Sh
IL ll_x
L.VTA
Md. Thal
R R_VTA
aILH Heb
IDHip
ISNC
mDHip
mSNC
Pri
-1.00
Results from our D4IC HSNR experiments are shown in Figure 4, where we see the REDCLIFF-S (cMLP) model achieves the highest average optimal fl-score with statisti- cally significant margin. While the improvements over com- peting methods are somewhat modest in this case, a brief investigation into the factor networks of D4IC HSNR sug- gests that four of the five factors in each repeat would have been rated as 'low complexity" according to Equation 11; importantly, REDCLIFF-S tends to make the least improve- ment baselines in the 'low complexity' regime (see Ap- pendix C.3). In all, our D4IC experimental results generally support those in Section 4.2, verifying that REDCLIFF-S is effective at hypothesizing dynamic causal relationships.
Driving Channels
Figure 5. Difference REDCLIFF-S' mean, normalized inter-
variable causal estimates in the Open Field vs Home Cage
paradigms of our TST Case Study.
Accumbens Core (Acb_Core predict behavior in the Pre- limbic and Infralimbic cortices, and from the Anterior-Left Lateral Habenula ('all.H_HAB') and Medial Dorsal Hip- pocampal (mDHip) channels to predict activity in several regions, including the Thalamus (Md_ Thal). Interestingly, our model's 's hypothesis that the Anterior Lateral Habe- nula ('alLH HAB') important for predicting LFP activ- ity across several brain regions in the more stressful OF paradigm appears to be validated by recent research impli- cating the Lateral Habenula in stress responses of mice (Tan et al., 2024). Previous research also identified a brain-wide "anxiety network' involving the Nucleus Accumbens Shell and the Prelimbic and Infralimbic cortices (Talbot et al., 2023), which our REDCLIFF-S model appears to express, in part, as pathways from the Nucleus Accumbens Core to the Prefrontal Cortex and from there the Infralimbic Cortex and then the Nucleus Accumbens Shell.
We also ran ablation tests to study the effect REDCLIFF- model parameters on performance, which we discuss in detail in Appendix C.6 and summarize in Table 3.
See further discussion in Appendix B.4.
5. Experiments on Real- World Datasets
We tested REDCLIFF-S on real-world data using the pub- licly available Tail Suspension Test (TST) dataset (Carlson al., 2023) which discuss here and the Social Pref- erence (SP) dataset described in Mague et al. (2022) (see Appendix C.5), Where possible, we combined multiple lo- cal field potential (LFP) recordings within single brain region by taking their average. The number factors and other hyperparameters were chosen based on grid searches minimizing the stopping criteria for training. Appendix covers TST preprocessing detail, and Appendix C.4 elaborates TST model selection and additional results.
6. Conclusion
We present the novel REDCLIFT-S hypothesis generation algorithm for temporal systems featuring dynamic causal interactions. By learning to combine nonlinear factors using weights conditioned signal history, REDCLIFF-S attains state-of- fatt--ofthe-are -f-hhearrt performance in estimating multiple causal graphs simultancously in various settings and in a way con- ducive to follow-up scientific inquiry. Experiments on real data suggest that REDCLIFF-S can provide scientific value to computational neuroscience and other fields.
In Figure 5 we show the difference between mean normal- ized causal estimates for the REDCLIFF-S factors assigned to the Open Field (OF) and Home Cage (HC) behavioral paradigms in this case study. The OF factor relies more heavily than the HC factor on information from the Nucleus

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Acknowledgements
Carlson, D., Kumar, S., and Dzirasa, K. Multi-region local field potential recordings during a tail-suspension test. Duke Research Data Repository, 2023.
We would like to thank our colleagues at the Collective for Psychiatric Veuro-Engineenn (CPNE) for supporting this work. In particular, conversations with Noah Lanier, Aaron Fleming, Casey Hanks, Kathryn Walder, and Hannah Soliman proved especially helpful in clarifying and commu- nicating our ideas. We also thank Carles Balsells-Rodas for his contributions during the early phases of this project, par- ticularly his guidance on SDCI and related concepts which informed our work. Conversations with Natalie Brown were also useful for clarifying key ideas. Finally, we thank the reviewers for providing their insight and helpful feedback.
Cheng, Y., Lyu, B., Xing, W., and Zhu, Z. Dycast: learning dynamic causal structure from time series. 2025.
D' Acunto, G., Di Lorenzo, P., and Barbarossa, S. Multiscale causal structure learning. arXiv preprint arXiv:2207.07990 2770297777798, 2022.
Dzirasa, K., Coque, L.., Sidor, M. M., Kumar, S., Dancy, E. A., Takahashi, J. S., McClung, C. A., and Nicolelis, M. A. Lithium ameliorates nucleus accumbens phase- signaling dysfunction in a genetic mouse model of mania. Journal of Neuroscience, (6348))6631411165 2010.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 2139754. Research reported in this publica- tion was also supported by the National Institute Of Mental Health of the National Institutes of Health under Award Number MH125543... The content is solely the responsi- bility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Fox, E., Sudderth, E. B., Jordan, M. I., and Willsky, A. S. Bayesian nonparametric inference of switching dynamic linear models. IEEE Transactions on signal processing,
59(4): 2011.
Friston, K., Moran, R., and Seth, A. K. Analysing connectiv- ity with granger causality and dynamic causal modelling. Current opinion in neurobiology, 23(2): 2013.
Impact Statement
Friston, K. J., Harrison, L., and Penny, W. Dynamic causal modelling. Neuroimage, 19(4):1273-1302 2003.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Fujiwara, D., Koyama, K., Kiritoshi, K., Okawachi, T.
Izumitani, T., and Shimizu, S. Causal discovery for non- stationary non-linear time series data using just-in-time modeling. In Conference on Causal Learning and Rea- soning, pp. 880-894 PMLR, 2023.
References
Allen, E. A., Liu, J., Kiehl, K. A., Gelernter, J., Pearl-
son, G. D., Perrone-Bizzozero, N. I., and Calhoun, V. D. Components of frose-fre modulation in health and disease. Frontiers in systems neuroscience, 5:59, 2011.
Gallagher, N., Dzirasa, K., and Carlson, D. Directed spec- trum measures improve latent network models of neural populations. Advances in neural information processing systems, 34:7421-7433 2021.
Assaad, C. K., Devijver, E., and Gaussier, E. Survey and evaluation of causal discovery methods for time series. Journal of Artificial Intelligence Reseurch, 73:767-819, 2022.
Gerhardus, A. and Runge, J. High-recall causal discovery for autocorrelated time series with latent confounders. Advances in Neural Information Processing Systems, 33: 12615-12625, 2020.
Balsells-Rodas, C., Tu, R., Li, Y., and Kjellstrom, H. Causal discovery from conditionally stationary time series. In UAI 2022 Workshop on Causal Representation Learning, 2022.
Ghimire, M. Switching neural network systems for non- linear tracking. Master's thesis, Wright State University, 2018.
Bussmann, B., Nys, J., and Latré, S. Neural additive vector autoregression models for causal discovery in time series. In Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11-13, 2021, Proceedings 24, pp. 446-460, 460. Springer, 2021.
Henckel, L.,, Würtzen, T., and Weichwald, S. Adjustment identification distance: A gadjid for causal structure learn- ing. In The 40th Conference on Uncertainty in Artificial Intelligence.
Huang, B., Zhang, K., Gong, M., and Glymour, C. Causal discovery and forecasting in nonstationary environments with state-space models. In Chaudhuri, K. and Salakhutdi- nov, R. (eds.), Proceedings of the 36th International Con ference on Machine Learning, volume 97 of Proceedings
Calhoun, V. D., Miller, R., Pearlson, G., and Adah, T. The chronnectome: time-varying connectivity networks as the next frontier in fmri data discovery. Neuron, 84(2): 262,274, 2014.
10

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
of Machine Learning Research, pp. 2901-2910. PMLR, 09-15 Jun 2019. URL https:/ htt.ttts///prrceeeii/e http///ccceed....... mlr. / v97 http///9///hhaggg htt.//gaaaggggggg/ html.
Runge, J. Discovering contemporaneous and lagged causal
relations in autocorrelated nonlinear time series datasets. In Conference Uncertainty in Artificial Intelligence, pp. 1388-1399 Pmlr, 2020.
Keshtkaran, M. R., Sedler, A. R.. Chowdhury, R. H., Tandon, R., Basrai, D., Nguyen, S. L., Sohn, H., Jazayeri, M., Miller, L. E., and Pandarinath, C. A large-scale neural network training framework for generalized estimation of single- trial population dynamics. Nature Methods, 19 (12):1572-1577, 2022.
Sadeghi, A., Gopal, A., and Fesanghary, M. Causal discov- ery in financial markets: A framework for nonstationary time-series data, 2024.
Saggioro, E., de Wiljes, J., Kretschmer, M., and Runge, J. Reconstructing regime-dependent causal relationships from observational time series. Chaos: An Interdisci- plinary Journal of Nonlinear Science, 30(11), 2020.
Kolda, T. G. and Bader, B. W. Tensor decompositions and applications. SIAM review, 51(3):555-00 2009.
Koutra, D. Exploring and making sense of large graphs. Computer Science Department, Carnegie Mellon Univer- sity, 2015.
Seth, A. Granger causality. Scholarpedia, 2(7): 2007.
Shojaie, A. and Fox, E. B. Granger causality: review and recent advances. Annual Review of Statisties and Its Application, 911::299-319 2022
Kugiumtzis, D. State space reconstruction parameters in the analysis of chaotic time series the role of the time window length. Physica D: Nonlinear Phenomena, 95 (1):13-28, 1996.
Song, T., Zheng, W., Song, P., and Cui, Z. Eeg emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing. 11 (3):532-541, 2018
Linderman, S., Johnson, M., Miller, A., Adams, R., Blei, D., and Paninski, L. Bayesian learning and inference in current switching linear dynamical systems. In Artificial intelligence and statistics, pp. 914-922. PMLR, 2017.
Stephan, K. E., Penny, W. D., Moran, R. J., den Ouden, H. E., Daunizeau, J., and Friston, J. Ten simple rules for dynamic causal modeling. Neuroimage, 49(4):1101- 3102, 2010
Löwe, S., Madras, D., Zemel, R., and Welling, M. Amor- tized causal discovery: Learning to infer causal graphs from time-series data. In Conference on Causal Learning and Reasoning, pp. 509-525. PMLR, 2022.
Talbot, A., Dunson, D., Dzirasa, K., and Carlson, D. Estimating a brain network predictive of stress and genotype with supervised autoencoders. Journal of the Royal Statistical Society Series C: Applied Statis- tics, 72(4):912-936, 05 2023. ISSN 0035-9254. doi: 10. 10.1099//rsssc///aa0033. URL https:://oi..o. 10. (tt.:////rsss///////
Maclaren, O. J. and Nicholson, R. What can be estimated? identifiability, estimability, causal inference and ill pospe inverse problems. arXiv preprint arXiv: /904::900.000 904.02826, 2019.
Mague, D., Talbot, A., Blount, C., Walder-Chrrstteseen, K. K., Duffney, L. J., Adamson, E., Bey, A. L., Ndubuizu, N., Thomas, G. E., Hughes, D. N., al. Brain-wide electrical dynamics encode individual appetitive social behavior. Newron, 10(10))1728 170::-1221----1 1741, 2022.
Tan, W., Ikoma, Y., Takahashi, Y., Konno, A., Hirai, H., Hirase, H., and Matsui, K. Anxiety control by astrocytes in the lateral habenula. Neuroscience Research, 2024.
Tank, A., Covert, I., Foti, N., Shojaie, A., and Fox, E. B. Neural granger causality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(8):4267-4279, 2021.
Marbach, D., Schaffter, T., Mattiussi, C., and Floreano, D. Generating realistic in silico gene networks for perfor- mance assessment of reverse enginecring methods. Jour- nal of computational biology, 16(2):229-233 239, 2009.
Weichwald, S., Jakobsen, M. E., Mogensen, P. B., Pe- tersen, L.. Thams, N., and Varando. G. Causal structure learning from time series: Large regression coefficients may predict causal links better in practice than small p- values. volume 123 of Proceedings of the NeurIPS 2019 Competition and Demonstration Track, Proceedings of Machine Learning Research, pp. 27-36. PMLR, 2020. URL http: http///eeeennns...e/ee mlr wwwppprrssssss we cewa d220a. html.
Pamfil, R., Sriwattanaworachai,, N., Desai, S., Pilgerstor- fer, P., Georgatzis, K., Beaumont, P., and Aragam, B. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics, pp. 1595-1605. PMLR, 2020.
Pearl, Causality. Cambridge university press, 2009.

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Yu, Y., Chen, J., Gao, T., and Yu, M. Dag-gnn: Dag structure learning with graph neural networks. In International conference machine learning, pp. 7154-7163. PMLR, 2019.
Zhang, K., Huang, B., Zhang, Glymour, C., and Schölkopf, B. Causal discovery from nonstation- ary/heterogeneous data: Skeleton estimation and orienta- tion determination. In IJCAI: Proceedings of the Confer- ence, volume 2017, pp. 1347, 2017a.
Zhang. K., Huang, B., Zhang. Glymour, C., and Schölkopf, B. Causal discovery from nonstation- ary/heterogeneous data: Skeleton estimation and orienta- tion determination. In IJCAI: Proceedings of the Confer- ence, volume 2017, pp. 1347. NIH Public Access, 2017b.
Zivot. E. and Wang. J. Vector autoregressive models for multivariate time series. Modeling financial time series with S-PLUSØ, pp. 385-429, 2006.
Zou, C. and Feng, Granger causality vs. dynamic bayesian network inference: a comparative study. BMC bioinfor- matics, 10: 10:1-17, 2009.
12

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
A. Supporting Theory and Proofs
A.1. Identifiability Analysis of Systems Featuring Nonlinear, Dynamic Causal Graphs
In this section, we provide theoretical arguments as to why the systems at the center of our work dynamical systems featuring nonlinear, dynamic causal graphs - are generally not identifiable. At the same time, we point out that prior work has demonstrated that using causal discovery methods can still benefit scientific research in real-world settings where nonlinear, dynamic causal graphs are the norm rather than the exception (Mague al., 2022). Throughout our discussion, we follow the common definition of identifiability, in which if the likelihood of two models is the same, then their parameterizations must also be the same (Maclaren & Nicholson, 2019).
There are three cases related to causal identifiability in our setting: the case where auxiliary variables are not available (the unsupervised case), that where some - but not all auxiliary variables are available (the partially supervised case), and the case where an auxiliary variable can be assigned to every system state (the fully supervised case). For the unsupervised and partially supervised cases, the lack of identifiability readily apparent in that we can swap various factors and factor weights tied to unsupervised states within our REDCLIFF(-S) architecture and arrive at functionally identical models.
The fully supervised case is also not identifiable, as we illustrate via counterexample. Consider the system with n = 2 states (0 and 1) and ne = 2 nodes (A and B3 where both nodes are self-connected. Assume all innovations in A and B follow a symmetric random distribution (e.g., the uniform or normal distributions) in both states 0 and 1 at each time step. Finally, assume the only difference between states 0 and 1 is that in state we have that A drives B (i.e., A + B) via the ReLU (that is (z)U(z) = x1 (max((x.0) (z)) function at lag
2, whereas in state we have A + B is the negative ReLU (that is (R) x1 [min((,00] (z)) function at lag t = 2. In this setting, the functional form of the A + B edge cannot be uniquely identified for either state 0 or 1 when given infinite number of samples. This is because for any given sequence of innovations S = ([Ac, B₁] > we have that 3,5 ([-A,, B,)) which satisfies
ReLU(A,) + B, = nReLUU - Bt
for any t E Ngr. In other words, for every instance in which it appears that A is influencing B via the ReLU update rule, we will observe another instance in which A is possibly influencing B via the nReL.U update with the exact same effect. Thus, the ReLU and nReLU update rules are equally likely in both state O and in state 1, rendering the two states unidentifiable by definition.
In summary, we have demonstrated that systems featuring nonlinear, dynamic causal graphs are not identifiable unless one makes sweeping assumptions, which preclude even fairly simple systems (such as our 2-node system with ReLU-like nonlinearities) from being considered. Despite this, previous work has shown that causal discovery methods still benefit scientific research in real-world settings where nonlinear, dynamic causal graphs are common (Mague et al., 2022).
A.2. Noisy Classification vs Generative Task Information Requirements
We now provide theoretical argument to why it may be necessary to provide additional system history to the REDCLIFF S) state model ge, and even to drastically limit its modeling capabilities (c.g. via the invertible 9a and gy functions of Section 3.5). Our argument centers on the implications of the parameters Tin and Ta (Eq. 4), and is thus related to long- oong-standing prior work investigating the effects of similar parameters on state-space models of dynamical systems (Kugiumtzis, 1996). Here, we argue that the task of predicting the state of a signal requires more historical time steps than is required to generate the same time series in the presence of nontrivial noise, at least for some dynamical systems. We now present a summary of our theoretical argument, followed by proof sketch, and finally a discussion of the theoretical implications.
1. MOTIVATING EXAMPLE (LEMMA 1 PRELIMINARIES)
As motivating example, we explore system for which classifier needs more historical information opposed to a generative model. In our example, there are at least two system states governed by different causal relationships operating on similar time scales, for which a classifier cannot both a) distinguish between these two states and b) be unbiased.
Let & be a system consisting of D E N interacting variables according to N E N states, in which the dynamics of are governed by M E N distinct causal graphs between variables. Define T as the maximal number of historical time steps required to generate the behavior of & across all states Assume 3 two states of & (call them states and j where i A j) for
"Ffffectively, T is the number of time steps required to reduce + to a Markov Chain.
13

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
which the generative functions can be expressed [((x)) A,x and (x( Axxe-r where both A, RDxD and A DDD are invertible with A; # Aj-
Finally, let X; R DxT be recorded signal of T E N observations made while & was in state i S N. We say that X; exhibits nontrivial noise if =bt another recording X, E RXT such that 0 < S NN with j o for which the initial conditions are independently, identically distributed with those of x,, that is: X i..d. X 7
A.2.2. LEMMA 1: STATEMENT AND PROOF SKETCH
Lemma To learn an accurate and unbiased state model go mapping recorded signals with nontrivial noise to states of (i.e., go Dxx Ngw), must be true that T' >
Proof Sketch Lemma I follows from Bayes' Theorem, which distribution featuring nontrivial noise cannot be readily distinguished by an unbiased model go.
show that two windows of length sampled from
Suppose the recording x E IR DxT with T < T is randomly sampled from & according to a uniform distribution over all N states. By way of contradiction, assume we have an unbiased classifier (i.c., the REDCLIFF-S state model go) for which X) # p(f,(X)
Notice that since both A; and Aj are invertible, we have both that X - (X.T) A ~ p(X(ft) well f()
as that X. T-r fj (X A X
Hence, we have
p(X p(X())).
(12)
Applying Bayes' Theorem, we have that
plSi, X)x() = -(X = -((x)) = X)(x
p(i.X) p(f))
plss X)
(13)
p(fi)
p(f))
p(f,)
Now recall that was randomly sampled according to uniform distribution over states. Therefore, p(fi, X) p(ff, which yields
-())) p(fi=
(14)
p(f)
p(J))
However, we can also apply Bayes' Theorem to the output of our classifier p(fi(X)= # p(fj(X). Doing this yields
p(fi. X)p(f)) x() # p(f)(x)
X(f,,X)p(ff)
(15)
x(X)
p(X)
As before, we use the fact that p(Ji, X) p(Jj, X) to obtain
plfi. X)p(ft) + plf,,X)((f,) = p(fi) + p(f))
(16)
But the finding that p(f)) A p(fz) contradicts our assumption that our classifier was unbiased, for which p(f:) p(fs).
A.2.3. DISCUSSION
Lemma has important implications for training REDCLIFF- models. Effectively, once the state model ge in a REDCLIFF model has enough capacity to accurately distinguish between states, Lemma 1 indicates that go may also have access (by necessity) to at least as much information as that required to forecast the evolution of (assuming the proper noise profile) in one or more states of the system, least for restricted (yet simple and arguably common) classes of temporal systems. These findings indicate that it may be necessary restrict the modeling capabilities of go to ensure the proper delineation of tasks within REDCLIFF-S S model.
This is relevant hypothesis generation, where we often start by assuming signals are generated from similarly distributed initial conditions, and we attempt to identify where this assumption fails.
14

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
A.3. 'Broadcast Multiplication': Formal Definition
Here we formally define broadcast multiplication, the linear operation used to combine outputs of our generative factors.
Definition: Let N-dimensional vectors v, v' R' be given along with 3-axis tensors Z, Z' I where both P.Q E N. Then "broadcast multiplication' denoted by of, is an operation such that
N
V Z
Vn
(17)
n=
B. Methodological Details
B.1. Factor-level Implementation - A Brief Review of cMLPs (Tank et al., 2021)
As mentioned in the main paper, we implement each factor of the REDCLIFF-S models in this work as a "component-wise Multi-Layer Perceptron" " or "cMLP" (Tank et al., 2021); here we briefly summarize the cMLP architecture and refer the reader to the original paper for more details. Essentially, the cMLP architecture is a 1D convolutional neural network that treats the first layer of weights as an information filter mapping prior variable values of dynamic system to the next predicted value of a single variable. Since the architecture allows for set of historical system observations to be input, the first layer of cMLP comes represent set of estimated, time-lagged Granger causal connections. One can view the magnitude of each weight in the first layer of parameters as the strength of a Granger causal connection from one of the input variables to the predicted variable. By instantiating a different cMI.P for each variable in a dynamical system, one can learn a sequence of time-lagged, full adjacency matrices mapping prior variable values to future variable values. We found this architecture to be flexible for our purposes and proved at least as effective in predictive tasks as other architectures we tested in preliminary experiments (namely the cLSTM (Tank et al., 2021) and DGCNN (Song et al., 2018) architectures).
B.2. Standardizing Causal Graph Estimates
To facilitate fair comparison between both baseline and our SEOOCIIFF-based algorithms, we needed to standardize the form in which each algorithm's 's estimated Granger causal graphs were presented. This was nontrivial, since some of the algorithms incorporated lag-dimension into their causal graph estimates (esp. those based the cMLP architecture from Tank et al. (2021)), while others (esp. DYNOTEARS from Pamfil et al. (2020)) treated each feature measured from a channel- - be it historical scalar value or power spectral feature as an individual node in the estimated graph, while yet others (esp. dCSFA-NMF in Gallagher et al. (2021) and REDCLIFFF-bsed models) learned multiple causal graph factors.
The standard we chose to adopt was that each estimated (and true) Granger causal graph would need to be presented as a simple adjacency matrix; that is, if there were n. variables in the recorded system, then the true/estimated graph(s) from each system/model were compressed to lie within 18 XMe In practice, this meant that graphs involving lags and/or node features had their representations compressed by summing over the lag and/or feature values corresponding to each node.
If an algorithm could only estimate a single graph for a system governed by m. >> 1 graphs, the estimated graph was copied Dy times and compared to each of the m true causal graphs, with results averaged across each comparison.
B.3. REDCLIFF-S Stopping Criteria and Model Selection
Here we describe how we arrived at the stopping criteria for REDCLIFF-S model training and how it was used in model selection. Our stopping criteria are summarized in Equation 10, which has three main components: a forecasting term, state classification term, and an estimated Granger causal graph similarity term. Intuitively, the two MSE terms (forecasting and state classification) are in the stopping criteria to ensure that our performance on the two main subtasks of our REDCLIFF-S models (predicting factor effect and relevance; see Figure 1A) is being optimized.
The third cosine similarity- -based term (estimated Granger causal graph similarity) is included for several reasons. For one thing, the very premise of REDCLIFF-S is that the target system being modeled has different causal relationships over time; hence, our third term implicitly assumes that factors should be different by minimizing this estimated Granger causal graph similarity term. A potentially useful side effect of minimizing similarity between factors is that it reduces the size and complexity of our REDCLIFF-S model.
In addition to these intuitive motivations for minimizing cosine similarity between factors, we also found this cosine
15

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
similarity- based term was correlated with the value of various graph similarity metrics comparing factor estimates with their ground truth representations over the course of training (for example, see Supplemental Table 4). Other interesting phenomena include our finding that troughs in our mean cosine similarity curve coincided with peaks in the DeltaConO similarity (a measure of functional similarity between two graphs (Koutra, 2015)) between estimated and true causal graphs, as well as the onset of instabilities in the ROC-AUC curves when comparing estimates to ground truth. These observations (along with similar ones from many other experiments) demonstrate why this penalty for the sum of cosine similarity between factor estimates served as reasonable proxy to determine when to stop training.
Table 4. Comparing correlation between stopping criteria elements and the ROC-AUC between final REDCLIFF-S factor predictions of causal relationships and the true factor graphs in our Synthetic Systems grid search over REDCLIFF-S model parameters. Values rounded three significant figures, with the strongest negative correlation bolded.
LOSS/PENALTY TERMS INCLUDED IN CRITERIA
FORECASTING STATE PREDICTION GRAPH SIMILARITY
PEARSON'S ' R VALUE
-0,368
0.529
-0,332
-0.176
-0.654
-0,201
-0.674
One open question is how to balance the significant differences in variation between the forecasting and state prediction penalties on the one hand and the significantly less-variable cosine similarity term the other. In selecting the final model for our D4IC experiment, we found that weighting the cosine similarity penalty by 60p (instead of just p) in our selection criteria balanced the three variation scales to all roughly lie within the same order of magnitude. More sophisticated balancing schemes would likely improve results.
B.4. Discussion: Limitations and Future Work
Most limitations of REDCLIFF-S arise as trade-off's for scientific utility. For example, the set of REDCLIFF-S factors has many hyperparameters that can be costly to tune; this cost restricted us to tuning on a single repeat of our 12-node, 33-edge, 3-factor system in Section 4.2, and may have limited REDCLIFF-S measured performance.
We also wish to draw attention to a general lack of methods for measuring functional similarity in nonlinear graphs. As discussed in relation Equation 5, we found it helpful to approximate the DeltaConO metric (which measures functional/comppsiiional similarity between lagged linear graphs) (Koutra, 2015) using cosine similarity penalty; however, to the best of our knowledge, the nonlinear version of DeltaConO has not been developed. This poses major obstacle to evaluating the performance of models on our Synthetic Systems task, and is a research question we hope our work motivates.
B.5. Comparing State Prediction Error with 1-Factor Models
To test the ability of the REDCLIFF-S algorithm to dynamically adjust factor weightings at each time step in preliminary experiments, we compared the mean squared error of each state label prediction ý made by REDCLIFF-S models against a 'Naive 1-Factor Model' state prediction. For this, we adopted the convention that a state label prediction made by a 1-factor model (such as the cMIP and cL.STM models by (Tank et al., 2021) or the DGCNN method by (Song et al., 2018)) would be equivalent to predicting that all system factors (represented by the model's single factor) would always be present; hence, ý 1 for any input X,.. The difference in MSE between these naïve predictions and the true label y was taken with the MSE obtained by the predictions of the REDCLIFF-S models across many samples and then averaged.
C. Additional Experimental Results
C.1. Pairwise Optimal F1-Score Improvement of REDCLIFF-S on Synthetic Systems' 6-2-2 Data
As we alluded to in the abstract of the main paper, REDCLIFF-S "improves fl-scores of predicted dynamic causal patterns by roughly 22-28% on average over baselines in some of our experiments, with some improvements reaching well over
16

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Improvement in Optimal F1-Score
Factor Instance
10
Avg. Optimal F1-Score
02-
02
00
cMLP
CLSTM
DGGNN
DCSFA -MM
Baseline Algorithm Name
Figure 6. Pairwise optimal F1-Score Improvement of REDCLIFF-S on the Synthetic Systems 6-2-2 experiment.
60%": Figure plots these levels of improvement explicitly.
C.2. Additional D4IC Results
report the main results of our D4IC Low Signal-to-Noise Ratio (LSNR) experiments in Figure 7 and those of our D4IC Moderate Signal-to-Noise Ratio (MSNR) experiments in Figure 8.
REDCLIFF-S (cMLP)
(cMLP)
<MLP-V1
cMLP-22
CLSTM
cLSTM
DGCNN
DOCNN
DYNOTEARS
DYNOTEARS
RAVAR-R
NAVAR-R
NAVAR-P
NAVAR-P
0.26 0.27 0.28 0.29 0.30
0.31
0.32 0.33 0.34 0.35
0.28
0.30
0.32
0.34
0.36
0.36
Avg. Optimal F1-Score Err. the Mean
Avg. Optimal F1-Score
Figure 8. Optimal f'l-ccor on D4IC-MSNR experiments. We also report ROC-AUC statistics corresponding REDCLIFF-S, cMLP-v2, and dCSFA-NMF models trained on the D4IC dataset in Table
Figure 7. Optimal f1-score on D4IC-LSNR experiments.
C.3. Additional Synthetic Systems Results
We now report additional results from our Synthetic Systems experiments for transparency.
17

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table Average inter-variable ROC-AUC score SEM the D4IC LSNR, MSNR, and HSNR datasets. Averages taken across factors and repeats. Note that predictions included edge weighting, but we converted true-edge weights into binary "present""" "nott-treentt values during evaluation facilitate ROC-AUC calculations.
D4IC NOISE LEVEL MSNR
ALGORITHM
LSNR
HSNR
CMLP-v2
0.550+0.016
0.594=0.017
0.567+00011
DCSFA-NMF
0.567==0.13
0.554+0.013
REDCLIFF-S (CMLP) 0.632.10.014
0.635.1.0016
0.629-10.018
System Complexity (G)
12
10
*
Not Significant for All Baselines
Significant for All Baselines
No Result
30
41122
3-1-3
3-1-4
3-1.5
32.
6-2
6.2
4
0.2
2-6
8
6.2.9
6.2 6.2.10
6.4.2
8.8.3
6-8-3
8:10-2
6-12.2
1.12.2
12-11-2
12-6
12:11-9
1211-
-3.3
33
12-5-22
3
Synthetic System Name (ne-ng-ngg)
Figure 9. Summary plot of pairwise optimal fl-score improvement REDCLIFF-S across Synthetic Systems experiments.
In Table we report ROC-AUC scores corresponding the models and optimal fl-score statistics in Figure 2 of the main manuscript.
We give additional results related to Table 2 in Supplemental Table 7. Note that between the two tables, the top performing basclines usually Pegime-PCMCI (Saggioro et al., 2020) and SLARAC (Weichwald et al., 2020) appear to 'trade off" between performance on metrics such as optimal f1-score and structural hamming distance (SHD) versus performance on the AID metrics. In contrast, REDCLIFF-S does not exhibit this trade-off, regularly attaining competitive performance (statistically similar to the best or second-best performance) across all metrics.
Regarding optimal fl-score performance, we provide a summary plot of these results Supplemental Figure and visualization comparing true to estimated factors in Supplemental Figure 10. Optimal fl-score and ROC-AUC score results are grouped by systems rated with similar complexity scores. Scores of C < 7.0 received 'Low' categorization; results on systems with this rating can be found in Supplementary Figures 11 through 16. Scores that satisfied 7.0 < e < 13.0 were categorized as 'Moderate' complexity; results on systems with this rating can be found Supplementary Figures 17 through 22. Scores were categorized as 'High' complexity they satisfied C > 13.0; the results on systems with this rating can be found in Supplementary Figures through 24.
18

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table Average inter-variahle ROC-AUC SEM the Synthetic System 6-2-2, 6-4-2, 12-11-2, and 12-11-5 datasets. Averages are taken across factors and repeats. Note that predictions included edge weighting, but we converted true-edge weights into binary "present''' not- porseeeeeett'' values during evaluation to facilitate ROC-AUC calculations.
SYNTHETIC SYSTEM 6-4-2
ALGORITHM
6-2-2
12-11-2
12-11-5
cMLP
0.544
0.591
0.572++.022
0.562=0.015
CLSTM
0.612=0.050
0,534+0.044
0.532:+0.024
0.552=0.020
DGCNN
0.556.10.072
0.566110.036
0.562
0.536:10.015
DCSFA-NMF
0.55710.012
REDCLIFF-S (CMLP)
0.79711.0336
0.68211.037
0.756
0.743
Table Supplemental results Table Note that the final column now gives the average placement across metrics from both tables combined. Again, note that performance is averaged across 10 random seeds, leading to slight differences with REDCLIFF performance given Table
(No THRESHOLD) ROC-AUC
ANCESTOR AID ERROR UPPER
OSET AID ERROR LOWER
OVERALL AVG. PLACE.
ALGORITHM
LOWER
UPPER
LASAR (SUP.)
0.457:0.043
1.961.466
2.940.433
1.731.331
2.940..33
3.8
QRBS (SUP.)
0.424+0.022
2.150.385
1.9+0.537
2.540.548
1.9+0.537
3.0
SLARAC (SUP.)
0.34710.023
0.810.7116
0.410.219
0.710.622
0.4:10.219
2.8
REGIME-PCMM
0.775
3.840.540
3.6.10.518
4.110.477
3.8:10.502
3.1
REDCLIFF-S
0.702
2.255.335
2.140.434
2.7:0.3990
2.3+0.522
2.3
C.4. Additional Region-Averaged TST 100 Hz Results
We provide visualization of our approach to model selection (specifically with respect determining the number of REDCLIFF-S factors) in Supplemental Figure 25. Plots of the supervised adjacency matrices learned by our REDCLIFF-S models trained the Region- Averraed TST 100 Hz dataset can be found in Supplemental Figures 26 through 28. We provide additional plots of the differences between the average Open Field (OF) and Tail Suspended (TS) supervised factors in Supplemental Figure 29 and between the average TS and Home Cage (HC) supervised factors in Supplemental Figure 30.
C.5. Region-Averaged Social Preference 100 Hz Results
In addition to the TST dataset, we also applied our REDCLIFF-S method to analyze the Social Preference (SP) dataset described in Mague et al. (2022). It should be noted that our analysis here is preliminary and was not as thorough as our analysis of the TST data due to time constraints. Supplemental Figure 31 provides a visualization of how we selected the number of factors for the REDCLIFF-S models trained on different folds of the SP data; note that the selection criteria history does not decrease monotonically as the number of factors increases (as it did on the TST data), suggesting that more detailed search of REDCLIFF-S" hyperparameter settings may yield more accurate results. Plots of the supervised adjacency matrices learned by REDCLIFF-S models trained on the Averied SP 100 Hz dataset can be found in Supplemental Figures 32 and 33.
We provide an additional plot of the mean normalized difference between Social Preference (SP) and Object Preference (OP) supervised factors in Supplemental Figure 34. While several of the connections found by REDCLIFF-S do appear to have been published previously (compare with Figure 4A in Mague et al. (2022)), we highlight the fact that the network identified by REDCLIFF-S seems to break from previously published results by including the Hippocampus in multiple network edges. Interestingly, Mague et al. (2022) report that elevated power in certain frequency bands recorded in the Hippocampus was implicated in social behavior, but they did not seem detect any coherence between activity in the Hippocampus and other regions. Given that our REDCLIFF-S method is designed to estimate both linear and nonlinear granger causal relationships, whereas the dCSFA- based technique employed by Mague et al. (2022) focuses exclusively on linear causal relationships, these findings may suggest that the Hippocampus shares nonlinear causal relationships with other regions of the brain during social activity. More follow-up work would need to be done to confirm this hypothesis, but we are excited to see that REDCLIFF-S appears to be identifying plausible, novel hypotheses.
19

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table Mean optimal fl-score SEM ohtained by REDCLIFF-S ablations across various datasets (summarized in Table 3 of the main paper). Values should be compared against those depicted in Figures 2A and of the main paper.
ABLATION
DATASET
0 (BQ. 5, 10)
1 (= EQ. 2)
a= 1 (EQ. 4)
A 0 (EQ. 9)
SYNTHETIC SYSTEM 6-2-2
0.463=0.055
0.279+0.044
0.291
0.279=0.044
SYNTHETIC SYSTEM 6-4-2
0.51710.054
0.33510.031
0.40710.037
0.335-10.031
SYNTHETIC SYSTEM 12-11-2
0.296=1.0037
0.188110033
0.208
0.188110.033
SYNTHETIC SYSTEM 12-11-5
0.362-10.31
0.15611.0114
0.197110011
0.156110.015
D4IC HSNR
0.370=0.014
0.304+0.002
0.336#0.009
0,342=0.010
C.6. Ablation Analyses
In Table 8 provide details of our ablation experiments summarized in Table of the main paper. With the exception of that the ablation scores are worse in many cases by large margin than those of the
P O ablation experiments, full REDCLIFF-S model, suggesting that these hyperparameters play important role in the performance of REDCLIFF-S.
The ablation experiments tell more nuanced story, with performance degrading marginally for three of the five systems tested and actually increasing when was ablated for two of the systems. These observations are somewhat unsurprising for few reasons. Firstly, we reiterate that we only tuned the hyperparameterss including p on a single synthetic system configuration due time constraints, so it is unsurprising that at least one of our parameter settings suboptimal in certain cases. Secondly, note that there appears be a pattern in Table 8 where the systems with relatively fewer edges and factors (esp. the 6-2-2 and 12-11-2 Synthetic Systems) tend perform worse after setting p 0, whereas those with more edges and factors (esp. the 6-4-2 and 12-11-5 Synthetic Systems) see improvements when p 0. This makes intuitive sense, since state graphs in systems with more edges and factors are simply more likely to share similar causal relationships than those with fewer connections and states. Finally, the CosSim loss term modulated by was introduced into the REDCLIFF training process after we observed that doing so was loosely correlated with improved ROC-AUC performance at test time preliminary experiments (see Appendix B.3); since the ROC-AUC and Fl metries are fundamentally different, the correlated relationship between minimized CosSim and ROC-AUC may not always indicate improved F1 performance. In summary, it would seem that p more sensitive than other hyperparameters to properties of the system being modeled and should thus be carefully tuned to suit the particular target system (and possibly dropped entirely for sufficiently dense or expressive systems).
20

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
True A
Est. A
Receiving Channels
1.0
0.8
0.6
True B
Est. B
Receiving Channels
0.4
0.2
0.0
Driving Channels
Driving Channels
Figure 10. Visualizing true (left column) and REDCLIFF& Top-10 estimated (right column) inter-variable causal relationships of factors ('A' also depicted in main paper Figure - and 'B') from Synthetic System "12-11-5". The color map shows (normalized) weight of vaterlvariable causal relationships.
21

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
REDCLIFF-S (cMLP)
12-22-2
12-22-2
<LSTM BCSFA-NMM
12-33/2
12-33-2
Synthelic System Name Intettitt)
12-46-2
Synthetic System Name (ft-fhu-hh
12-44-2
REDCLIFF-S (CMLP)
12-55-2
12-55-2
CLSTM
DGCNN
OCSFA- NMF
9-1-2
3-2-2
3-2-2
6-10-2
6:10-
-0.10
-0.05
0.00
0.05
0.30
0.15
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Avg. Difference Optimal F1-Score Std. Err. of the Mean
Avg. Optimal F1-Score = Std. Emr. of the Mean
Figure 12. Average improvement in optimal fl-scores + SEM by the REDCLIF BICILIF-SS algorithm baselines from Low- complexity Synthetic Systems experiments.
Figure 11. Average optimal fl-scores SEM from our Low- complexity Synthetic Systems experiments.
6-12-2
6.2
Synthetic System Name (nt-f
Synthelic System Name the
12-33-3 12-22-3
REDCLIFF-S (CMLP!
12.33-3 12-22-3
DGCNN
3-1.3
3-1-3
REDCLIFF
6-6-3
6-6-3
CUSTM
DOCNN
OCSFA-NMF
-0.05
0.00
0.05
0.10
0.15
0.20
0.25
0.0
0.1
0.2
0.3
0.4
0.5
066
Avg. Difference Optimal F1-Score Std. Err. of the Mean
Avg. Optimal F1-Score Std. Emr. of the Mean
Figure 14, Average improvement in optimal fl-scores + SEM by the REDCLIFF-S algorithm baselines from Low- complexity Synthetic Systems experiments.
Figure 13. Average optimal fl-scores + SEM from our complexity Synthetic Systems experiments.
22

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
REDCLIFF-S
CMLP
CLSTM
DOCNN OCSFA-AJME
12-22-4
12-22-4
Synthetic System Name (Nt-lle-tt))
Syntheiic System Name Intettant)))
3-1-4
6-6-4
6-0-4
12:22-5
12-22-5
3-155
REDCUFF-S (CMUP)
cMIP
DGCNN
NCSFA-NMF
-0. 10
-0.05
0.00
0.05
0.10
0.15
0.0
0.1
02
0.3
0.4
0.5
0.6
Avg. Difference Optimal F1-Score Std. En. of the Mean
Aug. Optimal F1-Score Std. Err. of the Mean
Figure 16. Average improvement in optimal fl-scores + SEM by the REDCLIFF& algorithm baselines from Low- complexity Synthetic Systems experiments.
Figure 15. Average optimal fl-scores SEM from our complexity Synthetic Systems experiments.
REDCLIFF S (cMLP)
12-13-2
CLSTM
DOCNN
OCSFA-NMF
12-12-2
12-12-2
Synthetic System Name (Nc-lt--t)))
Synthelic System Name (ne
6-5-2
6-4-2
REDCLIFF-S (CMLP)
12-11-3
12-11-3
CLSTM
DOCNN
OCSFA-NMF
12-12-3
6-433
6-4-3
12-11-4
12-11-4
0.000 0.025
0.150 0.175 0.200 Difference Optimal F1-Score Std. Err. of the Mean
0.050
0.075
0.100 0.125
0.0
0.1
0.2
Aug. Optimal F1-Score Std. Err of the Mean
Figure 18. Average improvement in optimal fl fl-scores + SEM by the REDCLIFF-S algorithm baselines from Moderate- complexity Synthetic Systems experiments.
Figure 17. Average optimal fl fl-scores SEM from our Moderate- complexity Synthetic Systems experiments.
23

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
REDCLIFF-S
12-12-4
cMLP
12-12-4
cLSTM
DGCNN
OCSFA-NIMF
6-4-4
6-2-4
Synthetic System Name (Rt-tle-ti))
12-11-5
Syntheic System Name Intertiniy
12-12-5 12-13-5
12-22-5
6-4-5
6-4-5
12-11-6
12-11-6
12-12-6
REDCLIFF-S (CHLU!!
cMLP
DGCNN
JCSFA-NMP
0.000 0.025
0.050
0.075
0.100
6.125
0.150
0.175 0.200
0.0
0.1
0.2
0.3
0.4
0.5
Avg. Difference Optimal F1-Score Std. Er. the Mean
Avg. Optimal F1-Score Std. Err. of the Mean
Figure 20. Average improvement in optimal fl fl-scores + SEM by the REDCLIFF-S algorithm baselines from Moderate- complexity Synthetic Systems experiments.
Figure 19. Average optimal Il fl-scores SEM from our Moderate- complexity Synthetic Systems experiments.
6-4-6
6-4-6
12-13-7
12-11-7
Synthetic System Name (nc-ne-th))
Synthetic System Name (n (----))
12-12-8 12-11-8 12-12-7
12-12-6 12-11-8 12-12-7
12-11-9
12-1:-4
12-12-9
(CHLP)
12-12-9
cMLP
<MUP
CLSTM
cLSTM
DGCNN
DGCNN
dCSIA-NMF
JCSFA-NMF
0.00
0.02
0.04
0.05
0.08
0.10
0.12
0.0
0.1
0.2
0.3
Avg. Difference Optimal F1-Score Std. Err. the Mean
Avg. Optimal F1-Score Std. Err. of the Mean
Figure 22. Average improvement in optimal fl fc-scores + SEM by the REDCLIFF-S algorithm baselines from Moderate- complexity Synthetic Systems experiments.
Figure 21. Average optimal fl f1-scores SEM from our Moderate- complexity Synthetic Systems experiments.
24

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
6.2.3
Synthetic System Name (nc-n,-n
Synthetic System Name Int-fiefht)
6-2.8
REDCUIF-S (CMLP)
REDCLIFF-S (CMLP)
cLSTM
cLSTM
DGCNN
dCSFA-NMF
OCSFA-NMF
0.0
Avg. Difference in Optimal F1-Score + Std. Err. of the Mean
Avg. Optimal
Figure 24. Average improvement in optimal fl-scores SEM by the REDCLIFF-S algorithm over baselines from our High- complexity Synthetic Systems experiments.
Figure 23. Average optimal fl-scores + SEM from High- complexity Synthetic Systems experiments.
2.00
Acb_Core Core
1.80
Selected Model
Acb_Sh
1.75
Avg. Stopping Criteria Performance Across
IL_Cx
1.75
Receiving Channels
VTA
1.50
Md. Thal
1.25
PrL_CK
1.65
R_VTA
1.60
aILH Hab
IDHip
0.75
1.55
ISNC
0.55
mDHip
mSNC
0.25
Acb Core
Acb Sh
IL C_
L VTA
Md. Thai
R VTA
aILH Hab
IDHip
ISNC
mDHip
mSNC
10
12
14
16
18
PrL
0.00
Number Factors in Model
Driving Channels
Figure Visualizing model selection Region-Averaged TST 100 Hz case study.
Figure 26. Average estimated strength of causal relationships he- tween channels recorded in the Home Cage state from the Region- Averaged TST 100 Hz experiment.
25

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
2.00
2.00
Acb_ xcb_Core
Acb_ Core
1.75
Acb_Sh
1.75
Acb_Sh
IL_Cx
CL_x
1.50
Receiving Channels
VTA
1.50
Receiving Channels
L_TT
Thal-
1.25
Md. Thal
1.25
PrL_Cx
PrL_Cr
1.00
R_ R_VTA
1.00
R_VTA
aILH Hab
aILH Hab
IDHip
0.75
IDHip
0.75
ISNC
0.50
ISNC
0.50
mDHip
mDHip
mSNc
0.25
mSNC
Acb Core
Acb Sh
IL Cx
LVT
Md. That
Pri Cx
RL RTA
aILH Hab
IDHip
ISNC
mDHip
mSNC
Acb Core
Acb Sh
IL Cx
LVTA
Md. Thai
Pri Cx
R. VTA
aILH Hab
IDHip
ISNC
mDHip
mSNC
0.00
0.00
Driving Channels
Driving Channels
Figure 27. Average estimated strength of causal relationships be- tween channels recorded Open Field state Region- Averaged TST 100 Hz experiment.
Figure 28. Average estimated strength of causal relationships be- tween channels recorded in Tail Suspended state from Region-Averaged TST 100 Hz experiment.
1.00
acb_Core
0.75
Acb, Core
0.75
Acb_Sh
Acb Sh
IL_Ga [L_x
0.50
IL Cx
0,50
Receiving Channels
L VTA
Receiving Channels
L LVVA
Md_ Thal
0.25
Md_ Thal
Prt_G CX
PrL_CC Cx
0.00
0.00
R_ R.VTA
R. VTA
alLH_Hab
alLH_ Hab
-0.25
IDHip
IDHip
ISNC
-0.50
ISNC
0.50
mDHip
mDHip
mSNC
-0.75
mSNC
0.75
Acb core
IL Cx
LVVTA
Md. That
PYL Cx
R_VT LVA
aILH Hab
IDHip
ISNC
mDHip
mSNC
Acb Core
Acb Sh
L. VVA
Md. That
Pri. Cx
R. VTA
aILH Hab
IDHip
ISNC
mDHip
mSNC
Acb
11.00
I1.
-1.00
Driving Channels
Driving Channels
Figure 29. Difference between Open Field vs Tail Suspended aver- age estimated strength of causal relationships between channels from the Region-Avee Averaged TST 100 Hz experiment.
Figure 30. Difference between Tail Suspended vs Home Cage av- erage estimated strength of causal relationships between channels from the Region-Averaged TST 100 Hz experiment.
26

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
3,70
Selected Model
Selected Model
3.65
3.60
Performance
Mean Performance
3,50
5
3.45
3.40
Number of Factors Tested
Number of Factors Tested
Figure 31. Visualizing model selection in the Region-Averaged SP 100 Hz case study.
3.0
Amy_Bn BLA
Amy BLA
2.0
2.5
Amy_CeA
Amy_CeA
Receiving Channels
Cg_ C2 R
Receiving Channels
Cg_Cx_R
2.0
Hipp
Hipp
NAr_Core
1.5
NAc_Core
-10
JAc_Shell-
NAc MAc_Shell
CX R
1.0
PrL_Cx_R
V
VTA_L
V R
0.5
VTA
Amy BLA
Amy CeA
cg_Ck_R
Hiop
NAC Core
NAC Shell
PrL CL_ R
VTA_
VTA. R
Amy BLA
Amy CeA
CQ Cx__R
Hipp
NAC Core
NAC. Shell
PrL CX_R
VTA. L
VTA R
0.0
0.0
Driving Channels
Driving Channels
Figure 32, Average estimated strength of causal relationships be- tween channels recorded in the Social Preference state from the Region-Averaged SP 100 Hz experiment.
Figure 33. Average estimated strength causal relationships be- tween channels recorded in the Object Preference state from the Region-Averaged SP 100 Hz experiment.
27

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
1.00
0.75
Amy_BLA
Amy_CeA
0.50
Receiving Channels
C9_Cx_R
0.25
Hipp
NAc_Core
0..00
NAc_Shell
-0.25
PrL_cx_R=
-0.50
VTA_L
VTA_R R
-0.75
Amy_Bi BLA
Amy_ CeA
Cg_cx_R R
Hipp
NAc_ NAc_Core Core
NAc NAc_Shell
Shell
CxL_ R
VTA_L
VTA_R
-1.00
Driving Channels
Figure 34, Difference in average estimated strength of causal relationships between channels predicted in the Social Preference vs Object
Preference behavioral paradigms the Aveerned 100 experiment.
28

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
D. Datasets Used: Additional Details
In this section, we report various hyperparameters used in curating/preparing the datasets used our experiments. Sup- plemental Table presents hyperparameters for the D4IC dataset (see Section 4.3). Supplemental Table 10 presents hyperparameters for the Synthetic Systems Dataset (sec Section 4.2). Supplemental Table 11 presents hyperparameterss for the Region-Averaged TST 100 Hz dataset (see Section 5). We briefly discuss the Region-Averaged SP 100 Hz dataset in Appendix D.2.
Note that some elements in these hyperparameter tables appear as functions/equations rather than numerical values or lists of numbers. Due to the vast number of models we had to train for our quantitative results, portions of our code were designed to compute various hyperparameters rather than have them hard-coded and/or passed as arguments. We release this code the project repository for the main paper, again, the repository can be accessed at giihub.com/carlsnn-ab/reddddd sypotheiziig-ddyamii--asssas
Table D4IC Dataset (Section 4.3) hyperparameters across experiments.
PARAMETER NAME
VALUE
NETWORK SIZE
10
NUMBER OF FACTORS PER SYSTEM
NUMBER OF CROSS VALIDATION FOLDS
DOMINANT COEFFICIENT
10
POSSIBLE BACKGROUND COEFF.S
[0.0, 0.1, 1.0]
Table 10. Synthetic Systems Dataset (Sections 4.1.2) hyperparameters across experiments. Of particular note, repeat number 3 (indexed from 0) of the n. = 1o-node, n. = 33- (inter-variable,, m 3-factor synthetic system dataset was used to perform grid searches over hyperparameters across Synthetic Systems experiments. Generally speaking, find that the greater the number of edges per node in the system, the more 'complex' the Granger causal estimation task.
PARAMETER NAME
VALUE
NUMBER OF REPEATS PER SYSTEM
NUMBER OF SECTOR/VAAR-MDDRR (Π)- = B)
n E (,,,8,,4,5,6,,788,,,
10
DIMENSIONALITY (n.)
De E 13,
6, 12
LAGGED DEPENDENCIES (T)
NUMBER TIMESTEPS RECORDINGS
100
TRAINING SAMPLES PER CLASS LABEL (TSCL)
1040
VALIDATION SAMPLES PER CLASS LABEL (VSCL)
240
NUMBER OF SAMPLES IN TRAINING SET
TSCL * (B +1)
NUMBER SAMPLES VALIDATION SET
VSCL (B + 1)
INNOVATIONS AMP_CSSFFS
-N..ONES 1))
INNOVATIONS.
NP.7EROS( 1))
INNOVATIONS MU
NP.ZEROS((n.: 1))
np.pi mp.array(i: 707 + i%2for i in range(n.)
BASE FREQUENCIES
).reshape(n. (n., 1)/120000
D.1. Preprocessing the TST 100 Hz. Datasets
As mentioned in the main paper, applied some preprocessing steps to the publicly available TST dataset (Carlson et al., 2023) to prepare it for our experiments. Specifically, we first sorted the publicly available local field potential (LFP) recordings and associated behavioral label files according to the identifier of the mouse being observed; this was done to ensure all mice were equally represented in each data repeat and so that different mice could be designated as 'holdout' subjects for each repeat. We then iterated through each subject mouse's files and performed the following operations on each recording:
Removed LFP recordings from any channel/brain--rgion that was not present across all recordings of all mice;
Marked any time point for which the LFP value was more than 15 median absolute deviations above the median value of the (filtered- see below) recorded signal as outliers (i.e., 'nan' values);
29

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table 11. Region-Averaged TST 100 Hz Dataset (Section 4.4) hyperparameters across experiments.
PARAMETER NAME
VALUE
NUM PROCESSED SAMPLES
10000
SAMPLE_ TEMP WINDOW SIIZE
1500
'AMPLE_FREQ
1000
POST PRRCERSSINGSSAPPPIIP FREQ
100
CUTOFF
35
MAD.THRESHOLD
15
ORDER
FILTER TYPE
LOWPASS
HOMECAGE STOP
SAMPLE FREQ
DOWNSAMPLING STEP SIZE
SAMPLE FREQ // POST PROCESSING SAMPLE FREQ
Filtered the resulting (outlier-marked) LFPs where we zero-ed out all outliers with low-pass Butterworth filter followed by IIR notch filter;
Filtered LFP channel recordings were then stacked into an ne x Tree recording where ne was the number of selected LFP channels and Tree was the full length of the recording.
Once a recording file had been run through the above steps, we drew samples evenly across the parts of the recording representing the a) Home Cage (HC), b) Open Field (OP), and c) Tail Suspended (TS) behavioral paradigms. Each sample was drawn in such a way that it was guaranteed have no marked outlier time points. Once sample had been drawn, one-hot vector label corresponding to the behavioral paradigm was attached to it, and the sampled signal was downsampled to reduce computational overhead in later experiments.
D.2. Preprocessing the SP 100 Hz Datasets
We used the same parameters to preprocess the Social Preference (SP) dataset (Mague al., 2022) were used to prepare the Region-Averaged TST 100 Hz dataset. The only fundamental differences between the preprocessed TST data and the Region-Averager SP 100 Hz dataset were 1) the labels in the datasets (three behavioral labels for TST and two for SP) and 2) the dimensionality of the time series being analyzed.
E. Model Hyperparameters Used in Experiments: Additional Details
In this section, report hyperparameters used configure and train the models used in experiments. Hyperparameters for the cLSTM models that appear in this paper (see Sections 4.2 and 4.3) are given in Supplemental Table 12. Hyper- parameters for the cMLP models that appear in this paper (see Sections 4.2 and 4.3) are given in Supplemental Table 13. Hyperparameters for the DGCNN models that appear in this paper (see Sections 4.2 and 4.3) are given in Supplemental Table 14. Hyperparameters for the DYNOTEARS models that appear in this paper (see Section 4.3) are given in Supplemental Table 15. Hyperparameters for the NAVAR-R (cLSTM) models that appear in this paper (see Section 4.3) are given in Supplemental Table 16. Hyperparameters for the NAVAR-P (cMLP) models that appear in this paper (see Section 4.3) are given in Supplemental Table 17. Hyperparameters for the dCSFA models that appear in this paper (see Sections 4.2 and 4.3) given in Supplemental Table 18. Hyperparameters for the REDCLIFF-S (cMLP) models that appear in this paper (see Sections 4.2, 4.3, and 5) are given in Supplemental Tables 19 and 20.
E.1. Identifying Datasets Used to Tune Parameters
Along with the hyperparameters for each model, report the dataset(s) used derive those parameters. Models represented by columns with "Synth. Systems" in the column name used hyperparameters selected by grid search(es) on Repeat 3 of the dataset for a (nonlinear, noisy) system with n. = 12, n. = 33, and ny = 3; code for generating this dataset is included in the project repository for the main paper. Models represented by columns with "D4IC" in the column name had their hyperparameters selected by grid search(es) performed on Repeat 0 of the Moderate SNR D4IC dataset (see Table 9), with the cMI.P model being less optimal than cMI.P v2 according to the original selection criteria.
30

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table 12, cLSTM hyperparameters across experiments.
PARAMETER NAME
SYNTH. SYSTEMS
D4IC
BATCH_ SIZE
128
128
MAX IIER
300
1000
NUM-SIMS
CONTEXT
2
2
LENGTH
4
4
FORECAST COBFF.
1.0
1.0
LI REGULARIZATION COFFF.
1.0
10.0
GEN_HIDDEN
25
25
GEN LR
0.0001
0.0005
GEN EPS
0.0001
0.0001
GEN WEIGHT DECAY
0.0001
0.0001
SMBED_HIDDEN SIZES
10
10
Table 13. cMLP hyperparameters across experiments.
PARAMETER NAME
SYNTH. SYSTEMS
D4IC (vI)
D4IC (v2)
BATCHLSIZE
128
128
128
MAX LITER
300
1000
1000
GEN LAG AND INPUINPUT.LE LEN
2
2
FORECAST COEFF.
1.0
1.0
1.0
ADJ. LI REGULARIZATION COFFF.
1.0
1.0
1.0
GEN_HIDDEN
25
50
50
GEN I.R
0.0001
0.0005
0.001
GEN EPS
0.0001
0.0001
0.0001
GEN WEIGHT DECAY
0.0001
0.0001
0.0001
JMBED_HIDDEN..N
10
60
10
E.2. Hyperparameters Represented by Functions
We noticed in preliminary experiments that some hyperparameters were highly sensitive to certain aspects of the dynamical systems being modeled; furthermore, we noticed these sensitivities could be described by fairly simple functions system aspects. Rather than set a fixed value for these model hyperparameters, we tuned imeta-hyperparameter's (c.g., coefficients) of the functions for these sensitive hyperparameters a particular dataset, and then recomputed the output of their functions each time the 'tuned' model was transferred new target dynamical system. We report the tuned functional form (with tuned ) in place of any fixed value for this type of sensitive hyperparameter.
31

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table 14. DGCNN hyperparameters across experiments.
PARAMETER NAME
SYNTH. SYSTEMS
D4IC
BATCH SIZE
MAX ITER
300
1000
NUM.CHANNELS
10
10
0.0001
0.0001
GEN_ WEIGHT.DECA
0.0001
0.0001
NUM FEATURES SEERRSSPER..EE PER_NODE
NUM NUM.CLASSS CLASSES
SAME AS DATASET
5
NUM. GRAPH_ .LAYERS
3
SUM_HIDDEN_NODEE
250
100
GEN
0.0001
0.0001
Table 15. DYNOTEARS hyperparameters experiments.
PARAMETER NAME
D4IC
MAX_ITER
100
LAMBDA_W
0.9
LAMBDA_A
0.1
TOL.
0.00000001
W THRESHOLD
0.0
TABU EDGES
NONE
TABU PARENT_NODES
NONE
BRBU_CHILD_OOO
NONE
LAG.SIZE
Table NAVAR-R (cLSTM) hyperparameters across experiments.
PARAMETER NAME
D4IC
BATCH_SIZE
128
EPOCHS
1000
HUM_NODER
MAXLAGS
2
NUM HIDDEN
256
HIDDEN.LAYER
1
LEARNING.RAID
0.0001
WEIGHT_DECHN
0
LAMBDA
0.0
Table NAVAR-P (cMLPP) P)- hyperparaameer across experiments.
PARAMETER NAME
D4IC
BATCH
128
EPOCHS
1000
NUM_NODES
10
MAXLAGS
20
NUM_HIDDEN
256
HIDDEN_LAYER
2
LEARNING,
0.0001
WEIGHT DECAY
0
LAMBDAI
0.0
32

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table 18. dCSFA hyperparameters across experiments.
PARAMETER NAME
SYNTH. SYSTEMS
D4IC
BATCH_SIZE
128
128
N EPOCHS
250
1000
N PRE EPOCHS
50
50
NMF MAX ITER
10
20
WUM.HICH.LEVBL...O BLLL
13
5
DEM.NODE_
N. E.COMPONENTS
SAME AS DATASET
5
N_SUP_NEE -EETONTORSKS
N COMMPONNSSSS
5
RECON. WEIGHT
RECON.WEIGHT
2.0
1.0
WEIGHT
1.0
2.0
SUP RECON. WEIGHT
1.0
1.0
SUP SMOTTHOTSSSSSS WEIGHT
1.0
2.0
H
256
256
MOMENTUM
0.9
0.5
LR
0.0005
0.001
100
100
MIN FREQ
0.0
0.0
MAX FREQ
50.0
50.0
DIRECTED SPECTRUM
TRUE
TRUE CONSTANT
DETREND
CONSTANT
WINDOW
HANN
HANN
NPERSEG
= NUM. BUIM.NODE_FEATUREE
FEATURES FUN.NODE.FFATTURES/2
= NODEE BEW.NODE.FEATURRS
NOVERLAP
NUUMLOODE_FEATURES//
NEFT
NONE
NONE
Table 19. REDCLIFF-S hyperparameters across experiments on synthetic data. Note that for the Synthetic Systems datasets we set Yee equal the dimensionality of the system, while for the D4IC datasets had
10.
PARAMETER NAME
SYNTH. SYSTEMS
D4IC
BATCHLSIZE
128
128
MAX_ITER
300
1000
nk B
SAME DATASET
Ilk
Th (EQ. 2)
4
4
(EQ. 5)
10.0
10.0
(EQ. 5)
0.1
E TT.
Pk
d.
n (EQ. 5)
0.00201
Di V. P-1
-
(Eq. 6)
0.001
0.001
(EQ. 9)
100.0
100.0
GEN HIDDEN
[25]
[100]
GEN LR
0.0005
0.0005
GEN EPS
0.0001
0.0001
WEIGHT-DECAY
0.0001
0.0001
NUM_PRETRAIN EEPCHS
100
50
ACCLACILINN S
EPOCHS
100
15
CESTOR-SCOREEEMEE EMBEDDER.TYPE
DGCNN
DGCNN
EMBED_NUM_NT NUM_HIDDER NNDDS
100
30
EMBED_NUN WWW..GRR..PL.CONL..AEEE
3
2
EMBED LR
0.0005
0.0002
EMBED RPS
0.0001
0.0001
EMBED WEIGHT DECAY
0.0001
0.0001
EMBED_LAG (Tin Telass)
16
20
33

---

Generating Hypotheses of Dynamic Causal Graphs in Neuroscience
Table 20. REDCLIFF-S hyperparameteers across experiments on real-world data. Note that for the Region-Averaged TST 100 Hz dataset had nr 12 while for the Region Averaged SP dataset had 9.
PARAMETER NAME
REG. AvG. TST 100 Hz
REG. AvG. SP 100 Hz
BATCH_SIZE
128
128
MAX_ITER
300
300
m
[2, 6, 18, 36]
Ta (EQ. 2)
w (EQ. 5)
10.0
10.0
p (EQ. 5)
0.0278
S* 0T
7
y (EQ.
0.000929
Ry V 9-1
y (Eq. 6)
0.001
0.001
X (EQ.
100.0
100.0
GEN HIDDEN
[25]
[25]
GEN LR
0.0005
0.0005
GEN GEN_EPS EPP
0.0001
0.0001
WENGWEIGHT.AO -ECCAY
0,0001
0.0001
NUM_PRETRAIN EPOCHS
100
100
AUMAACCLIMTTION.. EEPOCHS EPOCHS
100
100
TYPE
DGCNN
DGCNN
EMBED_NUM NUM_HIDENN _NODES
100
100
EMRED NUM GRAPH CONV LAYERS
3
EMBED
0.0005
0.0005
EMBED EPS
0.0001
0.0001
EMBED EMBED.WEIGHT.DD WEIGHT.DECAY
0.0001
0.0001
EMBED_LAC (Tm. + Telass
16
16
34


Reviews:
[{"review_id": "tbGhcMyuHS", "reviewer": "Reviewer_nDzu", "summary": "The submission propose a method for causal discovery (at least this is what I understand). \nSummarizing it seems that the proposed approach is another implementation of a non-linear Granger causality model with some particular choice of implementing architecture and loss function. \nThere are no theoretical results (identifiability, consistency etc..), instead the authors argue that such systems are non-identifiable in the appendix, and why it is still valuable. \nThe proposed approach is benchmarked against other approaches with a simulation experiment.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Question 1: Our Goal\n\nOur goal is to reduce the space of hypotheses that must be tested before a causal relationship is successfully identified. We allude to this in our abstract, but will make this more explicit in our final draft.\n\nQuestion 2: Our Assumptions\n\nWe only assume our data consists of regularly sampled time series of scalar-valued nodes (see Section 3.1). These time series may be noisy, and we make no assumptions regarding the underlying generative processes / graphs. In Sec. 3.5, we assume the availability of \"global state\" labels for each time step that relate to the dominant generative process.\n\nQuestion 3: Comparing REDCLIFF-S & Dynamic Causal Models\n\nThe REDCLIFF-S method should be viewed as a preparatory analysis technique preceding the implementation of a more sophisticated method such as Dynamic Causal Models (DCMs). As discussed in work cited by the reviewer [1,2], the Granger causal paradigm assumed by REDCLIFF-S precludes the availability of interventional inputs required to construct a DCM model [1], suggesting that Granger causal models (such as REDCLIFF-S) be employed as a complementary analysis prior to the implementation of DCMs [2]. \n\n - [1] pages 3101 & 3102 of Stephan, Klaas Enno, et al. \"Ten simple rules for dynamic causal modeling.\" Neuroimage 49.4 (2010): 3099-3109.\n\n - [2] page 175 of Friston, Karl, Rosalyn Moran, and Anil K. Seth. \"Analysing connectivity with Granger causality and dynamic causal modelling.\" Current opinion in neurobiology 23.2 (2013): 172-178.\n\nQuestion 4: PCMCI-style algorithms & simpler baselines (e.g. tidybench https://github.com/sweichwald)\n\nWe originally excluded PCMCI and similar methods over concerns with acyclicity constraints, particularly related to cross-frequency coupling phenomena we hoped to capture. Given the reviewer's feedback, we compared RECLIFF-S, Regime-PCMCI [3], and supervised versions of 'slarac', 'qrbs', and 'lasar' from the tidybench repository ('selvar' did not compile locally) on our '12-11-2' Synthetic Systems (Figure 2A), with results shown below. Values are given as the average (Avg) of the mean statistic of system factors across repeats and the standard error of the mean (SEM) for continuous-valued metrics or as the median (Med) statistic across repeats and factors for discrete metrics; 'Upper' and 'Lower' refer to the fact that we had to employ causal distance metrics [4] on the upper-triangular and lower-triangular portions of our adjacency matrices due to the presence of cycles (details in Sec. 4.1 & Appendix B.2).\n\nREDCLIFF-S \n - Opt. F1: 0.373 (Avg) 0.028 (SEM)\n - ROC-AUC: 0.687 (Avg) 0.017 (SEM)\n - Upper Ancestor Aid Errors: 3.0 (Med)\n - Lower Ancestor Aid Errors: 2.0 (Med)\n - Upper Parent Aid Errors: 4.5 (Med)\n - Lower Parent Aid Errors: 2.5 (Med)\n - Upper Oset Aid Errors: 3.0 (Med)\n - Lower Oset Aid Errors: 3.0 (Med)\n\nRegime-PCMCI \n - Opt. F1: 0.407 (Avg) 0.027 (SEM)\n - ROC-AUC: 0.673 (Avg) 0.026 (SEM)\n - Upper Ancestor Aid Errors: 4.0 (Med)\n - Lower Ancestor Aid Errors: 4.0 (Med)\n - Upper Parent Aid Errors: 6.0 (Med)\n - Lower Parent Aid Errors: 5.5 (Med)\n - Upper Oset Aid Errors: 4.0 (Med)\n - Lower Oset Aid Errors: 4.0 (Med)\n\n(Supervised) slarac\n - Opt. F1: 0.417 (Avg) 0.034 (SEM)\n - ROC-AUC: 0.598 (Avg) 0.015 (SEM)\n - Upper Ancestor Aid Errors: 6.0 (Med)\n - Lower Ancestor Aid Errors: 4.5 (Med)\n - Upper Parent Aid Errors: 8.0 (Med)\n - Lower Parent Aid Errors: 6.5 (Med)\n - Upper Oset Aid Errors: 6.0 (Med)\n - Lower Oset Aid Errors: 4.5 (Med)\n\n(Supervised) qrbs\n - Opt. F1: 0.451 (Avg) 0.029 (SEM)\n - ROC-AUC: 0.614 (Avg) 0.014 (SEM)\n - Upper Ancestor Aid Errors: 6.1 (Med)\n - Lower Ancestor Aid Errors: 5.0 (Med)\n - Upper Parent Aid Errors: 8.9 (Med)\n - Lower Parent Aid Errors: 7.0 (Med)\n - Upper Oset Aid Errors: 6.1 (Med)\n - Lower Oset Aid Errors: 5.0 (Med)\n\n(Supervised) lasar\n - Opt. F1: 0.283 (Avg) 0.009 (SEM)\n - ROC-AUC: 0.563 (Avg) 0.026 (SEM)\n - Upper Ancestor Aid Errors: 5.3 (Med)\n - Lower Ancestor Aid Errors: 4.7 (Med)\n - Upper Parent Aid Errors: 8.5 (Med)\n - Lower Parent Aid Errors: 6.7 (Med)\n - Upper Oset Aid Errors: 5.7 (Med)\n - Lower Oset Aid Errors: 4.7 (Med)\n\nNote Regime-PCMCI and REDCLIFF-S attain significantly higher ROC-AUC scores than the other baselines. While the Opt. F1 scores for REDCLIFF-S are similar to several baselines', we highlight REDCLIFF-S' lower median error across all of the causal metrics shown as evidence that it offers improvements in identifying functional relations in this setting. \n\n - [3] Elena Saggioro, Jana de Wiljes, Marlene Kretschmer, Jakob Runge; Reconstructing regime-dependent causal relationships from observational time series. Chaos 1 November 2020; 30 (11): 113115. https://doi.org/10.1063/5.0020538\n\n - [4] \"Adjustment Identification Distance: A gadjid for Causal Structure Learning\" https://openreview.net/forum?id=jO5UNNrjJr\n\nWe thank the reviewer for their helpful feedback which we will certainly incorporate into our paper, and we hope they found our response helpful.", "claims_and_evidence": "partially, in general there are no strong theoretical claims.\n\nNevertheless, I have concerns about:\n\n(1) \"The main strength of REDCLIFF-S is that its hypotheses can be easily tested\" (first sentence in discussion) what \"tested\" mean ? is it possible to do statistical inference ? why other methods hypothesis can't be tested? to my understanding the problem of post-selection inference in causal discovery is far from solved, except using the naive data-splitting, I am only aware of a Greedy Equivalent Search (GES) variant which allow for valid inference https://arxiv.org/pdf/2208.05949  . \n\n(2) In the conclusions:  \"attains state-of-the-art performance in estimating multiple causal graphs simultaneously in various settings, and in a way conducive to follow-up scientific inquiry\"  where is the evidence of that? what have been measured to prove that in the simulation or the real world data? Moreover  I think multiple baselines are missing, see next.", "methods_and_evaluation_criteria": "Yes the proposed data and benchmark makes sense. \nI would suggest the addition of some causality aware metrics between the learned graphs, apart from the F1 score. \n\nfor instance:\n\n- \"Structural intervention distance for evaluating causal graphs\"  https://dl.acm.org/doi/10.1162/NECO_a_00708\n- \"Adjustment Identification Distance: A gadjid for Causal Structure Learning\"  https://openreview.net/forum?id=jO5UNNrjJr\n- \" Separation-Based Distance Measures for Causal Graphs\"  https://openreview.net/forum?id=KO7fATqh2W", "theoretical_claims": "none", "experimental_designs_or_analyses": "correct", "relation_to_broader_scientific_literature": "I think numerous baselines are missing \nfor instance PCMCI variants are not considered in the experiments and in the Related Work they are partially dismissed because\n\"However, these methods tend to rely on assumptions that rule out hypotheses we would like to explore in neuroscience; for example, PCMCI+, LPCMCI, and DBNs all assume causal graphs are acyclical\" they assume the complete graph (expanded in time) is acyclic, they do not assume that there are no feedback loops in time for instance. \n\nMoreover some literature is not discussed (see next question).", "essential_references_not_discussed": "I think they main literature not discussed is the Dynamic Causal Model approach which was developed exactly for causal discovery in Neuroscience. \n\nsee for instance\n- Friston, Karl J., Lee Harrison, and Will Penny. \"Dynamic causal modelling.\" Neuroimage 19.4 (2003): 1273-1302.\n- Friston, Karl J., et al. \"Dynamic causal modelling of COVID-19.\" Wellcome open research 5 (2020): 89.\n- Stephan, Klaas Enno, et al. \"Ten simple rules for dynamic causal modeling.\" Neuroimage 49.4 (2010): 3099-3109.\n- Friston, Karl, Rosalyn Moran, and Anil K. Seth. \"Analysing connectivity with Granger causality and dynamic causal modelling.\" Current opinion in neurobiology 23.2 (2013): 172-178.\n\nfor a comprehensive recent reviews you can refer to:\n\n- Discovering causal relations and equations from data\nGustau Camps-Valls, Andreas Gerhardus, Urmi Ninad, Gherardo Varando, Georg Martius, Emili Balaguer-Ballester, Ricardo Vinuesa, Emiliano Diaz, Laure Zanna, and Jakob Runge  Physics Reports 1044   https://www.sciencedirect.com/science/article/pii/S0370157323003411\n\n- Causal Discovery from Temporal Data: An Overview and New Perspectives   https://dl.acm.org/doi/10.1145/3705297#sec-3\n\n- Review of Causal Discovery Methods Based on Graphical Models https://www.frontiersin.org/journals/genetics/articles/10.3389/fgene.2019.00524/full", "other_strengths_and_weaknesses": "The problem is not really clearly stated, for instance in sec. 3.1. Problem Statement I would expect a clear statement on which is the inference goal.", "other_comments_or_suggestions": "see above\n\n\n——-\n\nScore updated after rebuttal", "questions_for_authors": "- What is clearly the goal, recovering a causal graph? \n- what are the assumptions ?, of course different sets of assumptions would lead to different algorithms and methods\n- How the proposed approach compare to Dynamic Causal Models? \n- Why PCMCI style algorithms where not compared ? what about simpler baselines such as  tidybench https://github.com/sweichwald or a simple structural VAR? \n-", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r9mXxA4437", "reviewer": "Reviewer_Qgsb", "summary": "This paper proposes a novel hypothesis generation framework for dynamic causal graphs in neuroscience, aiming to improve the efficiency of causal discovery in time-dependent systems. The authors introduce REDCLIFF-S, a model designed to estimate state-dependent causal interactions by combining:\n\t1.\tFactor-based models for dynamic causal graphs.\n\t2.\tSupervised auxiliary variables (e.g., behavioral labels) to refine hypotheses.\n\t3.\tA flexible generative framework that adapts causal interactions over time.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful for the reviewer's thoughtful feedback. We respond to their concerns below.\n\nQuestion 1: \"Are there any theoretical guarantees on when REDCLIFF-S produces unique causal structures? Does the factorization step introduce ambiguities in the inferred causal graphs?\"\n\nAs we allude to in our introduction and expound upon in Appendix A.1, it is very difficult to prove identifiability for even the simplest nonlinear, state dependent systems (our example consists of 2 nodes A and B where A drives B according to 2 different functions depending on the global state). Thus, we focus our efforts on rigorous empirical testing across numerous dynamical systems (48 unique configurations each with 5 different repeats in our Synthetic Systems experiments - see Figure 9 in Appendix C.3).\n\nAs the reviewer rightly points out, our focus on emprical evidence may mean we miss ambiguities in our proposed REDCLIFF-S algorithm. However, due to the structure of our algorithm, these ambiguities are mostly related to the relative importance of each unsupervised factor as opposed to the relationship between a given factor and the input and output data (put another way, each factor has clear linear relationships to the input and output data - see Equation 4 as well as Appendix B.1 - but the weighting of each factor does not). For this reason, we outline our selection criteria in Equation 10 and demonstrate how we use it to restrict the number of unsupervised factors to the minimal amount necessary in our TST Case study in Section 6 (see also Figure 25 of the Appendices).\n\nQuestion 2: \"How much do behavioral labels contribute to improved performance? Would REDCLIFF-S still outperform unsupervised methods if auxiliary labels were unreliable or noisy?\"\n\nThis is an important question, and one which we should address more directly in our paper. In short, all of the Synthetic Systems experiments we present in this paper contain label noise, as do the D4IC MSNR and LSNR experiments presented in Appendix C.2. With regard to curating the labels for our Synthetic Systems experiments, we mention in the second paragraph of Section 4.1 that \"Recordings obtained from each VAR model were then weighted over time by linearly interpolated weights (randomly selected between 0 and 1). These weighted recordings were added together along with a level of Gaussian noise. We augmented labels by marking with a one-hot vector which factor weight was largest at each time step\". Thus, all of our labels in the Synthetic Systems experiments occluded information relating to how active each system was at a given time step. While we do not quantify how much information was occluded from the Synthetic Systems labels, we do provide a simpler exploration of this between our D4IC HSNR, MSNR, and LSNR datasets. As discussed in Section 4.3, the D4IC experiments each had \"down-weighted recordings from all but one\" system factor added on top of the recording from a \"main\" system factor (the identity of which determined the label for a given sample). This background noise was was weighted with a scalar coefficient valued at either 0.0, 0.1, or 1.0 while the recording of the 'main' system factor was multiplied by 10. As can be seen in Appendix C.2 (please forgive the typo in the caption of Figure 8, it should read 'D4IC-MSNR' instead of 'D4IC-HSNR') our proposed algorithm does not perform well under this particular form of noise. We believe that this is due to the fact that there was no variation/interpolation between the weight applied to each 'background' factor, making it difficult to ultimately distinguish them from the 'main' factor.\n\nWe thank the reviewer for mentioning additional literature that we can discuss in our paper, and we will certainly include it in our final draft. We hope our response has been helpful, and thank the reviewer again for their feedback.", "claims_and_evidence": "The claims made in the submission are supported by convincing evidence in general. \n\nPotential Weaknesses in Claims:\n* No analysis on how the model generalizes beyond neuroscience.\n* No formal guarantees on identifiability", "methods_and_evaluation_criteria": "Yes. The used datasets and evaluation metrics make sense for the problem. The used datasets include synthetic datasets (vector autoregressive models, nonlinear causal graphs), DREAM4 Insilico-Combined (D4IC) dataset (biological system benchmarks), and TST (Tail Suspension Test) neural recordings (real-world case study). Evaluation metrics include F1 score of causal graph estimation (Figures 2, 4), Computational efficiency (Table 1), and Qualitative validation with neuroscientific findings (Figure 5).\n\n\nLimitations\n* No evaluation on datasets outside neuroscience (e.g., economics, social systems).\n* No analysis of model sensitivity to hyperparameters or noise.", "theoretical_claims": "I did not check the proofs.\n\nThere is no formal guarantee on the identifiability.", "experimental_designs_or_analyses": "Strengths\n* Ablation study on synthetic systems shows clear performance trends (Figure 2).\n* DREAM4 dataset adaptation demonstrates the model’s effectiveness across different complexity levels.\n\nWeaknesses\n* No ablation study on the impact of supervised auxiliary labels.", "relation_to_broader_scientific_literature": "This paper extends causal discovery in time series (Runge, 2020; Gerhardus & Runge, 2020), generalizes dynamic graph models beyond PCMCI+ and DYNOTEARS, and ntroduces state-dependent causal inference relevant to neuroscience.", "essential_references_not_discussed": "No comparison to deep learning-based causal discovery approaches (e.g., DAG-GNN, Transformer-based models) and kernel-based method (e.g., CDNOD).", "other_strengths_and_weaknesses": "Strengths\n* Novel hypothesis generation for dynamic causal graphs.\n* Improves causal discovery accuracy in neuroscience applications.\n* Empirical validation on synthetic and real-world data.\n\nWeaknesses\n* No discussion on generalizability beyond neuroscience.\n* Limited theoretical discussion on identifiability.", "other_comments_or_suggestions": "No", "questions_for_authors": "The paper introduces dynamic causal graphs via factor models, but does not discuss formal identifiability conditions. Are there any theoretical guarantees on when REDCLIFF-S produces unique causal structures? Does the factorization step introduce ambiguities in the inferred causal graphs?\n\nThe supervised extension (REDCLIFF-S) incorporates behavioral labels for refining causal inference. How much do behavioral labels contribute to improved performance? Would REDCLIFF-S still outperform unsupervised methods if auxiliary labels were unreliable or noisy?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "17xvYlvs24", "reviewer": "Reviewer_xCcy", "summary": "This work introduces a deep generative factor model that uses weighted superposition of static graphs to achieve dynamic causal graph modeling and capture nonlinear relationships. For complex neural activity in the brain, the method does well in detecting time-varying interactions between neural variables. Through evaluation on synthetic datasets, this work shows that the proposed model achieves higher performance compared to leading methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Question 1 (Parameter selection and training cost): \n\nMost of our hyperparameters were chosen via grid-search, though we did identify some simple equations relating the $\\rho$ and $\\eta$ parameters in Eq. 5 to other parameters in the model and/or dataset that we used to adapt those specific parameters to new settings (see Appendix E.2 and corresponding Table 13). We discuss how the number of parameters affects our analyses in the first paragraph of Appendix B.4 - essentially, the high number of parameters in the REDCLIFF-S model does increase training costs and did restrict us to tuning hyperparameters on a subset of the Synthetic Systems datasets. However, we emphasize that there were 51 different systems each with 5 different repeats that we sought to use in those experiments (48 of which made it into the paper and/or appendices, see Appendix Figure 9).\n\nQuestion 2 (Ablation Studies): \n\nFollowing the reviewer's suggestion, we performed ablations on several REDCLIFF-S parameters across multiple systems. We will include the full extent of these experiments in the final paper, and report the following results obtained on our D4IC HSNR system (Sec. 4.3) here - given as the average (Avg) Optimal F1 score across repeats and factors and corresponding standard error of the mean (SEM):\n\n - Ablation: $\\rho=0$ (Eq. 5 & 10) :: 0.304 (Avg) 0.006 (SEM) \n - Ablation: $n_k=1$ (Eq. 2 equivalent) :: 0.316 (Avg) 0.011 (SEM)\n - Ablation: $\\alpha=1$ (Eq. 4) :: 0.338 (Avg) 0.009 (SEM)\n - Ablation: $\\lambda=0$ (Eq. 9) :: 0.332 (Avg) 0.006 (SEM)\n\nNote each ablation resulted in significantly worse performance than that reported in Figure 4 of our paper, verifying the utility of each parameter.\n\nConcern 1: Paper Organization\n\nWe thank the reviewer for raising valid feedback that will improve the readability of our paper; we will move Section 5 into Section 4 as a subsection and rename the section headings as suggested in the final paper.\n\nConcern 2: Related Works\n\nWe thank the reviewer for suggesting several related works for us to include in our paper, and we will be sure to incorporate them in our final draft.\n\nConcern 3: Additional Real-world (Neural) Datasets\n\nAs suggested by the reviewer, we ran an additional case study on the Social Preference dataset first used in [1]. We describe the results briefly here and will release a more detailed discussion of our findings in our final draft. We used a random subset of 20 out of the 28 mice for model cross validation, leaving the remaining 8 mice as holdouts. Regarding parameter selection, we borrowed the majority of parameters from our TST case study (see Appendix Table 13), with a few changes to reflect the Social Preference dataset. Specifically, we adjusted the number of supervised factors from 3 to 2 and assigned one to \"Social Preference\" label and the second factor to the \"Object Preference\" label. We then performed a 5-fold cross-validated grid search on the number of total factors, which resulted in the selection of the 18-factor model(s).\n\nAs with the TST case study, we averaged the predicted factors to arrive at our final predictions. We find the difference between the Social Preference and Object Preference factors resembles:\n\n - VTA_L -> Hipp\n - VTA_L -> NAc_Core\n - NAc_Core <--> NAc_Shell\n - NAc_Shell -> Amy_BLA\n - NAc_Shell -> VTA_R\n - VTA_R -> Hipp\n - Hipp -> PrL_Cx_R\n - PrL_Cx_R -> VTA_R\n\nInterestingly, the Hippocampus (Hipp) features prominently in REDCLIFF-S' predicted factor difference, whereas it does not seem as relevant in [1]. Given the complexities of studying sociability in mice and the significant differences between REDCLIFF-S and the methods used in [1], the discrepancies in these findings may be highlighting entirely different aspects of social behavior.\n\n - [1] Mague, S. D., et al. Brain-wide electrical dynamics encode individual appetitive social behavior. Neuron. 2022.\n\nConcern 4: Inclusion of ROC-AUC \n\nThe reviewer points out ROC-AUC is more popular in related works, and we will certainly include ROC-AUC performance as supplementary information in our final draft. Due to limited space, we simply note our analyses of our paper's '6-2-2', '6-4-2', and '12-11-2' Synthetic Systems (Figure 2A) suggests the reported Opt. F1 score performances largely reflect the ROC-AUC scores obtained using the same thresholded predictions (with the exception of DCSFA-NMF in the 6-4-2 system, which seems to perform similarly to REDCLIFF-S in terms of ROC-AUC, despite having a lower mean Opt. F1 score). As discussed regarding Reviewer Qgsb's Question 2, our Synthetic Systems labels omit information regarding the presence of 'background' factor activity, which limits the examples of 'true negative'/inactive relationships between nodes in our dataset and may reduce the applicability of ROC-AUC scores generally, whereas the F1-score more accurately reflects our data generation process.\n\nWe thank the reviewer again for their feedback, and hope they found our response helpful.", "claims_and_evidence": "A large number of experimental results on synthetic datasets demonstrate the effectiveness and uniqueness of the approach in this work. However, there are few experiments on real neural datasets (only one), and a lack of comparisons with other baseline models. Evaluation and comparison on more real datasets (e.g. the dataset used in [1]) could be more convincing of the model's contribution to the field of neuroscience.\n\n[1] Mague, S. D., et al. Brain-wide electrical dynamics encode individual appetitive social behavior. Neuron. 2022.", "methods_and_evaluation_criteria": "The use of dynamic coefficients to control the efficacy of multiple static models is novel to the field, although this practice is already common in machine learning. The use of behavioral information to guide dynamic coefficients is constructive and helpful. The objective function of the model is well-designed and theoretically supported.\n\nThis work uses the f1-score as the primary metric. Although the authors have claimed that ROC-AUC is less appropriate in this situation, I think this work should also report ROC-AUC results, as this metric is widely used in related work.", "theoretical_claims": "This work describes an accurate mathematical modeling of the method. A detailed discussion and derivation of the model design is also presented, which provides helpful theoretical support.", "experimental_designs_or_analyses": "As discussed in the comment of **Claims And Evidence**, more experiments on real neural datasets may help to strengthen the claim and contribution of this work.", "relation_to_broader_scientific_literature": "The key contribution of this work is to capture causal relationships between neural variables in the brain. The authors discuss broad scientific literature, including linear and nonlinear causal discovery methods, dynamic graph modeling and some applications of causal discovery in neuroscience. Some works [2-3] have proposed dynamical systems to extract low-dimensional latent dynamics from neural activity, although they did not discover causal relationships. It would be better to include these related works for discussion.\n\n[2] Linderman, S., et al. Bayesian Learning and Inference in Recurrent Switching Linear Dynamical Systems. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 2017.\n\n[3] Keshtkaran, M. R., et al. A large-scale neural network training framework for generalized estimation\nof single-trial population dynamics. Nature Methods. 2022.", "essential_references_not_discussed": "Please see the above comment of **Relation To Broader Scientific Literature**.", "other_strengths_and_weaknesses": "1. There is a lack of ablation studies for the method, such as an unsupervised version of REDCLIFF ($\\lambda$=0 in equation 9), a model without dynamic factors (coefficients are the same for all factors), a single-factor model, and a model without the cosine similarity penalty.\n2. The settings of the sections need to be fine-tuned. Section 5 is a discussion of Sections 4.1 to 4.3, which should be included as a subsection in Section 4. Section 6 is the experiment on a real-world dataset and is juxtaposed with Section 4. Therefore, it would be preferable to change the title of Section 4 to \"Experiments on simulated datasets\" and Section 6 to \"Experiments on real-world datasets\".", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. As there are many hyperparameters in the method, how do the authors perform the parameter search? Does it cause excessive training costs? How do these hyperparameters affect the model's performance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T09b7kUDqr", "reviewer": "Reviewer_GHbs", "summary": "The authors propose a factor model-based method to generate possible Granger causal hypotheses about time series data given by a time-dependent (i.e., non-static) data-generating process. The generated hypotheses are adjacency matrices and summarize possible causal effects among the observed system variables and their history. Intuitively, the proposed method learns a set of generative factor functions that predict the behavior of the time series/system given a history of $\\tau$ values as well as a state model that predicts a weighting vector of factors given said history. The factors are parameterized as neural networks where the first layer is interpreted as an adjacency matrix showing Granger causal relationships among variables while the state model is a neural network predicting the relevance of each factor given a history (thus introducing dynamic behavior and allowing to \"mix\" adjacency based on the model state).\nEmpirically, the proposed method seems to improve upon existing baselines on synthetic and real-world tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful for the reviewer's thoughtful feedback and suggestions on how we can further improve our paper. We respond to their concerns below.\n\nQuestion 1: \"with how many seeds were the experiments conducted?\"\n\nEach dataset was curated using a fixed random seed ('9999' for Synthetic Systems, '1337' for TST, '5' for D4IC) to ensure data could be generated/replicated on other devices (see provided code). With regards to model training, we used a single random seed ('0') throughout our experiments to ensure reproducibility, which can be seen in the included code (esp. at the beginning of \"_ _ main _ _\" in each of our python-based training files). In concurrence with best practices, these seeds were chosen arbitrarily so as to avoid 'seed'-hacking.\n\nQuestion 2: \"it is unclear what the authors mean by 'pairwise improvement' in Fig. 2B. Which metric is considered here exactly? Why is lower better than higher?\"\n\nWe thank the reviewer for giving us the opportunity to clarify our terminology. 'Pair-wise' here refers to the practice of matching predictions made by pairs of algorithms on the same graph/system and evaluating the differences. In the case of Fig. 2B, 'pair-wise improvement' is the difference between the optimal f1-score obtained by REDCLIFF-S (say, $f1_{redc}$) and the optimal f1-score obtained by another algorithm (say, $f1_{base}$) on a given graph $g$; that is, $\\mathrm{PWImprovement} = f1_{redc}(g) - f1_{base}(g)$. Thus, the higher the value, the better REDCLIFF-S did relative to the given baseline. In theory, the pair-wise improvement could take on any value between -1 (if REDCLIFF-S had an f1-score of 0 and the baseline's was 1) and 1 (with REDCLIFF-S' f1-score being 1 and the baseline being at 0).\n\nQuestion 3: \"what is the 'optimal f1-score'? Could the authors provide an exact definition?\"\n\nThe reviewer rightly points out an area which needs clearer discussion in our final draft. In essence, the f1-score compares positive predictions with positive labels, but this requires that a 'positive prediction' threshold be set for each algorithm. Rather than pick a threshold which may skew prediction results in our favor, we instead used a different threshold for each algorithm on each prediction such that the algorithm/baseline attained the highest possible f1-score performance on that prediction (i.e. the 'optimal' f1-score). Our current draft discusses the 'optimal f1-score' at the end of the second paragraph in Section 4 of our paper, in which we state that \"we compute each ... algorithm's f1-score using a classification threshold ... which yields the highest possible f1-score for that algorithm\", but we will absolutely expound on this in order to improve our paper. Are there any particular key points that the reviewer would like us to include in our final draft beyond what we have described above?\n\nConcern 1: Location of Figure 1\n\nWe thank the reviewer for their suggestion to improve the readability of our paper, and will adjust Fig. 1 so that it is located at the top of the page.\n\nConcern 2: Additional References\n\nWe thank the reviewer for calling our attention to the DyCAST paper, and will certainly include it in our discussion of related works.\n\nAgain, we thank the reviewer for their feedback and hope our response has been helpful.", "claims_and_evidence": "The paper claims that the proposed method generates hypotheses (i.e., Granger causal graphs) for nonlinear and state-dependent time series systems, thereby improving upon existing baselines that have stricter assumptions such as linearity or static-ness of the system.\n\nBy design, the method should be able to accurately capture nonlinear dependencies among system variables due to the use of neural networks. Also, an explicit state model is learned, representing the state of the modeled time series system and weighting a set of learned factor functions which include a representation of the Granger causal adjacency matrix. Combining these two components, the proposed method's design allows the modeling of nonlinear dependencies **and** model state-dependent time series systems.\n\nThe claims are supported by empirical evidence in the experimental section. The authors first test their method on artificially generated data that exhibits nonlinear dependencies among variables and state-dependent behavior.", "methods_and_evaluation_criteria": "The authors explicitly aim to construct a model that can be used for hypothesis generation in neuroscience applications. Hence, their evaluation of a real-world brain data problem is appropriate.", "theoretical_claims": "The authors claim and show that:\n- nonlinear, dynamic causal graphs are unidentifiable in general (App. A1)\n- assuming non-trivial noise, a window of size $\\tau$ is not sufficient to distinguish two (or more) factors if the classifier is assumed to be unbiased (App. A2)\n\nAlthough the authors show in A2 that for a time-window $T < \\tau$ it is impossible to have an accurate and unbiased estimator to distinguish between $f_i$ and $f_j$, it is not shown that having $T > \\tau$ is sufficient to achieve that goal.", "experimental_designs_or_analyses": "The experimental design is valid and sound. However, I am not familiar with neuroscience literature. Therefore, it is hard to tell whether the DREAM4 and TST datasets are appropriate choices to test the capabilities of the proposed method.", "relation_to_broader_scientific_literature": "As discussed by the authors, the work is mainly related to [1], [2], and [3], covering causal discovery and its application to neuroscience. In [1], the authors aim to discover causal models with nonlinear dependencies using local linear approximations. This restricts the expressiveness of the learned model. [2] and [3] discuss the necessity of using state-dependent models in modeling brain data and demonstrate the usefulness of modeling brain data causally.\n\n**References**\n\n[1] Fujiwara et al. Causal Discovery for Non-stationary Non-linear Time-series Data Using Just-In-Time Modeling. ICLR 2023.\n\n[2] Mague et al. Brain-wide electrical dynamics encode individual appetitive social behavior. Neuron 2022.\n\n[3] Talbot et al. Estimating a brain network predictive of stress and genotype with supervised autoencoders. Journal of the Royal Statistical Society. 2023.", "essential_references_not_discussed": "The approach is closely related to the method proposed in [1]. In [1], the authors propose to learn dynamic causal graphs from data using neural ordinary differential equations, thereby modeling temporal dependencies among variables.\n\n**References**\n\n[1] Cheng et al. DyCAST: Learning Dynamic Causal Structure from Time Series. ICLR 2025. \n\n> Note: The work in [1] was published in 2025. Thus, there is no need to incorporate it as an additional baseline, in my opinion. Nevertheless, it should be cited.", "other_strengths_and_weaknesses": "**Strengths**\n- the paper is generally clearly written\n- leveraging factor-based time series models to extract adjacency matrices is novel and innovative\n- the modeling choices are well-motivated\n- the authors test their method on real-world brain data and show significant improvements\n\n**Weaknesses**\n- Fig. 1 is too large and should be placed on the top of the page to improve readability\n- a rigor definition of \"optimal f1 score\" is missing (see questions) (Sec. 4.2)\n- it is unclear what the authors mean by \"pairwise improvement\" in Fig. 2", "other_comments_or_suggestions": "-", "questions_for_authors": "- with how many seeds were the experiments conducted?\n- it is unclear what the authors mean by \"pairwise improvement\" in Fig. 2B. Which metric is considered here exactly? Why is lower better than higher?\n- what is the \"optimal f1-score\"? Could the authors provide an exact definition?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a novel deep generative factor model for dynamic causal graph discovery, designed to address the challenges of modeling non-linear and state-dependent relationships in time series data, particularly within the context of neuroscience. The reviewers generally agree that the paper presents a promising approach with clear potential, supported by empirical results on both synthetic and real-world (though limited) datasets.

Reviewers raised concerns regarding missing experimental details, literature, and comparison and a lack of assessment of variability (e.g., with random seeds).

However, the strengths – an innovative method, a well-motivated approach, and demonstrated promising results – warrant acceptance contingent on the authors addressing the reviewers’ comments, particularly by providing better experimental details, including additional baseline experiments (e.g. in response to reviewer nDzu) and additional variability experiments.
Decision: Accept (poster)